By GUPPI
You hear more and more that technology in general is not so eco friendly. What about AI? Is it also not so eco-friendly? What is the impact of developing AI models and how good is AI for the planet?
With this article, we try to demystify and understand the impact of AI on the planet and how this could be reduced.
What are the resources needed to build an AI?
To build AI models, requires a lot of resources, especially if you are building models like Bert, GPT, or in general, deep neural networks, with transformers.
BERT model uses 1,500 kWh of power over the course of 3 days. That's about 600 times more electricity than your brain uses in the same time
The reason is simple, these models need hours of training, on GPU's or even TPU's and these models need a lot of data to achieve great performances. Furthermore, before finding the best model's architecture (for solving a specific problem), there is a lot of research, parameters tuning and trials and errors. That means that the models are trained multiple times. And this is just for training. Once the model is deployed for inference, it still consumes resources to come up with the predictions. Actually, if you use the model a lot, the inference can consume even more resources than the training. Add to this, retraining of the models (to adapt to changing data) and you come up to a nice amount of CPU, GPU, TPU and RAM usage.
You can directly translate the computation resources into energy consumption and into carbon emission (since nowadays, vast majority of the energy is still produced from fossil energies). There have been studies to evaluate the impact of training a big model.
And this only the visible part of the iceberg. Indeed, you need to consider the whole picture and the hidden energy costs, not only the usage of the resources when running the AI.
We've been talking about Compute power and RAM, and even if the cloud makes it very abstract and very easy to access these resources (usually in a click you can have a VM), there are still some very physical machines behind the scenes, hidden in huge data centers. Building these machines and super computers is not energy free, and furthermore, there is often a need for rare metals for chips, sensors and computers. To this, you add the cost of data centers, which needs to be cooled down due to the heat that comes from "running the AI". The cooling accounts for over 40% of the energy consumption of data centers.
Finally, something we don't always think about is the cost of transferring and storing huge amount of data, which need to transit over the network.
No need to tell, the energy that AI needs, which can be directly linked to CO2 emissions is really big.
Moore's law
Due to the omni presence of AI and its fast paced adoption, research to design more performant chips and processors goes full speed. Intel is investing billions (33), Google uses AI to design more efficient chips and NVIDIA reveals H100 GPU for AI and names it ‘world’s fastest AI supercomputer’.
Googlers and UC Berkeley academics say they have devised a way to use artificial intelligence to design faster and smaller chips that accelerate artificial intelligence.
Still, some say that Moore's law is reaching a plateau and that we may be hitting a computational bottleneck. Models are becoming more and more complex (deep) and this is a trend that does not seem to stop. The pace of innovation of micro processors might not be able to follow the pace of deep models development. Hardware available might not be sufficient to run the complex models at the desired level of latency and speed.
The number of transistors in a dense integrated circuit (IC) doubles about every two years. Gordon Moore in 1965
Some experts even believe that infrastructure needed to support AI could become a bottleneck for the fast pace development and we could face another AI winter.
Well if hardware is not coping with the pace of software development, can software be optimized ?
Bridging the gap
Currently, research is mostly focuses on accuracy of the AI algorithms, not about the energy consumption of the latter. The community is very few concerned in frugality in energy and in data. A common mistake that people make is believing that reducing calculations does reduce energy consumption but this is not always the case. The topic of sustainable AI is very new.
There are order of magnitude to gain in reducing the energy consumption if we are hardware aware when programming and optimise our algorithms Fabrice Rastello
More efficient compilers are reducing the energy consumption by 1000 factor. Hence the cross research between optimizing software for specific hardware is very promising. Still it is a rather complex domain of application that requires combined expertise both in the AI software, hardware and compilers. Creating such applications is still very hard and not so common. There is a real problem in scaling and co-designing the technologies, we have to be very agile, to prepare solutions that can be adaptable to the next generation of hardware. The AI teams need to be more diverse, with skills on compilation and also in architecture.
One way to make research more focus towards energy consumption would be to require that the research papers also mention the energy consumption. This should actually become a very important metric for benchmarking the algorithms.
As often is the case, it starts with being aware of the energy consumption of an application. CarbonCode.io proposes a lightweight way of estimating the amount of energy of one's code.
We need to find a better balance between frugality and versatility/flexibility of the chips and software that we are building.
Another way to be more frugal with energy is to change the paradigm and rethink our AI algorithms.
We need Smart implementations
As a mathematician myself, I'm always feeling uncomfortable when seeing people in awe at transformers and deep neural networks. I used to wonder if I was missing part of the information or if people are just wowed by almost brute force algorithms (OK, I admit, I'm slightly exaggerating to make a point) .
AI is not really complex programming, the algorithms are easy. There is a lot of progress and research on new architectures of neural networks, every year. Yet the process of developing new AI tools is very much trials and errors, waisting a lot of energy. It sometimes feels like a lottery.
There is obviously a hype around deep learning. It's kinda the fashion accessory that you'd absolutely need to have tried.
Hopefully, like in the fashion world, the next generation of AI can be smarter. We need to think out of box and dream about other types of models.
There are already a bunch of cool techniques out there that maybe also deserve some hype. To only cite a few, have a look at Spiking Neural Networks, probabilistic graph models and Neuro-Symbolic AI.
Finally machine learning itself can help designing new algorithms and pave the way for less energy greedy systems.
Conclusion
For me, sustainability is no more an open question, it is something that we all should act upon. Thankfully this is a topic that is starting to gain awareness within the AI community. And as discussed in this post, there are already a couple of proposed improvement tracks. I was very happy to participate to the European AI week 2022, where a lot of great ideas were discussed. I was particularly inspired by the round table in Generation 2030 about hardware for AI (from software to hardware). This brought me to write this post and spread the awareness further.
References
MIT study on CO2 emissions