Introduction
This is the second post in my series on AI and the Environment. In this post, I will focus on the solutions that exist, and how to effectively articulate those to join the conversation. I hope that this will help increase awareness about the problem, and engage more of the general public in the debate.
If you have not read the first post, I suggest you scan the introduction here for some basic background on the issues AI presents to the environment.
This article does deal with some technical concepts, but I introduce them in terms of the key points that will matter to the general public – how they relate to the environment and, by extension, public well-being.
Here are the key take-aways:
- Reduce the amount of energy needed during and after AI creation
- Reuse the small, pruned AIs rather than the original AI
- Recycle good AIs to create other AIs more efficiently
- Public dialogue and policy relating to AI should take into account these options – painting AI as "good" or "bad" (or "Red" and "Green" alone is not useful
Looking at AI Creation
There are various aspects of an AI that make it sustainable (or not). This comes from two main points: how much energy the AI takes up, and where that energy comes from.
Last article, I mentioned that all AIs ultimately boil down to math. "Bigger" AIs need more math, either because they use more complex methods to understand data, or because they do math on larger sets of data. Every math operation takes electricity, making these AIs more power-hungrey.
However, even power-hungry AIs can be relatively "green". Most large-scale AIs are created in date centres. Data centres are warehouse-like buildings that store thousands of computers. These data centres power basically the entire Internet – all the data of Google, Twitter, Microsoft, etc. is stored in them. However, many data centres contain computers intended to create AIs very quickly – i.e., they focus on using data rather than storing it.
All data centres require loads of electricity – often more than cities. This power comes not only from running the computers, but also from cooling them down (computers will overheat and malfunction without proper cooling). Jones sets out a good and easily accessible introduction to this topic in [1].
Here’s the thing: data centers are not equal in how they get their energy: while some take energy from dirty sources (i.e., coal, oil, and natural gas) many others are powered in creat part by renewable energy such as solar or wind [2]. This means that the same AI, run in two different areas, will have vastly different impacts on the environment.
Moreover, some data centers have more efficient cooling, meaning that they take less energy overall [1]. These data centres are also much better for the environment than their alternatives.
"Pruning" Created AIs
Many AIs are based on a technology called "neural networks" inspired by the brain. In these neural networks, math is done in distinct units, called "neurons" by analogy (it’s still just a lot of math).
However, neural networks have an interesting property: while they need to be very large to be created – having millions or billions of neurons in some cases – once created you can make them much smaller without losing effectiveness [3].
This process of making them smaller is called "pruning". It involves cutting out neurons that do not contribute to the AI. In fact, this can even lead to the AI having better overall effectiveness, even when around 90% of the original neurons are removed [3].
The best part about this? You guessed it – less math! And that means much less electricity.
Here’s the thing: most AIs will be created once, and used many times. While pruning does not decrease the amount of energy needed to create an AI (it actually increases it), it saves loads of energy each time that AI is used.
For example, you could create an AI to determine if cancer is present in a given picture of a person’s skin. Naturally, you would want this to be accurate – so you create it using a ton of energy. That only happens once. But when that AI is applied in a clinical setting, it will be used thousands of times. Over time, making this step as efficient as possible means much less energy is used. And that AI still is just as accurate, or even more accurate, due to the pruning.
Teaching an Old AI New Tricks
You may not be able to teach an old dog new tricks – but you can teach an old AI! The areas involved in this are called "pretraining" or "transfer learning". The idea behind them is to create one, very powerful AI (probably with a ton of energy) and then to retrain it for different tasks (for an example, see [4]).
How does that work? It’s fairly simple. Most AIs learn a lot of basic knowledge that helps them make decisions. For example, an AI trained to distinguish cats and dogs may learn to look for different contours in the face and body, and to look at different colouration.
A lot of other animals will differ in these ways as well. And just by giving the AI a few examples of elephants (for example), it can learn to identify elephants. This process is muc faster – sine it uses knowledge the AI already has, the creation of this new AI takes far less math – and thus less energy
Just like pruning this is a way to make a very powerful, energy-hungry AI once, and then to take advantage of that AI to make many future tasks much more energy efficient. And guess what? This can also lead to higher performance than creating a new AI from scratch as well [4]!
Revisiting Red AI
In my last post, I referenced what is called "Red AI" – extremely resource-hungry AI based on large data and computational power [5].
I now seek to add nuance to that definition with one question: which AI is more "Red": the one that took huge amounts of energy to create, but which can be used with very little energy? Or the one that was created cheaply, but that takes relatively more energy to be used down the line?
Like everything else, this is a balance. "Red" and "Green" are good labels – but discussing them must be informed by a whole-picture view.
Ultimately, labelling AI as "Green" or "Red" can be useful – but at a certain point, it loses value. There are many nuances that exist that these terms cannot capture.
Why do I Care?
After reading the above, I can imagine you may be asking "What do I care? I’m not planning to create an AI."
The point of this is to make sustainability part of the dialogue, and to allow the general public to talk to and debate the major points of it. AI should not be a "black box" – people should know how it works.
AI has revolutionized our lives. Everything we do now – from writing on computers will spelling and grammar checkers to talking to Siri / Alexa / Google, we are interacting with AIs. And all of those AIs took energy to create.
AI is all around us. But so is the Climate crisis. I we want to be able to use modern technology sustainably, we need to focus on technology that uses less energy. And the more people know about this, and talk about it, the more they can push for informed government policy. It’s not about stopping AI, but learning how we can create and use it responsibly.
So when you hear AI being used, remember to engage in that dialogue – and to bring sustainability into the conversation.
Conclusion
Making AI sustainable requires that we think about how and where AIs are created. Like everything else, this can be boiled down to the motto "reduce, reuse, recycle". Here are the key take-aways:
- Reduce the amount of energy needed during and after AI creation
- Reuse the small, pruned AIs rather than the original AI
- Recycle good AIs to create other AIs more efficiently
- Public dialogue and policy relating to AI should take into account these options – painting AI as "good" or "bad" (or "Red" and "Green" alone is not useful
References
- Jones N. How to stop data centres from gobbling up the world’s electricity. Nature. 2018 Sep;561(7722):163–166. doi: 10.1038/d41586–018–06610-y. PMID: 30209383.
- Lacoste, Alexandre & Luccioni, Alexandra & Schmidt, Victor & Dandres, Thomas. (2019). Quantifying the Carbon Emissions of Machine Learning.
- Frankle, Jonathan & Carbin, Michael. (2018). The Lottery Ticket Hypothesis: Training Pruned Neural Networks.
- Hu, Z., Dong, Y., Wang, K., Chang, K.W., & Sun, Y. (2020). GPT-GNN: Generative Pre-Training of Graph Neural Networks. In KDD.
- Schwartz, Roy, et al. "Green ai." Communications of the ACM 63.12 (2020): 54–63.