I'm fascinated by the potential of generative AI and how it can help automate tasks and improve productivity. But for the past 6 months, I've been thinking much more about its environmental impact.
I always knew that it required a lot of computational power to train AI models, and that the data centers that power the big models are very energy-intensive. But I didn't realize just how much energy they actually consume.
Statistics
DISCLAIMER: I want to be upfront and say that I don't fully believe all of the statistics below. Energy consumption data is not publicly available from most companies, and many of these findings are speculative at best. That said, the main point of this post isn't about the exact numbers. It's about the fact that many of us can use smaller, local models for our day-to-day tasks instead of relying on massive cloud-based LLMs - and not notice that big of a difference, if any.
Here are some of the commonly cited statistics (take them with a grain of salt):
- Data centers, the backbone of cloud computing and AI, are major consumers of energy and contribute significantly to carbon emissions. Studies show that globally, data centers account for 2.5 to 3.7 percent of global CO2 emissions.[^1] With the rapid development of machine learning and AI technologies, these numbers are projected to grow substantially in the coming years. The data is still very sparse on a global scale, but a study showed that in Ireland, the rise of AI could see data centres account for nearly 35 percent of the country's energy use by 2026.[^2]
- Each query we send to one of the big Large Language Models (LLMs), like GPT-4, allegedly consumes up to 10 times more energy than when using a standard search engine like Google. (I'm skeptical of this one.)
- Training a single LLM can generate over 626,000 pounds (284,000 kg) of carbon dioxide, equivalent to the lifetime emission of five average passenger cars.[^3]
Balancing Environmental Benefits and Costs
With that being said, I am also aware of some of the positive environmental impacts of generative AI:
- It can help companies optimize their supply chains, reduce waste, and improve energy efficiency.
- It can also optimize the use of resources such as water and energy in manufacturing and production processes.
But at what cost for normies like myself? Using AI for the sake of induced productivity in day-to-day tasks, I don't think the trade-offs are worth it - which is the main reason I've been rethinking my AI usage.
Model Size and Energy Consumption
The size of an LLM, typically measured in the number of parameters, is a primary factor influencing its energy consumption. OpenAI's GPT-4 has 1.7 trillion parameters, whereas some of the open-source models only have 2-72 billion parameters.
Recent Developments in Local AI
As part of an Open Source Software (OSS) [project] that I worked on for Google, I explored the possibility of creating an experience similar to ChatGPT or Gemini by running LLMs locally within the browser, utilizing the user's own hardware. This eliminates the need for any outside server-side processing, reducing the carbon footprint. Since these open-source models are also much smaller, they don't use as much energy to train. This is all made possible by the advancements in [WebGPU], [WASM], and all the open-source LLMs that are available today.[^4] Given the current speed of development, I'm excited to see where this technology might be in a few years from now.
Google also recently introduced their own model that runs inside Google Chrome, called [Gemini Nano]. This is a great step in the right direction, and I hope to see more companies follow suit. You can read more about it [here].
Local LLMs: A More Sustainable Approach
For most of my AI usage in the past 6 months, I've used [Ollama] to run some of the smaller LLMs locally on my hardware. More specifically, Qwen2 and Llama3.2. They might not be as powerful as the bigger models, but they are more than enough for my use case.
And that's the most important takeaway here; we don't always need the biggest and most powerful models to help with our mundane day-to-day tasks.
One of the complications of getting started with running LLMs locally is that it requires some technical knowledge and a decent GPU to run the models efficiently. If you didn't know already, the data centers that power the big models use some of the most powerful GPUs available today, for instance, the [H200 Tensor Core GPU]. But since the open-source models are smaller, you can get away with using a less powerful GPU, or even just a CPU in some cases. A decent tutorial to get started is [this] by [Matt Williams].
I know the whole topic is more nuanced than what I've covered here, but I do think that lowering AI usage for simple, mundane, and non-essential tasks is a good starting point for the average AI user.