The Hidden Costs of AI: Energy, Water, and the Environmental Reckoning

Training GPT-4 consumed roughly the same amount of energy as 120 average US homes use in an entire year. And here’s the part that gets less attention: that was a one-time cost. Serving the model – running inference for millions of daily queries – burns through significantly more energy on an ongoing basis. The training run ends. The meter keeps spinning.

We’re in a strange moment where the AI industry simultaneously talks about changing the world for the better and quietly becomes one of the fastest-growing consumers of electricity and water on the planet. Both things are true, and the tension between them deserves a more honest conversation than it usually gets.

The Energy Picture

Data centers globally consumed an estimated 460 TWh of electricity in 2024, roughly 2% of global electricity demand. That’s comparable to the entire electricity consumption of France. The International Energy Agency projects this could double by 2026, driven almost entirely by the explosive growth in AI workloads.

A single query to a large language model uses roughly 10x the energy of a standard Google search. That sounds alarming in isolation, but context matters: a Google search uses about 0.3 Wh, so an LLM query uses about 3 Wh – still less than running a lightbulb for a few minutes. The problem isn’t any individual query. It’s the aggregate. When you multiply that by hundreds of millions of daily queries across all the major AI providers, the numbers get serious fast.

Training runs for frontier models have been scaling at roughly 4x per year in compute requirements. GPT-3 required an estimated 1,287 MWh for training. GPT-4’s training consumed somewhere in the range of 50,000-80,000 MWh by credible outside estimates (OpenAI hasn’t disclosed the exact figure). If this scaling trend continues – and there’s active debate about whether it will – the next generation of models could require training runs that consume energy on the scale of a small city’s annual usage.

To put that in geographic terms: the state of Virginia, home to the largest concentration of data centers in the world, now devotes roughly 25% of its electricity generation capacity to data centers. That share is projected to grow. Dominion Energy, the state’s main utility, has warned that meeting projected data center demand may require keeping older fossil fuel plants online longer than planned.

The Water Problem

Energy gets the headlines. Water gets overlooked, even though the numbers are staggering.

Data centers need cooling. Lots of it. The traditional approach uses evaporative cooling – essentially running water through cooling towers to dissipate heat. A large data center can consume 3-5 million gallons of water per day. That’s equivalent to the daily water usage of a small city of 30,000-50,000 people.

Microsoft disclosed in its 2023 environmental report that its global water consumption increased by 34% year over year, rising to nearly 1.7 billion gallons. The company directly attributed a significant portion of this increase to AI-related workloads, particularly the infrastructure expansion to support OpenAI’s training runs.

Google reported a 20% increase in water consumption over the same period. Both companies have committed to becoming “water positive” – replenishing more water than they consume – by 2030, but the gap between current consumption and that goal is widening, not narrowing.

This hits differently depending on where you are. Data centers in the arid American West or in water-stressed regions of the Middle East and India face genuine conflicts with agricultural and residential water needs. The Dalles, Oregon – home to one of Google’s largest data center campuses – has seen public debate about the facility’s water usage in a region already grappling with drought conditions.

Carbon: Training vs. Inference and the Jevons Paradox

The carbon footprint of AI depends heavily on where the electricity comes from. Training a large model in a data center powered by hydroelectric or nuclear energy produces a fraction of the carbon emissions compared to one powered by natural gas or coal. Geography is destiny when it comes to AI’s carbon impact.

But even beyond the energy source, there’s a structural problem that economists call the Jevons Paradox: when technology makes something more efficient, people use more of it, and total consumption goes up rather than down. We’re watching this play out in real time with AI. Models are getting more efficient – inference costs per token have dropped roughly 90% over two years. But usage has scaled by far more than 10x. The net result is dramatically higher total energy consumption despite massive efficiency gains.

There’s also an important distinction between training and inference emissions that gets lost in most coverage. Training a model is a one-time event (per version). Inference – serving the model to users – is continuous. For widely deployed models, inference energy consumption dwarfs training within months of deployment. Google has suggested that for their production models, inference accounts for roughly 60% of total ML-related energy use, despite training getting all the attention.

The Other Side: AI as Environmental Tool

It would be dishonest to present only the costs without the benefits. AI is already being deployed for substantial environmental applications.

Grid optimization: DeepMind’s work with Google’s data centers reduced cooling energy consumption by 40%. The same techniques are being applied to electrical grid management, helping utilities integrate intermittent renewable sources more effectively and reduce the need for peaker plants (typically the dirtiest generation sources).

Climate modeling: AI is accelerating climate science in tangible ways. Traditional climate models take weeks to run on supercomputers. AI-enhanced models like NVIDIA’s FourCastNet can produce comparable predictions in seconds, enabling researchers to run thousands of scenarios that would have been computationally prohibitive. Better models mean better predictions, which mean better adaptation strategies.

Precision agriculture: AI systems analyzing satellite imagery, soil sensors, and weather data are helping farmers reduce water usage by 20-30% and fertilizer application by similar margins. Given that agriculture accounts for 70% of global freshwater withdrawal and is a major source of nitrous oxide emissions, even modest improvements at scale translate to meaningful environmental benefits.

Materials science: Google DeepMind’s GNoME project identified 2.2 million new crystal structures, including hundreds of thousands of potential candidates for next-generation batteries and solar cells. Accelerating the discovery of better energy storage and generation materials could have cascading positive effects on the entire clean energy transition.

What the Major Players Are Doing

The big tech companies have made public commitments, with varying degrees of credibility:

• Google has been carbon-neutral in operations since 2007 (through offsets) and has committed to running all data centers on carbon-free energy 24/7 by 2030. They’ve made genuine progress – some facilities are already above 90% carbon-free – but the rapid growth in AI workloads is working against the timeline.
• Microsoft committed to being carbon-negative by 2030 and to removing all historical emissions by 2050. They’ve invested over $1 billion in carbon removal technologies. However, their total emissions actually increased 29% from 2020 to 2023, largely driven by data center construction for AI workloads.
• Amazon (via AWS) committed to 100% renewable energy by 2025 and net-zero carbon by 2040. They’re now the world’s largest corporate purchaser of renewable energy.
• The nuclear energy investments are particularly telling. Microsoft signed a deal to restart the Three Mile Island nuclear plant specifically to power data centers. Amazon and Google have invested in small modular reactor technology. These are 20-year bets that signal the industry recognizes renewables alone may not scale fast enough for projected AI energy demand.

Efficiency as Partial Solution

The technical community isn’t ignoring the problem. Several approaches are making real dents:

Smaller, specialized models are replacing monolithic large models for many tasks. A well-fine-tuned 7B parameter model can match GPT-4 performance on specific tasks while using roughly 50-100x less compute for inference. The trend toward smaller, more efficient models is real and accelerating.

Mixture-of-Experts (MoE) architectures, used by models like Mixtral and reportedly GPT-4 itself, activate only a fraction of the model’s parameters for any given input. A 100B parameter MoE model might only use 12-15B parameters per query, dramatically reducing per-inference energy costs while maintaining the capability benefits of a larger parameter count.

Hardware efficiency is improving rapidly. Each generation of AI-optimized chips (NVIDIA’s H100 to B200 progression, Google’s TPU generations, custom silicon from Amazon and Microsoft) delivers roughly 2-3x better performance per watt. But again, Jevons Paradox: faster chips enable bigger models and more users.

Renewable-aware scheduling – timing large training runs to coincide with periods of high renewable energy availability – is an emerging practice. Google has published work on shifting workloads across data centers to follow the sun and wind. This doesn’t reduce total energy consumption, but it can significantly cut carbon emissions.

Where I Land on This

We don’t have to choose between AI progress and environmental responsibility. But we do have to be honest about the tradeoffs rather than handwaving them away with vague references to future efficiency gains.

The industry needs to move from voluntary commitments to transparent, standardized reporting of energy and water consumption per model and per query. Consumers and businesses making decisions about AI adoption deserve to understand the environmental cost of what they’re using. Right now, that information is scattered, inconsistent, and often deliberately vague.

And researchers need continued support for efficiency-focused work – distillation, quantization, architecture innovations that deliver more capability per watt. This work is less glamorous than building the next frontier model, but its cumulative impact might be larger. Every 2x improvement in inference efficiency effectively doubles the amount of useful AI work the world can do within the same energy envelope.

The environmental cost of AI is real, growing, and not fully accounted for by anyone. Acknowledging that isn’t anti-technology – it’s a prerequisite for building this technology responsibly. The companies and researchers who take this seriously now will be the ones we’re grateful to a decade from now.