Why AI needs energy and compute (without the panic theater)

A measured guide to why training and serving models use electricity: burst vs ongoing cost, size and scale, efficiency levers: with Luccioni et al. on BLOOM as the door into what was actually measured.

Primary source: Luccioni, Viguier & Ligozat, Estimating the Carbon Footprint of BLOOM (JMLR 2023): training is a measured burst; inference is ongoing; scope (power only vs fuller life cycle) changes the story: https://jmlr.org/papers/v24/23-0069.html

AI runs on computers. Computers use electricity. That sentence is almost boring, and it is the right place to start when headlines turn energy into theater.

Nothing here is conscious. Chatbots are one product family that burns compute answering prompts. Medical-scan models, spam filters, and recommendation systems also run on chips in buildings that draw power. Chat is not the whole of AI, and “AI” is not a separate physics from other data-center work.

The short version

1. Training is a concentrated burst of compute that builds (or updates) the model. Inference is the ongoing cost of answering queries once it is deployed.
2. More parameters and more data generally mean more compute, see What parameters mean.
3. Headlines often mix one training run with global electricity. Ask what was measured, for how long, and under which scope.
4. Efficiency matters: smaller or specialized models, better chips, smarter batching, cleaner grids, not only “never use AI.”
5. History and recipe doors: Transformers and scaling, Pretrain then finetune.

Two bills: build vs serve

Gloss in one line each:

• Training, the practice loop from How a model learns: guess, measure error, nudge billions of knobs, repeat across a huge pile of examples. Expensive when the model is large and the run is long.
• Inference, run the finished weights forward to produce an answer (or a scan score, or a ranking). Usually cheaper per request than a full training run, but it never stops if the product is popular.

Building a factory vs keeping the lights on: one is a project, the other an operating cost. Mixing them into one scary number without labels is how panic theater starts.

The primary source for this piece is Luccioni, Viguier, and Ligozat’s Estimating the Carbon Footprint of BLOOM (JMLR 2023). They try to quantify emissions for a large open language model across parts of its life cycle, not only the final training run, and they also measure energy for real-time inference through an API that received user queries. Their conclusion-facing point matches good reporting: precise footprints are hard; what you include (dynamic power only vs equipment manufacturing and other stages) changes the estimate a lot. They also note that keeping a large model loaded for serving still draws energy even when almost no one is querying, idle residency is part of the bill.

You do not need their exact tonnes to use the lesson. You need the habit: name training vs inference, name the boundary, and do not treat one lab’s training estimate as “all of AI forever.”

Why bigger models tend to cost more

What parameters and ‘billion-parameter’ actually mean is the door: parameters are adjustable knobs. More knobs plus more data plus more training steps mean more multiply-adds, more work for GPUs and their cousins, more electricity in the rack, more cooling in the building.

That is why scale shows up in energy talk. History Transformers and scaling covers how loss moves with size, data, and compute, not a moral ranking. Chasing larger models without watching the compute budget is how bills grow.

Pretrain then finetune adds a cost shape many products use: one expensive general pretraining, then cheaper specialization. Reuse can amortize the big burst across many jobs, and it can also multiply inference spend if every product keeps a giant model warm 24/7 when a smaller specialist would do.

Efficiency is not “stop thinking”

Measured concern is not the same as a ban. Teams already pull levers:

• Right-sized models, match capacity to the job; distill or finetune smaller stacks when the exam allows.
• Better hardware and utilization, chips that do more work per watt; keep GPUs busy instead of half-idle.
• Batching and caching, group requests or reuse work when the product allows delay.
• Where the electrons come from, the same kilowatt-hour is dirtier or cleaner depending on the grid and the data-center design.
• Fewer wasteful training runs, tune less blindly; reuse checkpoints; report energy so trade-offs are visible.

None of that requires pretending AI is free, or that every query is a coal plant.

What headlines blur

Three common mixes:

1. One training run quoted as if it were continuous global AI use.
2. Chat traffic treated as the only AI, ignoring other models that also sit in racks.
3. Narrow scope (GPU power during training) compared unfairly to a wide scope (manufacturing, idle servers, network) without saying so.

Luccioni and coauthors are useful precisely because they show how hard clean accounting is, and because they separate building BLOOM from serving it. When you see a footprint claim, ask: training or inference? Which hardware? Which region’s electricity mix? Dynamic power only, or embodied gear too? Why evaluation matters is the cousin habit for accuracy scores; energy reporting needs the same “name the test” reflex.

Easy → technical doors

The honest caveat

AI needs energy because it is computation at scale, training as a burst, inference as an operating cost, both sitting in real buildings on real grids. Bigger models and bigger traffic tend to cost more; efficiency and measurement can reduce waste without magic. Headlines that skip training-vs-serving and skip scope are selling heat, not light. On this site, Easy reading aims for clear claims, clear limits, and a path to the primary source when you want to verify.