Training vs inference (building the model vs using it)
A plain-language guide to the two AI bills: training as an expensive practice burst that updates weights, inference as the ongoing cost of running the fixed model: with Luccioni et al. on BLOOM as the door into why the split matters.
Primary source: Luccioni, Viguier & Ligozat, Estimating the Carbon Footprint of BLOOM (JMLR 2023): separates training burst from serving/inference; what you include in the boundary changes the estimate: https://jmlr.org/papers/v24/23-0069.html
Headlines often smash two different stories into one scary number. One story is building (or updating) a model. The other is using it to answer you. Those are not the same bill.
Training is practice that updates the model’s internal numbers. Inference is running those fixed numbers to produce an answer. Same family of math, different job, different cost shape.
Nothing in that picture is conscious. Chatbots are one product family that gets trained, then served. Medical-scan scorers, spam filters, and recommenders do the same split. Chat is not the whole of AI.
The short version
- Training = the practice loop that updates weights (an expensive burst when the model is large). Inference = run the fixed model to answer (an ongoing operating cost).
- Practice loop: How a model learns. Energy and scope: Why AI needs energy.
- Why normals care: your chat bill is mostly inference. “We trained a new model” is a different cost story from “millions of people typed today.”
- Finetune is more training, not inference. Specializing a pretrained stack still nudges weights.
- Bigger capacity and bigger traffic tend to cost more, see What parameters mean; reuse recipes live in Pretrain then finetune.
Training: practice that updates the knobs
Training is the loop from How a model learns: guess, measure how wrong, nudge the parameters a little, repeat across a huge pile of examples. The “aim” is millions or billions of adjustable knobs (see What parameters mean).
While that loop runs, the model is changing. Checkpoints get written. Gradients flow. GPUs stay busy for hours or days on a large run. That is why people call training a burst: a concentrated project cost, not a per-message fee you notice in a chat app.
When a lab says “we trained a new model,” they usually mean this kind of work: long pretraining, a mid-size adaptation, or a short specialized run. The shared feature is: weights move.
Inference: run the fixed model to answer
Inference is what happens after those weights are frozen for a product release. You send a prompt, a scan, or a ranking request. The system runs the network forward and returns an output. It does not, in the ordinary serving path, re-run the full training loop on your message.
Gloss once: inference = use the finished knobs, not practice them again.
Per request, that is usually cheaper than a full training run. Popular products still turn it into a large bill because the meter never really stops: many users, long contexts, and idle servers kept warm for a fast first reply. Building a factory vs keeping the lights on: one is a project, the other an operating cost.
Why your chat bill is mostly inference
If you pay for a chatbot plan, or if a company pays a cloud bill for an API, the day-to-day spend is mostly serving: tokens in, tokens out, chips spinning for answers. That is inference (plus keeping models loaded).
“We trained a new frontier model” is a different headline: rare, often huge, paid by a lab or partner, not the same line item as last Tuesday’s chat traffic. When news collapses both into “AI uses energy” without labels, ask: training burst, ongoing inference, or both mixed without saying so?
Why AI needs energy covers electricity and reporting scope. This piece stays on the name the stage habit: build vs use.
Finetune is still training
Pretrain then finetune is the common recipe: learn general patterns first, then specialize for a product job. Finetuning uses a smaller or more targeted set, and it is often cheaper than pretraining from scratch. It is still more training. Weights update. Calling finetune “just inference with vibes” mixes the dictionary.
After finetune, the product you meet again does inference on the new weights. Practice changes the knobs; the shift uses what practice already built. Instruction and preference tuning for chat tone is a flavor of that training family, not magic that skips the update step.
What a careful paper actually separates
The primary source for this piece is Luccioni, Viguier, and Ligozat’s Estimating the Carbon Footprint of BLOOM (JMLR 2023). They quantify emissions for a large open language model across parts of its life cycle, and they also measure energy for real-time inference through an API receiving user queries. The habit that matters: do not treat “the model’s footprint” as only the final training run, and do not pretend serving is free.
Clean numbers are still hard. Dynamic power only vs a fuller life cycle (including equipment manufacturing) changes the estimate a lot. A large model kept loaded still draws energy even when almost nobody is querying. You do not need their exact tonnes. You need the split: training burst vs serving / inference, plus an honest boundary for what was counted.
Easy → technical doors
| If you’re curious about… | Easy / History next step | Technical door |
|---|---|---|
| Guess → error → update | How a model learns | Optimizers / backprop via that post’s table |
| Knobs, size, capacity | What parameters mean | Kaplan scaling laws via that post |
| Expensive general → cheaper specialist | Pretrain then finetune | BERT / transfer via that post |
| Energy, grids, and reporting scope | Why AI needs energy | Same Luccioni et al. paper, energy framing |
| Measured training + inference footprint | This piece’s primary source | Luccioni et al., BLOOM carbon footprint (JMLR 2023) |
The honest caveat
Training updates the model; inference uses it. Your everyday chat cost is mostly the second story. A lab’s “we trained something new” is mostly the first. Finetune belongs with training, even when it is shorter than pretraining. Footprint claims still need a named boundary: stage, hardware, and what was included. None of that requires panic theater, and none of it implies a mind behind the weights. On this site, Easy reading aims for clear claims, clear limits, and a path to the primary source when you want to verify.