What is a neural network (without the scare quotes)
A plain-language tour of neural networks as stacked layers of simple units with adjustable weights: loosely neuron-inspired, not a brain: with doors into Rumelhart et al. and the History serial.
Primary source: Rumelhart, Hinton & Williams, Nature (1986): learning by back-propagating errors: https://doi.org/10.1038/323533a0
“Neural network” sounds like a tiny brain in a box. The engineering picture is quieter and more useful.
A neural network is stacked layers of simple units with adjustable weights that turn inputs into outputs. The name nods at biological neurons. The software is not a brain, not a mind, and not conscious. It is a patterned calculator that got good at a job because training nudged its numbers on lots of examples.
If you want the bigger map, pattern engines vs. sci-fi minds, start with What AI actually is. This piece zooms in on the wiring diagram people mean when they say “neural net.”
The short version
- Inputs (pixels, words, sensor readings…) go in; outputs (labels, next tokens, scores…) come out.
- In between sit layers of simple units. Each unit mixes what it sees with weights, adjustable numbers, and passes a transformed signal onward.
- Training nudges those weights so the outputs get closer to the right answers on examples. See How a model learns.
- Deep usually means many layers stacked. Depth is architecture, not depth of thought.
- Chatbots are one product family built this way. Medical readers, rankers, and other systems use related stacks with different data and risks. Chat ≠ all of AI.
Tiny knobs, not tiny minds
Picture a huge panel of tiny knobs. Each knob is a weight. Turn the panel one way and the network maps a photo toward “cat”; turn it another way and the same wiring leans toward “dog.” Nobody hand-sets millions of knobs. Training does, guess, measure how wrong, nudge the knobs a little, repeat.
That loop is the same learn-from-data story as What AI actually is. The network is the structure that holds the knobs. Learning is the practice that turns them. Neither step requires understanding, feelings, or a secret inner life.
Gloss in one line: a weight is “how strongly this signal counts when building the next one,” not “what the network believes.”
What a layer actually does
A single unit is almost boring: it takes several numbers in, multiplies them by its weights, adds them up, and runs the result through a simple squash-or-pass function so the signal stays usable. Stack many units side by side and you get a layer. Stack layers and you get a network.
Early layers often pick up simple patterns (edges in an image, local word habits in text). Later layers recombine those into bigger patterns. Useful cartoon, not a promise that layer three “means” something a human would name. The network is still doing weighted mixes, not explaining itself.
No math wall required. The primary source for this piece, Rumelhart, Hinton & Williams, Nature 1986, showed a practical way to train multilayer networks by back-propagating errors: push blame for a wrong answer backward so each weight gets a nudge. That letter helped make deep stacks a community tool. Same door as What AI actually is; continuity is deliberate.
What “deep” means in one line
Deep means many layers. “Deep learning” is the habit of training those tall stacks on large data with modern compute and tricks, not a claim that the model thinks deeply, feels deeply, or knows what it is doing.
Shallower nets still exist. Depth got famous because tall stacks plus data plus training worked startlingly well on many perception and language tasks. Door: The Deep Learning Turn.
A short history without the fog
Networks of simple threshold units are an old idea. Mid-century perceptrons showed what single-layer learners could and could not do; the critique that followed cooled the hype. Door: Perceptrons and the critique.
Multilayer, brain-inspired learning returned as connectionism, simple units whose knowledge lives in the weights. Door: Connectionism returns.
Training deep stacks needed a way to assign credit for errors across layers. That is Backpropagation, with Rumelhart et al. 1986 as a landmark primary source (linked above).
None of those chapters claim a brain was rebuilt in silicon. They claim a family of learning machines got better tools.
What a neural network is not
- Not a brain. Loose inspiration from neurons is not neuroscience in a box. Biology is messier; chips and training loops are different.
- Not magic. Failures usually have ordinary causes: bad or biased data, the wrong task setup, rare cases never seen in training, or asking for something outside the learned map. Same honesty as What AI actually is.
- Not only chatbots. Language models are neural nets (often transformers, stacks that mix attention with feed-forward layers). So are many vision models, speech systems, and recommenders. The product shape changes; the “layers + weights + training” idea often stays.
- Not a guarantee of truth. Fluent outputs can still be wrong. Fluency is a pattern skill, not a librarian stamp.
Easy → technical doors
| If you’re curious about… | Easy / History next step | Technical door |
|---|---|---|
| Pattern engines vs. movie AI | What AI actually is | Rumelhart et al., Nature 1986 (shared primary source) |
| Guess → error → nudge the knobs | How a model learns | Optimizers / backprop via that post’s table |
| Early single-layer limits | Perceptrons and the critique | Perceptron / Minsky–Papert trail via History |
| Multilayer learning comes back | Connectionism returns | Connectionist papers via History |
| Training deep stacks by error signals | Backpropagation | Rumelhart, Hinton & Williams, Nature 1986 |
| Why “deep” got famous | The Deep Learning Turn | AlexNet and later papers via History |
The honest caveat
A neural network is a learnable stack of simple units and weights, useful, measurable, and limited. The neuron metaphor helps you picture the parts; it overpromises if you treat the stack as a mind. On this site, Easy reading aims for clear claims, clear limits, and a path to the primary source when you want to check the wiring yourself.