Tag
The 2017 transformer reorganized sequence modeling around self-attention without recurrence; GPT and BERT are two objectives on that backbone; Kaplan and Chinchilla scaling laws are empirical regularities about compute, data, and parameters—not a slogan that bigger is always better.
What moved in 2012 was not a sudden invention of deep nets: ImageNet-scale labeled data, GPU training, ReLU, and dropout made a large ConvNet win a public, comparable benchmark—AlexNet as the ImageNet moment, not the first deep network.
An 85-task infra/eval benchmark spanning kernels, long-horizon repo work, and end-to-end optimization — topped by Claude Opus 5 at 36.53%, with a lot of headroom left.