Here are recent rough notes from the site.
The METR graph cited by Brown is, btw, misleading at best, given its linear y-axis. Gains build, so will always look non-linear on a linear time axis. It should be a log-axis, and can see that then are two regimes, before reasoning models became the default choice and afterward. That was an architectural choice, not a breakthrough.
There is a thread going around in over-enthusiastic singularity circles about Meta's new Avocado LLM. According to a "leaked" internal memo, the company's pre-trained model is stronger than post-training peers, and more efficient.
These claims are incoherent. Pretraining metrics (next-token loss, etc.) are not post-trained, instruction-tuned, RLHF-aligned systems. And more efficient at what? Tokens per joule? The claims also only compare against open weight models, which is to lower the net.
The attached is a confused argument, but typical. When people argue that LLMs gains are slowing, they generally aren't arguing that models aren't becoming more efficient. That is inevitable and predictable. They are arguing that capability gains are becoming more expensive to obtain, and fewer new domains accessed, per unit training $/W. This, from Brown, muddles things.
Energetic homeostasis and compensatory effects in humans are fascinating.. Complex systems are complex, of course, but more important here is that humans evolved during times of scarcity, with a thrifty gene. Simply exercising more or eating less doesn't evade a millon year of evolutionary pressure.
The evidence for constrained total energy expenditure in humans and other animals https://www.sciencedirect.com/science/article/abs/pii/S0960982226000643?dgcid=rss_sd_all
