A quiet story in AI isn't that large models keep getting better, it's that small ones are catching up quickly. In 2022, reaching 60% on MMLU, a test of academic reasoning, required a 540-billion-parameter model. By 2024, a 3.8-billion-parameter model hit the same threshold — a 142-fold parameter reduction in two years. The direction is clear, and the implications are wide-ranging.
0:00
/0:09
Some implications: