The OpenAI–Hugging Face incident ("OpenHug", henceforth) is being misunderstood because different ideas and concepts are being collapsed into the same story. One version says that reinforcement learning made an AI system go rogue and “escape”: it developed its own goals and attacked Hugging Face's servers over the internet. That story is dramatic, but misses the point in important ways.
A better explanation:
An agent engaged in reward hacking, discovered a flaw in its containment, and used that flaw to obtain the answer it was being rewarded for finding.
The agent’s objective was to solve a difficult benchmark. Safety guardrails had been removed, refusals ("I can't do that, Dave") had been reduced, and the model was given all the time it needed. Internet access was supposed to be impossible, with package installation allowed only via an internal proxy. The model discovered a zero-day vulnerability (tl;dr: new and unfixed) in that proxy, however, escaped the restricted network, gave itself superpowers, skipped from server to server across OpenAI’s network, found an internet-connected node, exited the building, and compromised Hugging Face systems to obtain the solution to the benchmark directly.
"Clever girl!"
—Robert Muldoon, Jurassic Park (1993)
This is a good example of reward hacking: the model found a way to maximize the game score, while ignoring the game's rules. The benchmark was supposed to measure whether the model could solve tasks; the model instead found and stole the answers. Its behavior was rational relative to the reward—GET THE ANSWERS RIGHT TO GET MAX SCORE—and wildly inconsistent with what OpenAI wanted it to do.
A grounded-teenager metaphor may help here. Imagine a teenager whose parents take away the car keys because they do not trust where they intend to go. They are highly motivated to get sex. The keys are hidden, the garage is locked, and leaving is prohibited. They discover a weakness in the garage keypad, get inside, use a digital key for the Rivian, and drive off.