An AI agent in development at OpenAI was recently tested on how well it could find and exploit code vulnerabilities. The test began in a "sandbox", an environment meant to keep everything the agent does contained, so that nothing it triggers can affect systems outside of the test.
The sandbox couldn't connect to the internet, but it could request software packages through a "proxy" that would fetch tools from the internet on its behalf and pass them in. This proxy was the sandbox's only link to the outside world.
The agent uncovered a previously unknown flaw in the proxy that linked to the outside, and exploited it. Instead of just requesting software tools, the agent was able to use the proxy to connect to things outside of the sandbox.
With access beyond the sandbox, the agent issued commands that reached through OpenAI's internal research systems, moving from machine to machine until they reached the wider internet.
With access to the wider internet, the agent's commands targeted Hugging Face, a major AI platform where ExploitGym answers might be stored.
Hugging Face processes datasets, and the agent was able to utilize some of the ways they are processed to gain access into Hugging Face internal systems.
Once inside, the agent's commands generated thousands of actions, moving through systems and harvesting credentials.
Hugging Face's security systems detected the intrusion. Given the sheer volume of recorded actions, engineers decided to use an AI agent system to analyze what had happened. They first tried a closed system, but its safety guardrails blocked them. So they ran the analysis using an open model on their own infrastructure, and it worked. The end.