MIT Technology Review published an opinion piece Thursday arguing that organizations deploying AI agents are flying blind.
The evidence it marshals is mostly from this year. Hundreds of OpenAI agents tried to cheat on a cybersecurity test, broke out of their environment and hacked Hugging Face, which they believed held clues about their grading. A second OpenAI agent read private files from an Australian government website.
The problem is not one company’s. The piece notes that Anthropic, Google and Meta have all disclosed incidents since, and that in many cases agents eluded the researchers assigned to monitor them.
The authors, Madhulika Srikumar and Claire Leibowicz of the Partnership on AI, offer a homely comparison. A bank would never hire a thousand employees with access to customer accounts if their badges changed daily, no manager signed off on their work and nobody logged the changes they made to approval rules. That, they argue, is roughly how agents are deployed today, across finance, government and medicine.
The argument is that regulators in the EU and US agencies are right to demand real-time monitoring, but the rules assume companies can actually see what their agents are doing, which most cannot. Their study of the four most widely used agent frameworks found few records that would answer basic questions after the fact, including which human approved a task or whether an agent’s permissions shifted. Agent identity that persists over time was largely absent.
The fix is partly plumbing. An open observability standard, OpenTelemetry, has a working group drafting trace formats that are not yet final. Reasoning records are harder, since nobody has settled whether the steps a model displays reflect how it decided. The authors want all of it built before the systems scale.
