Eighty-five percent of enterprises are piloting AI agents right now. Five percent have shipped them to production. That is not a readiness problem. It is a framing problem. And the clearest explanation of what is actually going wrong came from Amazon’s AGI lab this week.

Bryan Silverthorn, Director of AGI Autonomy at Amazon, told VB Transform 2026 on July 15 that the gap between piloting and shipping is not about capability. The models work. The demos impress. The prototypes pass internal evaluations. Then they hit the real world and break. Half of surveyed enterprises have shipped an agent that passed every internal test and still failed a customer. The pattern is consistent enough to name: the problem is not the agent. The problem is how you think about the agent.

Inside Amazon’s AGI lab, researchers call their agents “interns.” Not as a joke. As an operating philosophy. “I’ll have my intern talk to your intern” is real language inside the team. The framing carries a specific set of expectations. Interns are powerful. They can do real work. They also do things wrong in ways you would not predict. They need supervision, clear boundaries, backup plans, and someone paying attention to what they are doing before the work goes out the door.

That is the management layer most companies skip entirely.

The evaluation trap

Silverthorn described a customer that deployed an agent for software QA. It extracted serial numbers from screens. Worked perfectly for two months. Then it started reading wrong numbers intermittently. The cause was invisible to anyone looking at the agent’s outputs in testing. A software update shifted where serial numbers appeared on screen. The underlying system processed those locations differently. No benchmark caught it because no benchmark tested for that kind of drift.

His point was not that testing is useless. His point was that the tests most companies run measure the wrong things. They confirm that the agent can do the task. They do not confirm that the agent can do it correctly a thousand times in a row, across conditions that shift without warning.

VentureBeat’s own research backs this up. Most enterprises track whether their agents are online. Almost none track whether the answers are right. They are checking the pulse without checking the diagnosis.

Silverthorn breaks reliability into four dimensions, drawing from Princeton research: consistency, robustness, predictability, and safety. Most evaluations collapse all four into a single pass/fail gate. A team that separates them can spot exactly where an agent starts to drift before customers notice. A team that does not ends up with an agent that passed evaluation and failed production, which is exactly what half the industry is experiencing right now.

The frame determines the outcome

Here is where the insight gets practical. If you think of an AI agent as software, you test it like software. You run it through a QA cycle, it passes, you ship it, and you move on. When it breaks in production, you treat it as a bug and file a ticket. That is the cycle most companies are stuck in. It is why 85% pilot and 5% ship. The testing model assumes the agent is static. It is not.

If you think of an AI agent as a collaborator, you manage it like a person. You ask what could go wrong. You build in undo capability. You decide in advance what level of error is acceptable for this particular workflow. You check its work regularly, not just at deployment. You give it clear instructions and constraints before it starts, not after it fails.

Amazon’s lab has one agent running experiments around the clock on its own research plan. They accept that it will occasionally run the wrong experiment. That is a conscious tradeoff for velocity, not a failure of testing. The difference is that the team made the decision before the agent started working, not after it produced a wrong result and someone had to figure out who was responsible.

This is the shift most organizations have not made. They are still treating agents as products to ship. The companies actually getting agents into production are treating them as collaborators to manage.

What this means for your team

The practical takeaway is not “be more careful with testing.” It is a question of organizational posture. When your team deploys an agent, do they hand it off like a software release, or do they assign someone to manage it like a new hire?

The companies stuck at 85% have a testing problem they think is a capability problem. The models are good enough. The benchmarks are not the bottleneck. What is missing is the management infrastructure around the agent: who supervises it, how errors get caught before customers see them, what the acceptable failure rate is for each specific workflow, and who updates the agent’s constraints when the environment shifts.

Silverthorn put it cleanly: the enterprises that escape the 85% ceiling will not be the ones with the smartest agents. They will be the ones with the best managers.

That is not a technology investment. It is an operations decision. And every week a company runs another pilot without making it, the gap between experimenting and actually using agents grows wider. The intern is ready to work. The question is whether anyone on your team knows how to manage one.