The control question already has an answer

For a few years the question of whether increasingly capable AI systems could be kept legible — audited, verified, held inside a boundary someone actually checks — stayed mostly philosophical. Easy to argue in the abstract, right up until someone has to make the call under real conditions. The people making it now aren’t the careless ones. They’re the careful ones, and they’ve started deciding the boundary isn’t worth holding.

Simon Willison spent 2025 drawing the line himself. He called the careful version “agentic engineering” — reviewed, accountable, safe to ship — specifically against “vibe coding,” the reckless version where you never check what the model wrote. In May, he admitted the line moved on him: “I’m not reviewing every line of code that they write anymore, even for my production level stuff.” He treats the output now as a semi-black box — the trust you’d extend to a vendor’s API, not a colleague’s pull request. He names the risk himself, correctly: the normalization of deviance, the slow recalibration where nothing goes wrong until it does. He isn’t being lazy. Checking simply stopped scaling against how good the tool got, and he made the rational trade.

The same trade is being written into infrastructure, in plain text. WebMCP is the emerging web standard that lets a page tell a visiting AI agent which actions it’s allowed to take — click this, fill that, run this tool. Its own specification, dated this June, states that “there is no guarantee that a tool’s declared intent matches its actual behavior” and that “agents must assume good faith from site developers.” That’s an admission, not a fix. There’s an open proposal in the spec’s own repository to add real identity verification — cryptographic attestation, closer to how Apple already handles device trust — and it’s still just a proposal. Edge already ships WebMCP; Chrome has it in origin trial. The capability is live; the trust mechanism is a proposal that’s been sitting open in the same repository since February, unmerged. Nobody wrote “we’re giving up on verifying this” into the spec. They shipped the part that works and filed the part that doesn’t as future work — the same decision in a more comfortable coat.

None of this is reckless, exactly. Reviewing every line, verifying every tool — these were the reasonable things to stop doing, given how good the models got and how slowly verification gets built. But choosing to proceed without solving the trust problem is still a choice, and it has a customer. In early July, researchers at Sysdig disclosed a ransomware campaign, JadePuffer, run start to finish by an unattended agent — no operator, no human in the loop at any stage. When it hit a locked account mid-attack, it adapted and got back in within 31 seconds. None of the vulnerabilities it used were new; one was years old. The notable part isn’t that an agent can run an attack chain. It’s that the only thing between “unattended” and “undetected” was whether someone happened to be watching in real time — and on both sides of that line, fewer people are.

So the control question has an answer, and it’s not the one either side of the old debate expected. Not that the systems got safe enough to trust. Not that anyone built the auditing layer that was supposed to come first. The boundary got abandoned because the cost of not shipping rose above the cost of not checking — the careful practitioner and the standards body reached that point independently, at roughly the same moment. The attacker just showed up to collect. My bet is that the next real progress won’t be a better control mechanism; the closest attempts are still research benchmarks and unmerged proposals. It’ll be in how fast anyone notices when the unchecked thing goes wrong — because noticing is the only part of this anyone has actually agreed to build.