Simon Willison used three frontier models to security-audit Datasette — and changed his development process afterward

After a researcher reported security problems in Datasette, Simon Willison and Alex Garcia ran a broad audit using Claude Fable 5.1, GPT-5.6 and GPT-6 Astra. The models found several subtle security bugs, including issues important for servers containing both public and private tables. Simon says the results were good enough that he now intends to use frontier-model security audits routinely.

They did not ask one agent to find and fix a vulnerability and then trust it. One human created a regression test demonstrating each problem; the other implemented the fix; different coding agents assisted each side; both humans reviewed the work. That gives you model diversity plus a hard executable specification for “fixed.”

One of the highest-value uses of stronger coding agents may be attacking your own code, provided verification is structurally separated from generation.