An AI audit where neither side has to hand over its secrets

Independent evaluation has a two-sided confidentiality problem. Labs want to protect their weights. Evaluators want to keep test material out of future training data.

In a pilot highlighted by Zvi, AVERI, Google DeepMind, OpenMined and MLCommons evaluated Gemini 2.5 Flash-Lite inside a secure enclave. The evaluator’s new AILuminate prompts stayed hidden from Google; the model weights stayed hidden from the outside evaluators.

Both sides approved the computation. Hardware-backed isolation restricted access, and AVERI decrypted and graded the resulting answers. That provides a more concrete arrangement than a promise not to look at the other party’s information.

The achievement is a workable evaluation process with reciprocal confidentiality. It is not a comprehensive safety certification of the model. AVERI says the pilot did not eliminate every possible route for tampering, and auditing an entire AI company requires access to much more than model outputs.

Still, “we cannot expose our intellectual property” becomes a less complete answer to requests for independent testing.