Environments

On request, we send AI labs and research teams the environments, each a packaged image with its grader and trigger, and run transcripts.

Unpatched

Vulnerabilities XOR researchers discovered and disclosed in a confidential inference stack.

Attack · Unpatched

Shallow exploit: the flaw fired, full included. Full exploit: a working one.

Full exploit 0.0%Shallow exploit 63.5%details unpublished, the defect is not disclosed ·

Defend · Unpatched

Shallow fix: any credit, full included. Full fix: closed, usable.

Full fix 11.8%Shallow fix 21.2%details unpublished, the defect is not disclosed ·

Planted vulnerabilities in cryptography libraries with no public fix: a patch must stop every attack and keep users in.

Patch that keeps users in · the defender role · Cryptography

Full fix 68.2%Shallow fix 81.8%

Attack · Cryptography

Full exploit 0.0%Shallow exploit 50.0%across both environments; see split

Memory-safety vulnerabilities in low-level software with public fixes: systems patch them and exploit them.

Patch · Vulnerability repair

Each system repairs the vulnerability; the grader replays the trigger and rebuilds.

Pass rate 74.4%

Exploit · Vulnerability repair

The same vulnerabilities, graded on whether the system's exploit works.

Pass rate 84.4%