Vulnerability repair · attack side

Exploit leaderboard

The same vulnerabilities, graded on whether the system's exploit works.

The same vulnerabilities as the Patch leaderboard. Each system tries to write an input that triggers the vulnerability, and the grader runs it. Systems whose results the grader never checked show counts and no score. The set every system completed in both roles is on Attack versus defend.

Exploit is the attacker role; Patch is the defender role.

The environments and their graders are withheld; the leaderboards show aggregate results.

Method Add your system Attack versus defend

Leaderboard

Leaderboard3 systems

Each bar is the share of vulnerabilities where the system's exploit worked.

1
OpenCode GPT-5.5
96.2% 92.3–98.9, 95% CI 92.3–98.9
2
Claude Code Opus 4.7
79.7% 68.6–90.1, 95% CI 68.6–90.1
2
OpenCode GPT-5.3 Codex
77.3% 69.2–84.3, 95% CI 69.2–84.3

Leaderboard: Exploit · newest run 30 April 2026 · 3 systems · The same vulnerabilities as the Patch leaderboard. Each system tries to write an input that triggers the vulnerability, and the grader runs it. Systems whose results the grader never checked show counts and no score. The set every system completed in both roles is on Attack versus defend. The reasoning effort for these runs is not recorded.

How we measured this

Rank. One plus the number of systems whose whole range sits above this system's. Systems whose ranges overlap share a rank. A medal marks a rank held alone.
Score. Share of complete runs where the exploit worked.
Range. The thin line beside a score: the range the true rate probably sits in (its 95% confidence interval, CI). The unit of uncertainty is the codebase, because vulnerabilities in the same codebase are not independent draws.
Harness versions. A system is a harness at a version with a model.
OpenCode GPT-5.5
1.14.20
Claude Code Opus 4.7
2.1.114
OpenCode GPT-5.3 Codex
1.14.20
Newest run 30 April 2026 · Cite · Method. Runs from April 2026, published 6 September 2026.