Benchmarks
Compare systems
Pick up to five systems from one evaluation and compare their results.
Side by side3 of 8 systems on the Unpatched vulnerabilities · Repair leaderboard
Shallow patch: a nonzero score on the task’s checks, full included. Full patch: 100% on those checks.
SystemFull patch · Shallow patch
Codex GPT-6 Astra xhigh
35.3% · 41.2%17 graded runs
Claude Code Fable 5.1 high
28.6% · 42.9%7 graded runs
Codex GPT-5.6 Sol xhigh
16.7% · 29.2%24 graded runs
Leaderboard: Unpatched vulnerabilities, Repair· this leaderboard is not ranked · Open the leaderboard