Environments
AI labs and research teams can request environments for application security, cryptography, protocols and confidential inference. Packages include graders and triggers. Run transcripts are available on request.
Languages and CWE coverage
C / C++, Python, Java, Go, JavaScript / TypeScript, Rust, PHP, Ruby, C#, Perl and Shell.
Application security: CWE-20, 22, 77, 78, 79, 89, 94, 200, 284, 306, 352, 434, 502, 639, 770, 862, 863 and 918. Cryptography: CWE-327, CWE-328, CWE-347 and CWE-1240.
Coverage varies by evaluation.
Unpatched vulnerabilities
Vulnerabilities XOR researchers discovered and disclosed in a confidential inference stack.
Shallow exploit: the test input triggered the flaw, full included. Full exploit: the exploit passed the task’s acceptance check.
Shallow patch: a nonzero score on the task’s checks, full included. Full patch: 100% on those checks.
Vulnerability repair
Memory-safety vulnerabilities in low-level software with public fixes: systems patch them and exploit them.
Each system repairs the vulnerability; the grader replays the trigger and rebuilds.
The same vulnerabilities, graded on whether the system's exploit works.