Agent Cost Economics

What it costs to fix a bug with AI

We spent $0 running 15 agents across 128 real bugs. The cheapest agent fixes bugs for $2.64 each. The most accurate costs $52/fix. Growing to 6,138+ vulnerabilities across 250+ projects.

Cost matters because you will run this repeatedly. If a bug costs $5 to patch and you run this across 500 vulnerabilities, your spend is $2,500. The same bugs with a $0.50 agent cost $250. These numbers drive real procurement decisions. We measure actual token consumption from API logs, not estimates.

Budget with real data

Security ROI = (risk reduced - cost) / cost. These tested cost-per-fix numbers replace guesswork in your budget.

Cost vs Performance

Each dot is an agent. X-axis: cost per successful patch (log scale). Y-axis: pass rate. The dashed line shows the best trade-off - no agent below it is both cheaper and more accurate.

The scatter reveals agent clusters. Some agents cluster together at 40-50% pass rate with similar costs - they are functionally equivalent. But outliers exist: agents that are cheap but miss easy bugs, or expensive but uniquely capable on hard ones. Your choice depends on your budget and your bug distribution.

Pareto Frontier with Confidence Intervals

Cost efficiency frontier with 95% Wald confidence intervals on pass rates. The Pareto frontier identifies agents where no alternative is both cheaper AND more accurate. Every agent on this line represents a genuine trade-off decision: lower cost or higher accuracy, but not both.

Confidence intervals show the statistical range around each agent's pass rate. Wider intervals indicate greater uncertainty, typically from agents with more edge cases or lower sample counts. Agents on the frontier with tight confidence intervals are more reliable choices than those with wide bands.

Oracle Set Cover

Greedy set cover showing marginal value of adding each agent to the ensemble. This analysis answers a practical question: if you can run multiple agents on the same bug, which ones should you add to maximize coverage? Start with the best agent (highest pass rate) and add agents that fix bugs the leader misses.

The first agent covers maybe 60% of bugs. Adding the second agent might bring you to 72%. Adding a third might hit 78%. But at some point, the marginal gain from each new agent drops below the cost. This visualization helps you find the optimal ensemble size for your budget.

Cost Efficiency Rankings