Cybench
Professional capture-the-flag challenges solved without hints.
Agent Index weight
5%
in the Security & context learning pillar
Models tested
21
from the benchmark’s leaderboard and reports
Results
| # | Model | Score | Normalised | Harness | Cost/task | Date |
|---|---|---|---|---|---|---|
| 1 | Claude Opus 4.6 Supported Anthropic | 93.0% | source | — | — | |
| 2 | Claude Opus 4.5 (latest) Supported Anthropic | 82.0% | source | — | — | |
| 3 | Claude Sonnet 4.5 (latest) Supported Anthropic | 60.0% | source | — | — | |
| 4 | Grok 4 Supported xAI | 43.0% | source | — | — | |
| 5 | Claude Opus 4.1 (latest) Supported Anthropic | 42.0% | source | — | — | |
| 6 | Grok 4.1 xAI | 39.0% | source | — | — | |
| 7 | Claude-Opus-4 Supported Anthropic | 38.0% | source | — | — | |
| 8 | Claude Sonnet 4 Supported Anthropic | 35.0% | source | — | — | |
| 9 | Grok 4 Fast xAI | 30.0% | source | — | — | |
| 10 | o3-mini Estimated OpenAI | 22.5% | source | — | — | |
| 11 | Claude 3.7 Sonnet Supported Anthropic | 20.0% | source | — | — | |
| 12 | Claude-Sonnet-3.5 Estimated Anthropic | 17.5% | source | — | — | |
| 13 | GPT-4.5 OpenAI | 17.5% | source | — | — | |
| 14 | GPT-4o (2024-08-06) Supported OpenAI | 12.5% | source | — | — | |
| 15 | Claude 3 Opus Estimated Anthropic | 10.0% | source | — | — | |
| 16 | o1 Estimated OpenAI | 10.0% | source | — | — | |
| 17 | o1-mini OpenAI | 10.0% | source | — | — | |
| 18 | Gemini 1.5 Pro (Feb 2024) Google | 7.5% | source | — | — | |
| 19 | Llama 3.1-405B Meta | 7.5% | source | — | — | |
| 20 | Mixtral 8x22B Mistral | 7.5% | source | — | — | |
| 21 | Llama 3-70B Meta | 5.0% | source | — | — |
For models without a Cybench result, the index estimates one from Epoch’s Capabilities Index using a logistic fit on 20 models (residual SD 0.6 on the logit scale). Estimates are labelled and widen the model’s range.