MATH Level 5
The hardest tier of the MATH competition dataset.
Agent Index weight
0%
shown for context only
Models tested
83
run by Epoch AI on a shared harness
Results
| # | Model | Score | Normalised | Harness | Cost/task | Date |
|---|---|---|---|---|---|---|
| 1 | GPT-5 Supported OpenAI | 98.1% | source | — | Oct 29, 2025 | |
| 2 | GPT-5 Mini Estimated OpenAI | 97.8% | source | — | Oct 30, 2025 | |
| 3 | o4-mini Estimated OpenAI | 97.8% | source | — | Apr 16, 2025 | |
| 4 | o3 Supported OpenAI | 97.8% | source | — | Apr 16, 2025 | |
| 5 | Claude Sonnet 4.5 (latest) Supported Anthropic | 97.7% | source | — | Oct 21, 2025 | |
| 6 | Qwen3 Max Estimated Alibaba (Qwen) | 97.1% | source | — | Oct 9, 2025 | |
| 7 | DeepSeek-R1 Estimated DeepSeek | 96.6% | source | — | May 29, 2025 | |
| 8 | o3-mini Estimated OpenAI | 96.5% | source | — | Feb 13, 2025 | |
| 9 | Claude Haiku 4.5 Estimated Anthropic | 96.4% | source | — | Oct 22, 2025 | |
| 10 | Gemini 2.5 Pro Supported Google | 95.9% | source | — | May 8, 2025 | |
| 11 | GPT-5 Nano OpenAI | 95.2% | source | — | Aug 20, 2025 | |
| 12 | o1 Estimated OpenAI | 94.7% | source | — | Feb 13, 2025 | |
| 13 | Claude 3.7 Sonnet Supported Anthropic | 91.2% | source | — | Mar 13, 2025 | |
| 14 | Grok-3 mini xAI | 90.9% | source | — | Apr 10, 2025 | |
| 15 | DeepSeek-R1-Distill-Llama-70B DeepSeek | 89.9% | source | — | Mar 10, 2025 | |
| 16 | o1-mini OpenAI | 89.2% | source | — | Feb 13, 2025 | |
| 17 | Grok 3 xAI | 88.7% | source | — | Apr 10, 2025 | |
| 18 | GPT-4.1 mini OpenAI | 87.3% | source | — | Apr 14, 2025 | |
| 19 | DeepSeek-R1-Distill-Qwen-14B DeepSeek | 87.1% | source | — | Mar 11, 2025 | |
| 20 | Claude-Opus-4 Supported Anthropic | 85.0% | source | — | May 22, 2025 | |
| 21 | Claude Sonnet 4 Supported Anthropic | 84.4% | source | — | May 22, 2025 | |
| 22 | Gemini 2.0 Pro Google | 83.5% | source | — | Feb 6, 2025 | |
| 23 | GPT-4.1 OpenAI | 83.0% | source | — | Apr 14, 2025 | |
| 24 | Gemini-2.0-Flash Google | 82.2% | source | — | Feb 6, 2025 | |
| 25 | Mistral Medium 3 Mistral | 81.6% | source | — | May 7, 2025 | |
| 26 | GPT-4.5 OpenAI | 78.6% | source | — | Feb 28, 2025 | |
| 27 | DeepSeek V3 0324 DeepSeek | 75.5% | source | — | Apr 1, 2025 | |
| 28 | Gemma 3 27B Google | 74.0% | source | — | Mar 13, 2025 | |
| 29 | Llama 4 Maverick Meta | 73.0% | source | — | Apr 8, 2025 | |
| 30 | Gemini 1.5 Pro (Feb 2024) Google | 70.4% | source | — | Jan 27, 2025 | |
| 31 | GPT-4.1 nano OpenAI | 70.0% | source | — | Apr 14, 2025 | |
| 32 | Qwen3 235B A22B Alibaba (Qwen) | 68.9% | source | — | Jun 3, 2025 | |
| 33 | Qwen2.5-Max Alibaba (Qwen) | 67.2% | source | — | Apr 1, 2025 | |
| 34 | Qwen Plus Alibaba (Qwen) | 65.3% | source | — | Apr 7, 2025 | |
| 35 | Phi 4 Microsoft | 64.9% | source | — | Jan 31, 2025 | |
| 36 | DeepSeek-V3 DeepSeek | 64.9% | source | — | Jan 27, 2025 | |
| 37 | Grok-2 (Dec 2024) xAI | 63.5% | source | — | Feb 17, 2025 | |
| 38 | Qwen2.5-72B Estimated Alibaba (Qwen) | 63.2% | source | — | Jan 27, 2025 | |
| 39 | Llama 4 Scout Meta | 62.3% | source | — | Apr 8, 2025 | |
| 40 | Gemini 1.5 Flash (Sep 2024) Google | 61.9% | source | — | Jan 27, 2025 | |
| 41 | Claude-Sonnet-3.5 Estimated Anthropic | 56.9% | source | — | Jan 27, 2025 | |
| 42 | Qwen Turbo Alibaba (Qwen) | 56.2% | source | — | Apr 7, 2025 | |
| 43 | Qwen2.5-32B Alibaba (Qwen) | 56.1% | source | — | Jan 30, 2025 | |
| 44 | GPT-4o (2024-08-06) Supported OpenAI | 53.3% | source | — | Jan 27, 2025 | |
| 45 | GPT-4o mini OpenAI | 52.6% | source | — | Jan 27, 2025 | |
| 46 | Mistral Large 2 (Nov 2024) Mistral | 50.3% | source | — | Feb 25, 2025 | |
| 47 | Llama 3.1-405B Meta | 49.8% | source | — | Jan 27, 2025 | |
| 48 | Mistral Small 3.1 Mistral | 46.8% | source | — | Mar 18, 2025 | |
| 49 | GPT-4 Turbo OpenAI | 46.7% | source | — | Feb 27, 2025 | |
| 50 | Claude 3.5 Haiku Anthropic | 46.4% | source | — | Mar 12, 2025 | |
| 51 | Mistral Small 3 Mistral | 44.8% | source | — | Jan 30, 2025 | |
| 52 | Tulu 3 (Tülu 3) 70B Allen Institute for AI | 42.7% | source | — | Jan 27, 2025 | |
| 53 | Llama 3.3 70B Meta | 41.6% | source | — | Jan 27, 2025 | |
| 54 | Llama 3.2 90B Meta | 39.4% | source | — | Jan 27, 2025 | |
| 55 | Qwen2-72B Alibaba (Qwen) | 39.1% | source | — | Jan 27, 2025 | |
| 56 | Claude 3 Opus Estimated Anthropic | 37.5% | source | — | Jan 27, 2025 | |
| 57 | Llama 3.1-70B Meta | 36.7% | source | — | Jan 27, 2025 | |
| 58 | Gemma 2 27B Google | 27.9% | source | — | Jan 27, 2025 | |
| 59 | WizardLM-2 8x22B Microsoft | 25.7% | source | — | Jan 27, 2025 | |
| 60 | Yi-1.5-34B 01.AI | 25.5% | source | — | Jan 27, 2025 | |
| 61 | Mistral Large Mistral | 24.5% | source | — | Jan 27, 2025 | |
| 62 | Mixtral 8x22B Mistral | 24.2% | source | — | Jan 27, 2025 | |
| 63 | GPT-4 OpenAI | 23.0% | source | — | Jan 27, 2025 | |
| 64 | Llama 3.1-8B Meta | 22.9% | source | — | Jan 27, 2025 | |
| 65 | Hermes 2 Theta Llama-3 70B Nous Research | 22.7% | source | — | Jan 27, 2025 | |
| 66 | Llama 3-70B Meta | 22.6% | source | — | Jan 27, 2025 | |
| 67 | Gemma 2 9B Google | 21.0% | source | — | Jan 27, 2025 | |
| 68 | Claude 3 Sonnet Anthropic | 18.2% | source | — | Jan 27, 2025 | |
| 69 | phi-3-medium 14B Microsoft | 17.6% | source | — | Jan 31, 2025 | |
| 70 | GPT-3.5-turbo OpenAI | 15.9% | source | — | Jan 27, 2025 | |
| 71 | Ministral 3 8B 2512 Mistral | 14.9% | source | — | Jan 27, 2025 | |
| 72 | Claude-Haiku-3 Anthropic | 14.9% | source | — | Jan 27, 2025 | |
| 73 | Ministral 3B Mistral | 14.4% | source | — | Jan 27, 2025 | |
| 74 | Claude 2 Anthropic | 11.7% | source | — | Jan 27, 2025 | |
| 75 | DBRX Databricks | 11.7% | source | — | Jan 27, 2025 | |
| 76 | Gemini 1.0 Pro Google | 11.2% | source | — | Jan 27, 2025 | |
| 77 | Mistral Nemo Mistral | 10.8% | source | — | Jan 27, 2025 | |
| 78 | Mixtral 8x7B Mistral | 10.0% | source | — | Jan 27, 2025 | |
| 79 | DeepSeek LLM 67B DeepSeek | 6.4% | source | — | Jan 27, 2025 | |
| 80 | Llama 3-8B Meta | 6.1% | source | — | Jan 27, 2025 | |
| 81 | Yi-34B 01.AI | 5.1% | source | — | Jan 27, 2025 | |
| 82 | Mistral 7B v0.3 Mistral | 3.7% | source | — | Jan 27, 2025 | |
| 83 | Llama 2-70B Meta | 3.3% | source | — | Jan 27, 2025 |