SShogo Bench

MATH Level 5

The hardest tier of the MATH competition dataset.

Agent Index weight
0%
shown for context only
Models tested
83
run by Epoch AI on a shared harness
Normalised for the index
0 → 0, 1 → 100
Official site

Results

#ModelScoreNormalisedHarnessCost/taskDate
1GPT-5 Supported
OpenAI
98.1%source—Oct 29, 2025
2GPT-5 Mini Estimated
OpenAI
97.8%source—Oct 30, 2025
3o4-mini Estimated
OpenAI
97.8%source—Apr 16, 2025
4o3 Supported
OpenAI
97.8%source—Apr 16, 2025
5Claude Sonnet 4.5 (latest) Supported
Anthropic
97.7%source—Oct 21, 2025
6Qwen3 Max Estimated
Alibaba (Qwen)
97.1%source—Oct 9, 2025
7DeepSeek-R1 Estimated
DeepSeek
96.6%source—May 29, 2025
8o3-mini Estimated
OpenAI
96.5%source—Feb 13, 2025
9Claude Haiku 4.5 Estimated
Anthropic
96.4%source—Oct 22, 2025
10Gemini 2.5 Pro Supported
Google
95.9%source—May 8, 2025
11GPT-5 Nano
OpenAI
95.2%source—Aug 20, 2025
12o1 Estimated
OpenAI
94.7%source—Feb 13, 2025
13Claude 3.7 Sonnet Supported
Anthropic
91.2%source—Mar 13, 2025
14Grok-3 mini
xAI
90.9%source—Apr 10, 2025
15DeepSeek-R1-Distill-Llama-70B
DeepSeek
89.9%source—Mar 10, 2025
16o1-mini
OpenAI
89.2%source—Feb 13, 2025
17Grok 3
xAI
88.7%source—Apr 10, 2025
18GPT-4.1 mini
OpenAI
87.3%source—Apr 14, 2025
19DeepSeek-R1-Distill-Qwen-14B
DeepSeek
87.1%source—Mar 11, 2025
20Claude-Opus-4 Supported
Anthropic
85.0%source—May 22, 2025
21Claude Sonnet 4 Supported
Anthropic
84.4%source—May 22, 2025
22Gemini 2.0 Pro
Google
83.5%source—Feb 6, 2025
23GPT-4.1
OpenAI
83.0%source—Apr 14, 2025
24Gemini-2.0-Flash
Google
82.2%source—Feb 6, 2025
25Mistral Medium 3
Mistral
81.6%source—May 7, 2025
26GPT-4.5
OpenAI
78.6%source—Feb 28, 2025
27DeepSeek V3 0324
DeepSeek
75.5%source—Apr 1, 2025
28Gemma 3 27B
Google
74.0%source—Mar 13, 2025
29Llama 4 Maverick
Meta
73.0%source—Apr 8, 2025
30Gemini 1.5 Pro (Feb 2024)
Google
70.4%source—Jan 27, 2025
31GPT-4.1 nano
OpenAI
70.0%source—Apr 14, 2025
32Qwen3 235B A22B
Alibaba (Qwen)
68.9%source—Jun 3, 2025
33Qwen2.5-Max
Alibaba (Qwen)
67.2%source—Apr 1, 2025
34Qwen Plus
Alibaba (Qwen)
65.3%source—Apr 7, 2025
35Phi 4
Microsoft
64.9%source—Jan 31, 2025
36DeepSeek-V3
DeepSeek
64.9%source—Jan 27, 2025
37Grok-2 (Dec 2024)
xAI
63.5%source—Feb 17, 2025
38Qwen2.5-72B Estimated
Alibaba (Qwen)
63.2%source—Jan 27, 2025
39Llama 4 Scout
Meta
62.3%source—Apr 8, 2025
40Gemini 1.5 Flash (Sep 2024)
Google
61.9%source—Jan 27, 2025
41Claude-Sonnet-3.5 Estimated
Anthropic
56.9%source—Jan 27, 2025
42Qwen Turbo
Alibaba (Qwen)
56.2%source—Apr 7, 2025
43Qwen2.5-32B
Alibaba (Qwen)
56.1%source—Jan 30, 2025
44GPT-4o (2024-08-06) Supported
OpenAI
53.3%source—Jan 27, 2025
45GPT-4o mini
OpenAI
52.6%source—Jan 27, 2025
46Mistral Large 2 (Nov 2024)
Mistral
50.3%source—Feb 25, 2025
47Llama 3.1-405B
Meta
49.8%source—Jan 27, 2025
48Mistral Small 3.1
Mistral
46.8%source—Mar 18, 2025
49GPT-4 Turbo
OpenAI
46.7%source—Feb 27, 2025
50Claude 3.5 Haiku
Anthropic
46.4%source—Mar 12, 2025
51Mistral Small 3
Mistral
44.8%source—Jan 30, 2025
52Tulu 3 (Tülu 3) 70B
Allen Institute for AI
42.7%source—Jan 27, 2025
53Llama 3.3 70B
Meta
41.6%source—Jan 27, 2025
54Llama 3.2 90B
Meta
39.4%source—Jan 27, 2025
55Qwen2-72B
Alibaba (Qwen)
39.1%source—Jan 27, 2025
56Claude 3 Opus Estimated
Anthropic
37.5%source—Jan 27, 2025
57Llama 3.1-70B
Meta
36.7%source—Jan 27, 2025
58Gemma 2 27B
Google
27.9%source—Jan 27, 2025
59WizardLM-2 8x22B
Microsoft
25.7%source—Jan 27, 2025
60Yi-1.5-34B
01.AI
25.5%source—Jan 27, 2025
61Mistral Large
Mistral
24.5%source—Jan 27, 2025
62Mixtral 8x22B
Mistral
24.2%source—Jan 27, 2025
63GPT-4
OpenAI
23.0%source—Jan 27, 2025
64Llama 3.1-8B
Meta
22.9%source—Jan 27, 2025
65Hermes 2 Theta Llama-3 70B
Nous Research
22.7%source—Jan 27, 2025
66Llama 3-70B
Meta
22.6%source—Jan 27, 2025
67Gemma 2 9B
Google
21.0%source—Jan 27, 2025
68Claude 3 Sonnet
Anthropic
18.2%source—Jan 27, 2025
69phi-3-medium 14B
Microsoft
17.6%source—Jan 31, 2025
70GPT-3.5-turbo
OpenAI
15.9%source—Jan 27, 2025
71Ministral 3 8B 2512
Mistral
14.9%source—Jan 27, 2025
72Claude-Haiku-3
Anthropic
14.9%source—Jan 27, 2025
73Ministral 3B
Mistral
14.4%source—Jan 27, 2025
74Claude 2
Anthropic
11.7%source—Jan 27, 2025
75DBRX
Databricks
11.7%source—Jan 27, 2025
76Gemini 1.0 Pro
Google
11.2%source—Jan 27, 2025
77Mistral Nemo
Mistral
10.8%source—Jan 27, 2025
78Mixtral 8x7B
Mistral
10.0%source—Jan 27, 2025
79DeepSeek LLM 67B
DeepSeek
6.4%source—Jan 27, 2025
80Llama 3-8B
Meta
6.1%source—Jan 27, 2025
81Yi-34B
01.AI
5.1%source—Jan 27, 2025
82Mistral 7B v0.3
Mistral
3.7%source—Jan 27, 2025
83Llama 2-70B
Meta
3.3%source—Jan 27, 2025