SShogo Bench

FrontierMath Tiers 1-3

Hundreds of unpublished, expert-written math problems.

Agent Index weight
0%
shown for context only
Models tested
94
run by Epoch AI on a shared harness
Normalised for the index
0 → 0, 1 → 100
Official site

Results

#ModelScoreNormalisedHarnessCost/taskDate
1Claude Fable 5.2 Supported
Anthropic
95.1%source—Nov 6, 2026
2GPT-6.1 Sol Estimated
OpenAI
93.7%source—Sep 29, 2026
3GPT-6 Astra Supported
OpenAI
93.7%source—Aug 30, 2026
4GPT-6.2 Astra
OpenAI
92.6%source—Oct 27, 2026
5Claude Opus 5.5 Supported
Anthropic
91.2%source—Sep 22, 2026
6GPT-6.1 Terra Estimated
OpenAI
90.9%source—Oct 26, 2026
7Gemini 4 Carbon Supported
Google
90.5%source—Nov 2, 2026
8Claude Fable 5.1 Supported
Anthropic
90.2%source—Sep 1, 2026
9GPT-6 Sol Estimated
OpenAI
89.8%source—Sep 22, 2026
10GPT-5.6 Sol Supported
OpenAI
89.1%source—Jul 9, 2026
11Claude Sonnet 5.5 Estimated
Anthropic
88.8%source—Sep 29, 2026
12Gemini 4 Hydrogen Estimated
Google
88.1%source—Nov 13, 2026
13GPT-5.5 Pro
OpenAI
87.7%source—Jun 12, 2026
14Claude Fable 5 Supported
Anthropic
87.0%source—Jun 9, 2026
15Gemini 4 Argon Supported
Google
86.0%source—Oct 13, 2026
16GPT-5.6 Terra Estimated
OpenAI
86.0%source—Jul 9, 2026
17Claude Opus 5 Supported
Anthropic
85.6%source—Jul 24, 2026
18GPT-5.5 Supported
OpenAI
85.3%source—Jun 11, 2026
19GPT-5.4 Pro
OpenAI
82.5%source—Jun 13, 2026
20GPT-5.6 Luna Estimated
OpenAI
82.1%source—Jul 9, 2026
21Muse Wonder Estimated
Meta
82.1%source—Oct 12, 2026
22Claude Opus 4.8 Supported
Anthropic
80.0%source—Jun 10, 2026
23GPT-6 Luna
OpenAI
78.9%source—Sep 22, 2026
24GPT-5.4 Supported
OpenAI
78.6%source—Jun 11, 2026
25Kimi K3.1 Pro
Moonshot AI
78.2%source—Nov 6, 2026
26Claude Haiku 5.5 Estimated
Anthropic
75.1%source—Oct 7, 2026
27Qwen3.8 Max Preview Estimated
Alibaba (Qwen)
74.7%source—Aug 4, 2026
28Muse Spark 1.3 Estimated
Meta
74.4%source—Sep 16, 2026
29GPT-5.2 Pro
OpenAI
74.0%source—Jun 13, 2026
30Kimi K3 Supported
Moonshot AI
72.2%source—Jul 17, 2026
31Gemini 3.7 Flash Estimated
Google
71.6%source—Aug 14, 2026
32DeepSeek V4.1 Flash 1017 Estimated
DeepSeek
71.2%source—Oct 19, 2026
33Claude Opus 4.7 Supported
Anthropic
70.2%source—Jun 10, 2026
34Muse Spark Open
Meta
69.8%source—Oct 15, 2026
35GLM-5.3 Supported
Z.ai (Zhipu)
68.8%source—Aug 25, 2026
36Gemini 3.8 Flash Supported
Google
68.4%source—Sep 2, 2026
37GPT-5.2 Supported
OpenAI
67.4%source—Jun 11, 2026
38DeepSeek V4.1 Flash Estimated
DeepSeek
67.4%source—Oct 8, 2026
39Claude Opus 4.6 Supported
Anthropic
66.0%source—Jun 11, 2026
40Grok 4.6 Supported
xAI
66.0%source—Aug 14, 2026
41Claude Sonnet 5 Supported
Anthropic
65.6%source—Jun 30, 2026
42Qwen3.8 Max 0902
Alibaba (Qwen)
65.6%source—Sep 2, 2026
43DeepSeek V4 Pro 0813
DeepSeek
64.6%source—Aug 19, 2026
44Qwen3.7 Max
Alibaba (Qwen)
64.6%source—Jun 13, 2026
45Gemini 3.5 Flash Supported
Google
62.8%source—Jun 10, 2026
46Gemini 3.1 Pro Preview Supported
Google
59.6%source—Jun 11, 2026
47GLM-5.2 Supported
Z.ai (Zhipu)
59.2%source—Jun 19, 2026
48Gemini 3.6 Flash Estimated
Google
58.9%source—Aug 2, 2026
49Mistral Large 4
Mistral
58.6%source—Oct 12, 2026
50DeepSeek V4 Flash 0731
DeepSeek
57.5%source—Aug 2, 2026
51Grok 4.5 Estimated
xAI
57.2%source—Jul 9, 2026
52Kimi K2.6 Estimated
Moonshot AI
57.2%source—Jun 10, 2026
53GLM-5.3-Flash Supported
Z.ai (Zhipu)
55.8%source—Aug 27, 2026
54GPT-5 Pro
OpenAI
55.8%source—Jun 12, 2026
55GPT-5 Supported
OpenAI
55.4%source—Jun 10, 2026
56Kimi K2.7 Code Estimated
Moonshot AI
54.0%source—Jun 13, 2026
57Grok 4.7 Estimated
xAI
53.0%source—Sep 22, 2026
58Gemini 3 Flash Preview Supported
Google
51.2%source—Jun 11, 2026
59GPT-5.4 mini
OpenAI
51.2%source—Jun 12, 2026
60GPT-5 Mini Estimated
OpenAI
46.7%source—Jun 12, 2026
61Inkling Small
thinkingmachines
46.3%source—Aug 14, 2026
62DeepSeek V4 Pro Estimated
DeepSeek
45.3%source—Jun 17, 2026
63GPT-5.4 nano
OpenAI
44.9%source—Jun 12, 2026
64Grok 4.20 (Non-Reasoning) Estimated
xAI
44.9%source—Jul 13, 2026
65Grok 4.3 Beta
xAI
42.8%source—Jun 17, 2026
66Qwen3.6 Plus Estimated
Alibaba (Qwen)
38.2%source—Aug 30, 2026
67GLM-5.1 Estimated
Z.ai (Zhipu)
36.8%source—Aug 29, 2026
68o4-mini Estimated
OpenAI
36.1%source—Jun 11, 2026
69Qwen3.6 27B
Alibaba (Qwen)
35.1%source—Aug 28, 2026
70Claude Opus 4.5 (latest) Supported
Anthropic
34.4%source—Jun 11, 2026
71Qwen3.7 Plus
Alibaba (Qwen)
34.4%source—Aug 29, 2026
72Inkling Estimated
thinkingmachines
33.3%source—Aug 6, 2026
73o3 Supported
OpenAI
33.3%source—Aug 27, 2026
74Qwen3.5 397B-A17B
Alibaba (Qwen)
31.2%source—Aug 30, 2026
75Gemini 3.1 Flash Lite
Google
27.7%source—Aug 30, 2026
76GPT Chat Latest
OpenAI
26.3%source—Aug 2, 2026
77Gemini 3.5 Flash Lite
Google
26.0%source—Aug 2, 2026
78Gemini 2.5 Pro Supported
Google
24.6%source—Jun 11, 2026
79Claude Sonnet 4.5 (latest) Supported
Anthropic
23.9%source—Jun 11, 2026
80Qwen3.6 Flash
Alibaba (Qwen)
22.5%source—Aug 29, 2026
81Qwen3.6 35B-A3B
Alibaba (Qwen)
20.4%source—Aug 30, 2026
82GPT-5 Nano
OpenAI
20.0%source—Jun 12, 2026
83Qwen3.7 Flash
Alibaba (Qwen)
19.3%source—Aug 29, 2026
84Qwen3 Max Estimated
Alibaba (Qwen)
18.9%source—Aug 30, 2026
85o3-mini Estimated
OpenAI
18.6%source—Jun 11, 2026
86Qwen3.5 Flash
Alibaba (Qwen)
18.2%source—Aug 28, 2026
87o1 Estimated
OpenAI
14.7%source—Aug 28, 2026
88Claude Opus 4.1 (latest) Supported
Anthropic
12.6%source—Jun 11, 2026
89GPT-4.1 mini
OpenAI
6.7%source—Aug 27, 2026
90GPT-4.1
OpenAI
6.0%source—Aug 27, 2026
91GPT-4 Turbo
OpenAI
0.7%source—Aug 27, 2026
92GPT-4o mini
OpenAI
0.7%source—Aug 27, 2026
93GPT-4o (2024-08-06) Supported
OpenAI
0.4%source—Aug 27, 2026
94GPT-3.5-turbo
OpenAI
0.0%source—Aug 27, 2026