SShogo Bench

SimpleQA Verified

Short factual questions; measures hallucination.

Agent Index weight
0%
shown for context only
Models tested
95
run by Epoch AI on a shared harness
Normalised for the index
0 → 0, 1 → 100
Official site

Results

#ModelScoreNormalisedHarnessCost/taskDate
1Claude Fable 5.2 Supported
Anthropic
94.0%source—Nov 6, 2026
2GPT-6.2 Astra
OpenAI
78.5%source—Oct 27, 2026
3Gemini 4 Carbon Supported
Google
77.7%source—Nov 2, 2026
4Gemini 4 Argon Supported
Google
76.0%source—Oct 13, 2026
5GPT-6 Astra Supported
OpenAI
75.6%source—Aug 30, 2026
6GPT-6.1 Sol Estimated
OpenAI
73.9%source—Sep 29, 2026
7Gemini 3.1 Pro Preview Supported
Google
73.5%source—Aug 10, 2026
8Claude Opus 5.5 Supported
Anthropic
72.2%source—Sep 22, 2026
9Gemini 4 Hydrogen Estimated
Google
70.9%source—Nov 13, 2026
10Claude Fable 5.1 Supported
Anthropic
70.8%source—Sep 1, 2026
11Claude Fable 5 Supported
Anthropic
70.7%source—Aug 10, 2026
12Gemini 3.8 Flash Supported
Google
69.7%source—Sep 2, 2026
13GPT-5.6 Sol Supported
OpenAI
69.7%source—Aug 10, 2026
14Gemini 3.7 Flash Estimated
Google
69.2%source—Aug 27, 2026
15Gemini 3 Flash Preview Supported
Google
66.8%source—Aug 27, 2026
16Gemini 3.5 Flash Supported
Google
66.2%source—Aug 27, 2026
17Gemini 3.6 Flash Estimated
Google
66.2%source—Aug 27, 2026
18GPT-5.5 Supported
OpenAI
63.0%source—Aug 27, 2026
19Kimi K3.1 Pro
Moonshot AI
62.2%source—Nov 6, 2026
20GPT-6 Sol Estimated
OpenAI
60.7%source—Sep 22, 2026
21Muse Spark 1.2 Estimated
Meta
60.3%source—Aug 27, 2026
22Claude Opus 5 Supported
Anthropic
59.9%source—Aug 10, 2026
23Muse Spark 1.1 Estimated
Meta
57.8%source—Aug 31, 2026
24GPT-6.1 Terra Estimated
OpenAI
57.3%source—Oct 26, 2026
25Muse Spark Open
Meta
56.6%source—Oct 15, 2026
26Grok 4.7 Estimated
xAI
56.0%source—Sep 22, 2026
27Qwen3.7 Max
Alibaba (Qwen)
55.8%source—Aug 27, 2026
28Claude Opus 4.8 Supported
Anthropic
53.0%source—Aug 27, 2026
29DeepSeek V4 Pro 0813
DeepSeek
52.9%source—Aug 27, 2026
30Qwen3.6 Max Preview Estimated
Alibaba (Qwen)
52.0%source—Aug 27, 2026
31Claude Opus 4.7 Supported
Anthropic
51.7%source—Aug 27, 2026
32Kimi K3 Supported
Moonshot AI
50.6%source—Aug 27, 2026
33GPT-5 Supported
OpenAI
50.1%source—Aug 27, 2026
34o3 Supported
OpenAI
49.4%source—Aug 27, 2026
35Grok 4.6 Supported
xAI
49.3%source—Aug 27, 2026
36Qwen3 Max Estimated
Alibaba (Qwen)
48.7%source—Aug 27, 2026
37Grok 4.5 Estimated
xAI
48.3%source—Aug 27, 2026
38GPT-5.1 Supported
OpenAI
48.0%source—Aug 27, 2026
39Qwen3.8 Max 0902
Alibaba (Qwen)
47.3%source—Sep 2, 2026
40Claude Opus 4.6 Supported
Anthropic
47.0%source—Aug 27, 2026
41DeepSeek V4 Pro Estimated
DeepSeek
47.0%source—Aug 27, 2026
42Claude Sonnet 5.5 Estimated
Anthropic
46.5%source—Sep 29, 2026
43GPT-5.4 Pro
OpenAI
46.3%source—Aug 27, 2026
44Qwen3.8 Max Preview Estimated
Alibaba (Qwen)
45.8%source—Aug 27, 2026
45Claude Opus 4.5 (latest) Supported
Anthropic
45.7%source—Aug 27, 2026
46GPT-5.4 Supported
OpenAI
45.1%source—Aug 27, 2026
47Doubao Seed 2.1 Pro
ByteDance Seed
44.7%source—Oct 28, 2026
48Qwen3.6 Plus Estimated
Alibaba (Qwen)
44.1%source—Aug 27, 2026
49GPT-5.6 Terra Estimated
OpenAI
43.2%source—Aug 10, 2026
50DeepSeek V4.1 Flash Estimated
DeepSeek
42.9%source—Oct 13, 2026
51DeepSeek V4.1 Flash 1017 Estimated
DeepSeek
42.5%source—Oct 19, 2026
52Doubao Seed Evolving (Nov 2026)
ByteDance Seed
41.9%source—Nov 2, 2026
53GPT-6 Luna
OpenAI
41.4%source—Sep 22, 2026
54o1 Estimated
OpenAI
41.1%source—Aug 31, 2026
55GLM-5.3 Supported
Z.ai (Zhipu)
41.0%source—Aug 28, 2026
56GPT-5.6 Luna Estimated
OpenAI
41.0%source—Aug 10, 2026
57Qwen3 235B A22B Thinking 2507
Alibaba (Qwen)
40.4%source—Aug 27, 2026
58Inkling Estimated
thinkingmachines
40.3%source—Aug 27, 2026
59Doubao-Seed-2.0 Pro
ByteDance Seed
39.4%source—Oct 28, 2026
60GPT-5.2 Supported
OpenAI
37.1%source—Aug 27, 2026
61Kimi K2.7 Code Estimated
Moonshot AI
36.5%source—Aug 27, 2026
62Claude Sonnet 4.6 Supported
Anthropic
35.5%source—Aug 10, 2026
63Kimi K2.6 Estimated
Moonshot AI
34.9%source—Aug 10, 2026
64Kimi K2.5 Supported
Moonshot AI
34.3%source—Aug 27, 2026
65GLM-5.2 Supported
Z.ai (Zhipu)
34.2%source—Aug 27, 2026
66GLM-5.1 Estimated
Z.ai (Zhipu)
34.0%source—Aug 27, 2026
67Claude Sonnet 5 Supported
Anthropic
33.7%source—Aug 27, 2026
68DeepSeek V4 Flash 0731
DeepSeek
33.6%source—Aug 27, 2026
69Grok 4.3 Beta
xAI
33.2%source—Aug 27, 2026
70GLM-4.7 Estimated
Z.ai (Zhipu)
32.2%source—Aug 27, 2026
71Doubao Seed 2.0 Lite (Apr 2026)
ByteDance Seed
31.5%source—Oct 28, 2026
72GPT-4.1
OpenAI
31.1%source—Aug 31, 2026
73Claude Sonnet 4.5 (latest) Supported
Anthropic
30.7%source—Aug 27, 2026
74Grok 4.20 (Non-Reasoning) Estimated
xAI
30.2%source—Aug 27, 2026
75Doubao Seed 2.1 Turbo
ByteDance Seed
29.7%source—Oct 30, 2026
76GPT-5.4 mini
OpenAI
29.4%source—Aug 27, 2026
77GPT-4o (2024-08-06) Supported
OpenAI
26.0%source—Aug 31, 2026
78Qwen3.5 Plus Estimated
Alibaba (Qwen)
25.4%source—Aug 27, 2026
79Claude Haiku 5.5 Estimated
Anthropic
23.8%source—Oct 9, 2026
80Mistral Large 4
Mistral
22.4%source—Nov 2, 2026
81GPT-5 Mini Estimated
OpenAI
21.6%source—Aug 10, 2026
82Qwen3.5 Flash
Alibaba (Qwen)
20.3%source—Aug 27, 2026
83o4-mini Estimated
OpenAI
19.6%source—Aug 27, 2026
84Inkling Small
thinkingmachines
19.1%source—Aug 27, 2026
85Doubao-Seed 1.6
ByteDance Seed
17.4%source—Oct 30, 2026
86Qwen3.6 Flash
Alibaba (Qwen)
15.9%source—Aug 27, 2026
87o3-mini Estimated
OpenAI
15.3%source—Aug 31, 2026
88Claude Haiku 4.5 Estimated
Anthropic
13.2%source—Aug 10, 2026
89GPT-4.1 mini
OpenAI
12.7%source—Aug 31, 2026
90Claude 3 Opus Estimated
Anthropic
12.6%source—Aug 31, 2026
91GPT-5.4 nano
OpenAI
11.7%source—Aug 27, 2026
92GPT-5 Nano
OpenAI
11.7%source—Aug 10, 2026
93Gemma 4 31B IT
Google
10.4%source—Aug 27, 2026
94GPT-4o mini
OpenAI
8.3%source—Aug 31, 2026
95GPT-4.1 nano
OpenAI
6.0%source—Aug 31, 2026