DTBench
Decision-theory questions.
Agent Index weight
0%
shown for context only
Models tested
172
from the benchmark’s leaderboard and reports
Results
| # | Model | Score | Normalised | Harness | Cost/task | Date |
|---|---|---|---|---|---|---|
| 1 | Claude Fable 5.2 Supported Anthropic | 99.2% | source | — | — | |
| 2 | Claude Opus 5.5 Supported Anthropic | 98.9% | source | — | — | |
| 3 | Claude Fable 5 Supported Anthropic | 98.4% | source | — | — | |
| 4 | Claude Opus 5 Supported Anthropic | 97.9% | source | — | — | |
| 5 | Claude Fable 5.1 Supported Anthropic | 97.6% | source | — | — | |
| 6 | Gemini 4 Argon Supported Google | 97.6% | source | — | — | |
| 7 | Claude Sonnet 5.5 Estimated Anthropic | 97.3% | source | — | — | |
| 8 | GPT-6 Astra Supported OpenAI | 97.3% | source | — | — | |
| 9 | GPT-6 Sol Estimated OpenAI | 97.3% | source | — | — | |
| 10 | Grok 4.6 Supported xAI | 97.3% | source | — | — | |
| 11 | Gemini 3.1 Pro Preview Supported Google | 97.1% | source | — | — | |
| 12 | Gemini 3.7 Flash Estimated Google | 96.8% | source | — | — | |
| 13 | GPT-6.1 Sol Estimated OpenAI | 96.5% | source | — | — | |
| 14 | Grok 4.5 Estimated xAI | 96.5% | source | — | — | |
| 15 | Muse Spark 1.3 Estimated Meta | 96.5% | source | — | — | |
| 16 | GPT-5.5 Supported OpenAI | 96.0% | source | — | — | |
| 17 | GPT-5.5 Pro OpenAI | 96.0% | source | — | — | |
| 18 | GPT-5.6 Sol Supported OpenAI | 96.0% | source | — | — | |
| 19 | Grok 4.7 Estimated xAI | 96.0% | source | — | — | |
| 20 | Gemini 3.8 Flash Supported Google | 95.7% | source | — | — | |
| 21 | Gemini 3.6 Flash Estimated Google | 95.5% | source | — | — | |
| 22 | Muse Wonder Estimated Meta | 95.2% | source | — | — | |
| 23 | Claude Haiku 5.5 Estimated Anthropic | 95.0% | source | — | — | |
| 24 | Claude Opus 4.8 Supported Anthropic | 94.9% | source | — | — | |
| 25 | Gemini 3.5 Flash Supported Google | 94.7% | source | — | — | |
| 26 | Muse Spark 1.2 Estimated Meta | 94.7% | source | — | — | |
| 27 | Claude Opus 4.7 Supported Anthropic | 94.7% | source | — | — | |
| 28 | GPT-5.4 Supported OpenAI | 94.4% | source | — | — | |
| 29 | Muse Spark 1.1 Estimated Meta | 94.4% | source | — | — | |
| 30 | DeepSeek V4 Pro 0813 DeepSeek | 93.9% | source | — | — | |
| 31 | GLM-5.2 Supported Z.ai (Zhipu) | 93.6% | source | — | — | |
| 32 | GPT-5.6 Terra Estimated OpenAI | 93.3% | source | — | — | |
| 33 | Kimi K3.1 Pro Moonshot AI | 93.3% | source | — | — | |
| 34 | Claude Sonnet 5 Supported Anthropic | 92.5% | source | — | — | |
| 35 | Qwen3.7 Max Alibaba (Qwen) | 92.3% | source | — | — | |
| 36 | Qwen3.8 Max Preview Estimated Alibaba (Qwen) | 92.0% | source | — | — | |
| 37 | Claude Opus 4.6 Supported Anthropic | 91.2% | source | — | — | |
| 38 | Kimi K3 Supported Moonshot AI | 91.2% | source | — | — | |
| 39 | DeepSeek V4 Flash 0731 DeepSeek | 90.9% | source | — | — | |
| 40 | GPT-5.2 Supported OpenAI | 90.9% | source | — | — | |
| 41 | Kimi K2.6 Estimated Moonshot AI | 90.9% | source | — | — | |
| 42 | DeepSeek V4 Pro Estimated DeepSeek | 90.7% | source | — | — | |
| 43 | GPT-5 Supported OpenAI | 90.7% | source | — | — | |
| 44 | Grok 4.3 Beta xAI | 90.7% | source | — | — | |
| 45 | GPT-5.1 Supported OpenAI | 90.1% | source | — | — | |
| 46 | GPT-6 Luna OpenAI | 90.1% | source | — | — | |
| 47 | Grok 4.20 (Non-Reasoning) Estimated xAI | 90.1% | source | — | — | |
| 48 | Nemotron 3 Ultra NVIDIA | 90.1% | source | — | — | |
| 49 | Claude Opus 4.5 (latest) Supported Anthropic | 89.9% | source | — | — | |
| 50 | Claude Sonnet 4.6 Supported Anthropic | 89.9% | source | — | — | |
| 51 | DeepSeek V4.1 Flash Estimated DeepSeek | 89.9% | source | — | — | |
| 52 | Gemini 3 Flash Preview Supported Google | 89.1% | source | — | — | |
| 53 | GPT-5.6 Luna Estimated OpenAI | 89.1% | source | — | — | |
| 54 | Mistral Large 4 Mistral | 88.2% | source | — | — | |
| 55 | Qwen3.8 27B Alibaba (Qwen) | 88.0% | source | — | — | |
| 56 | DeepSeek V3.2 Supported DeepSeek | 87.7% | source | — | — | |
| 57 | Grok 4.1 Fast (Non-Reasoning) xAI | 87.7% | source | — | — | |
| 58 | GLM-5.3 Supported Z.ai (Zhipu) | 87.7% | source | — | — | |
| 59 | Inkling Estimated thinkingmachines | 87.5% | source | — | — | |
| 60 | Qwen3.5 397B-A17B Alibaba (Qwen) | 87.5% | source | — | — | |
| 61 | Qwen3.6 Max Preview Estimated Alibaba (Qwen) | 87.2% | source | — | — | |
| 62 | o3-pro OpenAI | 86.9% | source | — | — | |
| 63 | DeepSeek V4 Flash DeepSeek | 86.4% | source | — | — | |
| 64 | o3 Supported OpenAI | 84.8% | source | — | — | |
| 65 | MiMo-V2.5-Pro Xiaomi | 84.5% | source | — | — | |
| 66 | Qwen3.5 122B-A10B Alibaba (Qwen) | 84.3% | source | — | — | |
| 67 | Qwen3.7 Plus Alibaba (Qwen) | 84.0% | source | — | — | |
| 68 | Gemini 3.5 Flash Lite Google | 83.5% | source | — | — | |
| 69 | Claude Sonnet 4.5 (latest) Supported Anthropic | 83.2% | source | — | — | |
| 70 | Qwen3.5 Flash Alibaba (Qwen) | 82.9% | source | — | — | |
| 71 | DeepSeek V3.1 DeepSeek | 82.7% | source | — | — | |
| 72 | Gemma 4 31B IT Google | 82.7% | source | — | — | |
| 73 | Grok 4 Fast xAI | 82.7% | source | — | — | |
| 74 | Gemini 2.5 Pro Supported Google | 82.4% | source | — | — | |
| 75 | Qwen3.5 27B Alibaba (Qwen) | 82.4% | source | — | — | |
| 76 | Qwen3 Max Estimated Alibaba (Qwen) | 82.1% | source | — | — | |
| 77 | Qwen3.6 Plus Estimated Alibaba (Qwen) | 81.9% | source | — | — | |
| 78 | Claude-Opus-4 Supported Anthropic | 81.6% | source | — | — | |
| 79 | DeepSeek V3.1 Terminus DeepSeek | 81.3% | source | — | — | |
| 80 | Qwen Plus Alibaba (Qwen) | 81.1% | source | — | — | |
| 81 | GPT-5 Mini Estimated OpenAI | 80.5% | source | — | — | |
| 82 | Qwen3.5 Plus Estimated Alibaba (Qwen) | 80.5% | source | — | — | |
| 83 | GPT-5.4 nano OpenAI | 80.3% | source | — | — | |
| 84 | Qwen3 235B A22B Thinking 2507 Alibaba (Qwen) | 80.3% | source | — | — | |
| 85 | Claude Opus 4.1 (latest) Supported Anthropic | 80.0% | source | — | — | |
| 86 | GPT-5.4 mini OpenAI | 80.0% | source | — | — | |
| 87 | Qwen3.5 35B-A3B Alibaba (Qwen) | 80.0% | source | — | — | |
| 88 | MiniMax-M3 Estimated MiniMax | 78.9% | source | — | — | |
| 89 | Qwen3 235B A22B Instruct 2507 Alibaba (Qwen) | 78.4% | source | — | — | |
| 90 | Qwen3.6 27B Alibaba (Qwen) | 78.1% | source | — | — | |
| 91 | o4-mini Estimated OpenAI | 77.6% | source | — | — | |
| 92 | Claude Sonnet 4 Supported Anthropic | 77.1% | source | — | — | |
| 93 | Qwen3.6 Flash Alibaba (Qwen) | 77.1% | source | — | — | |
| 94 | Gemini 3.1 Flash Lite Google | 76.8% | source | — | — | |
| 95 | Gemini 2.5 Flash Estimated Google | 76.5% | source | — | — | |
| 96 | GPT OSS 120B Supported OpenAI | 76.3% | source | — | — | |
| 97 | Qwen3 235B A22B Alibaba (Qwen) | 75.7% | source | — | — | |
| 98 | Mistral Medium 3.5 Mistral | 75.5% | source | — | — | |
| 99 | Gemma 4 26B A4B Google | 74.9% | source | — | — | |
| 100 | o1 Estimated OpenAI | 74.7% | source | — | — | |
| 101 | Qwen3.6 35B-A3B Alibaba (Qwen) | 73.9% | source | — | — | |
| 102 | Claude Haiku 4.5 Estimated Anthropic | 73.6% | source | — | — | |
| 103 | Qwen3.5 9B Alibaba (Qwen) | 71.2% | source | — | — | |
| 104 | Mistral Small 4 Mistral | 70.9% | source | — | — | |
| 105 | Qwen3-30B-A3B-Thinking (Jul 2025) Alibaba (Qwen) | 69.3% | source | — | — | |
| 106 | GPT-4.1 mini OpenAI | 68.8% | source | — | — | |
| 107 | o3-mini Estimated OpenAI | 68.8% | source | — | — | |
| 108 | GPT-4.1 OpenAI | 68.3% | source | — | — | |
| 109 | GPT OSS 20B OpenAI | 68.0% | source | — | — | |
| 110 | Claude-Sonnet-3.5 Estimated Anthropic | 67.8% | source | — | — | |
| 111 | Mistral Medium 3.1 Mistral | 67.5% | source | — | — | |
| 112 | Qwen3 32B Alibaba (Qwen) | 67.5% | source | — | — | |
| 113 | Qwen3 30B A3B Instruct 2507 Alibaba (Qwen) | 67.2% | source | — | — | |
| 114 | Grok-2 (Dec 2024) xAI | 65.2% | source | — | — | |
| 115 | Mistral Large 3 Mistral | 65.1% | source | — | — | |
| 116 | DeepSeek V3 0324 DeepSeek | 64.8% | source | — | — | |
| 117 | GPT-4o (2024-08-06) Supported OpenAI | 64.5% | source | — | — | |
| 118 | Qwen3 14B Alibaba (Qwen) | 64.0% | source | — | — | |
| 119 | Gemini-2.0-Flash Google | 63.2% | source | — | — | |
| 120 | Qwen2.5-72B Estimated Alibaba (Qwen) | 62.9% | source | — | — | |
| 121 | Gemini 2.5 Flash-Lite Google | 62.8% | source | — | — | |
| 122 | GPT-4 OpenAI | 62.7% | source | — | — | |
| 123 | GPT-5 Nano OpenAI | 62.7% | source | — | — | |
| 124 | Mistral Medium 3 Mistral | 62.3% | source | — | — | |
| 125 | Llama 4 Maverick Meta | 61.9% | source | — | — | |
| 126 | Claude 3 Opus Estimated Anthropic | 61.6% | source | — | — | |
| 127 | GPT-4 Turbo OpenAI | 61.6% | source | — | — | |
| 128 | Llama 3.1-405B Meta | 61.4% | source | — | — | |
| 129 | Cohere Command A Cohere | 61.3% | source | — | — | |
| 130 | Magistral Small 1.2 Mistral | 61.3% | source | — | — | |
| 131 | Mistral Large 2 (Nov 2024) Mistral | 61.2% | source | — | — | |
| 132 | Qwen3 30B A3B Alibaba (Qwen) | 60.3% | source | — | — | |
| 133 | Llama 3.1-70B Meta | 60.0% | source | — | — | |
| 134 | Mistral Small 3.2 Mistral | 59.9% | source | — | — | |
| 135 | Qwen3 8B Alibaba (Qwen) | 59.7% | source | — | — | |
| 136 | Llama 3.3 70B Meta | 59.5% | source | — | — | |
| 137 | Gemini 1.5 Pro (Feb 2024) Google | 59.0% | source | — | — | |
| 138 | Mistral Small 3.1 Mistral | 58.6% | source | — | — | |
| 139 | Llama 4 Scout Meta | 57.9% | source | — | — | |
| 140 | Claude 3.5 Haiku Anthropic | 56.7% | source | — | — | |
| 141 | Mistral Large Mistral | 55.9% | source | — | — | |
| 142 | Mixtral 8x22B Mistral | 55.1% | source | — | — | |
| 143 | Command R+ Cohere | 54.9% | source | — | — | |
| 144 | GPT-4o mini OpenAI | 54.4% | source | — | — | |
| 145 | Llama 3-70B Meta | 54.2% | source | — | — | |
| 146 | Gemini 1.5 Flash (Sep 2024) Google | 53.8% | source | — | — | |
| 147 | Claude 3 Sonnet Anthropic | 53.6% | source | — | — | |
| 148 | Gemini-2.0-Flash-Lite Google | 52.5% | source | — | — | |
| 149 | Gemma 3 27B Google | 52.5% | source | — | — | |
| 150 | GPT-4.1 nano OpenAI | 52.5% | source | — | — | |
| 151 | Claude 2 Anthropic | 51.9% | source | — | — | |
| 152 | Ministral 3B Mistral | 51.7% | source | — | — | |
| 153 | Claude 2.1 Anthropic | 51.0% | source | — | — | |
| 154 | Gemma 3 4B Google | 50.9% | source | — | — | |
| 155 | Llama 3.1-8B Meta | 50.9% | source | — | — | |
| 156 | Claude-Haiku-3 Anthropic | 50.1% | source | — | — | |
| 157 | Gemini 1.5 Flash 8B Google | 50.0% | source | — | — | |
| 158 | Mixtral 8x7B Mistral | 49.6% | source | — | — | |
| 159 | Mistral Small v24.09 Mistral | 49.5% | source | — | — | |
| 160 | Gemma 3 12B Google | 48.8% | source | — | — | |
| 161 | Mistral Nemo Mistral | 48.6% | source | — | — | |
| 162 | GPT-3.5-turbo OpenAI | 48.5% | source | — | — | |
| 163 | Gemma 2 27B Google | 48.0% | source | — | — | |
| 164 | Qwen2.5-7B Alibaba (Qwen) | 47.7% | source | — | — | |
| 165 | c4ai-command-r-08-2024 Unknown | 46.4% | source | — | — | |
| 166 | Gemini 1.0 Pro Google | 45.9% | source | — | — | |
| 167 | Claude Instant Anthropic | 45.8% | source | — | — | |
| 168 | Ministral 3 8B 2512 Mistral | 45.7% | source | — | — | |
| 169 | Llama 3-8B Meta | 43.9% | source | — | — | |
| 170 | Mistral 7B v0.3 Mistral | 42.5% | source | — | — | |
| 171 | Llama 2-13B Meta | 42.2% | source | — | — | |
| 172 | Llama 2-70B Meta | 41.6% | source | — | — |