BALROG
Agentic play across text and grid games (NetHack, Crafter, BabyAI).
Agent Index weight
0%
shown for context only
Models tested
37
from the benchmark’s leaderboard and reports
Results
| # | Model | Score | Normalised | Harness | Cost/task | Date |
|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra Supported OpenAI | 68.3% | source | — | Sep 18, 2026 | |
| 2 | Claude Opus 5 Supported Anthropic | 63.4% | source | — | Sep 20, 2026 | |
| 3 | GPT-5.6 Sol Supported OpenAI | 60.0% | source | — | Sep 18, 2026 | |
| 4 | Gemini 3 Pro Preview Supported Google | 58.1% | source | — | Feb 3, 2026 | |
| 5 | Gemini 3.1 Pro Preview Supported Google | 57.0% | source | — | Feb 25, 2026 | |
| 6 | GPT-5.6 Terra Estimated OpenAI | 53.2% | source | — | Sep 20, 2026 | |
| 7 | Gemini 3 Flash Preview Supported Google | 48.1% | source | — | Feb 13, 2026 | |
| 8 | GPT-5.6 Luna Estimated OpenAI | 45.6% | source | — | Sep 18, 2026 | |
| 9 | Grok 4 Supported xAI | 43.6% | source | — | Jul 23, 2025 | |
| 10 | Claude Opus 4.5 (latest) Supported Anthropic | 43.5% | source | — | Feb 24, 2026 | |
| 11 | Gemini 2.5 Pro Supported Google | 43.3% | source | — | Apr 25, 2025 | |
| 12 | DeepSeek-R1 Estimated DeepSeek | 34.9% | source | — | Apr 10, 2025 | |
| 13 | Gemini 2.5 Flash Estimated Google | 33.5% | source | — | Jul 22, 2025 | |
| 14 | GPT-5 Supported OpenAI | 32.8% | source | — | Aug 19, 2025 | |
| 15 | Claude-Sonnet-3.5 Estimated Anthropic | 32.6% | source | — | Nov 11, 2024 | |
| 16 | GPT-4o (2024-08-06) Supported OpenAI | 32.3% | source | — | Nov 11, 2024 | |
| 17 | Claude Haiku 4.5 Estimated Anthropic | 31.2% | source | — | Feb 24, 2026 | |
| 18 | Grok 3 xAI | 29.5% | source | — | Apr 25, 2025 | |
| 19 | Reka Flash 3 rekaai | 29.2% | source | — | Mar 13, 2025 | |
| 20 | Llama 3.1-70B Meta | 27.9% | source | — | Nov 11, 2024 | |
| 21 | Llama 3.2 90B Meta | 27.3% | source | — | Nov 11, 2024 | |
| 22 | Llama 3.3 70B Meta | 23.0% | source | — | Dec 9, 2024 | |
| 23 | Gemini 1.5 Pro (Feb 2024) Google | 21.0% | source | — | Nov 11, 2024 | |
| 24 | DeepSeek-R1-Distill-Qwen-32B DeepSeek | 19.5% | source | — | Jan 26, 2025 | |
| 25 | Claude 3.5 Haiku Anthropic | 19.3% | source | — | Dec 11, 2024 | |
| 26 | Mistral Nemo Mistral | 17.6% | source | — | Dec 9, 2024 | |
| 27 | GPT-4o mini OpenAI | 17.4% | source | — | Nov 11, 2024 | |
| 28 | Llama 3.2 11B Meta | 16.8% | source | — | Nov 11, 2024 | |
| 29 | Qwen2.5-72B Estimated Alibaba (Qwen) | 16.2% | source | — | Nov 25, 2024 | |
| 30 | Llama 3.1-8B Meta | 15.1% | source | — | Nov 11, 2024 | |
| 31 | Gemini 1.5 Flash (Sep 2024) Google | 14.6% | source | — | Nov 11, 2024 | |
| 32 | Qwen2-VL-72B-Instruct Unknown | 12.8% | source | — | Nov 25, 2024 | |
| 33 | Phi 4 Microsoft | 11.6% | source | — | Jan 13, 2025 | |
| 34 | Llama 3.2 3B Instruct Meta | 10.1% | source | — | Nov 11, 2024 | |
| 35 | Qwen2.5-7B Alibaba (Qwen) | 7.8% | source | — | Nov 25, 2024 | |
| 36 | Llama 3.2 1B Instruct Meta | 6.6% | source | — | Nov 11, 2024 | |
| 37 | Qwen2-VL-7B-Instruct Unknown | 3.7% | source | — | Nov 25, 2024 |