Fiction.LiveBench
Deep comprehension of long stories.
Agent Index weight
0%
shown for context only
Models tested
51
from the benchmark’s leaderboard and reports
Results
| # | Model | Score | Normalised | Harness | Cost/task | Date |
|---|---|---|---|---|---|---|
| 1 | GPT-5 Supported OpenAI | 97.2% | source | — | — | |
| 2 | o3-pro OpenAI | 97.2% | source | — | — | |
| 3 | Grok 4 Supported xAI | 94.4% | source | — | — | |
| 4 | Grok 4 Fast xAI | 94.4% | source | — | — | |
| 5 | Gemini 2.5 Pro Supported Google | 91.7% | source | — | — | |
| 6 | o3 Supported OpenAI | 88.9% | source | — | — | |
| 7 | Kimi K2.5 Supported Moonshot AI | 86.1% | source | — | — | |
| 8 | Claude 3.7 Sonnet Supported Anthropic | 83.3% | source | — | — | |
| 9 | DeepSeek V3.2 Supported DeepSeek | 83.3% | source | — | — | |
| 10 | o1 Estimated OpenAI | 83.3% | source | — | — | |
| 11 | Qwq 32B Alibaba (Qwen) | 83.3% | source | — | — | |
| 12 | Gemini 2.5 Flash Estimated Google | 77.8% | source | — | — | |
| 13 | o4-mini Estimated OpenAI | 77.8% | source | — | — | |
| 14 | DeepSeek-R1 Estimated DeepSeek | 75.0% | source | — | — | |
| 15 | Qwen3 235B A22B Thinking 2507 Alibaba (Qwen) | 75.0% | source | — | — | |
| 16 | Qwen3 32B Alibaba (Qwen) | 74.2% | source | — | — | |
| 17 | GPT-5 Mini Estimated OpenAI | 69.4% | source | — | — | |
| 18 | MiniMax M1 MiniMax | 69.4% | source | — | — | |
| 19 | Qwen3 235B A22B Alibaba (Qwen) | 67.7% | source | — | — | |
| 20 | GPT-4o (2024-08-06) Supported OpenAI | 66.7% | source | — | — | |
| 21 | Grok-3 mini xAI | 66.7% | source | — | — | |
| 22 | Kimi K2 0905 Estimated Moonshot AI | 66.7% | source | — | — | |
| 23 | Qwen2.5-Max Alibaba (Qwen) | 66.7% | source | — | — | |
| 24 | Qwen3 Max Estimated Alibaba (Qwen) | 66.7% | source | — | — | |
| 25 | GPT-4.1 OpenAI | 63.9% | source | — | — | |
| 26 | GPT-4.5 OpenAI | 63.9% | source | — | — | |
| 27 | Qwen3 14B Alibaba (Qwen) | 62.5% | source | — | — | |
| 28 | Qwen3 8B Alibaba (Qwen) | 62.1% | source | — | — | |
| 29 | Claude-Opus-4 Supported Anthropic | 61.1% | source | — | — | |
| 30 | Gemini-2.0-Flash Google | 61.1% | source | — | — | |
| 31 | chutes/GLM-4.5-FP8 Unknown | 58.3% | source | — | — | |
| 32 | Grok 3 xAI | 58.3% | source | — | — | |
| 33 | Qwen3-Next 80B-A3B Instruct Alibaba (Qwen) | 55.6% | source | — | — | |
| 34 | Qwen3 235B A22B Instruct 2507 Alibaba (Qwen) | 52.9% | source | — | — | |
| 35 | DeepSeek V3.1 DeepSeek | 52.8% | source | — | — | |
| 36 | Gemini 2.0 Flash Thinking (Jan 2025) Google | 52.8% | source | — | — | |
| 37 | DeepSeek V3 0324 DeepSeek | 50.0% | source | — | — | |
| 38 | o3-mini Estimated OpenAI | 50.0% | source | — | — | |
| 39 | Gemini 2.5 Flash-Lite Google | 47.2% | source | — | — | |
| 40 | Claude Sonnet 4 Supported Anthropic | 46.9% | source | — | — | |
| 41 | Llama 4 Maverick Meta | 46.2% | source | — | — | |
| 42 | GPT-4.1 mini OpenAI | 44.4% | source | — | — | |
| 43 | GPT-5 Nano OpenAI | 44.4% | source | — | — | |
| 44 | GPT OSS 120B Supported OpenAI | 44.4% | source | — | — | |
| 45 | Gemini 2.0 Pro Google | 41.7% | source | — | — | |
| 46 | Qwen3 30B A3B Alibaba (Qwen) | 40.6% | source | — | — | |
| 47 | Llama 4 Scout Meta | 36.0% | source | — | — | |
| 48 | Gemma 3 27B Google | 33.3% | source | — | — | |
| 49 | Llama 3.3 70B Meta | 33.3% | source | — | — | |
| 50 | GPT-4.1 nano OpenAI | 25.0% | source | — | — | |
| 51 | Nvidia Nemotron Nano 9B V2 NVIDIA | 25.0% | source | — | — |