SShogo Bench

Claude 3 Opus

Claude 3 Opus from Anthropic.

Shogo Agent Index

Estimated#92 of 92
11.0range 7.9–14.1 · 15% of weight measured
Agentic coding
11
estimated
Long-horizon autonomy
16
50% measured
Knowledge-work agents
12
estimated
Computer use
2
estimated
Security & context learning
10
50% measured

Price

No public API price.

Agent-effective = 20 input tokens per output token, 85% cache hits. Why.

Agentic benchmarks

BenchmarkScoreNormalisedIndex weightHarness / source
SWE-bench Verifiedest.8%predicted 35
Terminal-Bench 2.0est.8%predicted 2
DeepSWEest.6%predicted 3
FrontierSWEest.5%predicted 0
CursorBenchest.4%predicted 11
GSO-Benchest.4%predicted 0
METR Time Horizons29.5%10%source
Vending-Bench 2est.6%predicted 0
PostTrainBenchest.4%predicted 7
APEX-Agentsest.8%predicted 5
GDPvalest.7%predicted 9
Remote Labor Indexest.5%predicted 0
DeepResearch Benchest.5%predicted 38
OSWorld 2.0est.6%predicted 0
OSWorldest.4%predicted 5
Cybench10.0%5%source
CL-benchest.5%predicted 9

Other benchmarks

BenchmarkCategoryScoreSource
Chess PuzzlesReasoning4.0%source
DTBenchKnowledge61.6%source
GPQA DiamondReasoning47.2%source
LMCAKnowledge17.0%source
MATH Level 5Math37.5%source
MMLULegacy84.6%source
OTIS Mock AIMEMath4.7%source
SimpleBenchReasoning23.5%source
SimpleQA VerifiedKnowledge12.6%source