Benchmarks / Artificial Analysis
Reported by Artificial Analysis
Artificial Analysis
Share of questions answered correctly over long documents, as run by Artificial Analysis with its own harness and prompts.
- Results dated
- 1 Oct 2026
- Models
- 309
- Unit
- % of questions
Full results
| # | Model | Artificial Analysis: Long-context reasoning (AA-LCR) % of questions, higher is better |
|---|---|---|
| 1 | Kimi K3 (max)Moonshot AI |
88.7%
|
| 2 | Claude Fable 5.1 (max)Anthropic |
85.3%
|
| 3 | Claude Fable 5.1 (medium reasoning)Anthropic |
84.7%
|
| 3 | Claude Opus 5.5 (max reasoning)Anthropic |
84.7%
|
| 3 | Claude Opus 5.5 (xhigh reasoning)Anthropic |
84.7%
|
| 6 | Claude Opus 5.5 (medium reasoning)Anthropic |
84.3%
|
| 6 | GPT-5.5 (high)OpenAI |
84.3%
|
| 6 | GPT-5.5 (xhigh reasoning)OpenAI |
84.3%
|
| 9 | DeepSeek V4.1 Flash (max)DeepSeek |
84.0%
|
| 9 | Gemini 3.8 Flash (medium reasoning)Google |
84.0%
|
| 9 | GPT-5.6 Sol (max)OpenAI |
84.0%
|
| 9 | GPT-6.1 Sol (low)OpenAI |
84.0%
|
| 13 | Claude Fable 5.1 (high reasoning)Anthropic |
83.7%
|
| 13 | GPT-5.6 Luna (max)OpenAI |
83.7%
|
| 13 | GPT-6 Sol (high)OpenAI |
83.7%
|
| 13 | GPT-6 Sol (max)OpenAI |
83.7%
|
| 17 | muse-glimmer (high)Meta |
83.3%
|
| 17 | GPT-5.3-Codex (xhigh)OpenAI |
83.3%
|
| 17 | GPT-6.1 Sol (medium)OpenAI |
83.3%
|
| 17 | GPT-6 Luna (max)OpenAI |
83.3%
|
| 21 | Claude Fable 5.1 (xhigh)Anthropic |
83.0%
|
| 21 | Gemini 3.7 Flash (medium reasoning)Google |
83.0%
|
| 21 | Muse Spark 1.3 (max)Meta |
83.0%
|
| 21 | Muse Spark 1.3 (xhigh)Meta |
83.0%
|
| 21 | GPT-5.5 (medium reasoning)OpenAI |
83.0%
|
| 21 | GPT-5.6 Terra (max)OpenAI |
83.0%
|
| 21 | GPT-6.1 Sol (max)OpenAI |
83.0%
|
| 28 | Claude Opus 5.5 (high)Anthropic |
82.7%
|
| 28 | Claude Sonnet 5.5 (max)Anthropic |
82.7%
|
| 28 | GPT-5.2 (xhigh)OpenAI |
82.7%
|
| 31 | Claude Fable 5.1 (low reasoning)Anthropic |
82.3%
|
| 31 | Claude Fable 5 (max reasoning)Anthropic |
82.3%
|
| 31 | GPT-5.2-Codex (xhigh)OpenAI |
82.3%
|
| 31 | GPT-5.6 Sol (xhigh)OpenAI |
82.3%
|
| 31 | GPT-6.1 Sol (high)OpenAI |
82.3%
|
| 31 | GPT-6 Sol (medium)OpenAI |
82.3%
|
| 37 | Qwen3.8 27B (xhigh)Alibaba |
82.0%
|
| 37 | Claude Opus 5 (medium)Anthropic |
82.0%
|
| 37 | Claude Sonnet 5 (max reasoning)Anthropic |
82.0%
|
| 37 | Gemini 3.1 Pro PreviewGoogle |
82.0%
|
| 37 | GPT-5.4 (xhigh reasoning)OpenAI |
82.0%
|
| 42 | Gemini 3.7 Flash (high)Google |
81.7%
|
| 42 | GPT-5.6 Luna (xhigh)OpenAI |
81.7%
|
| 42 | GPT-5.6 Sol (high reasoning)OpenAI |
81.7%
|
| 45 | Claude Opus 5 (low)Anthropic |
81.3%
|
| 45 | DeepSeek V4 Flash Vision (experimental, max)DeepSeek |
81.3%
|
| 45 | Gemini 3.8 Flash (high)Google |
81.3%
|
| 45 | GPT-6 Sol (xhigh)OpenAI |
81.3%
|
| 49 | Kimi K2.6Moonshot AI |
81.0%
|
| 49 | GPT-5.5 (low reasoning)OpenAI |
81.0%
|
| 49 | Grok 4.6 (medium)xAI |
81.0%
|
| 49 | Grok 4.6 (xhigh)xAI |
81.0%
|
| 53 | Qwen3.6 Max PreviewAlibaba |
80.7%
|
| 53 | Claude Opus 5.5 (low reasoning)Anthropic |
80.7%
|
| 53 | Gemini 3.8 Flash (low reasoning)Google |
80.7%
|
| 53 | GPT-6 Astra (max)OpenAI |
80.7%
|
| 53 | Grok 4.6 (low)xAI |
80.7%
|
| 58 | Qwen3.8 2.4T A95BAlibaba |
80.3%
|
| 58 | Qwen3.8 Max (0902)Alibaba |
80.3%
|
| 58 | Claude Opus 5 (xhigh)Anthropic |
80.3%
|
| 58 | DeepSeek V4 Pro 0423 (max)DeepSeek |
80.3%
|
| 58 | GPT-5.6 Luna (high reasoning)OpenAI |
80.3%
|
| 58 | GPT-5.6 Sol (medium reasoning)OpenAI |
80.3%
|
| 58 | Grok 4.6 (high)xAI |
80.3%
|
| 65 | Claude Sonnet 4.6 (max reasoning)Anthropic |
80.0%
|
| 65 | Gemini 3.6 Flash (high)Google |
80.0%
|
| 65 | GPT-5.1 (high)OpenAI |
80.0%
|
| 65 | GPT-6 Astra (high reasoning)OpenAI |
80.0%
|
| 65 | GPT-6 Astra (low reasoning)OpenAI |
80.0%
|
| 65 | GPT-6 Astra (xhigh reasoning)OpenAI |
80.0%
|
| 65 | GPT-6 Luna (xhigh)OpenAI |
80.0%
|
| 65 | GLM 5.3 FlashZ.ai |
80.0%
|
| 73 | Qwen3.8 27B (medium)Alibaba |
79.7%
|
| 73 | Qwen3.8 Flash NextAlibaba |
79.7%
|
| 73 | Claude Sonnet 5.5 (xhigh)Anthropic |
79.7%
|
| 73 | DeepSeek V4 Flash 0423 (max)DeepSeek |
79.7%
|
| 73 | Gemini 4 Argon (high)Google |
79.7%
|
| 73 | GPT-6.1 Sol (xhigh)OpenAI |
79.7%
|
| 73 | GPT-6 Astra (medium reasoning)OpenAI |
79.7%
|
| 73 | GLM 5.3 (max)Z.ai |
79.7%
|
| 81 | Claude Opus 5 (max)Anthropic |
79.3%
|
| 81 | Kimi K2.7 CodeMoonshot AI |
79.3%
|
| 81 | Kimi K3 (low reasoning)Moonshot AI |
79.3%
|
| 81 | GPT-6 Luna (high)OpenAI |
79.3%
|
| 81 | GPT-6 Sol (low)OpenAI |
79.3%
|
| 81 | Grok 4.5 (high)xAI |
79.3%
|
| 87 | Qwen3.7 MaxAlibaba |
79.0%
|
| 87 | Claude Opus 5 (high)Anthropic |
79.0%
|
| 87 | Muse Spark 1.2Meta |
79.0%
|
| 87 | GPT-5.6 Terra (xhigh)OpenAI |
79.0%
|
| 91 | Claude Opus 4.7 (max reasoning)Anthropic |
78.7%
|
| 91 | Gemini 3.7 Flash (low reasoning)Google |
78.7%
|
| 93 | Qwen3.6 PlusAlibaba |
78.3%
|
| 93 | Qwen3.8 Max (0803)Alibaba |
78.3%
|
| 93 | GPT-6 Luna (medium)OpenAI |
78.3%
|
| 93 | GLM 5.2 (max)Z.ai |
78.3%
|
| 97 | GPT-5 (high)OpenAI |
78.2%
|
| 98 | Claude Opus 4.6 (max reasoning)Anthropic |
78.0%
|
| 98 | Claude Sonnet 5.5 (high)Anthropic |
78.0%
|
| 98 | gemini-3-flash (thinking reasoning)Google |
78.0%
|
| 98 | muse-sparkMeta |
78.0%
|
| 98 | Kimi K2.5Moonshot AI |
78.0%
|
| 98 | GPT-5.6 Sol (low reasoning)OpenAI |
78.0%
|
| 104 | Qwen3.5-27BAlibaba |
77.7%
|
| 104 | Claude Opus 4.8 (max reasoning)Anthropic |
77.7%
|
| 104 | Muse Spark 1.1 (xhigh)Meta |
77.7%
|
| 104 | GPT-5.6 Terra (high reasoning)OpenAI |
77.7%
|
| 108 | Qwen3.5 397B A17BAlibaba |
77.3%
|
| 108 | Qwen3.6 27BAlibaba |
77.3%
|
| 108 | Qwen3.8 27B (low)Alibaba |
77.3%
|
| 108 | Claude Opus 4.5 (thinking reasoning)Anthropic |
77.3%
|
| 112 | GPT-5.4 Mini (xhigh)OpenAI |
77.0%
|
| 112 | Grok 4.7 (high)xAI |
77.0%
|
| 114 | Claude Sonnet 5 (high)Anthropic |
76.7%
|
| 114 | Claude Sonnet 5 (xhigh reasoning)Anthropic |
76.7%
|
| 114 | GPT-5.4 Nano (xhigh)OpenAI |
76.7%
|
| 114 | GPT-5.4 (low reasoning)OpenAI |
76.7%
|
| 114 | Grok 4.7 (xhigh)xAI |
76.7%
|
| 119 | Qwen3.5-122B-A10BAlibaba |
76.3%
|
| 119 | Claude Sonnet 5.5 (medium)Anthropic |
76.3%
|
| 121 | Claude Opus 4.1 (thinking reasoning)Anthropic |
76.0%
|
| 121 | Claude Sonnet 5.5 (low reasoning)Anthropic |
76.0%
|
| 121 | Gemini 3.5 Flash LiteGoogle |
76.0%
|
| 121 | gemini-3-pro (high)Google |
76.0%
|
| 121 | GPT-5 (medium)OpenAI |
76.0%
|
| 126 | Claude Opus 4.7 (off)Anthropic |
75.7%
|
| 126 | GLM 5Z.ai |
75.7%
|
| 128 | GPT-5.6 Luna (medium reasoning)OpenAI |
75.0%
|
| 128 | Grok 4.3 (medium)xAI |
75.0%
|
| 130 | deepseek-v4-pro-preview (max reasoning)DeepSeek |
74.7%
|
| 130 | Grok Build 0.1xAI |
74.7%
|
| 132 | Qwen3 Max ThinkingAlibaba |
74.3%
|
| 132 | Claude Haiku 4.5 (thinking reasoning)Anthropic |
74.3%
|
| 132 | deepseek-v4-flash-preview (max reasoning)DeepSeek |
74.3%
|
| 132 | Gemini 3.1 Flash Lite PreviewGoogle |
74.3%
|
| 132 | Gemini 3.5 Flash (medium)Google |
74.3%
|
| 137 | gemini-3-pro (low)Google |
74.0%
|
| 137 | GPT-5.6 Terra (medium reasoning)OpenAI |
74.0%
|
| 137 | GPT-6 Luna (low)OpenAI |
74.0%
|
| 137 | grok-4-1-fast-reasoningxAI |
74.0%
|
| 137 | Grok 4.3 (low)xAI |
74.0%
|
| 142 | Claude Sonnet 5 (medium reasoning)Anthropic |
73.7%
|
| 142 | grok-4-fast-reasoningxAI |
73.7%
|
| 142 | GLM 5.1Z.ai |
73.7%
|
| 145 | DeepSeek V3.2DeepSeek |
73.3%
|
| 145 | Gemini 3.5 Flash (high)Google |
73.3%
|
| 147 | Qwen3.7 PlusAlibaba |
73.0%
|
| 147 | Grok 4.3 (high)xAI |
73.0%
|
| 149 | GLM 5.3 (low reasoning)Z.ai |
72.7%
|
| 150 | Claude Sonnet 4.5 (thinking reasoning)Anthropic |
72.3%
|
| 150 | DeepSeek V3.2 ExpDeepSeek |
72.3%
|
| 150 | GPT-5 Mini (high)OpenAI |
72.3%
|
| 150 | GPT-5 Mini (medium)OpenAI |
72.3%
|
| 154 | Qwen3 235B A22B Thinking 2507Alibaba |
72.0%
|
| 154 | Qwen3.5-35B-A3BAlibaba |
72.0%
|
| 154 | deepseek-v4-flash-preview (high reasoning)DeepSeek |
72.0%
|
| 154 | Kimi K2 ThinkingMoonshot AI |
72.0%
|
| 158 | Qwen3.6 35B A3BAlibaba |
71.7%
|
| 158 | GPT-5-Codex (high)OpenAI |
71.7%
|
| 158 | GLM 5 TurboZ.ai |
71.7%
|
| 161 | GPT-5.6 Terra (low reasoning)OpenAI |
71.3%
|
| 162 | gemini-2.5-flash-preview-09-2025 (thinking reasoning)Google |
71.0%
|
| 162 | GLM 4.7Z.ai |
71.0%
|
| 164 | Claude Opus 4.5 (no reasoning)Anthropic |
70.7%
|
| 165 | deepseek-v4-pro-preview (high reasoning)DeepSeek |
70.3%
|
| 165 | GPT-5.2 (medium reasoning)OpenAI |
70.3%
|
| 165 | GLM 5V TurboZ.ai |
70.3%
|
| 168 | Qwen3.5-9BAlibaba |
70.0%
|
| 168 | Claude Sonnet 5 (off)Anthropic |
70.0%
|
| 168 | deepseek-v3.2-specialeDeepSeek |
70.0%
|
| 168 | GPT-5.5 Instant (26 June)OpenAI |
70.0%
|
| 168 | GPT-5.6 Luna (low reasoning)OpenAI |
70.0%
|
| 173 | Gemma 4 31BGoogle |
69.7%
|
| 173 | Kimi K2.6 (no reasoning)Moonshot AI |
69.7%
|
| 175 | Qwen3.8 27B (off)Alibaba |
69.3%
|
| 175 | DeepSeek V3.1 TerminusDeepSeek |
69.3%
|
| 175 | Mistral Medium 3.5Mistral |
69.3%
|
| 175 | GPT-5.1-Codex (high)OpenAI |
69.3%
|
| 179 | Grok 4.20xAI |
69.0%
|
| 180 | Claude Sonnet 4.6 (off)Anthropic |
68.3%
|
| 181 | grok-4xAI |
68.0%
|
| 182 | grok-4.20-0309-reasoningxAI |
67.7%
|
| 183 | Claude Sonnet 5 (low reasoning)Anthropic |
67.3%
|
| 183 | Kimi K2.5Moonshot AI |
67.3%
|
| 183 | GPT-5.4 Nano (medium)OpenAI |
67.3%
|
| 186 | Claude Opus 4.6 (off)Anthropic |
67.0%
|
| 186 | GPT-5.4 Mini (medium)OpenAI |
67.0%
|
| 188 | Qwen3.6 27B (no reasoning)Alibaba |
66.7%
|
| 188 | GPT-5.1-Codex-Mini (high)OpenAI |
66.7%
|
| 190 | GPT-5.5 Instant (26 May)OpenAI |
66.3%
|
| 191 | Gemma 4 26B A4BGoogle |
65.7%
|
| 192 | gpt-5-chatOpenAI |
65.0%
|
| 193 | gemini-2.5-flash-lite-preview-09-2025 (thinking reasoning)Google |
64.7%
|
| 194 | Qwen3.5 397B A17B (no reasoning)Alibaba |
64.3%
|
| 194 | Qwen3.6 35B A3B (no reasoning)Alibaba |
64.3%
|
| 196 | Qwen3.5-27B (no reasoning)Alibaba |
64.0%
|
| 196 | GPT-5.5 (no reasoning)OpenAI |
64.0%
|
| 196 | GPT-6 Sol (no reasoning)OpenAI |
64.0%
|
| 199 | Qwen3.5 Omni PlusAlibaba |
63.7%
|
| 199 | Qwen3 Next 80B A3B ThinkingAlibaba |
63.7%
|
| 199 | Qwen3 VL 235B A22B ThinkingAlibaba |
63.7%
|
| 199 | gemma-4-12b-itGoogle |
63.7%
|
| 203 | Qwen3.5-35B-A3B (no reasoning)Alibaba |
63.0%
|
| 203 | qwen3.5-4b (thinking reasoning)Alibaba |
63.0%
|
| 205 | GPT-5.6 Sol (no reasoning)OpenAI |
62.3%
|
| 206 | Qwen3 Max Thinking (preview)Alibaba |
62.0%
|
| 207 | Qwen3 30B A3B Thinking 2507Alibaba |
61.3%
|
| 207 | Qwen3.5-122B-A10B (no reasoning)Alibaba |
61.3%
|
| 207 | Gemini 3.5 Flash (minimal reasoning)Google |
61.3%
|
| 210 | gemini-2.5-flash-preview-09-2025 (no reasoning)Google |
60.0%
|
| 211 | GPT-5.6 Terra (no reasoning)OpenAI |
58.7%
|
| 212 | GPT-5.4 (no reasoning)OpenAI |
58.3%
|
| 213 | DeepSeek V3.1DeepSeek |
56.7%
|
| 214 | qwen3-vl-32b-thinkingAlibaba |
55.3%
|
| 214 | DeepSeek V4.1 Flash (off)DeepSeek |
55.3%
|
| 214 | gemini-3-flash (off)Google |
55.3%
|
| 217 | Claude Sonnet 4.5 (no reasoning)Anthropic |
54.0%
|
| 217 | GLM 4.6 (thinking reasoning)Z.ai |
54.0%
|
| 219 | GLM 5.1 (no reasoning)Z.ai |
53.3%
|
| 220 | deepseek-v4-pro-preview (no reasoning)DeepSeek |
53.0%
|
| 220 | Magistral Medium 1.2Mistral |
53.0%
|
| 220 | Kimi K2 0905Moonshot AI |
53.0%
|
| 220 | Kimi K2 0711Moonshot AI |
53.0%
|
| 220 | Grok Code Fast 1xAI |
53.0%
|
| 225 | Qwen3 Next 80B A3B InstructAlibaba |
52.7%
|
| 225 | GLM 4.5Z.ai |
52.7%
|
| 227 | Qwen3.5 Omni FlashAlibaba |
52.0%
|
| 227 | gpt-oss-120b (high)OpenAI |
52.0%
|
| 229 | DeepSeek V4 Pro 0423 (no reasoning)DeepSeek |
50.7%
|
| 230 | Qwen3 MaxAlibaba |
50.0%
|
| 231 | Claude Haiku 4.5 (no reasoning)Anthropic |
49.7%
|
| 231 | gemini-2.5-flash-lite-preview-09-2025 (no reasoning)Google |
49.7%
|
| 231 | Mistral Small 4Mistral |
49.7%
|
| 234 | GLM 4.6V (thinking reasoning)Z.ai |
48.7%
|
| 235 | Qwen3 Coder NextAlibaba |
47.0%
|
| 235 | DeepSeek V3.1 (no reasoning)DeepSeek |
47.0%
|
| 237 | Gemma 4 31B (off)Google |
46.7%
|
| 237 | GLM 4.5 AirZ.ai |
46.7%
|
| 239 | Qwen3.5-9B (no reasoning)Alibaba |
46.0%
|
| 239 | gpt-oss-120b (low)OpenAI |
46.0%
|
| 241 | Qwen3 Coder 480B A35BAlibaba |
45.7%
|
| 241 | DeepSeek V3.2 (no reasoning)DeepSeek |
45.7%
|
| 241 | GPT-5.2 (no reasoning)OpenAI |
45.7%
|
| 244 | DeepSeek V3.1 Terminus (no reasoning)DeepSeek |
45.3%
|
| 245 | GPT-5.1 (off)OpenAI |
45.0%
|
| 245 | GPT-5 Nano (high)OpenAI |
45.0%
|
| 247 | DeepSeek V3.2 Exp (no reasoning)DeepSeek |
44.7%
|
| 248 | GPT-5 Nano (medium)OpenAI |
43.7%
|
| 248 | GLM 5 (no reasoning)Z.ai |
43.7%
|
| 250 | qwen3-max-previewAlibaba |
43.0%
|
| 251 | Mistral Medium 3.1Mistral |
42.7%
|
| 251 | GPT-5.6 Luna (no reasoning)OpenAI |
42.7%
|
| 253 | Gemma 4 26B A4B (off)Google |
42.3%
|
| 253 | GLM 5.2 (no reasoning)Z.ai |
42.3%
|
| 255 | deepseek-v4-flash-preview (no reasoning)DeepSeek |
41.7%
|
| 255 | GLM 4.7 FlashZ.ai |
41.7%
|
| 257 | GLM 4.7 (no reasoning)Z.ai |
40.7%
|
| 258 | GPT-6 Luna (no reasoning)OpenAI |
39.7%
|
| 259 | GPT-5 Mini (minimal)OpenAI |
39.3%
|
| 260 | qwen3-4b-thinking-2507Alibaba |
37.3%
|
| 261 | GPT-5.4 Mini (no reasoning)OpenAI |
37.0%
|
| 262 | Mistral Large 3 2512Mistral |
36.0%
|
| 263 | gemma-4-12b-it (no reasoning)Google |
35.0%
|
| 264 | qwen3.5-4b (no reasoning)Alibaba |
34.7%
|
| 264 | gpt-oss-20b (high reasoning)OpenAI |
34.7%
|
| 266 | Qwen3 235B A22B Instruct 2507Alibaba |
33.9%
|
| 267 | Qwen3 VL 8B ThinkingAlibaba |
33.3%
|
| 268 | Qwen3 Coder 30B A3B InstructAlibaba |
32.7%
|
| 268 | Qwen3 VL 235B A22B InstructAlibaba |
32.7%
|
| 270 | Devstral 2 2512Mistral |
32.3%
|
| 270 | Grok 4.3 (off)xAI |
32.3%
|
| 272 | gemma-4-e4b-itGoogle |
32.0%
|
| 272 | Devstral MediumMistral |
32.0%
|
| 274 | grok-4-1-fast-non-reasoning (no reasoning)xAI |
31.3%
|
| 275 | gpt-oss-20b (low reasoning)OpenAI |
31.0%
|
| 276 | GPT-5.4 Nano (no reasoning)OpenAI |
29.7%
|
| 277 | Mistral Small 4 (no reasoning)Mistral |
28.3%
|
| 278 | devstral-small-2-24b-instruct-2512Mistral |
28.0%
|
| 278 | kimi-linear-48b-a3b-instructMoonshot AI |
28.0%
|
| 280 | Qwen3 30B A3B Instruct 2507Alibaba |
26.3%
|
| 280 | Ministral 3 14B 2512Mistral |
26.3%
|
| 280 | GLM 4.6 (no reasoning)Z.ai |
26.3%
|
| 283 | Ministral 3 8B 2512Mistral |
25.7%
|
| 284 | grok-4.20-0309-non-reasoning (no reasoning)xAI |
25.3%
|
| 285 | gemma-4-e4b-it (no reasoning)Google |
24.0%
|
| 285 | grok-4-fast-non-reasoning (no reasoning)xAI |
24.0%
|
| 287 | Grok 4.20 (off)xAI |
23.3%
|
| 288 | qwen3-vl-4b-thinkingAlibaba |
21.3%
|
| 289 | qwen3.5-2b (thinking reasoning)Alibaba |
21.0%
|
| 290 | GLM 4.7 Flash (no reasoning)Z.ai |
20.3%
|
| 291 | GPT-5 Nano (minimal)OpenAI |
20.0%
|
| 292 | DiffusionGemma 26B A4BGoogle |
19.7%
|
| 293 | magistral-small-2509Mistral |
19.3%
|
| 294 | Devstral Small 1.1Mistral |
18.7%
|
| 295 | Ministral 3 3B 2512Mistral |
17.0%
|
| 295 | GLM 4.6V (no reasoning)Z.ai |
17.0%
|
| 297 | Qwen3 VL 8B InstructAlibaba |
16.7%
|
| 298 | gemma-4-e2b-it (no reasoning)Google |
16.3%
|
| 298 | gemma-4-e2b-itGoogle |
16.3%
|
| 300 | qwen3.5-2b (no reasoning)Alibaba |
14.0%
|
| 300 | qwen3-vl-4b-instructAlibaba |
14.0%
|
| 302 | qwen3-4b-instruct-2507Alibaba |
11.3%
|
| 303 | qwen3.5-0.8b (thinking reasoning)Alibaba |
9.0%
|
| 304 | qwen3.5-0.8b (no reasoning)Alibaba |
8.0%
|
| 305 | Qwen3 Omni 30B A3B InstructAlibaba |
0.0%
|
| 305 | qwen3-omni-30b-a3b-thinkingAlibaba |
0.0%
|
| 305 | Gemma 3 270MGoogle |
0.0%
|
| 305 | GLM 4.5V (off)Z.ai |
0.0%
|
| 305 | GLM 4.5V (thinking reasoning)Z.ai |
0.0%
|
Results as published by Artificial Analysis; we do not re-run them.
What it measures
Share of questions answered correctly over long documents, as run by Artificial Analysis with its own harness and prompts.
What it does not measure
Not retrieval over your own document store.
Source
Failures
Failures count against a model: a product with no usable output is a failed listing. They are listed here so you can see why.
Not ranked
- Command A: creator outside the providers the catalogue covers
- GPT-4 Turbo: released before 2025-07-01
- GPT-4o (2024-05-13): released before 2025-07-01
- GPT-4o (2024-08-06): released before 2025-07-01
- Hy3: creator outside the providers the catalogue covers
- Hy3 preview: creator outside the providers the catalogue covers
- Inkling: creator outside the providers the catalogue covers
- Inkling Small: creator outside the providers the catalogue covers
- Llama 4 Maverick: released before 2025-07-01
- Llama 4 Scout: released before 2025-07-01
- Mercury 2: creator outside the providers the catalogue covers
- MiniMax M2: creator outside the providers the catalogue covers
- MiniMax M3: creator outside the providers the catalogue covers
- Mistral Large 2407: released before 2025-07-01
- Mistral Medium 3: released before 2025-07-01
- Phi 4: creator outside the providers the catalogue covers
- QwQ-32B-Preview: released before 2025-07-01
- R1: released before 2025-07-01
- Reka Flash 3: creator outside the providers the catalogue covers
- Solar Pro 3: creator outside the providers the catalogue covers
- Solar Pro 4: creator outside the providers the catalogue covers
- Trinity Large Thinking: creator outside the providers the catalogue covers
- a-x-k2: creator outside the providers the catalogue covers
- apertus-70b-instruct: creator outside the providers the catalogue covers
- apertus-8b-instruct: creator outside the providers the catalogue covers
- apodex-1-1: creator outside the providers the catalogue covers
- apriel-v1-5-15b-thinker: creator outside the providers the catalogue covers
- apriel-v1-6-15b-thinker: creator outside the providers the catalogue covers
- arctic-instruct: creator outside the providers the catalogue covers
- celeris-1: creator outside the providers the catalogue covers
- claude-2: released before 2025-07-01
- claude-21: released before 2025-07-01
- claude-3-5-haiku: released before 2025-07-01
- claude-3-7-sonnet: released before 2025-07-01
- claude-3-7-sonnet-thinking: released before 2025-07-01
- claude-3-haiku: released before 2025-07-01
- claude-3-opus: released before 2025-07-01
- claude-3-sonnet: released before 2025-07-01
- claude-35-sonnet: released before 2025-07-01
- claude-35-sonnet-june-24: released before 2025-07-01
- claude-4-opus: released before 2025-07-01
- claude-4-opus-thinking: released before 2025-07-01
- claude-4-sonnet: released before 2025-07-01
- claude-4-sonnet-thinking: released before 2025-07-01
- claude-instant: released before 2025-07-01
- claude-sonnet-4-6-non-reasoning-low-effort: non-standard setting combination
- cogito-v2-1-reasoning: creator outside the providers the catalogue covers
- command-a-plus: creator outside the providers the catalogue covers
- command-r-03-2024: creator outside the providers the catalogue covers
- command-r-plus-04-2024: creator outside the providers the catalogue covers
- dbrx: creator outside the providers the catalogue covers
- deephermes-3-llama-3-1-8b-preview: creator outside the providers the catalogue covers
- deephermes-3-mistral-24b-preview: creator outside the providers the catalogue covers
- deepseek-coder-v2: released before 2025-07-01
- deepseek-coder-v2-lite: released before 2025-07-01
- deepseek-llm-67b-chat: released before 2025-07-01
- deepseek-r1-0120: released before 2025-07-01
- deepseek-r1-distill-llama-70b: released before 2025-07-01
- deepseek-r1-distill-llama-8b: released before 2025-07-01
- deepseek-r1-distill-qwen-1-5b: released before 2025-07-01
- deepseek-r1-distill-qwen-14b: released before 2025-07-01
- deepseek-r1-distill-qwen-32b: released before 2025-07-01
- deepseek-r1-qwen3-8b: released before 2025-07-01
- deepseek-v2: released before 2025-07-01
- deepseek-v2-5: released before 2025-07-01
- deepseek-v2-5-sep-2024: released before 2025-07-01
- deepseek-v3: released before 2025-07-01
- deepseek-v3-0324: released before 2025-07-01
- devstral-small-2505: released before 2025-07-01
- doubao-seed-code: creator outside the providers the catalogue covers
- ernie-4-5-300b-a47b: creator outside the providers the catalogue covers
- ernie-5-0-thinking-preview: creator outside the providers the catalogue covers
- exaone-4-0-1-2b: creator outside the providers the catalogue covers
- exaone-4-0-1-2b-reasoning: creator outside the providers the catalogue covers
- exaone-4-0-32b: creator outside the providers the catalogue covers
- exaone-4-0-32b-reasoning: creator outside the providers the catalogue covers
- exaone-4-5-33b: creator outside the providers the catalogue covers
- falcon-h1r-7b: creator outside the providers the catalogue covers
- g9v3-39a5b: creator outside the providers the catalogue covers
- g9v3-3b: creator outside the providers the catalogue covers
- gemini-1-0-pro: released before 2025-07-01
- gemini-1-0-ultra: released before 2025-07-01
- gemini-1-5-flash: released before 2025-07-01
- gemini-1-5-flash-8b: released before 2025-07-01
- gemini-1-5-flash-may-2024: released before 2025-07-01
- gemini-1-5-pro: released before 2025-07-01
- gemini-1-5-pro-may-2024: released before 2025-07-01
- gemini-2-0-flash: released before 2025-07-01
- gemini-2-0-flash-experimental: released before 2025-07-01
- gemini-2-0-flash-lite-001: released before 2025-07-01
- gemini-2-0-flash-lite-preview: released before 2025-07-01
- gemini-2-0-flash-thinking-exp-0121: released before 2025-07-01
- gemini-2-0-flash-thinking-exp-1219: released before 2025-07-01
- gemini-2-0-pro-experimental-02-05: released before 2025-07-01
- gemini-2-5-flash: released before 2025-07-01
- gemini-2-5-flash-04-2025: released before 2025-07-01
- gemini-2-5-flash-lite: released before 2025-07-01
- gemini-2-5-flash-lite-reasoning: released before 2025-07-01
- gemini-2-5-flash-reasoning: released before 2025-07-01
- gemini-2-5-flash-reasoning-04-2025: released before 2025-07-01
- gemini-2-5-pro: released before 2025-07-01
- gemini-2-5-pro-03-25: released before 2025-07-01
- gemini-2-5-pro-05-06: released before 2025-07-01
- gemma-3-12b: released before 2025-07-01
- gemma-3-1b: released before 2025-07-01
- gemma-3-27b: released before 2025-07-01
- gemma-3-4b: released before 2025-07-01
- gemma-3n-e2b: released before 2025-07-01
- gemma-3n-e4b: released before 2025-07-01
- gemma-3n-e4b-preview-0520: released before 2025-07-01
- gpt-35-turbo: released before 2025-07-01
- gpt-4: released before 2025-07-01
- gpt-4-1: released before 2025-07-01
- gpt-4-1-mini: released before 2025-07-01
- gpt-4-1-nano: released before 2025-07-01
- gpt-4-5: released before 2025-07-01
- gpt-4o: released before 2025-07-01
- gpt-4o-chatgpt: released before 2025-07-01
- gpt-4o-chatgpt-03-25: released before 2025-07-01
- gpt-4o-mini: released before 2025-07-01
- granite-3-3-8b-instruct: creator outside the providers the catalogue covers
- granite-4-0-350m: creator outside the providers the catalogue covers
- granite-4-0-h-350m: creator outside the providers the catalogue covers
- granite-4-0-h-nano-1b: creator outside the providers the catalogue covers
- granite-4-0-h-small: creator outside the providers the catalogue covers
- granite-4-0-micro: creator outside the providers the catalogue covers
- granite-4-0-nano-1b: creator outside the providers the catalogue covers
- granite-4-1-30b: creator outside the providers the catalogue covers
- granite-4-1-3b: creator outside the providers the catalogue covers
- granite-4-1-8b: creator outside the providers the catalogue covers
- granite-4-2-30b: creator outside the providers the catalogue covers
- granite-4-2-3b: creator outside the providers the catalogue covers
- granite-4-2-8b: creator outside the providers the catalogue covers
- grok-1: released before 2025-07-01
- grok-2-1212: released before 2025-07-01
- grok-3: released before 2025-07-01
- grok-3-mini-reasoning: released before 2025-07-01
- grok-3-reasoning: released before 2025-07-01
- grok-beta: released before 2025-07-01
- hermes-3-llama-3-1-70b: creator outside the providers the catalogue covers
- hermes-4-llama-3-1-405b: creator outside the providers the catalogue covers
- hermes-4-llama-3-1-405b-reasoning: creator outside the providers the catalogue covers
- hermes-4-llama-3-1-70b: creator outside the providers the catalogue covers
- hermes-4-llama-3-1-70b-reasoning: creator outside the providers the catalogue covers
- hy3-non-reasoning: creator outside the providers the catalogue covers
- hyperclova-x-seed-think-32b: creator outside the providers the catalogue covers
- hypernova-60b: creator outside the providers the catalogue covers
- intellect-3: creator outside the providers the catalogue covers
- jamba-1-5-large: creator outside the providers the catalogue covers
- jamba-1-5-mini: creator outside the providers the catalogue covers
- jamba-1-6-large: creator outside the providers the catalogue covers
- jamba-1-6-mini: creator outside the providers the catalogue covers
- jamba-1-7-large: creator outside the providers the catalogue covers
- jamba-1-7-mini: creator outside the providers the catalogue covers
- jamba-reasoning-3b: creator outside the providers the catalogue covers
- jt-35b-flash: creator outside the providers the catalogue covers
- jt-4-1-flash-236b-a21b: creator outside the providers the catalogue covers
- jt-mini: creator outside the providers the catalogue covers
- jt236b: creator outside the providers the catalogue covers
- k-exaone: creator outside the providers the catalogue covers
- k-exaone-2-0-0803: creator outside the providers the catalogue covers
- k-exaone-non-reasoning: creator outside the providers the catalogue covers
- k2-horizon-0-9b: creator outside the providers the catalogue covers
- k2-horizon-3-7b: creator outside the providers the catalogue covers
- k2-horizon-375b-a23b: creator outside the providers the catalogue covers
- k2-horizon-7b: creator outside the providers the catalogue covers
- k2-horizon-mova-36b-a4b: creator outside the providers the catalogue covers
- k2-think-v2: creator outside the providers the catalogue covers
- k2-v2: creator outside the providers the catalogue covers
- k2-v2-low: creator outside the providers the catalogue covers
- k2-v2-medium: creator outside the providers the catalogue covers
- kat-coder-pro-v1: creator outside the providers the catalogue covers
- kat-coder-pro-v2: creator outside the providers the catalogue covers
- lfm-40b: creator outside the providers the catalogue covers
- lfm2-1-2b: creator outside the providers the catalogue covers
- lfm2-2-6b: creator outside the providers the catalogue covers
- lfm2-24b-a2b: creator outside the providers the catalogue covers
- lfm2-5-1-2b-instruct: creator outside the providers the catalogue covers
- lfm2-5-1-2b-thinking: creator outside the providers the catalogue covers
- lfm2-5-2-6b: creator outside the providers the catalogue covers
- lfm2-5-8b-a1b: creator outside the providers the catalogue covers
- lfm2-5-vl-1-6b: creator outside the providers the catalogue covers
- lfm2-8b-a1b: creator outside the providers the catalogue covers
- ling-1t: creator outside the providers the catalogue covers
- ling-2-6-1t: creator outside the providers the catalogue covers
- ling-2-6-flash: creator outside the providers the catalogue covers
- ling-3-0-flash: creator outside the providers the catalogue covers
- ling-3-0-flash-fin: creator outside the providers the catalogue covers
- ling-3-0-flash-vl: creator outside the providers the catalogue covers
- ling-3-0-tiny: creator outside the providers the catalogue covers
- ling-flash-2-0: creator outside the providers the catalogue covers
- ling-mini-2-0: creator outside the providers the catalogue covers
- llama-2-chat-13b: released before 2025-07-01
- llama-2-chat-70b: released before 2025-07-01
- llama-2-chat-7b: released before 2025-07-01
- llama-3-1-instruct-405b: released before 2025-07-01
- llama-3-1-instruct-70b: released before 2025-07-01
- llama-3-1-instruct-8b: released before 2025-07-01
- llama-3-1-nemotron-instruct-70b: creator outside the providers the catalogue covers
- llama-3-1-nemotron-nano-4b-reasoning: creator outside the providers the catalogue covers
- llama-3-1-nemotron-ultra-253b-v1-reasoning: creator outside the providers the catalogue covers
- llama-3-2-instruct-11b-vision: released before 2025-07-01
- llama-3-2-instruct-1b: released before 2025-07-01
- llama-3-2-instruct-3b: released before 2025-07-01
- llama-3-2-instruct-90b-vision: released before 2025-07-01
- llama-3-3-instruct-70b: released before 2025-07-01
- llama-3-3-nemotron-super-49b: creator outside the providers the catalogue covers
- llama-3-3-nemotron-super-49b-reasoning: creator outside the providers the catalogue covers
- llama-3-instruct-70b: released before 2025-07-01
- llama-3-instruct-8b: released before 2025-07-01
- llama-65b: released before 2025-07-01
- llama-nemotron-super-49b-v1-5: creator outside the providers the catalogue covers
- llama-nemotron-super-49b-v1-5-reasoning: creator outside the providers the catalogue covers
- longcat-2-0: creator outside the providers the catalogue covers
- longcat-flash-lite: creator outside the providers the catalogue covers
- magistral-medium: released before 2025-07-01
- magistral-small: released before 2025-07-01
- mercury-2-5: creator outside the providers the catalogue covers
- mi-dm-k-2-5-pro-dec28: creator outside the providers the catalogue covers
- midm-250-pro-rsnsft: creator outside the providers the catalogue covers
- mimo-v2-0206: creator outside the providers the catalogue covers
- mimo-v2-5-0424: creator outside the providers the catalogue covers
- mimo-v2-5-pro: creator outside the providers the catalogue covers
- mimo-v2-5-pro-non-reasoning: creator outside the providers the catalogue covers
- mimo-v2-6-flash: creator outside the providers the catalogue covers
- mimo-v2-6-pro: creator outside the providers the catalogue covers
- mimo-v2-flash: creator outside the providers the catalogue covers
- mimo-v2-flash-reasoning: creator outside the providers the catalogue covers
- mimo-v2-omni: creator outside the providers the catalogue covers
- mimo-v2-omni-0327: creator outside the providers the catalogue covers
- mimo-v2-pro: creator outside the providers the catalogue covers
- minicpm-v4-6-1-3b: creator outside the providers the catalogue covers
- minicpm5-1b: creator outside the providers the catalogue covers
- minicpm5-1b-non-reasoning: creator outside the providers the catalogue covers
- minicpm5-2b: creator outside the providers the catalogue covers
- minimax-m1-40k: creator outside the providers the catalogue covers
- minimax-m1-80k: creator outside the providers the catalogue covers
- minimax-m2-1: creator outside the providers the catalogue covers
- minimax-m2-5: creator outside the providers the catalogue covers
- minimax-m2-7: creator outside the providers the catalogue covers
- mistral-7b-instruct: released before 2025-07-01
- mistral-8x22b-instruct: released before 2025-07-01
- mistral-large: released before 2025-07-01
- mistral-large-2: released before 2025-07-01
- mistral-medium: released before 2025-07-01
- mistral-saba: released before 2025-07-01
- mistral-small: released before 2025-07-01
- mistral-small-2402: released before 2025-07-01
- mistral-small-3: released before 2025-07-01
- mistral-small-3-1: released before 2025-07-01
- mistral-small-3-2: released before 2025-07-01
- mixtral-8x7b-instruct: released before 2025-07-01
- molmo-7b-d: creator outside the providers the catalogue covers
- molmo2-8b: creator outside the providers the catalogue covers
- motif-0714: creator outside the providers the catalogue covers
- motif-2-12-7b: creator outside the providers the catalogue covers
- motif-3: creator outside the providers the catalogue covers
- nanbeige4-1-3b: creator outside the providers the catalogue covers
- nemotron-3-5-lightning: creator outside the providers the catalogue covers
- nemotron-3-nano-omni-30b-a3b: creator outside the providers the catalogue covers
- nemotron-cascade-2-30b-a3b: creator outside the providers the catalogue covers
- nex-n2-pro: creator outside the providers the catalogue covers
- north-mini-code: creator outside the providers the catalogue covers
- nova-2-0-lite: creator outside the providers the catalogue covers
- nova-2-0-lite-reasoning: creator outside the providers the catalogue covers
- nova-2-0-lite-reasoning-low: creator outside the providers the catalogue covers
- nova-2-0-lite-reasoning-medium: creator outside the providers the catalogue covers
- nova-2-0-omni: creator outside the providers the catalogue covers
- nova-2-0-omni-reasoning-low: creator outside the providers the catalogue covers
- nova-2-0-omni-reasoning-medium: creator outside the providers the catalogue covers
- nova-2-0-pro: creator outside the providers the catalogue covers
- nova-2-0-pro-reasoning-low: creator outside the providers the catalogue covers
- nova-2-0-pro-reasoning-medium: creator outside the providers the catalogue covers
- nova-lite: creator outside the providers the catalogue covers
- nova-micro: creator outside the providers the catalogue covers
- nova-premier: creator outside the providers the catalogue covers
- nova-pro: creator outside the providers the catalogue covers
- nvidia-nemotron-3-nano-30b-a3b: creator outside the providers the catalogue covers
- nvidia-nemotron-3-nano-30b-a3b-reasoning: creator outside the providers the catalogue covers
- nvidia-nemotron-3-nano-4b: creator outside the providers the catalogue covers
- nvidia-nemotron-3-super-120b-a12b: creator outside the providers the catalogue covers
- nvidia-nemotron-3-ultra-550b-a55b: creator outside the providers the catalogue covers
- nvidia-nemotron-nano-12b-v2-vl: creator outside the providers the catalogue covers
- nvidia-nemotron-nano-12b-v2-vl-reasoning: creator outside the providers the catalogue covers
- nvidia-nemotron-nano-9b-v2: creator outside the providers the catalogue covers
- nvidia-nemotron-nano-9b-v2-reasoning: creator outside the providers the catalogue covers
- o1: released before 2025-07-01
- o1-mini: released before 2025-07-01
- o1-preview: released before 2025-07-01
- o1-pro: released before 2025-07-01
- o3: released before 2025-07-01
- o3 Mini: released before 2025-07-01
- o3 Mini High: released before 2025-07-01
- o3 Pro: released before 2025-07-01
- o4 Mini: released before 2025-07-01
- olmo-2-32b: creator outside the providers the catalogue covers
- olmo-2-7b: creator outside the providers the catalogue covers
- olmo-3-1-32b-instruct: creator outside the providers the catalogue covers
- olmo-3-1-32b-think: creator outside the providers the catalogue covers
- olmo-3-32b-think: creator outside the providers the catalogue covers
- olmo-3-7b-instruct: creator outside the providers the catalogue covers
- olmo-3-7b-think: creator outside the providers the catalogue covers
- openchat-35: creator outside the providers the catalogue covers
- palm-2: released before 2025-07-01
- phi-3-mini: creator outside the providers the catalogue covers
- phi-4-mini: creator outside the providers the catalogue covers
- phi-4-multimodal: creator outside the providers the catalogue covers
- pixtral-large-2411: released before 2025-07-01
- quasar-438b: creator outside the providers the catalogue covers
- qwen-2-5-max: released before 2025-07-01
- qwen-chat-14b: released before 2025-07-01
- qwen-chat-72b: released before 2025-07-01
- qwen-turbo: released before 2025-07-01
- qwen1.5-110b-chat: released before 2025-07-01
- qwen2-5-72b-instruct: released before 2025-07-01
- qwen2-5-coder-32b-instruct: released before 2025-07-01
- qwen2-5-coder-7b-instruct: released before 2025-07-01
- qwen2-72b-instruct: released before 2025-07-01
- qwen2.5-32b-instruct: released before 2025-07-01
- qwen3-0.6b-instruct: released before 2025-07-01
- qwen3-0.6b-instruct-reasoning: released before 2025-07-01
- qwen3-1.7b-instruct: released before 2025-07-01
- qwen3-1.7b-instruct-reasoning: released before 2025-07-01
- qwen3-14b-instruct: released before 2025-07-01
- qwen3-14b-instruct-reasoning: released before 2025-07-01
- qwen3-235b-a22b-instruct: released before 2025-07-01
- qwen3-235b-a22b-instruct-reasoning: released before 2025-07-01
- qwen3-30b-a3b-instruct: released before 2025-07-01
- qwen3-30b-a3b-instruct-reasoning: released before 2025-07-01
- qwen3-32b-instruct: released before 2025-07-01
- qwen3-32b-instruct-reasoning: released before 2025-07-01
- qwen3-4b-instruct: released before 2025-07-01
- qwen3-4b-instruct-reasoning: released before 2025-07-01
- qwen3-8b-instruct: released before 2025-07-01
- qwen3-8b-instruct-reasoning: released before 2025-07-01
- qwq-32b: released before 2025-07-01
- r1-1776: creator outside the providers the catalogue covers
- reka-flash: creator outside the providers the catalogue covers
- ring-1t: creator outside the providers the catalogue covers
- ring-2-6-1t: creator outside the providers the catalogue covers
- ring-flash-2-0: creator outside the providers the catalogue covers
- sarvam-105b: creator outside the providers the catalogue covers
- sarvam-30b: creator outside the providers the catalogue covers
- sarvam-m-reasoning: creator outside the providers the catalogue covers
- seed-oss-36b-instruct: creator outside the providers the catalogue covers
- solar-mini: creator outside the providers the catalogue covers
- solar-mini4: creator outside the providers the catalogue covers
- solar-open-100b-reasoning: creator outside the providers the catalogue covers
- solar-open2-250b: creator outside the providers the catalogue covers
- solar-pro-2: creator outside the providers the catalogue covers
- solar-pro-2-preview: creator outside the providers the catalogue covers
- solar-pro-2-preview-reasoning: creator outside the providers the catalogue covers
- solar-pro-2-reasoning: creator outside the providers the catalogue covers
- sonar: creator outside the providers the catalogue covers
- sonar-pro: creator outside the providers the catalogue covers
- sonar-reasoning: creator outside the providers the catalogue covers
- sonar-reasoning-pro: creator outside the providers the catalogue covers
- step-3-5-flash: creator outside the providers the catalogue covers
- step-3-5-flash-0202: creator outside the providers the catalogue covers
- step-3-7-flash: creator outside the providers the catalogue covers
- step-3-vl-10b: creator outside the providers the catalogue covers
- step-5: creator outside the providers the catalogue covers
- tiny-aya-global: creator outside the providers the catalogue covers
- tri-21b-think-preview: creator outside the providers the catalogue covers
- tri-21b-think-v0-5: creator outside the providers the catalogue covers
- tulu3-405b: creator outside the providers the catalogue covers
Data sourced from Artificial Analysis. Licence: Artificial Analysis commercial data licence.