AI model benchmarks
What each benchmark measures, who measured it, and how today's models do. For one score across all of them, see the overall leaderboard; for launches and pace, models over time.
Measured by Spring Prompt
What each of our own benchmarks tests, how it is scored and what it leaves out: our benchmarks, explained.
- Speed under time pressure BulletBench When thinking time comes off the clock, which models are quick enough to still make good decisions? Leader: Gemini 3.5 Flash-Lite 89245 modelsLast updated 19 Nov 2026
- Product listings CatalogBench Can a model turn a product feed and photos into a listing that is ready to publish, without making things up? Leader: Gemini 4 Neon 78.6%35 modelsLast updated 19 Nov 2026
- Slide decks DeckBench Can a model turn an analysis into a deck you could present without fixing it? Leader: Qwen4-Max 1,56336 modelsLast updated 19 Nov 2026
- Ad budgets ROASBench Given a year of ad spend decisions, which models grow revenue and which ones burn the budget? Leader: Claude Fable 5.2 61.336 modelsLast updated 18 Nov 2026
- Survey research SurveyBench Can a model plan a user survey and read its results without being misled? Leader: Claude Fable 5.2 1,67027 modelsLast updated 19 Nov 2026
One number from these benchmarks: the Spring Prompt Score, each model's average percentile across them, with equal weights.
Which model for the job
From our own benchmarks: the top result, and the cheapest model that gets at least 70% of its score. Costs are at the run date's prices.
- Product listings (CatalogBench) Best: Gemini 4 Neon, 78.6% at $652.78 per 10,000 products. Best value: GPT-6 Luna, 55.4% at $4.77 per 10,000 products.
- Slide decks (DeckBench) Best: Qwen4-Max, 1,563 at $0.18 a deck. Best value: GPT-6 Luna, 1,252 at $0.0087 a deck.
- Survey research (SurveyBench) Best: Claude Fable 5.2, 1,670 at $1.51 a task. Best value: Claude Haiku 5.5, 1,233 at $0.0185 a task.
- Ad budgets (ROASBench) Best: Claude Fable 5.2, 61.3 at $1.76 a simulated year. Best value: GPT-6 Luna, 53.4 at $0.0142 a simulated year.
- Speed under time pressure (BulletBench) Best: Gemini 3.5 Flash-Lite, 892 at $0.0070 a game. Best value: Gemini Decision 1, 755 at $0.0018 a game.
Reported by others
- Professional tasks APEX-Agents Share of investment banking, consulting and corporate law tasks an agent completes in a simulated workplace with files and apps, graded against expert criteria (one attempt). Leader: Claude Sonnet 5.5 (max reasoning) 75.5%37 modelsLast updated 1 Oct 2026
- People's preference Arena (formerly LMArena) Head-to-head human preference across all text prompts. Leader: Gemini 4 Argon (high reasoning) 1,525387 modelsLast updated 2 Oct 2026
- General intelligence Artificial Analysis Share of expert-written exam questions answered correctly, as run by Artificial Analysis with its own harness and prompts. Leader: Claude Opus 5.5 (max reasoning) 61.4%250 modelsLast updated 9 Nov 2026
- Tool and API calls Berkeley Function Calling Leaderboard (BFCL) V4 BFCL's own weighted average across its test categories. Leader: Claude Opus 4.5 77.5%81 modelsData older than 90 days: last updated 16 Dec 2025
- Document work GDP.pdf Rubric score on 100 real professional tasks that need reasoning over PDFs (finance, legal, insurance, engineering, HR and more). Leader: GPT-5.6 Sol (max reasoning) 30.7%32 modelsLast updated 1 Oct 2026
- Customer service agents Microsoft STATE-Bench Share of tasks completed correctly, averaged over repeated runs. Leader: GPT-5.4 (high reasoning) 55.7%4 modelsData older than 90 days: last updated 1 Jul 2026
- Coding agents OpenHands Index The average of five coding-agent tasks, weighted equally: resolving GitHub issues, front-end work, building a project from scratch, writing tests and gathering information. Leader: Claude Fable 5 81.0%34 modelsData older than 90 days: last updated 30 Jun 2026
- Freelance projects Remote Labor Index Share of 240 real paid freelance projects (design, software, data, architecture and more) an agent delivers at a standard a client would accept. Leader: GPT-6 Astra 20.8%13 modelsLast updated 1 Oct 2026
- Factual knowledge SimpleQA Verified (Epoch AI) Share of short factual questions answered correctly without search, as run by Epoch AI. Leader: GPT-6 Astra (max reasoning) 75.6%76 modelsLast updated 29 Sep 2026
- Customer service agents tau2-bench Share of banking questions answered from a policy knowledge base tasks completed within policy, averaged over trials. Leader: Qwen3.8-Max (0902, extra-high reasoning) 55.2%21 modelsData older than 90 days: last updated 3 Aug 2026
- Writing UGI Leaderboard UGI's blend of intelligence, style, repetition and length adherence in writing, tuned to average human preference. Leader: Gemini 3.8 Flash (medium reasoning) 78.6230 modelsLast updated 2 Oct 2026
- Hallucination Vectara Hallucination Leaderboard Share of document summaries that contain something the document does not support. Leader: Finix-S1-32B 1.8%104 modelsLast updated 22 Sep 2026
- Running a business Vending-Bench 2 Bank balance after an agent runs a simulated vending business for a year: ordering stock, setting prices and dealing with suppliers. Leader: GPT-6 Astra $15,51558 modelsLast updated 1 Oct 2026
Do general leaderboards predict our results?
How closely each general leaderboard's order matches ours, on the models both measured (each at its best setting). 1 is the same order, 0 no relation. With this few models in common, only large differences mean much.
| Our benchmark | Compared with | Models in both | Rank correlation | Reading |
|---|---|---|---|---|
| CatalogBench | Artificial Analysis | n = 25 | +0.62 | moderately in step |
| DeckBench | Artificial Analysis | n = 25 | +0.65 | moderately in step |
| SurveyBench | Artificial Analysis | n = 16 | +0.46 | moderately in step |
| ROASBench | Artificial Analysis | n = 25 | +0.65 | moderately in step |
| BulletBench | Artificial Analysis | n = 14 | −0.16 | barely related |
| CatalogBench | Arena (formerly LMArena) | n = 19 | +0.37 | weakly in step |
| DeckBench | Arena (formerly LMArena) | n = 18 | +0.22 | weakly in step |
| SurveyBench | Arena (formerly LMArena) | n = 9 | +0.58 | moderately in step |
| ROASBench | Arena (formerly LMArena) | n = 18 | +0.33 | weakly in step |
| BulletBench | Arena (formerly LMArena) | n = 11 | +0.08 | barely related |
Quick answers
Which benchmarks does Spring Prompt run itself?
BulletBench, CatalogBench, DeckBench, ROASBench and SurveyBench. We design and run these on business work, such as product listings, decks and ad budgets, and publish every model's output with the scores.
Where do the other results come from?
13 leaderboards run by other teams, including APEX-Agents, Arena (formerly LMArena) and Artificial Analysis. Each page links to the original and shows the dates of its results.
Which AI benchmark should I look at for my task?
Pick the one closest to the work: for customer support agents, tau2-bench and Microsoft STATE-Bench; for agents doing office and professional work, APEX-Agents and Remote Labor Index; for coding agents, OpenHands Index; for tool and function calling, Berkeley Function Calling Leaderboard (BFCL) V4; for made-up facts in summaries, Vectara Hallucination Leaderboard. For one score across all of them, see the overall leaderboard.