AI model benchmarks

What each benchmark measures, who measured it, and how today's models do. For one score across all of them, see the overall leaderboard; for launches and pace, models over time.

Measured by Spring Prompt

What each of our own benchmarks tests, how it is scored and what it leaves out: our benchmarks, explained.

One number from these benchmarks: the Spring Prompt Score, each model's average percentile across them, with equal weights.

Which model for the job

From our own benchmarks: the top result, and the cheapest model that gets at least 70% of its score. Costs are at the run date's prices.

Reported by others

Do general leaderboards predict our results?

How closely each general leaderboard's order matches ours, on the models both measured (each at its best setting). 1 is the same order, 0 no relation. With this few models in common, only large differences mean much.

Our benchmarkCompared withModels in bothRank correlationReading
CatalogBenchArtificial Analysisn = 25+0.62moderately in step
DeckBenchArtificial Analysisn = 25+0.65moderately in step
SurveyBenchArtificial Analysisn = 16+0.46moderately in step
ROASBenchArtificial Analysisn = 25+0.65moderately in step
BulletBenchArtificial Analysisn = 14−0.16barely related
CatalogBenchArena (formerly LMArena)n = 19+0.37weakly in step
DeckBenchArena (formerly LMArena)n = 18+0.22weakly in step
SurveyBenchArena (formerly LMArena)n = 9+0.58moderately in step
ROASBenchArena (formerly LMArena)n = 18+0.33weakly in step
BulletBenchArena (formerly LMArena)n = 11+0.08barely related

Quick answers

Which benchmarks does Spring Prompt run itself?

BulletBench, CatalogBench, DeckBench, ROASBench and SurveyBench. We design and run these on business work, such as product listings, decks and ad budgets, and publish every model's output with the scores.

Where do the other results come from?

13 leaderboards run by other teams, including APEX-Agents, Arena (formerly LMArena) and Artificial Analysis. Each page links to the original and shows the dates of its results.

Which AI benchmark should I look at for my task?

Pick the one closest to the work: for customer support agents, tau2-bench and Microsoft STATE-Bench; for agents doing office and professional work, APEX-Agents and Remote Labor Index; for coding agents, OpenHands Index; for tool and function calling, Berkeley Function Calling Leaderboard (BFCL) V4; for made-up facts in summaries, Vectara Hallucination Leaderboard. For one score across all of them, see the overall leaderboard.