Benchmarks / Open data

Measured by Spring Prompt

Open benchmark data

The public tasks of our own benchmarks and every model's results on them, to download and reuse. For what each benchmark tests and how it is scored, see our benchmarks, explained. The results come from the release the site is serving, 2026-11-19-3608133e8331, built 19 November 2026.

Licence
Creative Commons Attribution 4.0 International (CC BY 4.0)
Credit
Spring Prompt (springprompt.com)
Everything
results.csv · results.json

Downloads

BenchmarkFileRows
BulletBench bulletbench/results.csv Results, CSV 745
bulletbench/results.json Results, JSON 745
CatalogBench catalogbench/results.csv Results, CSV 1185
catalogbench/results.json Results, JSON 1185
catalogbench/tasks.json Public tasks, JSON 56
catalogbench/tasks.csv Public tasks, CSV 56
DeckBench deckbench/results.csv Results, CSV 645
deckbench/results.json Results, JSON 645
deckbench/tasks.json Public tasks, JSON 6
deckbench/tasks.csv Public tasks, CSV 6
ROASBench roasbench/results.csv Results, CSV 342
roasbench/results.json Results, JSON 342
SurveyBench surveybench/results.csv Results, CSV 575
surveybench/results.json Results, JSON 575
surveybench/tasks.json Public tasks, JSON 9
surveybench/tasks.csv Public tasks, CSV 9

A results row is one measure for one model configuration: the value, its 95% range where there is one, the date, and the sample. Costs are at each run's prices. Each task file holds the full public task as the models saw it, with what it was scored against. Task CSVs carry each task as JSON in one column.

Spring Prompt Score

The Spring Prompt Score for each of the 35 models it ranks, with its percentile on each benchmark, how many benchmarks it covers and the release: score.csv · score.json. Each of our benchmarks becomes a percentile from 0 to 100 among the models with one of our commerce benchmarks (CatalogBench or ROASBench), each model at its best setting, on the headline measure the overall leaderboard uses. The score is the plain average of the percentiles a model has; it needs at least 3. No third-party results are used. Values are to one decimal; the site shows whole numbers.

Using it

You may share and adapt the data for any purpose, including commercially, if you credit Spring Prompt (springprompt.com) and link to this page. Please say which release you used: results change as new runs arrive.

The public tasks are public so you can see exactly what was tested. If you train on them, results on them stop meaning much; our private tasks are held back for that reason.

What is not included

  • Third-party benchmark results. Theirs to publish, under their own terms; see each benchmark page for the source and licence.
  • Private and development tasks. Held back so the benchmarks stay honest; they never appear on the site.
  • Model outputs. Shown on each benchmark page; not in the download yet.
  • CatalogBench product images. Generated for the benchmark; not in the download yet.
  • Google product categories. CatalogBench's category answers come from Google's product taxonomy, whose terms we have not checked for republishing.
  • ROASBench and BulletBench tasks. Their tasks are a market simulator and chess games against Stockfish, run by code rather than stored as task files; their results are included.
  • Where's Waldo study images. Third-party images; the study is not a catalogue benchmark.

Score badges

Every model page has a badge for each of our benchmarks it was ranked on, with the code to embed it: /badge/<benchmark>/<model>.svg. It shows the model's rank and score from the current release and links to the benchmark page. A model with a Spring Prompt Score also has /badge/score/<model>.svg.