What we found when we tested the job, not the model.
Each experiment takes one common AI task, runs it on at least seven models, rewrites the prompt from what failed, and tests it again on cases the rewrite never saw. These are our own test cases on invented data, not customer results.
Testing your own prompt? You can import a Langfuse or LangSmith dataset, or a Helicone log export, so the cases are real inputs from your product. Test your prompt →
Can a cheap model run a hotel's guest chat?
An assistant answering guests from the hotel's own information, without inventing policy or promising outcomes.
- Best value
- $0.25
- One model, same setting, new prompt
- 11% → 81% 4 of 36 → 29 of 36
Which model should reply to your ecommerce customers?
Support replies from order details and a returns policy, without promising what the policy doesn't cover.
- Best value
- $0.39
- One model, same setting, new prompt
- 0% → 94% 0 of 36 → 34 of 36
Which model writes meeting notes you can trust?
Decisions and action items from a transcript, without turning ideas into decisions or inventing owners.
- Best value
- $0.18
- One model, same setting, new prompt
- 31% → 81% 11 of 36 → 29 of 36
Can you trust a support agent to pick the right action?
An agent choosing between refunds, address changes and handing to a person, without claiming it's done something it hasn't.
- Best value
- $0.06
- One model, same setting, new prompt
- 11% → 92% 4 of 36 → 33 of 36
Can AI write marketing emails without making things up?
Emails from a short brief, without inventing offers, dates or claims the shop never made.
- Best value
- $0.57
- One model, same setting, new prompt
- 0% → 75% 0 of 36 → 27 of 36
Which model should answer your phones?
Short spoken replies for an invented dental clinic: only real slots, one question at a time, no medical advice.
- Best value
- $0.48
- One model, same setting, new prompt
- 0% → 100% 0 of 36 → 36 of 36
Can AI pitch a journalist without inventing anything?
Short pitches from a story and a journalist's beat, without made-up numbers, quotes or past articles.
- Best value
- $0.52
- One model, same setting, new prompt
- 0% → 78% 0 of 36 → 28 of 36
Can AI turn group chat messages into clean order rows?
Orders from messy shop chats, without guessing amounts or missing corrections.
- Best value
- $0.18
- One model, same setting, new prompt
- 0% → 83% 0 of 36 → 30 of 36
Which model turns notes into an honest slide outline?
Outlines from rough notes that stick to the facts, keep the caveats and end on the decision.
- Best value
- $0.87
- One model, same setting, new prompt
- 3% → 67% 1 of 36 → 24 of 36
Want a hand setting it up?
Tell us what your prompt does, and we'll set the test up with you inside your own free trial account: checks and test cases, other models with cost and speed, and a rewrite, the way these experiments were made. Your prompt stays in your account.