Experiments

What we found when we tested the job, not the model.

Each experiment takes one common AI task, runs it on at least seven models, rewrites the prompt from what failed, and tests it again on cases the rewrite never saw. These are our own test cases on invented data, not customer results.

Testing your own prompt? You can import a Langfuse or LangSmith dataset, or a Helicone log export, so the cases are real inputs from your product. Test your prompt →

Experiment 01Tested 4 November 2026

Can a cheap model run a hotel's guest chat?

An assistant answering guests from the hotel's own information, without inventing policy or promising outcomes.

Best value
$0.25
One model, same setting, new prompt
11% → 81%
4 of 36 → 29 of 36
Read the experiment →
Experiment 02Tested 4 November 2026

Which model should reply to your ecommerce customers?

Support replies from order details and a returns policy, without promising what the policy doesn't cover.

Best value
$0.39
One model, same setting, new prompt
0% → 94%
0 of 36 → 34 of 36
Read the experiment →
Experiment 03Tested 4 November 2026

Which model writes meeting notes you can trust?

Decisions and action items from a transcript, without turning ideas into decisions or inventing owners.

Best value
$0.18
One model, same setting, new prompt
31% → 81%
11 of 36 → 29 of 36
Read the experiment →
Experiment 04Tested 4 November 2026

Can you trust a support agent to pick the right action?

An agent choosing between refunds, address changes and handing to a person, without claiming it's done something it hasn't.

Best value
$0.06
One model, same setting, new prompt
11% → 92%
4 of 36 → 33 of 36
Read the experiment →
Experiment 05Tested 4 November 2026

Can AI write marketing emails without making things up?

Emails from a short brief, without inventing offers, dates or claims the shop never made.

Best value
$0.57
One model, same setting, new prompt
0% → 75%
0 of 36 → 27 of 36
Read the experiment →
Experiment 06Tested 4 November 2026

Which model should answer your phones?

Short spoken replies for an invented dental clinic: only real slots, one question at a time, no medical advice.

Best value
$0.48
One model, same setting, new prompt
0% → 100%
0 of 36 → 36 of 36
Read the experiment →
Experiment 07Tested 4 November 2026

Can AI pitch a journalist without inventing anything?

Short pitches from a story and a journalist's beat, without made-up numbers, quotes or past articles.

Best value
$0.52
One model, same setting, new prompt
0% → 78%
0 of 36 → 28 of 36
Read the experiment →
Experiment 08Tested 4 November 2026

Can AI turn group chat messages into clean order rows?

Orders from messy shop chats, without guessing amounts or missing corrections.

Best value
$0.18
One model, same setting, new prompt
0% → 83%
0 of 36 → 30 of 36
Read the experiment →
Experiment 09Tested 4 November 2026

Which model turns notes into an honest slide outline?

Outlines from rough notes that stick to the facts, keep the caveats and end on the decision.

Best value
$0.87
One model, same setting, new prompt
3% → 67%
1 of 36 → 24 of 36
Read the experiment →
Free until 23 November 2026 · No credit card needed

Want a hand setting it up?

Tell us what your prompt does, and we'll set the test up with you inside your own free trial account: checks and test cases, other models with cost and speed, and a rewrite, the way these experiments were made. Your prompt stays in your account.

Ask for a hand Takes a minute. We reply by email.