Unit Test Models: Read the Receipt Before You Pick

01 / Price one test batch, not one million tokens

The cheapest model for unit test generation is not necessarily the one with the lowest output rate. It is the one that produces useful tests with the smallest combined bill for tokens, retries, and developer attention.

Start with a specific purchasing unit: one test batch for one function or small module. Include the implementation, relevant interfaces, existing test conventions, and the generated tests. Otherwise, a cheap-looking request can become an expensive conversation.

The receipt below uses illustrative rates and workloads, not verified CheapRelay prices or measured model performance. Replace these assumptions with current rates and your own token logs before making a purchasing decision.

02 / A 1,000-batch receipt starts at $12

Assume your first-pass candidate costs $0.50 per million input tokens and $2.00 per million output tokens. Each batch consumes 12,000 input tokens and 3,000 output tokens. Assume uncached requests throughout this example.

• ■ Input per batch: 12,000 ÷ 1,000,000 × $0.50 = $0.006.
• ■ Output per batch: 3,000 ÷ 1,000,000 × $2.00 = $0.006.
• ■ First-pass total: $0.012 per batch.
• ■ Monthly workload: 1,000 batches × $0.012 = $12.00.

That is the attractive headline. It is not the complete receipt. A generated file may fail to compile, mock the wrong boundary, or assert behavior the implementation never promised. Budget for handling those failures rather than assuming every response is finished work.

03 / Escalating 200 difficult batches adds $20.40

Now assume 200 batches need a second model. This 20% escalation rate is a planning assumption, not a benchmark. Give each escalation the original context plus a concise failure report: 14,000 input tokens and 4,000 output tokens.

At illustrative escalation rates of $3.00 per million input tokens and $15.00 per million output tokens, the additional receipt looks like this:

• ■ Escalation input: 14,000 ÷ 1,000,000 × $3.00 = $0.042.
• ■ Escalation output: 4,000 ÷ 1,000,000 × $15.00 = $0.060.
• ■ Escalation total: 200 × $0.102 = $20.40.
• ■ Combined token bill: $12.00 + $20.40 = $32.40.

You have bought 1,200 generation attempts for 1,000 requested batches. The cost is $0.0324 per requested batch, not per accepted batch. If only 900 batches meet your acceptance criteria, the token cost per accepted batch becomes $0.036.

04 / Developer review can erase $69.60 in token savings

For comparison, sending all 1,000 batches through the escalation path at that same assumed workload would cost $102.00. The two-stage route saves $69.60 in tokens under these assumptions. That is a scenario difference, not a provider discount.

At an assumed loaded developer cost of $60 per hour, $69.60 buys 69.6 minutes. If the cheaper route creates 70 additional minutes of review and repair, its token savings disappear. Count incremental human work, not review both routes require.

05 / Choose candidates with the same acceptance checklist

You might shortlist Gemini 2.5 Flash and Claude Haiku 4.5, but do not assign either the illustrative prices or success rates above. Run both against the same repository tasks and record their actual usage.

• ■ Does the generated test compile and run?
• ■ Does it assert meaningful behavior rather than merely execute code?
• ■ Does it fail when you deliberately introduce a relevant bug?
• ■ How many repair attempts and review minutes does it require?

Keep straightforward cases and difficult dependency-heavy cases separate. Select the default model using accepted-test cost, then escalate failures with a bounded retry budget.

06 / Keep the relay bill separate from the quality score

CheapRelay offers pay-as-you-go access through one key, without a monthly fee or subscription. Billing covers real token usage, including cache reads and writes. If you enable caching, itemize those charges separately rather than treating reused context as free.

Sign up for CheapRelay to run a small, capped comparison using your own test tasks. Keep the receipt beside the acceptance results: tokens show what you spent; working, meaningful tests show what you bought.