Four Few-Shot Examples: A $200 Prompt Receipt

Four examples can turn a $128 batch into a $200 batch. Whether that extra $72 is sensible depends on the errors those examples prevent. Counting examples alone tells you little; counting their tokens and measuring their effect gives you a purchasing decision.

Here is a receipt for 100,000 requests, followed by a practical way to decide how many examples deserve space in your prompt.

01 / The $200 receipt starts with explicit assumptions

This calculation uses illustrative rates of $1 per million input tokens and $4 per million output tokens. These are arithmetic assumptions, not a CheapRelay price quote or a claim about any particular model. Substitute the current rates for your chosen model before budgeting.

Assume each request contains 800 input tokens before examples, each complete example adds 180 tokens, and the response uses 120 output tokens. The example allowance includes its input, ideal answer and separators. Assume every request succeeds once, with no caching or retries.

• ■ Base input: 800 tokens × $1 per million = $0.00080 per request.
• ■ Four examples: 720 tokens × $1 per million = $0.00072 per request.
• ■ Output: 120 tokens × $4 per million = $0.00048 per request.
• ■ Total: 1,520 input tokens plus 120 output tokens = $0.00200 per request.
• ■ Batch receipt: 100,000 requests × $0.00200 = $200.

The examples account for $72 of that batch. That is a 56.25% increase over the $128 bill without examples, even though the output length stays unchanged.

02 / Zero, two, four and eight examples on the same bill

Holding everything else constant makes the tradeoff visible. Under those same assumptions, the receipts are:

• ■ Zero examples: 800 input tokens; $0.00128 per request; $128 per 100,000 requests.
• ■ Two examples: 1,160 input tokens; $0.00164 per request; $164 per 100,000 requests.
• ■ Four examples: 1,520 input tokens; $0.00200 per request; $200 per 100,000 requests.
• ■ Eight examples: 2,240 input tokens; $0.00272 per request; $272 per 100,000 requests.

Each additional 180-token example costs $18 across this batch. Two examples that repeat an already clear pattern therefore cost another $36. They need to produce enough useful improvement to justify that charge.

03 / The third and fourth examples need to earn $36

Suppose a failed response requires manual review costing $0.20. Moving from two examples to four adds $36 per 100,000 requests. Avoiding 180 reviews saves exactly $36, so break-even requires a reduction of 0.18 percentage points in the review rate.

That is a threshold, not a measured improvement. If your test shows only 100 fewer reviews, the added examples save $20 in review work while adding $36 in API charges. If they prevent 300 reviews, the $60 review saving exceeds the added charge by $24.

Use your actual failure cost. An automatic retry, a support ticket and a manual correction have different prices. Include any change in output length, too: examples that encourage longer answers can increase both sides of the token bill.

04 / Buy coverage before buying more demonstrations

Start with a small test set drawn from real requests, including awkward cases. Compare zero, two, four and eight examples using the same inputs and grading rules. For structured extraction, score required fields and valid output. For classification, score the chosen label against an agreed answer.

Choose examples that resolve different decisions: a normal case, a boundary case, missing information and a case where the correct response is to abstain. These are candidates to test, not a universal four-example recipe. Keep test cases separate from prompt examples, and check small apparent gains on more data.

05 / Read the billed receipt before expanding the batch

CheapRelay provides pay-as-you-go API access with no monthly fee or subscription. Billing follows real token usage, including cache reads and writes. That makes actual usage records useful for checking whether your estimated example cost matches the charge.

Caching can change this receipt, but reuse alone does not establish a saving: apply the relevant cache rates to recorded usage. Sign up for CheapRelay to run a small comparison, then choose the example count with the lowest combined API and failure cost.