Small Routers, Big Answers: A $652.80 AI Receipt

01 / The $810 bill before adding a router

A small routing model should make a cheap decision, not write an expensive answer twice. Its job is to identify requests that need a capable answer model and separate them from requests your application can handle without generation.

Start with 100,000 requests in a planning period. Without routing, every request reaches the answer model, averaging 1,200 input tokens and 300 output tokens. For this worked example, assume answer-model rates of $3 per million input tokens and $15 per million output tokens.

These are illustrative budget assumptions, not quoted CheapRelay prices or measured model performance. The request mix and token counts are assumptions too. Replace them with current rates and your own logs before making a purchasing decision.

• ■ Answer input: 120 million tokens × $3 = $360.
• ■ Answer output: 30 million tokens × $15 = $450.
• ■ Total model bill: $810.
• ■ Cost per 1,000 requests: $8.10.

02 / A $4.80 sorting step before generation

Now place a small model before the answer model. Give it a compact instruction, the relevant request text, and a short list of permitted destinations. Require a tiny structured response containing a destination and, where applicable, an approved response identifier.

Assume the router averages 160 input tokens and 20 output tokens per request. Budget its input at $0.20 per million tokens and output at $0.80 per million. Again, those are hypothetical rates, not a price claim about any named model.

• ■ Router input: 16 million tokens × $0.20 = $3.20.
• ■ Router output: 2 million tokens × $0.80 = $1.60.
• ■ Total routing charge: $4.80.

Suppose 20,000 requests qualify for deterministic handling: an approved help message, a navigation destination, or an application-cached response that passes freshness and authorization checks. The remaining 80,000 requests go to the big answer model. This example assumes the router itself generates no user-facing answers.

03 / The routed receipt totals $652.80

Keep the answer workload unchanged for requests that escalate: 1,200 input tokens and 300 output tokens each. Count any routing metadata passed downstream inside that input budget. Routing does not magically shrink those answers.

• ■ Small-model routing: $4.80.
• ■ Answer input: 96 million tokens × $3 = $288.
• ■ Answer output: 24 million tokens × $15 = $360.
• ■ Total model bill: $652.80.
• ■ Difference versus sending everything: $157.20, or approximately 19.4%.

The resulting cost is $6.528 per 1,000 incoming requests. This receipt excludes hosting, engineering time, retrieval services, retries, and provider token-cache activity. Application caching means reusing an answer without another generation call; it is not the same as discounted token-cache reads.

04 / The router must eliminate at least 593 answer calls

At the assumed rates, one answer call costs $0.0081: $0.0036 for input and $0.0045 for output. Divide the $4.80 routing bill by $0.0081. The router needs to eliminate 593 average answer calls per 100,000 requests to cover its model cost, before other expenses.

If every request still reaches the answer model with the same token counts, the router increases the bill to $814.80. Classification is not automatically optimization. The saving comes from work genuinely avoided, not from adding another model to the diagram.

05 / Keep uncertain requests on the answer path

Use an explicit fallback: uncertain classifications escalate. Validate the router's output against allowed destinations, and enforce permissions in application code. A plausible response identifier must never bypass authorization or freshness checks.

Log routing tokens, downstream tokens, retries, corrected classifications, and user-visible failures. Test representative requests before deployment. A cheap router that sends difficult questions to canned responses can lower token spending while making the product worse.

06 / Put both stages on one auditable bill

CheapRelay offers one key across multiple model families through OpenAI-compatible routes. Its pay-as-you-go billing charges for real tokens, including cache reads and writes, with no monthly fee or subscription. That makes both routing and answering visible costs, rather than treating the first stage as free.

Sign up for CheapRelay, run a capped trial, and replace every assumption in this receipt with observed usage. Keep the router only if avoided work pays for its tokens, operational overhead, and mistakes.