Cut Your AI Bill 66 Percent with 200-Token Routers

01 / The 66 Percent Saving Comes from One Asymmetry

The trick behind small models for routing and big models for answers is not cleverness. It is arithmetic. A big model might cost roughly twenty times a small model per thousand tokens. If you send every prompt to the big model, you pay twenty times for everything, including the questions a fast small model could have closed in one word.

So you split the work. A tiny routing call reads the user prompt, predicts which answerer fits, and returns a single label. Only then does the expensive call fire. The router is maybe 200 tokens. The routed answer is the same 2,400 tokens it always was. You added 200 tokens of overhead and removed thousands of expensive tokens from the majority of requests.

On a tiny budget this matters more than model quality debates. If you have ten dollars to spend this month, every unnecessary expensive token is a request you cannot make later. CheapRelay bills per real token with no monthly fee and no subscription, which is exactly the shape a router needs: you can run a cheap model for pennies of classification and still reach Claude, GPT, Gemini, DeepSeek, Kimi, GLM, Qwen, Grok and more through one key.

The worked example below uses a 20:1 price ratio as an illustration. Replace it with the live prices you see on the site before you budget anything.

02 / A 200-Token Router Prompt That Actually Works

Keep the router prompt brutally short. Long router prompts silently eat the saving you were chasing. A prompt of about 200 tokens can hold a role, four or five route labels, and an instruction to answer with the label only.

• ■ simple: greetings, yes or no checks, formatting, single fact lookups, short rewrites
• ■ code: small functions, regex, shell commands, bug fixes in one file
• ■ reason: multi-step planning, math, trade-off analysis, long documents
• ■ unsafe: requests you would rather not answer at all

The output should be one of those words and nothing else. Do not let the router explain itself. An explanation is fifty extra tokens per request, and at ten thousand requests a month that is half a million tokens of narration you did not need.

Two practical rules keep the router honest. First, make the default the expensive path: if the label is missing or malformed, escalate rather than guess. Second, give the router only the user message, not the whole conversation history. Routing a three-message thread means billing three messages twice.

A 200-token router plus a 10-token label costs about 210 tokens per request. That is your fixed tax. Everything else is savings you earn by keeping simple traffic off the big model.

03 / Receipt Math for 1,000 Requests in a Month

Assume 1,000 requests, each with 2,000 input tokens and 400 output tokens. That is 2,400 tokens per request, or 2.4 million tokens for the month. Assume the small model costs 1 unit per thousand tokens and the big model costs 20 units per thousand. These are placeholders, not published rates.

Without routing: 2.4 million tokens at 20 units per thousand equals 48 units.

With routing, split traffic 70 and 30.

• ■ Router: 1,000 calls at 210 tokens equals 210,000 tokens, about 0.21 units.
• ■ Small answers: 700 requests at 2,400 tokens equals 1.68 million tokens, about 1.68 units.
• ■ Escalated answers: 300 requests at 2,400 tokens equals 720,000 tokens, about 14.4 units.
• ■ Total with routing: about 16.29 units, a saving of about 31.7 units, or 66 percent.

Now be honest about the fragility. If the real price ratio is 5:1 instead of 20:1, the same setup saves closer to half. If your traffic is mostly hard reasoning and 70 percent escalates, the router becomes pure overhead and you are worse off than before. The saving is a property of your traffic mix, not of the architecture.

Log three numbers every week: requests routed small, requests escalated, and tokens billed. Three lines of a notebook beat a dashboard you never open.

04 / Cache Reads and Writes Are Real Tokens on the Bill

Beginners forget caching until the bill arrives. CheapRelay bills per real token including cache reads and writes, so a cached prompt is not a free prompt. It is usually cheaper, not zero.

Take a system prompt of 1,200 tokens plus 800 tokens of fresh user input and 400 tokens of output. On the first request you pay for a cache write of 1,200 tokens. On requests two through one thousand you pay for cache reads of 1,200 tokens each. The token total is unchanged at 2.4 million, but the composition is roughly 1,200 cache-write tokens, 1.199 million cache-read tokens, 800,000 fresh input tokens, and 400,000 output tokens.

Two consequences follow. First, a big system prompt repeated on every router call is a recurring cost, so keep the router prompt small and un-cached or cached once and reused. Second, cache efficiency favors stable prefixes: put instructions first and volatile content last. Reordering a prompt can change the cache hit pattern without changing a single word of meaning.

Also count the router itself in cache math. If the router prompt is identical across calls, it is a natural cache hit. That is the cheapest token you will spend all month.

05 / Wiring Two Models into One OpenAI-Compatible Call

You do not need a framework. The whole router is a conditional around two calls using any OpenAI SDK, pointed at the OpenAI-compatible routes under /v1. Gemini native routes live under /v1beta if you prefer that shape.

• ■ Step one: send the 200-token router prompt to Claude Haiku 4.5 and capture the single-word label.
• ■ Step two: map the label to a target model. Send simple traffic to a small answerer and reserve Claude Opus 5 for reasoning, long context, and anything the user marked as important.
• ■ Step three: record input tokens, output tokens, cache reads, cache writes, and the chosen label for every call.
• ■ Step four: review the log after a week and move labels around based on real failures, not guesses.

Keep the code boring. A dictionary mapping labels to model names, a try and except that escalates on malformed labels, and a plain text log file are enough for the first month. You can move to structured logging later when the traffic justifies it.

One more detail: send the same temperature and max-token settings to both paths where possible. Different defaults across models are the most common reason a routed pipeline feels inconsistent.

06 / Three Cases Where Routing Wastes Money Instead

Routing is a tool, not a virtue. Three situations make it cost more than it saves.

• ■ Short sessions. If a user sends two messages and leaves, the router overhead is a large share of the total. Skip routing below roughly a dozen requests per session.
• ■ Escalation chains. If the big model re-reads the router transcript and the small model draft, you paid for the work twice. Route first, then answer once.
• ■ Highly uniform traffic. If every prompt is hard, the router is a tax with no benefit. Sample ten prompts by hand first and count how many a small model could honestly close.

Run that sample before you build anything. Ten prompts, five minutes, and you will know whether your traffic is 70 percent simple or 20 percent simple. That single number decides the design.

If the sample is favorable, sign up at CheapRelay and try the split on a five dollar month before you commit anything larger. Once it works, the referral program returns 20 percent of what invited users pay, which quietly subsidizes the next month of experiments.

The discipline is simple: small models decide, big models answer, cache reads are counted, and every escalation is justified in the log. Do that and a tiny budget behaves like a much bigger one.