8 Million Fewer Tokens From One Prompt Trim

01 / A repeated 800-token cut adds up fast

Cut a system prompt from 1,000 tokens to 200, and you remove 800 input tokens each time that prompt is sent. Across 10,000 requests, that is 8 million fewer input tokens. At an illustrative uncached input price of $1 per million tokens, the difference would be $8. That price is a calculation example, not a quoted model rate.

For a tiny project, start with the cost of one request. A long instruction block can accompany every short question, so you may repeatedly pay for text that adds little value. Shortening it is a practical experiment you can run without building new infrastructure.

02 / Count the system prompt before editing it

A system prompt is the standing instruction block your application sends with a request. It might define the assistant’s job, tone, output format, and boundaries. It consumes input tokens alongside the user’s message and any conversation history.

Copy your current system prompt into a separate file and keep the original. Measure its tokens with a tokenizer suitable for your chosen model, if available. Otherwise, compare request usage before and after editing while keeping the user message identical. Token counts vary by model; word count is only a rough guide.

Record three things: system prompt tokens, total input tokens, and output tokens. This prevents a misleading claim later. Removing 80% of the system prompt does not mean removing 80% of the bill.

03 / Replace repeated prose with observable instructions

Look for sentences that repeat the same requirement. “Be concise,” “avoid unnecessary detail,” and “keep answers brief” probably do not all need to appear. Replace vague praise such as “provide an exceptional experience” with a behavior you can actually check.

Consider this padded instruction: “You are a friendly and highly knowledgeable support assistant. Always maintain a polite, friendly, approachable tone. Keep answers brief and concise, avoiding unnecessary length. If information is missing, never invent it and instead ask the customer for clarification.”

A shorter version is: “Answer customer support questions politely in up to 120 words. Ask for missing details instead of inventing them.”

The rewrite keeps the useful behaviors and adds a measurable length limit. It also removes “highly knowledgeable,” which supplies no product facts. Treat this as an editing example, not a verified token reduction.

04 / Move occasional details out of every request

A refund policy belongs in requests about refunds. A shipping table belongs in requests about delivery. If your application already knows the topic, include only the relevant material instead of attaching the entire handbook.

Keep shared requirements in the system prompt: the assistant’s role, required output format, important boundaries, and rules for missing information. Put changing customer facts in the appropriate request context. Moving the same text into another message does not itself save tokens; omitting unnecessary text does.

Do not delete essential rules just to hit a token target. Authorization checks and other application controls should also be enforced in code. A shorter prompt is useful only while the application still behaves correctly.

05 / Check ten questions before spending on traffic

Build a small set of ten representative questions. Include an ordinary request, missing information, an unsupported request, and a case requiring your exact output format. Run each against both prompt versions using the same model and settings.

• ■ Check factual accuracy and required formatting.
• ■ Check whether missing information triggers a useful question.
• ■ Compare total input and output tokens, including any retries.
• ■ Restore a specific instruction when its removal causes a failure.

Ten questions will not prove reliability, but they provide an affordable first check. A prompt that saves input tokens while causing longer answers or repeated requests may cost more overall.

06 / Price the reduction using your actual billing

Calculate uncached input savings as removed tokens per request multiplied by request count, divided by one million, then multiplied by the input price per million tokens. If prompt caching applies, use the relevant cache read and write rates instead. Removing cached text may save less than the uncached example suggests.

CheapRelay offers pay-as-you-go access with no monthly fee or subscription, and bills real token usage, including cache reads and writes. Sign up for CheapRelay to compare your original and trimmed prompts with a small test budget. Keep the version that lowers measured cost while passing your checks, then review it again when your application’s requirements change.