$0.20 to $0.08: Stop Buying the Same AI Answer

If 100 AI requests cost $0.002 each, the bill is $0.20. If 60 requests repeat answers you already saved, you send only 40 requests and spend $0.08. That saves $0.12, or 60%, before local storage costs. These are example figures, not a model price or a promised hit rate.

The useful part is the mechanism: when your application can reuse a saved answer, it does not need to make another API call. On a tiny budget, start with storage you already have.

01 / Save finished answers before buying another service

An answer cache is a small lookup table. Your application checks it before contacting the AI API. A matching, unexpired entry supplies the saved answer. A missing entry means you call the API, save the successful result, and return it.

This differs from provider prompt caching, which can reduce the cost of processing repeated input while still making a request. CheapRelay bills real tokens, including cache reads and writes. A local answer hit avoids the API request altogether, so that lookup creates no relay token charge.

Start with repeated, stable work: explaining a fixed error message, classifying the same product description, or rewriting an unchanged template. Live prices, account balances, and answers requiring fresh research are poor first candidates.

02 / Build the first version with one SQLite file

If your application uses Python, its standard library includes SQLite and SHA-256 hashing. You can keep a small cache in one database file without renting a database server. Your existing machine still has storage and operating costs, but you avoid adding a separate service.

• ■ Create a table named answer_cache with three fields: cache_key as a text primary key, answer as text, and expires_at as an integer timestamp.
• ■ Before each API request, create a stable representation of everything that could change the answer, then hash it to produce cache_key.
• ■ Look up that key. If the entry exists and expires_at is later than the current time, return its answer.
• ■ Otherwise, send the request. After a complete, successful response, insert or replace the entry with its answer and expiry time.

Keep your existing API configuration. With CheapRelay, OpenAI-compatible requests use https://api.cheapapirelay.com/v1. Put the cache lookup immediately before the function that sends those requests. If you need API access, sign up for CheapRelay and use its pay-as-you-go billing with no monthly fee or subscription.

03 / Make the key describe the whole request

Using only the latest question as the key is a common mistake. “Explain this simply” means different things after different conversations. Build the key from the model identifier, complete message history, system instructions, generation settings, and a version number for your application’s prompt.

For example, if you use Gemini 2.5 Flash, include that identifier so a later model change produces a different key. Serialize the request fields consistently, with sorted object keys, before hashing. Keep message order intact.

Include source text, document versions, and tool definitions when they affect the answer. Keep your API secret out of the key material. For private or personalized answers, add a user or tenant identifier to prevent reuse across accounts. A hash identifies a request; it does not encrypt the saved answer.

04 / Start with a one-day expiry and count real hits

A 24-hour expiry is a reasonable trial for stable text tasks, not a universal rule. Choose shorter expiry periods when information changes quickly, and bypass caching when freshness is essential. Periodically delete expired rows so the file stays small.

Save only complete, successful answers. Skip errors, interrupted streams, and responses that trigger actions such as sending messages or changing records. Replaying text cannot safely reproduce those actions. Also remember that reusing an answer removes the variation a fresh generation might provide.

Track three counters: total requests, local hits, and API calls. After 100 requests, 60 hits would reproduce the example’s 60% reduction in calls. Dollar savings depend on the actual token charges of the requests avoided. Compare measured spending with local storage costs, then keep the cache only where the savings justify it.