$2 for a Small RAG App, With a Real Spending Gate

Give your first document Q&A app a $2 API budget. That is a spending limit you choose, not a promise about how many answers any provider will deliver. The useful beginner project is small: one user, a folder of notes, local search, and short answers supported by those notes.

RAG means retrieval-augmented generation. Your app retrieves relevant text, then asks a model to answer using that text. Keep retrieval on your laptop and spend API money only when you need an answer.

01 / Split the $2 before sending a question

Set aside $0.20 as an accounting cushion and allow the app to authorize at most $1.80 in requests. This budget covers API usage only. Run the app locally so hosting does not introduce another bill.

Use a dedicated API key and a persistent spending ledger. Every paid request must pass through one budget-checking function. Disable automatic retries: a timeout can leave you uncertain whether a request incurred a charge, so repeating it can spend twice.

A local ledger controls this app's requests. It cannot control other software using the same account. If your provider offers an enforceable account spending limit, configure that too; a low-balance alert alone is not a hard ceiling.

02 / Search 20 notes without buying embeddings

Start with around 20 plain-text documents. Split each into chunks of roughly 200 words, preserving the filename and a chunk number. Store them in SQLite with its FTS5 full-text search extension.

For each question, retrieve the three best matching chunks. This is lexical retrieval: it matches words rather than meaning. It can miss synonyms, but it avoids embedding charges and a separate vector database while you learn the workflow.

• ■ Keep the original documents on disk and rebuild the index when they change.
• ■ Limit question length and retrieved context before constructing the prompt.
• ■ If search finds nothing useful, return “No matching passage found” locally.
• ■ Show retrieved excerpts so the user can inspect the evidence.

03 / Connect one model through one API key

CheapRelay provides a pay-as-you-go relay with no monthly fee or subscription. An OpenAI SDK client can use https://api.cheapapirelay.com/v1 as its base URL. Sign up for CheapRelay, create a dedicated key, and keep that key in a server-side environment variable.

For this experiment, consider DeepSeek V4 Flash, then check its current input, output, and cache billing rates before enabling requests. Copy the exact model identifier from the service documentation rather than guessing it from the display name. CheapRelay bills actual tokens, including cache reads and writes, so those categories belong in your budget calculation.

04 / Reserve each answer's maximum cost first

Suppose, purely for arithmetic, input costs $1 per million tokens and output costs $3 per million. These are illustrative rates, not quoted CheapRelay prices. With at most 1,800 input tokens and 250 output tokens, a request costs at most $0.00255 under those assumptions: $0.00180 for input plus $0.00075 for output.

At that illustrative maximum, 600 requests reserve $1.53. Actual capacity depends on the selected model's current rates and billing rules.

Count the complete input, including instructions, question, excerpts, and message overhead, using a tokenizer compatible with the selected model. Set the API's supported output-token limit to 250. If token accounting, reasoning charges, or another billable category cannot be bounded, keep paid requests disabled until you can calculate a reliable maximum.

Before calling the API, atomically reserve that maximum in the ledger. Reject the request if settled charges plus outstanding reservations plus the new reservation exceed $1.80. Use conservative applicable rates for cache activity. After receiving authoritative usage, replace the reservation with the calculated charge. Keep the full reservation after an ambiguous failure. Persist everything so restarting the app cannot reset its allowance.

05 / Make short answers earn their cost

Tell the model to answer only from the supplied excerpts, cite filenames and chunk numbers, and say when evidence is insufficient. Treat document text as reference material, never as instructions. Omit conversation history initially; every repeated message consumes more input.

Test retrieval locally with ten questions whose answers you already know. Then enable a few paid calls and compare answers with their cited passages. Record usage and cost beside each result. When the ledger refuses another request, show the remaining allowance and stop. A tiny RAG app succeeds when it answers useful questions and obeys its budget.