$1 Budget? Start With 2 Parallel API Requests

With a $1 test budget, start with two requests running at once. That is a conservative starting point, not a universal limit. It gives you enough parallelism to test a small workflow while keeping mistakes easier to catch. Increase it only after you know what one completed task costs.

Concurrency means the number of requests currently in flight. If you have 100 prompts and a concurrency limit of two, send two, then start another whenever one finishes. Keep the remaining prompts in a queue instead of sending all 100 immediately.

01 / Two parallel requests control your spending speed

Parallelism mainly changes how quickly work finishes and charges accumulate. If every request uses identical tokens at identical rates, running 100 requests sequentially or in parallel has the same token cost. In practice, retries, changing output lengths, and abandoned work can change that total.

Suppose a request takes eight seconds. One active request could complete roughly 7.5 requests per minute; two could complete roughly 15. These are arithmetic examples, not measured performance. Actual results depend on the model, response length, network conditions, and service limits.

The budget problem is reaction time. If a prompt accidentally generates a long answer, two active requests leave fewer expensive responses underway than 20 active requests when you notice.

02 / Turn a $1 allowance into a task budget

Before adjusting concurrency, estimate the cost per completed task. Include input tokens, output tokens, any billed cache reads or writes, and retries. If one task makes three API calls, count all three.

For illustration, assume your measured average is $0.002 per task. This is a hypothetical cost, not a CheapRelay price quote. Under that assumption:

• ■ 100 tasks cost about $0.20 before unexpected extra work.
• ■ A $1 budget covers about 500 tasks at the same average cost.
• ■ Reserving $0.20 leaves $0.80, or roughly 400 planned tasks.

Your average is an estimate, not a ceiling. Use a supported output-token limit to constrain individual responses, then track actual usage. A short prompt can still produce a costly answer if you allow a large output.

03 / Use one queue and two workers

A beginner implementation needs a queue, two workers, and a spending counter. Each worker takes one task, sends its request, records usage, and then takes the next task. Two workers means at most two active requests across that queue.

Keep the queue bounded, too. A queue containing thousands of tasks can quietly keep spending long after you stop watching. For an initial experiment, load a batch of 20 tasks and inspect the results before adding more.

Stop launching new work before your budget is exhausted. Completed-request totals exclude charges still accumulating in active requests. Leave room for those requests to finish, and remember that cancelling a request does not guarantee that already processed tokens are free.

04 / Treat rate limits as separate constraints

A concurrency limit of two does not guarantee compliance with requests-per-minute or tokens-per-minute limits. Two fast requests can still produce many starts per minute; two large requests can consume substantial token capacity.

If you receive a 429 response, follow any Retry-After guidance. Otherwise, use increasing delays with a little random variation. Cap retries—for example, at two additional attempts—and make every attempt pass through the same concurrency control. Avoid retrying authentication errors or invalid requests without fixing them.

05 / Raise the limit only when the small batch works

After your first batch, compare successful tasks, total charges, retry counts, and completion time. If costs fit your allowance and limits permit it, try four workers on another small batch. Keep two if that already meets your deadline.

CheapRelay offers pay-as-you-go AI API access with no monthly fee or subscription. Billing follows actual tokens, including cache reads and writes, so account for those categories when estimating your workload. Sign up for CheapRelay, set a small test allowance, and measure your own cost per completed task before increasing concurrency.