$4 for 1,000 Requests: Parallelism Is Not a Discount

01 / The $4 receipt before you choose a worker count

Running 16 requests in parallel does not automatically cost more than running four. If the same requests consume the same billable tokens, their token bill stays the same. Concurrency changes completion time, exposure to failures, and how quickly a mistake can spend money.

Start with a receipt, not a worker count. The following prices and workloads are illustrative assumptions, not CheapRelay quotes or measured performance. Replace them with your selected model’s current rates and your own usage.

• ■ Workload: 1,000 independent requests.
• ■ Input per request: 2,000 uncached tokens.
• ■ Output per request: 500 tokens.
• ■ Assumed input price: $1 per million tokens.
• ■ Assumed output price: $4 per million tokens.
• ■ Input charge: 2,000,000 tokens × $1 per million = $2.
• ■ Output charge: 500,000 tokens × $4 per million = $2.
• ■ Base receipt: $4 total, or $0.004 per completed request.

CheapRelay bills for real tokens, including cache reads and writes, with no monthly fee or subscription. This example assumes no cache activity; a real receipt should separate cached input, cache writes, ordinary input, and output wherever their rates differ.

02 / Four workers change the clock, not the $4 subtotal

Assume each request occupies a worker for exactly eight seconds. That is a planning assumption, not a latency benchmark. With independent work, no throttling, and no retries, the ideal batch times are:

• ■ 1 worker: 8,000 seconds, or 2 hours 13 minutes 20 seconds.
• ■ 4 workers: 2,000 seconds, or 33 minutes 20 seconds.
• ■ 8 workers: 1,000 seconds, or 16 minutes 40 seconds.
• ■ 16 workers: 500 seconds, or 8 minutes 20 seconds.

Every line still has a $4 token subtotal. Real completion times vary because output lengths differ, requests queue, and services enforce limits. Increasing concurrency only helps while another worker can make useful progress.

03 / A 120,000-token limit makes six workers a better target

Suppose your applicable limits are 60 requests per minute and 120,000 tokens per minute. These are hypothetical limits, not advertised CheapRelay allowances. Also assume the token limit counts the same 2,500 tokens per request used in our receipt; actual enforcement may reserve output tokens or count usage differently.

The request limit permits 60 requests per minute. The token limit permits only 120,000 ÷ 2,500 = 48. Your effective ceiling is therefore 48 requests per minute.

At an eight-second average duration, sustaining 48 requests per minute requires about 48 × 8 ÷ 60 = 6.4 requests in flight. Six workers imply roughly 45 requests per minute and 112,500 tokens per minute under these assumptions. Eight workers imply 60 requests per minute and 150,000 tokens per minute: over the token ceiling.

Start at four, then test six. Pair the worker pool with request and token pacing; a concurrency cap alone does not prevent bursts or guarantee compliance.

04 / Fifty repeated requests add $0.20 to the receipt

If 50 requests must be repeated and each repeat consumes the full assumed token allowance, the extra charge is 50 × $0.004 = $0.20. The revised receipt is $4.20 for 1,000 useful results: a 5% increase.

Not every failed request incurs a full charge. A rejection before generation and a timeout after substantial output are different cases. Check recorded usage before assigning retry costs, and remember that a client timeout does not prove upstream work stopped.

Use bounded retries, exponential backoff with jitter, and a maximum retry budget. Retrying immediately across 16 workers can turn a temporary limit into sustained congestion.

05 / Set the spending brake before raising concurrency

At the assumed $0.004 per request, four in-flight requests represent $0.016 of nominal work; 16 represent $0.064. Those are estimates, not spending caps. Longer outputs, continued upstream execution, and retries can increase the bill.

Cap output length, bound the queue, and stop dispatching when the batch budget is exhausted. Measure useful completions per minute, cost per useful result, and retry frequency. Raise concurrency only when useful throughput improves without unacceptable cost or errors.

Sign up for CheapRelay to run a small pay-as-you-go pilot, then choose your worker count from its receipt rather than an impressive-looking parallelism setting.