A retry policy that knows when to stop

Separate temporary failures

The DeepSeek error table associates 429 with rate or concurrency limits and 503 with overload. In contrast, 400 and 422 require a corrected request, 401 requires a valid credential and 402 requires sufficient balance. Repeating the same payload does not resolve those conditions.

Maintain an explicit list of temporary errors and check which retries the SDK already performs. If the SDK makes three attempts and your wrapper makes four, one operation may produce twelve requests. Put the policy in an observable place, with counters that reveal the actual behavior rather than just the configuration you intended.

Use three limits together

Set a maximum attempt count, a total time budget and a concurrency limit. Attempts bound extra traffic; the time budget protects the user’s wait; concurrency prevents a crowd of queued requests from restarting together. A timeout for each individual attempt does not limit the duration of the whole operation.

Exponential backoff widens the delay after failures. Jitter randomizes the wait within a range, spreading restarts. If the server sends Retry-After, treat it as a minimum delay for that attempt. When it will not fit inside the remaining budget, stop rather than retrying earlier than the server requested.

A server-side wrapper

This Node example uses fetch, AbortSignal.timeout and AbortSignal.any. It targets non-streaming responses and does not retry ambiguous network failures. The body must be a reusable JSON string, not an already-consumed stream.

import { setTimeout as delay } from 'node:timers/promises';

function retryAfterMs(header) {
  if (!header) return 0;
  const seconds = Number(header);
  if (Number.isFinite(seconds)) return Math.max(0, seconds * 1000);
  const date = Date.parse(header);
  return Number.isNaN(date) ? 0 : Math.max(0, date - Date.now());
}

export async function fetchWithRetry(url, init, {
  signal, attempts = 4, budgetMs = 15000
} = {}) {
  const deadline = performance.now() + budgetMs;
  for (let attempt = 0; attempt < attempts; attempt++) {
    signal?.throwIfAborted();
    const left = Math.floor(deadline - performance.now());
    if (left <= 0) throw new Error('Retry budget exhausted');
    const timeout = AbortSignal.timeout(left);
    const combined = signal
      ? AbortSignal.any([signal, timeout]) : timeout;
    const response = await fetch(url, { ...init, signal: combined });
    if (![429, 500, 503].includes(response.status)) return response;
    const wait = Math.max(
      retryAfterMs(response.headers.get('retry-after')),
      Math.random() * Math.min(4000, 300 * 2 ** attempt)
    );
    if (attempt + 1 === attempts ||
        wait >= deadline - performance.now()) return response;
    await response.body?.cancel();
    await delay(wait, undefined, { signal });
  }
  throw new Error('No attempts remaining');
}

The caller must check response.ok and consume the response within its associated time signal. For a more elaborate policy, explicitly include body processing in the budget as well. Transport errors propagate because the client cannot know whether the server already processed the request. Adding automatic retries there requires a separate decision about duplicate work.

After the first streamed fragment

Once a streaming answer has started, automatically restarting it can duplicate text and cost. Keep the partial answer and mark the interruption. A Retry button can create a clearly separate generation while leaving the previous text available for inspection.

The same issue appears in agents that use tools. A payment, message or external write may have completed before a timeout. Deduplicating those actions requires identifiers and support from the destination system. Adding an arbitrary key to the model request does not make the entire chain idempotent. Handle side effects at the layer that performs them, where completion can actually be established.

Manage the queue

If 429s rise, reduce simultaneous work instead of adding retries. A bounded queue makes load explicit and allows early rejection of operations that cannot finish in time. Long-running batches can slow down; an interactive application may be better served by a clear error than an extended, unproductive wait.

Record HTTP status, attempt count, accumulated delay and total duration without including prompts or credentials in routine logs. If few retries recover successfully, the policy is increasing cost without improving service. That is a reason to revisit concurrency, timeouts or capacity, rather than raising the attempt limit once again. Recovery rate matters more than how persistent the loop appears.