Skip to main content
At scale, some Fetch requests meet a transient condition: a page that loads slowly on one attempt, a busy moment, a connection that fails to open. Retry those with growing waits and they go through. Requests that don’t succeed aren’t charged, so a retry costs the attempt that succeeds. Uses: Fetch with Adaptive Stealth Mode (mode=auto), the API error codes and the SDKs’ Python retries, Node.js retries and Go retries

Ask your AI agent

With the Zenrows MCP server connected to Claude, Cursor or any MCP client, paste this prompt. Your agent picks the right Zenrows tools and settings on its own.
The MCP server’s scrape and extract tools send each call once by default. On the local server, set ZENROWS_FETCH_RETRIES to make them retry a 429, 502, 503 or 504, and a connection that failed before the request was sent, before your agent sees the error. See Retries in scrape and extract.
Retries stack. With ZENROWS_FETCH_RETRIES=2, each scrape or extract call can send up to 3 requests, so “retry up to 3 times” (4 calls) can send up to 12 requests for one URL. The same holds for your own retry loop around an SDK client with retries turned on.

Which errors to retry

Retry the errors that can succeed on the next attempt. Fix the ones that return the same answer every time. Every error body is JSON with a code field: log it with the URL so you can group failures by cause. A read timeout on your side does not stop the request at Zenrows. Zenrows may still finish and charge it, and the request keeps its concurrency slot until it ends. A blind retry can pay twice and run into 429. If your dashboard shows no matching request, it never reached Zenrows, and you can send it again. See Cancelled request behavior.

Retry with backoff in your own code

This function retries the retryable statuses with waits of 1, 2 and 4 seconds plus random jitter, switches a fixed configuration to mode=auto after a 422, and stops at once on errors that need a fix. It retries a connection that never reached Zenrows, and raises a read timeout or a dropped connection instead of retrying it.

Example output

A page that loads on the first attempt returns at once:
A target that keeps failing, here https://httpbin.io/status/503, comes back from Zenrows as 422 with code RESP001 on every attempt. The function retries it with growing waits, then reports it:
None of those attempts is charged.

Retry with a Requests session

If you already use a requests.Session, mount urllib3’s Retry on it and every call through the session retries on its own. backoff_jitter needs urllib3 2.x, which current requests installs.
Python (Requests)
This retries with the same parameters each time. Use the function above when you also want the switch to mode=auto on 422.

Use the SDKs’ built-in retries

The SDK clients send each request once by default. Pass a retry count to turn retries on: retries in Python and Node.js, WithMaxRetryCount in Go. The client then retries for you, with a random wait that grows with each retry. Install with pip install zenrows, npm install zenrows or go get github.com/zenrows/zenrows-go-sdk/service/api.
Output: 200 1 credit. When every attempt fails, the client returns the last response, so check its status as usual. All three return Extract’s 422 with code no_data_extracted at once, because it is a final answer. None retries a POST or PUT, any other network error such as a read timeout, or a status from the target site (original_status, allowed_status_codes). They wait the API’s Retry-After in whole seconds, from 0.25 to 30 seconds, and a retry starts only if its wait ends within 90 seconds of the first try. See Python retries, Node.js retries and Go retries for every option. Combine retries with concurrency to run many requests safely, see Run many requests concurrently.

Customize it

Troubleshooting

  • Every retry returns the same 422: the configuration is too light for that site. Switch to mode=auto, or add js_render=true and premium_proxy=true. When content loads late, add wait_for.
  • 429 keeps coming back during retries: retries add to the requests in flight. Lower your pool size and keep the backoff.
  • A 400 comes back without a retry: the request itself needs a fix. Read the code and see API error codes.
  • 499 or timeouts on your side: your client gave up before Zenrows finished. Set the timeout to 180 seconds or more before adding retries.

Resolve common response codes

The fix for each error code you are likely to see.

Run many requests concurrently

Size your pool to your plan and keep 429s away.

Scrape pages protected by anti-bot systems

Let Adaptive Stealth Mode pick the right configuration per page.

Scrape a list of URLs

Let Batch queue, retry and store results for you.