> ## Documentation Index
> Fetch the complete documentation index at: https://docs.zenrows.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> The Zenrows HTTP API serves Fetch and Extract (extract=auto) at https://api.zenrows.com/v1/, Crawl at https://api.zenrows.com/v1/crawls, and Batch at https://async.api.zenrows.com/v1. Fetch, Extract, Crawl and Batch are described in OpenAPI 3.1 at https://www.zenrows.com/openapi.json and listed in the RFC 9727 API catalog at https://www.zenrows.com/.well-known/api-catalog. Every endpoint also has its own page under /api-reference with its schema. Use the spec for exact parameter names and types. Crawl's endpoints, parameters and errors are also at https://docs.zenrows.com/crawl/endpoints.md. To call Zenrows from an agent, connect the MCP server at https://mcp.zenrows.com/mcp. To search these docs from an agent, connect the docs MCP server at https://docs.zenrows.com/mcp. Agent skills for Zenrows are published at https://github.com/ZenRows/cli and install with `npx skills add ZenRows/cli`.

# Prepare any website for Extract

> Add any website to Zenrows Extract in minutes, then get its pages as clean JSON.

Extract learns each website's page shapes once, then returns clean JSON for every page of each page type. To add a new site, prepare it yourself: send 1 to 5 example URLs (we recommend 3 or more of the same page type), poll the status for a few moments, and start extracting. Every Zenrows account can use the domain once it's ready.

**Uses:** the [Extract endpoints](https://docs.zenrows.com/extract/endpoints.md) (`GET /v1/extract/domains`, `POST /v1/extract/prepared-domains`, `GET /v1/extract/prepared-domains/{domain}`) and [Extract](https://docs.zenrows.com/extract/setup.md) (`extract=auto`)

## Ask your AI agent

With the [Zenrows MCP server](https://docs.zenrows.com/mcp/overview.md) connected to Claude, Cursor or any MCP client, paste this prompt. Your agent picks the right Zenrows tools and settings on its own.

```text theme={"theme":"css-variables"}
Use Zenrows to get https://www.allbirds.com/products/womens-tree-runners as structured JSON:
the product name, price and every size with its SKU and availability.
```

When a site is new to Extract, the agent still returns JSON. To prepare the domain itself, run the steps below once; the prompt then gets site-tailored fields on every page of the page types you prepared, for one product or thousands.

## How it works

1. **Check coverage.** `GET /v1/extract/domains` lists every domain Extract serves now. It's free.
2. **Ask for the domain.** `POST /v1/extract/prepared-domains` with the domain and 1 to 5 `example_urls`. One works; we recommend 3 or more of the same page type (e.g. 3 product pages), because Extract compares pages of one type to check the fields it found and fill in fields one page is missing.
3. **Poll the status.** `GET /v1/extract/prepared-domains/{domain}` moves through `received`, `preparing` and `verifying` to `ready`. Preparation usually takes seconds to minutes.
4. **Extract.** Send `extract=auto` to any page of the page types you prepared.

The three Extract endpoints are free: none of them fetches a page.

## Check whether a domain is covered

```python Python (Requests) theme={"theme":"css-variables"}
import requests

response = requests.get(
    "https://api.zenrows.com/v1/extract/domains",
    headers={"X-API-Key": "YOUR_ZENROWS_API_KEY"},
    timeout=30,
)
response.raise_for_status()

covered = {entry["domain"] for entry in response.json()["domains"]}
print(len(covered), "domains;", "allbirds.com" in covered)
```

```javascript Node.js theme={"theme":"css-variables"}
const response = await fetch("https://api.zenrows.com/v1/extract/domains", {
  headers: { "X-API-Key": "YOUR_ZENROWS_API_KEY" },
});
if (!response.ok) throw new Error(`${response.status}: ${await response.text()}`);

const { domains } = await response.json();
const covered = new Set(domains.map((entry) => entry.domain));
console.log(covered.size, "domains;", covered.has("allbirds.com"));
```

```bash cURL theme={"theme":"css-variables"}
curl "https://api.zenrows.com/v1/extract/domains?apikey=YOUR_ZENROWS_API_KEY"
```

```text Output (before preparing) theme={"theme":"css-variables"}
423 domains; False
```

Sending `extract=auto` to a new domain returns `403` with [`REQS007`](https://docs.zenrows.com/api-error-codes#REQS007), free of charge, and the message points to the next step:

```json theme={"theme":"css-variables"}
{
  "code": "REQS007",
  "detail": "Extract has not been prepared for example-shop.com yet. Ask for it with POST /v1/extract/prepared-domains, then retry once it reports ready. GET /v1/extract/domains lists what Extract serves today.",
  "status": 403
}
```

## Prepare the domain and wait until it's ready

Send the domain without a scheme, and 1 to 5 example URLs. One works; we recommend 3 or more of the same page type (e.g. 3 product pages), because Extract compares pages of one type to check the fields it found and fill in fields one page is missing. Several page types can share the 5 (e.g. 3 product pages + 2 search-result pages). This example sends three product pages. Then poll every 30 seconds and branch on `status`: `preparing`, `ready` or `failed`.

```python Python (Requests) theme={"theme":"css-variables"}
import time
import requests

headers = {"X-API-Key": "YOUR_ZENROWS_API_KEY"}
base = "https://api.zenrows.com/v1/extract/prepared-domains"
domain = "allbirds.com"

# 1. Ask for the domain, with three pages of the same page type
response = requests.post(
    base,
    headers=headers,
    json={
        "domain": domain,
        "example_urls": [
            "https://www.allbirds.com/products/mens-tree-runners",
            "https://www.allbirds.com/products/mens-wool-runners",
            "https://www.allbirds.com/products/mens-wool-cruiser",
        ],
    },
    timeout=30,
)
print(response.status_code, response.json()["state"])

# 2. Poll until it is ready
while True:
    body = requests.get(f"{base}/{domain}", headers=headers, timeout=30).json()
    print(body["status"], body["state"])
    if body["status"] != "preparing":
        break
    time.sleep(30)

if body["status"] == "failed":
    print(body["failure"]["code"], body["failure"]["detail"])
```

```javascript Node.js theme={"theme":"css-variables"}
const headers = { "X-API-Key": "YOUR_ZENROWS_API_KEY", "Content-Type": "application/json" };
const base = "https://api.zenrows.com/v1/extract/prepared-domains";
const domain = "allbirds.com";

// 1. Ask for the domain, with three pages of the same page type
const submit = await fetch(base, {
  method: "POST",
  headers,
  body: JSON.stringify({
    domain,
    example_urls: [
      "https://www.allbirds.com/products/mens-tree-runners",
      "https://www.allbirds.com/products/mens-wool-runners",
      "https://www.allbirds.com/products/mens-wool-cruiser",
    ],
  }),
});
console.log(submit.status, (await submit.json()).state);

// 2. Poll until it is ready
let body;
while (true) {
  body = await (await fetch(`${base}/${domain}`, { headers })).json();
  console.log(body.status, body.state);
  if (body.status !== "preparing") break;
  await new Promise((resolve) => setTimeout(resolve, 30_000));
}
if (body.status === "failed") console.log(body.failure.code, body.failure.detail);
```

```bash cURL theme={"theme":"css-variables"}
# 1. Ask for the domain
curl -X POST "https://api.zenrows.com/v1/extract/prepared-domains?apikey=YOUR_ZENROWS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"domain": "allbirds.com", "example_urls": ["https://www.allbirds.com/products/mens-tree-runners", "https://www.allbirds.com/products/mens-wool-runners", "https://www.allbirds.com/products/mens-wool-cruiser"]}'

# 2. Check progress until status is ready
curl "https://api.zenrows.com/v1/extract/prepared-domains/allbirds.com?apikey=YOUR_ZENROWS_API_KEY"
```

The statuses from the test run, which went from submission to `ready` in under a minute:

```text Output theme={"theme":"css-variables"}
202 received
preparing verifying
ready ready
```

The final status record, with one `roles` entry for each example URL you sent:

```json theme={"theme":"css-variables"}
{
  "domain": "allbirds.com",
  "status": "ready",
  "state": "ready",
  "roles": [
    {
      "role": "detail",
      "status": "ready",
      "url": "https://www.allbirds.com/products/mens-tree-runners",
      "cluster_id": "ea44bc9b0c5f",
      "detail": "ready; fields the page evidence could not verify were dropped and will return empty: aggregate_rating.rating_value, product.availability"
    },
    {
      "role": "detail",
      "status": "ready",
      "url": "https://www.allbirds.com/products/mens-wool-runners",
      "cluster_id": "ea44bc9b0c5f",
      "detail": "ready; fields the page evidence could not verify were dropped and will return empty: aggregate_rating.rating_value, product.availability"
    },
    {
      "role": "detail",
      "status": "ready",
      "url": "https://www.allbirds.com/products/mens-wool-cruiser",
      "cluster_id": "ea44bc9b0c5f",
      "detail": "ready; fields the page evidence could not verify were dropped and will return empty: aggregate_rating.rating_value, product.availability"
    }
  ],
  "submitted_at": "2026-10-15T00:49:32.818952Z",
  "updated_at": "2026-10-15T00:49:37.161497Z"
}
```

The `detail` sentence names any field that comes back empty on this site, so you know which fields to read before you build on them. Submitting a domain that's already ready returns `202` with this same record, so a retry is always safe.

## Extract pages from the new domain

Once the status reads `ready`, `extract=auto` works on every page of the prepared page type, not just the example URLs. This request reads a different product than the ones used for preparation:

```python Python (Requests) theme={"theme":"css-variables"}
import json
import requests

params = {
    "apikey": "YOUR_ZENROWS_API_KEY",
    "url": "https://www.allbirds.com/products/womens-tree-runners",
    "mode": "auto",
    "extract": "auto",
}
response = requests.get("https://api.zenrows.com/v1/", params=params, timeout=180)
response.raise_for_status()
print(json.dumps(response.json()["parsed"], indent=2))
```

```javascript Node.js theme={"theme":"css-variables"}
const params = new URLSearchParams({
  apikey: "YOUR_ZENROWS_API_KEY",
  url: "https://www.allbirds.com/products/womens-tree-runners",
  mode: "auto",
  extract: "auto",
});
const response = await fetch(`https://api.zenrows.com/v1/?${params}`, {
  signal: AbortSignal.timeout(180_000),
});
if (!response.ok) throw new Error(`${response.status}: ${await response.text()}`);

const { parsed } = await response.json();
console.log(JSON.stringify(parsed, null, 2));
```

```bash cURL theme={"theme":"css-variables"}
curl "https://api.zenrows.com/v1/?apikey=YOUR_ZENROWS_API_KEY&url=https%3A%2F%2Fwww.allbirds.com%2Fproducts%2Fwomens-tree-runners&mode=auto&extract=auto"
```

```bash CLI theme={"theme":"css-variables"}
zenrows extract https://www.allbirds.com/products/womens-tree-runners --out tree-runner.json
```

## Example output

The `parsed` field for the Women's Tree Runner page, trimmed:

```json theme={"theme":"css-variables"}
{
  "product": {
    "name": "Women's Tree Runner",
    "price": 100,
    "currency": "USD",
    "gender": "womens",
    "material": "tree",
    "best_for": ["Traveling", "Walking", "Commuting", "Everyday"],
    "care_instructions": "Machine Washable",
    "url": "https://www.allbirds.com/products/womens-tree-runners"
  },
  "materials": [
    { "component": "Upper", "material_name": "Tree Knit" },
    { "component": "Midsole", "material_name": "SweetFoam®" }
  ],
  "technical_specs": {
    "country_of_origin": "Vietnam",
    "heel_toe_drop": "9.0mm",
    "weight": "8.6oz (M9), 7.3oz (W7)"
  },
  "variants": [
    { "available": true, "price": 100, "size": "7", "sku": "TR3WMST070" },
    { "available": true, "price": 100, "size": "8", "sku": "TR3WMST080" }
  ]
}
```

Fields are tailored to the site: here `materials`, `technical_specs` and per-size `variants` with SKUs. The domain also appears in `GET /v1/extract/domains` now (`424 domains; True`).

## Customize it

| Change | How |
| - | - |
| Search and category pages too | Add search or category URLs to `example_urls` (for example 3 product pages and 2 search-result pages), then use [Turn search and listing pages into JSON](https://docs.zenrows.com/academy/extract/listing-pages.md) |
| Safe retries of the submission | Send an `Idempotency-Key` header, see [Getting a domain prepared](https://docs.zenrows.com/extract/endpoints.md#getting-a-domain-prepared) |
| A progress bar | Show `state` and each entry in `roles`, see [Reading the response](https://docs.zenrows.com/extract/endpoints.md#reading-the-response) |
| Domains your account has submitted | `GET /v1/extract/prepared-domains`, see [the endpoint reference](https://docs.zenrows.com/extract/endpoints.md) |
| Just a few fields | Add `extract_fields`, see [Get structured JSON from any page](https://docs.zenrows.com/academy/extract/auto-extraction.md#keep-just-the-fields-you-need) |
| Thousands of pages from the new domain | Put `"extract": "auto"` in a Batch job's `zenrows_params`, see [Scrape a list of URLs](https://docs.zenrows.com/academy/scrape-from-a-list-of-urls.md) |

## Troubleshooting

* **`failed` with `unpreparable` and a role detail of "the target could not be fetched"**: that example URL returned no usable page. Resubmit with a different URL of the same page type, or with the URLs that worked.
* **A role reads "this page was prepared, but the domain was not finalized"**: another URL in the same submission failed. Resubmit without that URL.
* **`failed` with `discovery_failed`**: resubmit with `example_urls`, so Zenrows prepares the exact page types you point at.
* **`409` with `in_progress`**: a preparation for that domain is already running. Poll its status instead of resubmitting.
* **`REQS007` on a page of a ready domain**: read the response's `reason`. It is usually `page_not_recognised`: the page may have nothing to extract, such as a home page. Try a product, article or listing page instead, or retry once in a few minutes. Preparing the domain again does not change it, because a ready domain learns other page types on their first request. See [REQS007](https://docs.zenrows.com/api-error-codes.md#REQS007) for every case.
* **A field is always empty**: check the `detail` sentence on the role. It names the fields that site's pages leave empty.

See [When a preparation fails](https://docs.zenrows.com/extract/endpoints.md#when-a-preparation-fails) for every failure code and the next step for each.

## Related recipes

> [card] [Get structured JSON from any page](https://docs.zenrows.com/academy/extract/auto-extraction.md)
>
> How extract=auto works, with extract\_fields and the SDKs.

> [card] [Turn search and listing pages into JSON](https://docs.zenrows.com/academy/extract/listing-pages.md)
>
> Collect every item across search and category pages.

> [card] [Validate extracted data with a schema](https://docs.zenrows.com/academy/extract/validate-with-a-schema.md)
>
> Type and check every row before you store it.

> [card] [Scrape a list of URLs](https://docs.zenrows.com/academy/scrape-from-a-list-of-urls.md)
>
> Run Extract on thousands of pages as one Batch job.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.