> ## Documentation Index
> Fetch the complete documentation index at: https://docs.zenrows.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> The Zenrows HTTP API serves Fetch and Extract (extract=auto) at https://api.zenrows.com/v1/, Crawl at https://api.zenrows.com/v1/crawls, and Batch at https://async.api.zenrows.com/v1. Fetch, Extract, Crawl and Batch are described in OpenAPI 3.1 at https://www.zenrows.com/openapi.json and listed in the RFC 9727 API catalog at https://www.zenrows.com/.well-known/api-catalog. Every endpoint also has its own page under /api-reference with its schema. Use the spec for exact parameter names and types. Crawl's endpoints, parameters and errors are also at https://docs.zenrows.com/crawl/endpoints.md. To call Zenrows from an agent, connect the MCP server at https://mcp.zenrows.com/mcp. To search these docs from an agent, connect the docs MCP server at https://docs.zenrows.com/mcp. Agent skills for Zenrows are published at https://github.com/ZenRows/cli and install with `npx skills add ZenRows/cli`.

# Automate scraping from your terminal with the CLI

> Fetch, extract, crawl, run batch jobs and drive a browser from your terminal with the Zenrows CLI, and schedule it all with cron.

The `zenrows` CLI wraps every Zenrows primitive in one command, writes clean output to stdout and progress to stderr, and keeps a trace of every run. That makes it a natural fit for shell scripts, `jq` pipelines and cron jobs.

**Uses:** [`zenrows fetch`](https://docs.zenrows.com/cli/fetch/introduction), [`zenrows extract`](https://docs.zenrows.com/cli/extract/introduction), [`zenrows batch`](https://docs.zenrows.com/cli/batch/introduction), [`zenrows crawl`](https://docs.zenrows.com/cli/crawl/introduction), [`zenrows browser`](https://docs.zenrows.com/cli/browser/introduction), [traces](https://docs.zenrows.com/cli/troubleshooting.md)

## Ask your AI agent

With the [Zenrows MCP server](https://docs.zenrows.com/mcp/overview.md) connected, or the CLI available to a coding agent such as Claude Code, paste this prompt:

```text theme={"theme":"css-variables"}
Use Zenrows to find the product pages on https://www.scrapingcourse.com/ecommerce/ (up to 10 page loads),
scrape them all in one batch as markdown, and save each page to a folder called pages/.
```

## Install and log in

Install the CLI once, then set up a workspace with your API key. `zenrows init` creates the local `.zenrows/` folder that holds your config, policy and run traces.

```bash CLI theme={"theme":"css-variables"}
npm install -g @zenrows/cli
zenrows init --yes --api-key YOUR_ZENROWS_API_KEY
```

In CI or containers, export the key instead and tell the CLI to read it:

```bash CLI theme={"theme":"css-variables"}
export ZENROWS_API_KEY=YOUR_ZENROWS_API_KEY
zenrows login --env
```

## Fetch and extract into jq

`zenrows fetch` and `zenrows extract` use Adaptive Stealth Mode (`mode=auto`) by default. The page body goes to stdout and the status line goes to stderr, so pipes stay clean.

`zenrows extract` returns structured JSON, so you can pipe it straight into `jq`. With `--css`, you name the fields and their selectors:

```bash CLI theme={"theme":"css-variables"}
zenrows extract https://www.scrapingcourse.com/ecommerce/ \
  --css '{"name":".product-name","price":".product .price"}' \
  | jq '[.name, .price] | transpose | map({name: .[0], price: .[1]}) | .[:3]'
```

```json theme={"theme":"css-variables"}
[
  { "name": "Abominable Hoodie", "price": "$69.00" },
  { "name": "Adrienne Trek Jacket", "price": "$57.00" },
  { "name": "Aeon Capri", "price": "$48.00" }
]
```

Without `--css`, `zenrows extract` sends `extract=auto` for site-tailored fields on any prepared domain. On a domain that isn't prepared yet, it fails with `EXTRACT_DOMAIN_NOT_PREPARED` and prints the `zenrows extract prepare` command to run; it never falls back to Autoparse.

Save any page as markdown with `--output markdown` and `--out`:

```bash CLI theme={"theme":"css-variables"}
zenrows fetch https://www.scrapingcourse.com/ecommerce/ --output markdown --out catalog.md
```

```text theme={"theme":"css-variables"}
→ Protected Fetch https://www.scrapingcourse.com/ecommerce/ (auto)…
✓ Wrote 6763 bytes → catalog.md
✓ HTTP 200 · 6763 bytes · cost $0.0010 · 1 credit · run r-1a9dfd1e
```

## Script with --json

Every command accepts `--json` and prints a structured result instead of the body or the status line. Use it to branch on status or log cost in a script:

```bash CLI theme={"theme":"css-variables"}
zenrows fetch https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/ --json \
  | jq '{ok, httpStatus, costCredits, runId}'
```

```json theme={"theme":"css-variables"}
{
  "ok": true,
  "httpStatus": 200,
  "costCredits": 1,
  "runId": "r-bcdfc63b"
}
```

## Run a URL list as one Batch job

For tens to thousands of URLs, submit them as one [Batch](https://docs.zenrows.com/batch/introduction.md) job instead of a loop of fetches. The CLI reads a JSONL file with one `{"url": ...}` object per line.

**1. Build the job file from a plain URL list**

```bash CLI theme={"theme":"css-variables"}
jq -Rc '{url: .}' urls.txt > jobs.jsonl
```

**2. Estimate the cost locally**

`estimate` validates the file and prints an upper bound, with no cloud call and no credits spent. Tasks in Adaptive Stealth Mode count at their highest possible cost.

```bash CLI theme={"theme":"css-variables"}
zenrows batch estimate jobs.jsonl
```

```text theme={"theme":"css-variables"}
Job spec: jobs.jsonl
valid jobs: 3/3
estimated credits, upper bound (1x basic / 5x js / 10x proxy / 25x both or auto): 75
✓ Spec is valid.
```

**3. Submit and wait for the run**

`--follow` polls until the run finishes. `--output markdown` sets the format of every result.

```bash CLI theme={"theme":"css-variables"}
zenrows batch create jobs.jsonl --output markdown --follow
```

```text theme={"theme":"css-variables"}
→ Submitting batch job (3 tasks, ~75 credits)…
✓ Submitted job 01M4Y9ACBYZZFWWBY7JA8KE6QQ · status: completed
  3/3 completed · 3 successful · 0 failed
```

**4. Download every result**

```bash CLI theme={"theme":"css-variables"}
zenrows batch results 01M4Y9ACBYZZFWWBY7JA8KE6QQ --download pages
```

`pages/` gets one file per task plus `_manifest.jsonl`, which maps each file to its URL and status:

```json theme={"theme":"css-variables"}
{"task_id":"01M4Y9ACBYS5SHV7V54EGYVW2D","url":"https://www.scrapingcourse.com/ecommerce/product/abominable-hoodie/","status":"successful","type":"markdown","file":"01M4Y9ACBYS5SHV7V54EGYVW2D.md"}
```

The run above billed 3 credits in total: the estimate is a ceiling, and `mode=auto` bills only the configuration that succeeds. Check the real spend with `zenrows batch status <job-id> --json`, which returns `stats.spend.credits`.

## Crawl a site, then batch what you found

`zenrows crawl create` discovers URLs from one start page. Feed the results straight into a Batch job:

```bash CLI theme={"theme":"css-variables"}
zenrows crawl create https://www.scrapingcourse.com/ecommerce/ \
  --max-pages 10 --include-patterns /product/ --include-patterns /page/ --follow

zenrows crawl results <crawl-id> --out crawl.jsonl
jq -c 'select(.url | contains("/product/")) | {url}' crawl.jsonl > jobs.jsonl
```

```text theme={"theme":"css-variables"}
→ Starting crawl of https://www.scrapingcourse.com/ecommerce/ (no depth limit)…
✓ Crawl c_01M4Y9B3GA71076Z7S3MZ2XGP7 finished · status: completed (max_items)
  100 item(s) found · 10 page(s) fetched · 0 failed
```

Including `/page/` lets the crawl walk the paginated listing, and the `jq` filter keeps only product URLs (88 of the 100 in this run). See [Crawl from a seed URL](https://docs.zenrows.com/academy/scrape-and-crawl-from-a-seed-url.md) for more patterns.

## Drive a browser from the shell

When a page needs clicks or typing, open a Zenrows Browser session. Each verb takes the `--session` id that `open` returns. Always close the session when you finish.

```bash CLI theme={"theme":"css-variables"}
SESSION=$(zenrows browser open https://www.scrapingcourse.com/button-click --json | jq -r .session_id)

zenrows browser click --session "$SESSION" --selector "#load-more-btn"
zenrows browser wait --session "$SESSION" --ms 2000
zenrows browser query-selector-all --session "$SESSION" --selector ".product-name" --json \
  | jq -r '.result.elements[].text'

zenrows browser close --session "$SESSION"
```

```text theme={"theme":"css-variables"}
Chaz Kangeroo Hoodie
Teton Pullover Hoodie
Bruno Compete Hoodie
Frankie Sweatshirt
```

For longer flows, put the steps in a JSON file and run them with `zenrows browser run script.json`, which closes the session for you. See the [Browser CLI guide](https://docs.zenrows.com/cli/browser/introduction.md).

## Schedule it with cron

The CLI keeps your key and policy in `.zenrows/`, so a cron job only needs to run from that folder. This script refreshes the product pages every morning:

```bash scrape.sh theme={"theme":"css-variables"}
#!/usr/bin/env bash
set -euo pipefail
cd /home/me/catalog

JOB=$(zenrows batch create jobs.jsonl --output markdown --follow --timeout 2h --json | jq -r .job_id)
zenrows batch results "$JOB" --download "pages/$(date +%F)"
```

```bash crontab theme={"theme":"css-variables"}
0 6 * * * /home/me/catalog/scrape.sh >> /home/me/catalog/scrape.log 2>&1
```

For schedules that run on Zenrows' side with no machine of your own, see [Run scraping jobs on a schedule](https://docs.zenrows.com/academy/batch/scheduled-jobs.md).

## Read a run's trace

Every `fetch`, `extract`, `batch`, `crawl` and `browser` run writes a record under `.zenrows/runs/<run-id>/`. The run id is in the status line (`run r-1a9dfd1e`).

```bash CLI theme={"theme":"css-variables"}
zenrows trace explain r-1a9dfd1e
zenrows trace inspect r-1a9dfd1e
```

```text theme={"theme":"css-variables"}
Trace explain · r-1a9dfd1e
what happened:   zenrows fetch on https://www.scrapingcourse.com/ecommerce/ succeeded (HTTP 200, 6763 bytes).
failure reason:  none
evidence:        status=ok cost=$0.0010 · 1 credit cap=protected_fetch
next action:     Reuse these parameters for similar targets; consider capturing them as a skill.
```

`trace inspect` prints the full record as JSON, including the exact request parameters and `costCredits`. `trace replay` prints the command to run it again.

## Customize it

| Change | How |
| - | - |
| Geo-target a request | `--proxy-country us` on `fetch` or `batch create`, see [CLI fetch flags](https://docs.zenrows.com/cli/fetch/introduction.md#flags) |
| Pick your own render and proxy settings | `--manual --js-render --premium-proxy`, see [CLI fetch](https://docs.zenrows.com/cli/fetch/introduction.md) |
| Extract with CSS selectors | `zenrows extract <url> --css '{"title":"h1","price":".price"}'`, see [CLI extract](https://docs.zenrows.com/cli/extract/introduction.md) |
| Only the failed tasks again | `zenrows batch retry-failed <job-id>`, see [CLI batch](https://docs.zenrows.com/cli/batch/introduction.md) |
| Cap what one run may spend | `zenrows policy set max_credits_per_run 500`, see [Track usage and control spend](https://docs.zenrows.com/academy/production/control-costs.md) |
| Check credits left | `zenrows usage` |

## Troubleshooting

* **`AUTH_MISSING`**: no key in this workspace. Run `zenrows login --api-key YOUR_ZENROWS_API_KEY` from the project folder.
* **`No workspace found` on `trace`**: traces live in `.zenrows/`. Run `zenrows init --yes` in the project folder, then run the command again.
* **`PARAM_CONFLICT_AUTO_MANUAL`**: `--js-render` or `--premium-proxy` on `fetch` or `extract` needs `--manual`. Add it, or drop those flags to keep `mode=auto`.
* **`POLICY_LIMIT_EXCEEDED` on `batch create`**: the estimate is above your `max_credits_per_run`. Split the file or raise the cap with `zenrows policy set max_credits_per_run <n>`.
* **The cron job finds no key**: cron starts in your home folder. `cd` into the project folder first, as in the script above.
* **The cron job reports `zenrows: command not found`**: cron runs with a minimal `PATH`. Call the CLI by its full path, which `which zenrows` prints.

More codes and fixes: [CLI troubleshooting](https://docs.zenrows.com/cli/troubleshooting.md).

## Related recipes

> [card] [Scrape a list of URLs](https://docs.zenrows.com/academy/scrape-from-a-list-of-urls.md)
>
> Batch from the API and SDKs, with webhooks and reruns.

> [card] [Prompts that put Zenrows MCP to work](https://docs.zenrows.com/academy/surfaces/mcp-prompts.md)
>
> The same jobs, done by your AI agent.

> [card] [Track usage and control spend](https://docs.zenrows.com/academy/production/control-costs.md)
>
> Read per-request cost and set caps.

> [card] [Resolve common response codes](https://docs.zenrows.com/academy/production/fix-common-errors.md)
>
> Symptom-to-fix for 422, 429, 402 and more.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.