POST/v1/crawl

Bounded async crawl with a job id, a page cap quoted up front, and per-page Markdown.

$0.003/page
async job
robots

What it does

One paragraph, no pitch.

Bounded async crawl with a job id, a page cap quoted up front, and per-page Markdown.

Price

Quoted by the 402 before anything runs.

$0.003/pageper page
whole-call budget
billedonly when content is delivered
no keythe payment is the credential

x402 settlement is live on eip155:8453: sign a USDC transfer and retry with PAYMENT-SIGNATURE.

When to call it

Cheapest endpoint that answers the question, first.

Call it when the sentence above is the whole job. Its neighbours solve the same URL with a different trade-off — a browser instead of plain HTTP, a site instead of one page, one section instead of a document. /choose walks the decision in thirteen routes, cheapest first, browser last.

Not sure the page is worth paying for at all? /v1/probe is free and returns the status, content type, size, title, language and whether a bot wall is waiting — no body.

Request fields

JSON body. Send this to https://readfetch.com/v1/crawl.

Field	Type	Default	What it does
`url` required	string	—
`egress`	string	—	Egress pool to leave through: `direct` (the default route, no proxy) or any pool the operator configured, e.g. `residential` (the home exit) or `local`. Omit to route by host rules.
`max_pages`	number	20
`max_depth`	number	2
`include`	array	—	Path prefixes; * suffix wildcard.
`exclude`	array	—

Make the call

Unpaid, this answers 402 with the quote. Sign accepts[0] and resend with PAYMENT-SIGNATURE.

curl

curl -s https://readfetch.com/v1/crawl -H 'content-type: application/json' -d '{"url":"https://example.com"}'

json body

{
  "url": "https://example.com"
}

Response

Fields shown are the ones a caller branches on.

200 ok

{
  "hint": "GET /v1/crawl/{id} for progress",
  "job_id": "job_ab12",
  "pages_quoted": 20,
  "price_usd": "0.0500",
  "status": "queued"
}

Limits

Requests are JSON, 1 MiB maximum. Redirects are re-validated at every hop and capped at 5; the final URL is reported. is the whole-call budget — DNS, connect, redirects, retries and any render share it, and no step starts a fresh clock.

Failures come back as {"error": "…", "hint": "…"} with a status to branch on: 402 for the quote, 429 with Retry-After, 502 and 504 for the upstream. /docs lists the shapes.

Billing

Settled after the work, only when something was delivered.

$0.003 per delivered page. A response with billed: false settles nothing and names the next call to try.

billed: false means nothing settled: a bot wall, an empty extraction, an unsupported type or a transport failure. The hint on that response names the next call to try.

Every settlement carries a signed RECEIPT bound to the transaction. If a paid call delivered nothing, a claim issues a single-use credit for it — the rail cannot refund.