# readfetch.com

> Web reading for AI agents: readable text and site maps without a browser,
> browser rendering when JavaScript is required, and raw HTTP for everything
> else.

Base URL: https://readfetch.com
Skill (start here): https://readfetch.com/SKILL.md
MCP: https://readfetch.com/mcp  (tools: https://readfetch.com/.well-known/mcp.json)
OpenAPI: https://readfetch.com/openapi.json
Manifest: https://readfetch.com/.well-known/service.json
Resources: https://readfetch.com/.well-known/x402-resources.json
Health: https://readfetch.com/healthz

## Endpoints

  POST /v1/text          $0.005  Article text and metadata, boilerplate stripped. No browser by default; falls back to one when a page blocks plain HTTP.
  POST /v1/map           $0.005  Sitemaps, robots rules, feeds and link counts for a whole site, in one call. Plan a crawl before paying for it.
  POST /v1/scrape        $0.005  Chromium renders the page, runs interactions, and returns Markdown, text or HTML with its links.
  POST /v1/scrape/residential      $0.02  Scrape through a residential address pool, for targets that refuse a datacentre address. Same contract as /v1/scrape, one attempt, priced for the bandwidth.
  POST /v1/fetch         $0.003  Raw HTTP through the same egress. JSON APIs, GraphQL, PDFs, byte ranges - anything that is not a page.
  POST /v1/batch     $0.0015/page  Up to 50 URLs in one call and one payment. Results in input order; undelivered pages come back as credit.
  POST /v1/pdf       $0.005/document  Documents, not pages: page text plus one Markdown body, priced per document rather than per page.
  POST /v1/section       $0.003  Read one heading or one selector instead of the whole page - ten to a hundred times fewer tokens.
  POST /v1/diff          $0.003  Stateless change detection. Send the hash you hold, get back whether it changed and a bounded diff.
  POST /v1/screenshot     $0.003  PNG, JPEG or PDF of a rendered page: viewport, full page, or a single element.
  POST /v1/search        $0.005  Web search without a search subscription, returning titles, URLs and snippets.
  POST /v1/crawl     $0.003/page  Bounded async crawl with a job id, a page cap quoted up front, and per-page Markdown.
  POST /v1/probe           free  Metadata before spending: status, type, title, size, language, whether a bot wall is waiting. No page body.

Every POST takes `content-type: application/json` and `x-payment-proof` in test mode, or
`PAYMENT-SIGNATURE` when settlement is live. The 402 response quotes the exact price and
x402 `accepts` block for that endpoint. Prices are USD per call.

## Guides

- /choose — which endpoint to use for a job.
- /docs — how the payment header, quoting and billing work.
- /examples — a request and response per endpoint.
- /integrations — wiring readfetch into an agent or an app.
- /faq — short answers to the questions that come up first.

### POST /v1/text

Fetch a public URL and return its main readable content as Markdown or plain text, plus title, author, published date and site name. Strips navigation, ads and boilerplate. No JavaScript rendering, so it costs a fraction of a browser scrape; use /v1/scrape when the content only appears after JavaScript runs.

Body:
  accept_language any      default: null; Accept-Language header, e.g. 'de-DE,de;q=0.9'. Defaults to en-US.
  egress       string   Exit to leave through: `direct` (this service's own datacentre egress) or `residential` (a residential address pool, for targets that refuse datacentre addresses). Omit to use the default route.
  egress_fallback string   Pool to retry through once when the first attempt is refused (bot check, wall, empty extraction, origin 403/429/5xx); must differ from `egress`. Absent keeps one exit for the whole call.
  format       string   markdown | text; default: markdown; Output flavour: Markdown keeps headings, lists and tables.
  max_bytes    integer  1024-20000000; default: 5000000; Abort if the response body exceeds this many bytes.
  timeout_s    number   1-60; default: 30; Total budget for the whole call in seconds: DNS, redirects, the render fallback and any retry share it.
  url          string   required; 1-2083 chars; Public http(s) URL to read.

Billing: a call that delivers nothing returns `billed: false` and is not
charged. A `hint` names the next call to try.

### POST /v1/map

Discover a site's crawlable surface without crawling it: parse robots.txt allow/disallow rules, collect URLs from sitemap.xml and sitemap indexes, find RSS, Atom and JSON feeds, and count internal and external links on the homepage. Use the returned URLs to plan a /v1/scrape or /v1/crawl.

Body:
  egress       string   Exit to leave through: `direct` (this service's own datacentre egress) or `residential` (a residential address pool, for targets that refuse datacentre addresses). Omit to use the default route.
  egress_fallback string   Pool to retry through once when the first attempt is refused (bot check, wall, empty extraction, origin 403/429/5xx); must differ from `egress`. Absent keeps one exit for the whole call.
  limit        integer  1-1000; default: 200; Maximum number of URLs to return (1-1000).
  max_bytes    integer  1024-20000000; default: 5000000; Abort if any single document exceeds this many bytes.
  timeout_s    number   1-95; default: 60; Total budget for the whole call in seconds; every sitemap fetch and the browser fallback share it. 95 s is the edge ceiling (Cloudflare closes at 100 s).
  url          string   required; 1-2083 chars; Public http(s) URL of the site to map.

Billing: a call that delivers nothing returns `billed: false` and is not
charged. A `hint` names the next call to try.

### POST /v1/scrape

JavaScript rendering and web scraping for agents: load a public URL in a real browser session, optionally scroll, wait for selectors, run JS, set headers, cookies or auth, and return Markdown, text or HTML with the page's links.

Body:
  advanced     any      default: null
  auth         any      default: null
  egress       string   Exit to leave through: `direct` (this service's own datacentre egress) or `residential` (a residential address pool, for targets that refuse datacentre addresses). Omit to use the default route.
  egress_fallback string   Pool to retry through once when the first attempt is refused (bot check, wall, empty extraction, origin 403/429/5xx); must differ from `egress`. Absent keeps one exit for the whole call.
  format       string   markdown | text | html; default: markdown
  fresh        boolean  default: false
  mode         string   fast | render | full; default: render
  options      any      default: null
  url          string   required; 1-2083 chars

Billing: a call that delivers nothing returns `billed: false` and is not
charged. A `hint` names the next call to try.

### POST /v1/scrape/residential

Identical request and response to /v1/scrape, but the render leaves through a residential address instead of the datacentre. For targets that block by IP reputation (DataDome, PerimeterX, Cloudflare, AWS WAF). One attempt, no retries: residential bandwidth is the cost, and four times the scrape price is what it takes to cover it honestly.

Body:
  advanced     any      default: null
  auth         any      default: null
  format       string   markdown | text | html; default: markdown
  fresh        boolean  default: false
  mode         string   fast | render | full; default: render
  options      any      default: null
  url          string   required; 1-2083 chars

Billing: a call that delivers nothing returns `billed: false` and is not
charged. A `hint` names the next call to try.

### POST /v1/fetch

Send an arbitrary HTTP request (GET, POST, PUT, PATCH, DELETE, HEAD) and return the response with status, headers and body, as text or base64. Intended for APIs, GraphQL, PDFs, byte ranges and large payloads where a browser is not needed.

Body:
  binary       boolean  default: false
  body         any      default: null
  egress       string   Exit to leave through: `direct` (this service's own datacentre egress) or `residential` (a residential address pool, for targets that refuse datacentre addresses). Omit to use the default route.
  egress_fallback string   Pool to retry through once when the first attempt is refused (bot check, wall, empty extraction, origin 403/429/5xx); must differ from `egress`. Absent keeps one exit for the whole call.
  follow_redirects boolean  default: true
  headers      any      default: null
  json_body    any      default: null
  max_bytes    integer  1024-50000000; default: 10000000
  method       string   GET | POST | PUT | PATCH | DELETE | HEAD; default: GET
  timeout_s    number   1-95; default: 95; Total budget for the whole call in seconds, redirect hops included.
  url          string   required; 1-2083 chars

Billing: a call that delivers nothing returns `billed: false` and is not
charged. A `hint` names the next call to try.

### POST /v1/batch

Up to 50 URLs in one call and one payment. Results in input order; an undelivered page is not billed and is credited.

Body:
  accept_language any      default: null
  egress       string   Exit to leave through: `direct` (this service's own datacentre egress) or `residential` (a residential address pool, for targets that refuse datacentre addresses). Omit to use the default route.
  format       string   markdown | text; default: markdown
  fresh        boolean  default: false
  pipeline     string   text | render; default: text; text = HTTP-first; render needs the browser tier.
  timeout_s    number   1-95; default: 95; Total budget for the whole batch in seconds; every item shares it and no item restarts it.
  urls         array    required; URLs to read, deduped before quoting.

Billing: a call that delivers nothing returns `billed: false` and is not
charged. A `hint` names the next call to try.

### POST /v1/pdf

Documents, not pages: page text plus one Markdown body, priced per document.

Body:
  content_base64 string   Inline PDF, up to 20 MB.
  egress       string   Exit to leave through: `direct` (this service's own datacentre egress) or `residential` (a residential address pool, for targets that refuse datacentre addresses). Omit to use the default route.
  filename     string   
  max_pages    number   1-200; default: 50
  url          string   PDF URL, fetched through egress.

Billing: a call that delivers nothing returns `billed: false` and is not
charged. A `hint` names the next call to try.

### POST /v1/section

One heading or one selector instead of the whole page. A heading that is not found returns 200, an empty body and the list of headings that do exist.

Body:
  egress       string   Exit to leave through: `direct` (this service's own datacentre egress) or `residential` (a residential address pool, for targets that refuse datacentre addresses). Omit to use the default route.
  format       string   markdown | text; default: markdown
  heading      any      Case-insensitive, whitespace-normalised.
  max_chars    number   100-20000; default: 8000
  mode         string   fast | render; default: render
  selector     any      CSS selector; up to 20 matches concatenated.
  url          string   required

Billing: a call that delivers nothing returns `billed: false` and is not
charged. A `hint` names the next call to try.

### POST /v1/diff

Stateless change detection: send the hash you hold, get changed/unchanged and a bounded diff. A no-change answer is billable.

Body:
  egress       string   Exit to leave through: `direct` (this service's own datacentre egress) or `residential` (a residential address pool, for targets that refuse datacentre addresses). Omit to use the default route.
  format       string   text | markdown; default: text
  previous_content any      Previous text, for an added/removed diff.
  previous_hash any      sha256:... from a previous call.
  timeout_s    number   1-120
  url          string   required

Billing: a call that delivers nothing returns `billed: false` and is not
charged. A `hint` names the next call to try.

### POST /v1/screenshot

PNG or JPEG of a rendered page: viewport, full page, or one element.

Body:
  delay_ms     number   0-30000
  egress       string   Exit to leave through: `direct` (this service's own datacentre egress) or `residential` (a residential address pool, for targets that refuse datacentre addresses). Omit to use the default route.
  format       string   png | jpeg; default: png
  full_page    boolean  default: true
  height       number   1-2160
  mode         string   fast | render | full; default: render
  selector     any      Capture one element.
  url          string   required
  width        number   1-3840

Billing: a call that delivers nothing returns `billed: false` and is not
charged. A `hint` names the next call to try.

### POST /v1/search

Web search without a search subscription, returning titles, URLs and snippets.

Body:
  egress       string   Exit to leave through: `direct` (this service's own datacentre egress) or `residential` (a residential address pool, for targets that refuse datacentre addresses). Omit to use the default route.
  lang         any      e.g. en, de-DE
  limit        number   1-20; default: 5
  query        string   required
  tbs          string   h | d | w | m | y; Time range.

Billing: a call that delivers nothing returns `billed: false` and is not
charged. A `hint` names the next call to try.

### POST /v1/crawl

Bounded async crawl with a job id, a page cap quoted up front, and per-page Markdown.

Body:
  egress       string   Exit to leave through: `direct` (this service's own datacentre egress) or `residential` (a residential address pool, for targets that refuse datacentre addresses). Omit to use the default route.
  exclude      array    
  include      array    Path prefixes; * suffix wildcard.
  max_depth    number   1-3; default: 2
  max_pages    number   1-100; default: 20
  url          string   required

Billing: a call that delivers nothing returns `billed: false` and is not
charged. A `hint` names the next call to try.

### POST /v1/probe

Metadata before spending: status, content type, size, title, language and whether a bot wall is waiting. No page body, no browser, always free.

Body:
  accept_language any      default: null; Accept-Language header, e.g. 'de-DE,de;q=0.9'. Defaults to en-US.
  egress       string   Exit to leave through: `direct` (this service's own datacentre egress) or `residential` (a residential address pool, for targets that refuse datacentre addresses). Omit to use the default route.
  timeout_s    number   1-60; default: 10; Total budget for the whole call in seconds.
  url          string   required; 1-2083 chars; Public http(s) URL to inspect.

Billing: a call that delivers nothing returns `billed: false` and is not
charged. A `hint` names the next call to try.

## Recovery

- `billed: false` means nothing was charged: read `hint`, then retry the
  suggested endpoint.
- 429 or 403 from a target: retry with /v1/scrape, which renders in a browser.
- Prices and schemas: /.well-known/x402-resources.json