# Read a public web page into structured text

> Fetch one public web page and return its title, description, headings, readable text, links and embedded JSON-LD, honouring robots.txt.

Decant: Files in. Rows out. Runs instantly.

## What it is for

An agent or analyst often needs a page's actual content in a clean structure (headings, text, links, schema.org data) rather than raw HTML, to quote it, compare offers or extract terms. This fetches one public page and returns that structure, refusing pages the site asks robots not to fetch.

Developers and agents that need page content without running a browser, and researchers collecting a few pages' text with their sources.

## What happens next

The page is fetched once from the public internet, after its robots.txt is checked for our user agent. Private, internal and this service's own addresses are refused before any request; at most three redirects are followed, each checked again; 5 MB and 10 seconds are the limits. The answer is built from that one response; JavaScript is not run.

## What you get

A JSON answer with the final URL, status, title, meta description, canonical link, headings, readable text (up to 100,000 characters), up to 300 links and any JSON-LD blocks.

## Scope

Any public website reachable over HTTP or HTTPS on the standard ports.

- One page per call; links are listed, not followed.
- Pages that need JavaScript, a login or a CAPTCHA come back incomplete or refused.
- HTML, plain text and JSON only; PDFs and spreadsheets go to document conversion.

## Operation: POST /web-extraction/op/extract

One public page as title, description, headings, text, links and JSON-LD.

Price: free within a daily allowance.

- `url` (string): http or https URL of a public page

```json
{
  "url": "https://example.com/",
  "final_url": "https://example.com/",
  "status": 200,
  "content_type": "text/html; charset=UTF-8",
  "title": "Example Domain",
  "description": null,
  "canonical": null,
  "headings": [
    {
      "level": 1,
      "text": "Example Domain"
    }
  ],
  "text": "Example Domain This domain is for use in documentation examples without needing permission.",
  "links": [
    {
      "href": "https://iana.org/domains/example",
      "text": "Learn more"
    }
  ],
  "json_ld": [],
  "robots": "allowed",
  "truncated": false
}
```

## Send a requirement

Call POST /web-extraction/op/extract with {"url": "https://..."} (free, within a daily budget). Or describe a crawl or extraction job this does not cover.

POST https://decant.forevenza.com/web-extraction/requests with JSON like:

```json
{
  "action": "retrieve",
  "need": "Read the pricing page at https://www.cloudflare.com/plans/ and give me its headings and text.",
  "specifications": {
    "url": "https://www.cloudflare.com/plans/",
    "fields": [
      "headings",
      "text"
    ]
  }
}
```

Schema: https://decant.forevenza.com/web-extraction/schema.json

## Who runs it, and what it costs

Decant. Operated by Tempered Technologies LLC, 8 The Green STE B, Dover, DE 19901. Privacy questions and deletion requests: privacy@forevenza.com.

Free to use within a daily allowance; sending a requirement is free.

Privacy notice: https://decant.forevenza.com/privacy/