TinyFish
Search
Fast, structured web search
Fetch
Any URL to clean content
Agent
Multi-step web automation
Browser
Stealth Chromium sessions
All products share one API keyView docs →
Documentation
API reference and guides
Integrations
Connect with your stack
Blog
Product updates and insights
Cookbook
Open-source examples
Pricing
Overview
Enterprise-grade web data
Use Cases
What teams are building
Customers
See who builds with TinyFish
Log InContactContact
Products
SearchFast, structured web search
FetchAny URL to clean content
AgentMulti-step web automation
BrowserStealth Chromium sessions
Resources
DocumentationAPI reference and guides
IntegrationsConnect with your stack
BlogProduct updates and insights
CookbookOpen-source examples
PricingPlans, credits, and billing
Enterprise
OverviewEnterprise-grade web data
Use CasesWhat teams are building
CustomersSee who builds with TinyFish
Log InContact
TinyFish

Web APIs built for agents.

Product
  • Enterprise
  • Use Cases
  • Customers
  • Pricing
  • Integrations
  • Docs
  • Trust
Resources
  • Cookbook
  • Blog
  • Current
  • Accelerator
Connect
  • X/Twitter
  • LinkedIn
  • Discord
  • GitHub
  • Contact Us
© 2026 TinyFish·Privacy·Terms
Engineering

Production-Grade Web Fetching for AI Agents

Chenlu Ji·ML Engineer·Apr 14, 2026
Share
Production-Grade Web Fetching for AI Agents

Some fetch failures are obvious. The page times out, the batch stalls, the error is logged. The harder ones look like successes. Status 200. Content returned. 14 characters of text.

In our own testing across 200 URLs, over 10% of "successful" fetches came back nearly empty - enough content to pass, not enough to reason on.

Fetching the page was never the hard part. Getting something an agent can actually use is.

Today we’re launching TinyFish Fetch API to solve that layer end-to-end.

Fetch API is a dual-layer web page fetching and content extraction service built for AI agents. It renders URLs through Chromium, extracts structured content, and returns it in Markdown, HTML, or JSON through a single API.

It is built for reading web pages: articles, documentation, product pages, pricing pages, and other content-heavy URLs that agents need to understand quickly and reliably.

Why Web Fetching Normally Breaks for Agents

Modern agents need web access that is accurate after JavaScript execution, deterministic and structured for downstream tools, resilient when running autonomous workflows, and safe by default when handling arbitrary URLs.

The modern web is not optimised for that kind of consumption.

What looks like a readable page to a person often looks like noise to an agent: JavaScript shells, navigation chrome, cookie banners, injected widgets, ad blocks, and fragmented content spread across inconsistent HTML structures.

Even when the page does render, the result is often still hard to use. One site wraps key information in semantic markup; another buries it in nested divs. One page loads cleanly; another stalls behind retries, incomplete hydration, or bot defenses.

Web fetching is a systems problem. Not just getting HTML back. Getting the right content, in a usable form, with enough reliability for autonomous workflows.

How Fetch API Fixes It

Fetch API was designed as infrastructure. It removes parser work from the caller by returning agent-ready content plus metadata such as title, description, author, published date, and language in one response.

Fetch API combines two layers that are often stitched together in typical stacks:

  1. Rendering layer (browser-native)
  2. Extraction layer (structure-aware content processing)

Rendering alone is not enough. Extraction alone is not enough.

At the API level, the contract stays simple: send one request with one or more URLs, and get back structured content plus metadata.

curl -X POST <https://api.fetch.tinyfish.ai> \\\\
  -H "X-API-Key: $TINYFISH_API_KEY" \\\\
  -H "Content-Type: application/json" \\\\
  -d '{ "urls": ["<https://example.com>"] }'

When a request comes in, Fetch first solves the rendering problem, then the extraction problem, and finally the output-format problem.

If any one of those stages breaks down, the workflow becomes unreliable. Fetch is built to handle them as one pipeline.

Step 1: Rendering That Survives the Real Web

Under the hood, Fetch API runs on TinyFish Browser, backed by a warm Chromium pool.

  • Full JavaScript rendering for SPAs and modern frameworks (React, Vue, Next.js)
  • Infrastructure-level handling to improve fetch reliability on protected sites
  • Rotation strategies to improve reliability in autonomous runs
  • Warm browser pool to avoid cold-start penalties on request paths
  • Automatic recovery from incomplete renders by detecting skeleton/preloader states and retrying with deeper capture (including same-origin iframes and shadow DOM)

For agents, that means access to post-JavaScript page state even on sites where static fetches miss the content that actually matters.

Step 2: Extraction Built for Agent Inputs

After the page is rendered, Fetch moves from visibility to usability.

Its extraction pipeline is tuned for noisy and irregular HTML.

  • Primary extraction pass with automatic fallback when output falls short of expected completeness
  • Fallback pipeline includes semantic targeting, noise stripping, link-density filtering, and div-to-paragraph normalization
  • Native PDF path: PDF URLs bypass browser slots and are downloaded and parsed directly
  • Rich metadata extraction, including title, description, author, published_date, language, links, and image links

This is where many fetch pipelines break down. Getting the page is only the first half. The harder part is turning a messy, dynamic document into an agent-ready result.

Agents can choose output by workload:

  • Markdown for LLM-friendly context windows
  • HTML for raw fidelity
  • JSON document tree for structured programmatic processing

The goal is simple: a docs page, a blog post, and a pricing page may all look different in raw HTML, but the caller should get the same kind of usable output back.

Extraction Quality for Agent Inputs

For agents, a page is only useful if its meaning survives contact with the model.

Different sites can express the same content through completely different DOM structures. Fetch is designed to normalize those differences into outputs that are easier to ground on, route through retrieval systems, and process in downstream automation.

Reliability and Safety in Production

Agent pipelines need systems that continue running even when the web is inconsistent.

Reliability at Batch Scale

Fetch API includes reliability defaults built for unattended jobs:

  • Batch fetch up to 10 URLs per request, processed in parallel
  • Per-URL isolation so one failing target does not collapse the entire batch
  • Per-URL errors are returned in a structured errors array, so agent workflows can handle failures without losing successful results
  • Latency behavior tied to the slowest URL in a batch (not additive across all URLs)
  • Production profile observed across diverse traffic: p50 around 12s, p95 around 30s. Initial per-attempt error rate is under 5%; with retry logic enabled, transient failures are usually recovered, resulting in high end-to-end completion rates.
  • Clear error categories (timeouts, invalid URLs, bot blocks, and more) to simplify agent fallback policies

In practice, that means one bad page does not poison the entire job, and transient failures usually recover on the next attempt instead of forcing manual cleanup.

Safety and Operational Defaults

When agents fetch arbitrary URLs, safety controls cannot be optional.

Fetch API applies SSRF protections before fetch execution:

  • URL validation blocks private/reserved IP ranges
  • Cloud metadata endpoints are denied
  • Internal browser service boundary supports safer deployment patterns

Operationally, Fetch API is built for production integration:

  • Plain HTTP interface (POST <https://api.fetch.tinyfish.ai>) that maps directly to tool-calling architectures
  • Container-friendly deployment model with health endpoints
  • Logs, metrics, and traces for monitoring extraction quality, latency, and failures

Fetch in Practice

Beyond the core fetch flow, Fetch exposes a number of capabilities that other products do not provide in the same form.

The table below compares Fetch API with Exa, Firecrawl, and Crawl4AI based on publicly documented capabilities.

✓ = first-class documented support, ~ = supported through a separate flow or different integration model, — = not exposed as a first-class documented feature

Comparison based on publicly documented product behaviour as of April 13, 2026.

Where Fetch Fits in the TinyFish Stack

When the job is to read and extract content from a URL, Fetch is the right layer. When the job requires clicking, navigating, logging in, or filling forms, that is where the broader TinyFish browser and agent stack comes in.

  • Web Agent: give it a goal, and it handles browser actions autonomously
  • Web Search: structured web search built for agent workflows
  • Web Fetch: clean, structured content extraction from live pages
  • Web Browser: direct browser control for interactive workflows
  • Vault: secure credential storage for authenticated automation

Fetch is the reading layer in that stack: the part that turns live pages into stable inputs for reasoning and downstream workflows.

What's Next

We are already working on the next set of capabilities for Fetch API, including smarter caching, subdocument querying, richer output options, and lower-latency controls. The direction is simple: make Fetch faster on repeat work, more selective in what it returns, and easier to tune for different agent workloads.

Try It Now

AI agents are only as good as the web substrate beneath them.

TinyFish Fetch API is live today. Get an API key and let us know if it makes your agent's life better.

Get started

Start building.

No credit card. No setup. Run your first operation in under a minute.

Get 500 free creditsRead the docs
More Articles
Why Stitched Web Stacks Fail in Production
Product and Integrations

Why Stitched Web Stacks Fail in Production

Keith Zhai·Apr 14, 2026
We Shipped an MCP Server. Then We Shipped a CLI. The CLI Won.
Engineering

We Shipped an MCP Server. Then We Shipped a CLI. The CLI Won.

Divya Lath·Apr 4, 2026
TinyFish × n8n: Web Agents Just Landed in Your Workflow Canvas
Product & Integrations

TinyFish × n8n: Web Agents Just Landed in Your Workflow Canvas

Sky Zhang·Mar 18, 2026