Firecrawl

Address it

import { create, readUrl } from "@agntn/web";

const firecrawl = await create("firecrawl"); // FIRECRAWL_API_KEY
const results = await firecrawl.search("vercel ai sdk tools", {
  sources: ["web", "news"],
  categories: ["research"],
});
const page = await readUrl("https://sdk.vercel.ai/docs", { provider: "firecrawl", targetSelector: "main" });

Bearer auth against api.firecrawl.dev.

What it reads

What comes back

snippet is a passage relevant to the query, Markdown included when the source has it. Sometimes that passage is the whole page, almost 80,000 characters of Node.js release notes, so it stops at 500 characters with …. text is the scraped Markdown, image and metadata when present. highlights: false turns the passage scraping off. Domain filters, sources (web, news, images) and categories (research, pdf, developer) work. The singular category is not forwarded, Firecrawl wants the plural.

Gotchas

• searchDetailed exposes response metadata: request id, warning and creditsUsed. web search --provider firecrawl --json prints it.
• The developer category cannot be mixed with the others. The adapter rejects the combination before the request instead of letting Firecrawl answer with a 400.
• A read comes back with the URL Firecrawl landed on after redirects. Its metadata is only about the fetch: originalUrl, statusCode, error, contentType, language, cachedAt, creditsUsed and warning. The page's meta tags are left out, because they ran to dozens of keys (71 on a GitHub repo page) that mostly repeat the title and description.
• An explicit read rejects maxTokens instead of ignoring it. An automatic read that lands here leaves it out and says so in ignoredOptions. Use maxChars for a bound that means the same thing everywhere.

Where it lives

src/providers/firecrawl.ts.