[Integrations](https://spiral.ai/integrations) › [dots](https://spiral.ai/integrations/dots) › [Firecrawl batch web scraping](https://spiral.ai/integrations/dots/firecrawl-batch-web-scraping)

# How to Use Firecrawl batch web scraping in dots

Fetch content from a supplied list of web pages as Markdown, HTML, links, screenshots, or structured JSON. Use it when you already know which URLs to read, such as documentation pages or product pages you want to compare.

|  |  |
| --- | --- |
| Model | Firecrawl batch web scraping |
| Available through | Firecrawl |
| Category | Web |
| Price | $0.001746 per URL ($0.00873 with JSON output). PII redaction, a prompt injection check or lockdown: $0.006984 more per URL each. |
| Returns | json |
| Runs | In the background |
| Setup | About 2 minutes, with a Spiral API key |

## Prompts to try with Firecrawl batch web scraping in dots

- Every morning, use Firecrawl batch web scraping on https://www.python.org/downloads/ and https://nodejs.org/en/download. Request fresh page content and report changes to the visible release names and version numbers since the previous run.
- Every Monday, use Firecrawl batch web scraping on https://react.dev/reference/react/useState and https://react.dev/reference/react/useEffect. Request fresh content and summarize changed guidance or examples compared with the previous run.
- Every week, use Firecrawl batch web scraping on https://www.mozilla.org/en-US/privacy/ and https://www.mozilla.org/en-US/about/. Request fresh content and show any changed passages with their source URLs.
- Every Friday, use Firecrawl batch web scraping on https://docs.python.org/3/library/json.html and https://docs.python.org/3/library/csv.html. Request fresh content and report changes to documented behavior since the previous run.
- Every morning, use Firecrawl batch web scraping on https://nodejs.org/en/blog/release and https://www.python.org/downloads/. Request fresh content and report newly visible release headings and links compared with the previous run.

## How to set up Firecrawl batch web scraping in dots

Pick the way that suits you. Each takes about two minutes.

### Use the Spiral skill

1. **Send dots this message**: Your agent reads Spiral's skill and sets itself up.

```
set up https://spiral.ai/SKILL.md
```

2. **Sign in or paste a key**: Approve Spiral in the browser, or give the agent a key from the console.

### Use Spiral's MCP server

1. **Create an API key**: In the console, open API keys and create a key for your dot. Copy it.

2. **Send your dot this message**: Your dot reads Spiral's skill and asks for your API key.

```text
set up https://spiral.ai/SKILL.md
```

3. **Give your dot the key**: When your dot asks for the key, add it where your dot keeps secrets, never in the chat itself.

### Use the Spiral CLI

1. **Install and sign in**: Installs the Spiral CLI, signs you in and adds its skill to your coding agents.

```bash
npx -y spiral-cli@latest init
```

### Call the Spiral API

1. **Create an API key**: In the console, under API keys.

2. **Run Firecrawl batch web scraping**: One request; the result comes back by polling or a webhook.

```bash
curl https://api.spiral.ai/v1/runs \
  -H "Authorization: Bearer $SPIRAL_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $(uuidgen)" \
  -d '{
    "provider": "firecrawl",
    "model": "firecrawl/batch-scrape",
    "input": {
      "urls": [
        "<urls>"
      ]
    }
  }'
```

Then paste one of the prompts above: dots finds Firecrawl batch web scraping on Spiral, shows the price and runs it.

Setup guide in the docs: https://spiral.ai/docs/quickstarts/dots

## Firecrawl batch web scraping tools and prices

- [Firecrawl Batch Scrape · Many Pages](https://spiral.ai/tools/firecrawl/firecrawl/batch-scrape) (Firecrawl): $0.001746 per URL ($0.00873 with JSON output). PII redaction, a prompt injection check or lockdown: $0.006984 more per URL each.

## Settings you can ask for

| Setting | Allowed values | Default | Description |
| --- | --- | --- | --- |
| proxy | basic, enhanced, auto | auto | Specifies the type of proxy to use. - **basic**: Proxies for scraping sites with none to basic anti-bot solutions. Fast and usually works. - **enhanced**: Enhanced proxies for scraping sites with advanced anti-bot solutions. Slower, but more reliable on certain sites. Billed at the same credit cost as basic. - **auto**: Firecrawl will automatically retry scraping with enhanced proxies if the basic proxy fails. Enhanced proxies carry no credit surcharge, so either way only the regular cost is billed. |
| maxAge |  | 172800000 | Returns a cached version of the page if it is younger than this age in milliseconds. If a cached version of the page is older than this value, the page will be scraped. If you do not need extremely fresh data, enabling this can speed up your scrapes by 500%. Defaults to 2 days. |
| minAge |  | — | When set, the request only checks the cache and never triggers a fresh scrape. The value is in milliseconds and specifies the minimum age the cached data must be. If matching cached data exists, it is returned instantly. If no cached data is found, a 404 with error code SCRAPE_NO_CACHED_DATA is returned. Set to 1 to accept any cached data regardless of age. |
| mobile |  | false | Set to true if you want to emulate scraping from a mobile device. Useful for testing responsive pages and taking mobile screenshots. |
| actions |  | — | Actions to perform on the page before grabbing the content |
| formats |  | [{"type":"markdown"}] | Output formats to include in the response. You can specify one or more formats, either as strings (e.g., `'markdown'`) or as objects with additional options (e.g., `{ type: 'json', schema: {...} }`). Some formats require specific options to be set. Example: `['markdown', { type: 'json', schema: {...} }]`. |

## What can you do with Firecrawl batch web scraping in dots?

Receive a digest of changes across selected release notes or public policy pages. Have the agent fetch the same URLs on each run and compare the returned text with its previous snapshot.

### Build a documentation reference

Collect a selected set of documentation URLs as page text, then organize the returned content into a searchable reference with links to the original pages.

### Compare public product descriptions

Read several public product pages and build a table of stated features, supported platforms, and published specifications.

### Prepare a migration review

Fetch the documentation pages relevant to a planned code change and compare their examples and requirements with the project's implementation.

### Create a release digest

Fetch selected release-note pages on a schedule and compare their text with saved snapshots to summarize newly published changes.

### Review public policy changes

Collect a fixed list of public terms or policy pages and compare successive snapshots, highlighting the passages that changed.

### Audit selected website pages

Inspect returned page text and links across an approved URL list to find inconsistent descriptions, outdated references, or unexpected navigation links.

### Compare page presentation

Collect screenshots and page text for selected pages, then review how their visible presentation matches their stated content.

## Overview

- [Firecrawl](https://spiral.ai/providers/firecrawl)
- [All dots guides](https://spiral.ai/integrations/dots)

## Firecrawl batch web scraping in other agents

- [Amp](https://spiral.ai/integrations/amp/firecrawl-batch-web-scraping)
- [ChatGPT](https://spiral.ai/integrations/chatgpt/firecrawl-batch-web-scraping)
- [Claude](https://spiral.ai/integrations/claude-desktop/firecrawl-batch-web-scraping)
- [Claude Code](https://spiral.ai/integrations/claude-code/firecrawl-batch-web-scraping)
- [Cline](https://spiral.ai/integrations/cline/firecrawl-batch-web-scraping)
- [Codex](https://spiral.ai/integrations/codex/firecrawl-batch-web-scraping)
- [Cursor](https://spiral.ai/integrations/cursor/firecrawl-batch-web-scraping)
- [Devin](https://spiral.ai/integrations/devin/firecrawl-batch-web-scraping)
- [Factory Droid](https://spiral.ai/integrations/factory-droid/firecrawl-batch-web-scraping)
- [Gemini](https://spiral.ai/integrations/gemini/firecrawl-batch-web-scraping)
- [Gemini CLI](https://spiral.ai/integrations/gemini-cli/firecrawl-batch-web-scraping)
- [Google Antigravity](https://spiral.ai/integrations/antigravity/firecrawl-batch-web-scraping)
- [goose](https://spiral.ai/integrations/goose/firecrawl-batch-web-scraping)
- [Grok](https://spiral.ai/integrations/grok/firecrawl-batch-web-scraping)
- [Grok Build](https://spiral.ai/integrations/grok-build/firecrawl-batch-web-scraping)
- [Hermes](https://spiral.ai/integrations/hermes/firecrawl-batch-web-scraping)
- [Instinct](https://spiral.ai/integrations/instinct/firecrawl-batch-web-scraping)
- [Manus](https://spiral.ai/integrations/manus/firecrawl-batch-web-scraping)
- [Muse](https://spiral.ai/integrations/muse/firecrawl-batch-web-scraping)
- [Muse Code](https://spiral.ai/integrations/muse-code/firecrawl-batch-web-scraping)
- [n8n](https://spiral.ai/integrations/n8n/firecrawl-batch-web-scraping)
- [Notion](https://spiral.ai/integrations/notion/firecrawl-batch-web-scraping)
- [OpenClaw](https://spiral.ai/integrations/openclaw/firecrawl-batch-web-scraping)
- [opencode](https://spiral.ai/integrations/opencode/firecrawl-batch-web-scraping)
- [Perplexity](https://spiral.ai/integrations/perplexity/firecrawl-batch-web-scraping)
- [Raycast](https://spiral.ai/integrations/raycast/firecrawl-batch-web-scraping)
- [VS Code](https://spiral.ai/integrations/vscode/firecrawl-batch-web-scraping)
- [Windsurf](https://spiral.ai/integrations/windsurf/firecrawl-batch-web-scraping)
- [Zapier](https://spiral.ai/integrations/zapier/firecrawl-batch-web-scraping)

## More dots guides

- [Exa web search](https://spiral.ai/integrations/dots/exa-web-search)
- [Firecrawl developer search](https://spiral.ai/integrations/dots/firecrawl-developer-search)
- [Exa web research](https://spiral.ai/integrations/dots/exa-web-research)
- [Firecrawl web page content](https://spiral.ai/integrations/dots/firecrawl-web-page-content)
- [Exa web page content](https://spiral.ai/integrations/dots/exa-web-page-content)
- [Firecrawl web research](https://spiral.ai/integrations/dots/firecrawl-web-research)

## Questions about Firecrawl batch web scraping in dots

**Can dots use Firecrawl batch web scraping?**

Yes. Add Spiral to dots as an MCP server, and dots can find and run Firecrawl batch web scraping like any built-in tool.

**How much does Firecrawl batch web scraping cost in dots?**

$0.001746 per URL ($0.00873 with JSON output). PII redaction, a prompt injection check or lockdown: $0.006984 more per URL each. You pay from your Spiral balance as you go, with no subscription, and a failed run is not charged. Your dots plan is separate.

**Do I need a Firecrawl account to use Firecrawl batch web scraping in dots?**

You need no Firecrawl account. dots uses one Spiral API key for every tool, and Spiral runs Firecrawl batch web scraping on its own Firecrawl account.

**How long does it take to set up Firecrawl batch web scraping in dots?**

About two minutes: create an API key, then send your dot this message, then give your dot the key.

**Does dots wait while Firecrawl batch web scraping runs?**

Firecrawl batch web scraping runs in the background. dots starts the run, checks on it and gives you the json when it's ready.

**Does Firecrawl batch scraping collect an entire website?**

It processes an explicit list of URLs. Its coverage follows that list, so include each page you want represented; some websites are also unsupported.

**How fresh is the returned page content?**

Firecrawl's default cache freshness window is two days. A new request can therefore return a recent cached copy rather than newly fetched website content.

**Can one result response contain the whole batch?**

Large results require pagination: Firecrawl provides a continuation URL for the next 10 MB of data. The response also reports how many pages were successfully scraped.

**How long can I retrieve completed results?**

Batch results are available through the API for 24 hours after completion. Save the returned content within that window if you need it for later comparisons or indexing.

**Can structured JSON include hidden HTML attributes?**

JSON extraction works from the page's Markdown conversion, which preserves visible text. HTML attributes are unavailable to that extraction, so use original HTML when those values matter.

