[Integrations](https://spiral.ai/integrations) › [Cline](https://spiral.ai/integrations/cline) › [Firecrawl batch web scraping](https://spiral.ai/integrations/cline/firecrawl-batch-web-scraping)

# How to Use Firecrawl batch web scraping in Cline

Fetch content from a supplied list of web pages as Markdown, HTML, links, screenshots, or structured JSON. Use it when you already know which URLs to read, such as documentation pages or product pages you want to compare.

|  |  |
| --- | --- |
| Model | Firecrawl batch web scraping |
| Available through | Firecrawl |
| Category | Web |
| Price | $0.001746 per URL ($0.00873 with JSON output). PII redaction, a prompt injection check or lockdown: $0.006984 more per URL each. |
| Returns | json |
| Runs | In the background |
| Setup | About 2 minutes, no API key |

## Prompts to try with Firecrawl batch web scraping in Cline

- Use Firecrawl batch web scraping on https://react.dev/reference/react/useState and https://react.dev/reference/react/useEffect. Compare the returned guidance with this project's React components and identify changes worth reviewing.
- Use Firecrawl batch web scraping on https://nodejs.org/api/fs.html and https://nodejs.org/api/path.html. Review this project's file handling against the documented behavior and cite the relevant sections.
- Use Firecrawl batch web scraping on https://docs.python.org/3/library/json.html and https://docs.python.org/3/library/csv.html. Compare the documented parsing behavior with our import code and flag mismatches.
- Use Firecrawl batch web scraping on https://www.postgresql.org/docs/current/sql-select.html and https://www.postgresql.org/docs/current/sql-insert.html. Review the SQL in this repository against the returned documentation.
- Use Firecrawl batch web scraping on https://react.dev/reference/react/useState and https://react.dev/reference/react/useReducer. Create a short reference in the project's docs explaining which existing components could benefit from a reducer.

## How to set up Firecrawl batch web scraping in Cline

Pick the way that suits you. Each takes about two minutes.

### Use Spiral's MCP server

1. **Add the server**: In the Cline panel, click the MCP Servers icon and open the Remote Servers tab. Enter:

```bash
cline mcp add spiral https://api.spiral.ai/mcp --transport http --yes
```

2. **Sign in**: Open the Configure tab and click Authenticate on the Spiral server. Sign in to Spiral, pick the organization and click Authorize. No API key.

3. **Check it**: Spiral shows as connected on the Configure tab. In the CLI:

```bash
cline config mcp
```

### Use the Spiral skill

1. **Send Cline this message**: Your agent reads Spiral's skill and sets itself up.

```
set up https://spiral.ai/SKILL.md
```

2. **Sign in or paste a key**: Approve Spiral in the browser, or give the agent a key from the console.

### Use the Spiral CLI

1. **Install and sign in**: Installs the Spiral CLI, signs you in and adds its skill to your coding agents.

```bash
npx -y spiral-cli@latest init
```

### Call the Spiral API

1. **Create an API key**: In the console, under API keys.

2. **Run Firecrawl batch web scraping**: One request; the result comes back by polling or a webhook.

```bash
curl https://api.spiral.ai/v1/runs \
  -H "Authorization: Bearer $SPIRAL_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $(uuidgen)" \
  -d '{
    "provider": "firecrawl",
    "model": "firecrawl/batch-scrape",
    "input": {
      "urls": [
        "<urls>"
      ]
    }
  }'
```

Then paste one of the prompts above: Cline finds Firecrawl batch web scraping on Spiral, shows the price and runs it.

Setup guide in the docs: https://spiral.ai/docs/quickstarts/cline

## Firecrawl batch web scraping tools and prices

- [Firecrawl Batch Scrape · Many Pages](https://spiral.ai/tools/firecrawl/firecrawl/batch-scrape) (Firecrawl): $0.001746 per URL ($0.00873 with JSON output). PII redaction, a prompt injection check or lockdown: $0.006984 more per URL each.

## Settings you can ask for

| Setting | Allowed values | Default | Description |
| --- | --- | --- | --- |
| proxy | basic, enhanced, auto | auto | Specifies the type of proxy to use. - **basic**: Proxies for scraping sites with none to basic anti-bot solutions. Fast and usually works. - **enhanced**: Enhanced proxies for scraping sites with advanced anti-bot solutions. Slower, but more reliable on certain sites. Billed at the same credit cost as basic. - **auto**: Firecrawl will automatically retry scraping with enhanced proxies if the basic proxy fails. Enhanced proxies carry no credit surcharge, so either way only the regular cost is billed. |
| maxAge |  | 172800000 | Returns a cached version of the page if it is younger than this age in milliseconds. If a cached version of the page is older than this value, the page will be scraped. If you do not need extremely fresh data, enabling this can speed up your scrapes by 500%. Defaults to 2 days. |
| minAge |  | — | When set, the request only checks the cache and never triggers a fresh scrape. The value is in milliseconds and specifies the minimum age the cached data must be. If matching cached data exists, it is returned instantly. If no cached data is found, a 404 with error code SCRAPE_NO_CACHED_DATA is returned. Set to 1 to accept any cached data regardless of age. |
| mobile |  | false | Set to true if you want to emulate scraping from a mobile device. Useful for testing responsive pages and taking mobile screenshots. |
| actions |  | — | Actions to perform on the page before grabbing the content |
| formats |  | [{"type":"markdown"}] | Output formats to include in the response. You can specify one or more formats, either as strings (e.g., `'markdown'`) or as objects with additional options (e.g., `{ type: 'json', schema: {...} }`). Some formats require specific options to be set. Example: `['markdown', { type: 'json', schema: {...} }]`. |

## What can you do with Firecrawl batch web scraping in Cline?

Build a documentation reference that matches the code you are changing. Firecrawl supplies content from selected URLs so a coding agent can compare published guidance with the project's implementation.

### Build a documentation reference

Collect a selected set of documentation URLs as page text, then organize the returned content into a searchable reference with links to the original pages.

### Compare public product descriptions

Read several public product pages and build a table of stated features, supported platforms, and published specifications.

### Prepare a migration review

Fetch the documentation pages relevant to a planned code change and compare their examples and requirements with the project's implementation.

### Create a release digest

Fetch selected release-note pages on a schedule and compare their text with saved snapshots to summarize newly published changes.

### Review public policy changes

Collect a fixed list of public terms or policy pages and compare successive snapshots, highlighting the passages that changed.

### Audit selected website pages

Inspect returned page text and links across an approved URL list to find inconsistent descriptions, outdated references, or unexpected navigation links.

### Compare page presentation

Collect screenshots and page text for selected pages, then review how their visible presentation matches their stated content.

## Overview

- [Firecrawl](https://spiral.ai/providers/firecrawl)
- [All Cline guides](https://spiral.ai/integrations/cline)

## Firecrawl batch web scraping in other agents

- [Amp](https://spiral.ai/integrations/amp/firecrawl-batch-web-scraping)
- [ChatGPT](https://spiral.ai/integrations/chatgpt/firecrawl-batch-web-scraping)
- [Claude](https://spiral.ai/integrations/claude-desktop/firecrawl-batch-web-scraping)
- [Claude Code](https://spiral.ai/integrations/claude-code/firecrawl-batch-web-scraping)
- [Codex](https://spiral.ai/integrations/codex/firecrawl-batch-web-scraping)
- [Cursor](https://spiral.ai/integrations/cursor/firecrawl-batch-web-scraping)
- [Devin](https://spiral.ai/integrations/devin/firecrawl-batch-web-scraping)
- [dots](https://spiral.ai/integrations/dots/firecrawl-batch-web-scraping)
- [Factory Droid](https://spiral.ai/integrations/factory-droid/firecrawl-batch-web-scraping)
- [Gemini](https://spiral.ai/integrations/gemini/firecrawl-batch-web-scraping)
- [Gemini CLI](https://spiral.ai/integrations/gemini-cli/firecrawl-batch-web-scraping)
- [Google Antigravity](https://spiral.ai/integrations/antigravity/firecrawl-batch-web-scraping)
- [goose](https://spiral.ai/integrations/goose/firecrawl-batch-web-scraping)
- [Grok](https://spiral.ai/integrations/grok/firecrawl-batch-web-scraping)
- [Grok Build](https://spiral.ai/integrations/grok-build/firecrawl-batch-web-scraping)
- [Hermes](https://spiral.ai/integrations/hermes/firecrawl-batch-web-scraping)
- [Instinct](https://spiral.ai/integrations/instinct/firecrawl-batch-web-scraping)
- [Manus](https://spiral.ai/integrations/manus/firecrawl-batch-web-scraping)
- [Muse](https://spiral.ai/integrations/muse/firecrawl-batch-web-scraping)
- [Muse Code](https://spiral.ai/integrations/muse-code/firecrawl-batch-web-scraping)
- [n8n](https://spiral.ai/integrations/n8n/firecrawl-batch-web-scraping)
- [Notion](https://spiral.ai/integrations/notion/firecrawl-batch-web-scraping)
- [OpenClaw](https://spiral.ai/integrations/openclaw/firecrawl-batch-web-scraping)
- [opencode](https://spiral.ai/integrations/opencode/firecrawl-batch-web-scraping)
- [Perplexity](https://spiral.ai/integrations/perplexity/firecrawl-batch-web-scraping)
- [Raycast](https://spiral.ai/integrations/raycast/firecrawl-batch-web-scraping)
- [VS Code](https://spiral.ai/integrations/vscode/firecrawl-batch-web-scraping)
- [Windsurf](https://spiral.ai/integrations/windsurf/firecrawl-batch-web-scraping)
- [Zapier](https://spiral.ai/integrations/zapier/firecrawl-batch-web-scraping)

## More Cline guides

- [Exa web search](https://spiral.ai/integrations/cline/exa-web-search)
- [Firecrawl developer search](https://spiral.ai/integrations/cline/firecrawl-developer-search)
- [Exa web research](https://spiral.ai/integrations/cline/exa-web-research)
- [Firecrawl web page content](https://spiral.ai/integrations/cline/firecrawl-web-page-content)
- [Exa web page content](https://spiral.ai/integrations/cline/exa-web-page-content)
- [Firecrawl web research](https://spiral.ai/integrations/cline/firecrawl-web-research)

## Questions about Firecrawl batch web scraping in Cline

**Can Cline use Firecrawl batch web scraping?**

Yes. Add Spiral to Cline as an MCP server, and Cline can find and run Firecrawl batch web scraping like any built-in tool.

**How much does Firecrawl batch web scraping cost in Cline?**

$0.001746 per URL ($0.00873 with JSON output). PII redaction, a prompt injection check or lockdown: $0.006984 more per URL each. You pay from your Spiral balance as you go, with no subscription, and a failed run is not charged. Your Cline plan is separate.

**Do I need a Firecrawl account to use Firecrawl batch web scraping in Cline?**

No. Spiral runs Firecrawl batch web scraping on its own Firecrawl account. You sign in to Spiral once and never paste a key.

**How long does it take to set up Firecrawl batch web scraping in Cline?**

About two minutes: add the server, then sign in, then check it.

**Does Cline wait while Firecrawl batch web scraping runs?**

Firecrawl batch web scraping runs in the background. Cline starts the run, checks on it and gives you the json when it's ready.

**Does Firecrawl batch scraping collect an entire website?**

It processes an explicit list of URLs. Its coverage follows that list, so include each page you want represented; some websites are also unsupported.

**How fresh is the returned page content?**

Firecrawl's default cache freshness window is two days. A new request can therefore return a recent cached copy rather than newly fetched website content.

**Can one result response contain the whole batch?**

Large results require pagination: Firecrawl provides a continuation URL for the next 10 MB of data. The response also reports how many pages were successfully scraped.

**How long can I retrieve completed results?**

Batch results are available through the API for 24 hours after completion. Save the returned content within that window if you need it for later comparisons or indexing.

**Can structured JSON include hidden HTML attributes?**

JSON extraction works from the page's Markdown conversion, which preserves visible text. HTML attributes are unavailable to that extraction, so use original HTML when those values matter.

