[Integrations](https://spiral.ai/integrations) › [Instinct](https://spiral.ai/integrations/instinct) › [Firecrawl website URLs](https://spiral.ai/integrations/instinct/firecrawl-website-urls)

# How to Use Firecrawl website URLs in Instinct

Find website URLs with titles and descriptions when available, using sitemap, search engine, and previously crawled data. Use it to inventory a site or shortlist pages for research; discovery can miss links.

|  |  |
| --- | --- |
| Model | Firecrawl website URLs |
| Available through | Firecrawl |
| Category | Web |
| Price | $0.0017459 per run. |
| Returns | json |
| Runs | In the background |
| Setup | About 2 minutes, with a Spiral API key |

## Prompts to try with Firecrawl website URLs in Instinct

- Every Monday, use Firecrawl website URLs on https://docs.stripe.com to find webhook-related URLs. Compare with the previous run and report newly discovered URLs and URLs no longer returned.
- Each week, use Firecrawl website URLs on https://www.patagonia.com to find repair-related URLs. Report changes in the returned list with titles where available.
- Every Friday, use Firecrawl website URLs on https://www.nasa.gov to find Artemis-related URLs. Compare with the prior run and label additions as newly discovered, without assigning publication dates.
- Each month, use Firecrawl website URLs on https://docs.github.com to find authentication-related URLs. Report changes in the returned paths and include descriptions where available.
- Every Wednesday, use Firecrawl website URLs on https://www.firecrawl.dev to find blog URLs. Compare with the previous run and report URLs newly appearing in the results.

## How to set up Firecrawl website URLs in Instinct

Pick the way that suits you. Each takes about two minutes.

### Use the Spiral skill

1. **Send Instinct this message**: Your agent reads Spiral's skill and sets itself up.

```
set up https://spiral.ai/SKILL.md
```

2. **Sign in or paste a key**: Approve Spiral in the browser, or give the agent a key from the console.

### Use Spiral's MCP server

1. **Create an API key**: In the console, open API keys and create a key for Instinct. Copy it.

2. **Send Instinct this message**: Instinct reads Spiral's skill and asks for your API key.

```text
set up https://spiral.ai/SKILL.md
```

3. **Give Instinct the key**: When Instinct asks for the key, add it where Instinct keeps secrets, never in the chat itself.

### Use the Spiral CLI

1. **Install and sign in**: Installs the Spiral CLI, signs you in and adds its skill to your coding agents.

```bash
npx -y spiral-cli@latest init
```

### Call the Spiral API

1. **Create an API key**: In the console, under API keys.

2. **Run Firecrawl website URLs**: One request; the result comes back by polling or a webhook.

```bash
curl https://api.spiral.ai/v1/runs \
  -H "Authorization: Bearer $SPIRAL_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $(uuidgen)" \
  -d '{
    "provider": "firecrawl",
    "model": "firecrawl/map",
    "input": {
      "url": "<url>"
    }
  }'
```

Then paste one of the prompts above: Instinct finds Firecrawl website URLs on Spiral, shows the price and runs it.

Setup guide in the docs: https://spiral.ai/docs/quickstarts/instinct

## Firecrawl website URLs tools and prices

- [Firecrawl Map · Site URLs](https://spiral.ai/tools/firecrawl/firecrawl/map) (Firecrawl): $0.0017459 per run.

## Settings you can ask for

| Setting | Allowed values | Default | Description |
| --- | --- | --- | --- |
| limit | … to 100000 | 5000 | Maximum number of links to return |
| search |  | — | Specify a search query to order the results by relevance. Example: 'blog' will return URLs that contain the word 'blog' in the URL ordered by relevance. |
| sitemap | skip, include, only | include | Sitemap mode when mapping. If you set it to `skip`, the sitemap won't be used to find URLs. If you set it to `only`, only URLs that are in the sitemap will be returned. By default (`include`), the sitemap and other methods will be used together to find URLs. |
| timeout |  | — | Timeout in milliseconds. There is no timeout by default. |
| location |  | — | Location settings for the request. When specified, this will use an appropriate proxy if available and emulate the corresponding language and timezone settings. Defaults to 'US' if not specified. |
| ignoreCache |  | false | Bypass the sitemap cache to retrieve fresh URLs. Sitemap data is cached for up to 7 days; use this parameter when your sitemap has been recently updated. |

## What can you do with Firecrawl website URLs in Instinct?

Flag newly discovered documentation or business URLs by comparing scheduled Firecrawl results. The agent can report changes in the returned inventory while keeping discovery changes separate from confirmed publication or deletion.

### Find documentation before an integration

Discover URLs related to authentication, webhooks, or refunds on a vendor's documentation site. Build a reading list before fetching the selected pages.

### Prepare a website migration

Create an initial inventory of discovered URLs and organize them by path. Compare that inventory with planned routes to identify pages that need migration review.

### Select sources for a knowledge base

Shortlist documentation and help-center URLs for later ingestion. Let a reviewer choose relevant sections before collecting page content.

### Watch for newly discovered URLs

Compare repeated maps of a public business website to flag URLs that newly appear or disappear from results. Treat these as discovery changes requiring review.

### Review a site's content structure

Group discovered URLs by directory and use available titles to outline documentation, blog, product, and support sections.

### Build a public research reading list

Find topic-related URLs on a known organization website and present them with available titles and descriptions. Keep the shortlist separate from conclusions that require reading the pages.

## Overview

- [Firecrawl](https://spiral.ai/providers/firecrawl)
- [All Instinct guides](https://spiral.ai/integrations/instinct)

## Firecrawl website URLs in other agents

- [Amp](https://spiral.ai/integrations/amp/firecrawl-website-urls)
- [ChatGPT](https://spiral.ai/integrations/chatgpt/firecrawl-website-urls)
- [Claude](https://spiral.ai/integrations/claude-desktop/firecrawl-website-urls)
- [Claude Code](https://spiral.ai/integrations/claude-code/firecrawl-website-urls)
- [Cline](https://spiral.ai/integrations/cline/firecrawl-website-urls)
- [Codex](https://spiral.ai/integrations/codex/firecrawl-website-urls)
- [Cursor](https://spiral.ai/integrations/cursor/firecrawl-website-urls)
- [Devin](https://spiral.ai/integrations/devin/firecrawl-website-urls)
- [dots](https://spiral.ai/integrations/dots/firecrawl-website-urls)
- [Factory Droid](https://spiral.ai/integrations/factory-droid/firecrawl-website-urls)
- [Gemini](https://spiral.ai/integrations/gemini/firecrawl-website-urls)
- [Gemini CLI](https://spiral.ai/integrations/gemini-cli/firecrawl-website-urls)
- [Google Antigravity](https://spiral.ai/integrations/antigravity/firecrawl-website-urls)
- [goose](https://spiral.ai/integrations/goose/firecrawl-website-urls)
- [Grok](https://spiral.ai/integrations/grok/firecrawl-website-urls)
- [Grok Build](https://spiral.ai/integrations/grok-build/firecrawl-website-urls)
- [Hermes](https://spiral.ai/integrations/hermes/firecrawl-website-urls)
- [Manus](https://spiral.ai/integrations/manus/firecrawl-website-urls)
- [Muse](https://spiral.ai/integrations/muse/firecrawl-website-urls)
- [Muse Code](https://spiral.ai/integrations/muse-code/firecrawl-website-urls)
- [n8n](https://spiral.ai/integrations/n8n/firecrawl-website-urls)
- [Notion](https://spiral.ai/integrations/notion/firecrawl-website-urls)
- [OpenClaw](https://spiral.ai/integrations/openclaw/firecrawl-website-urls)
- [opencode](https://spiral.ai/integrations/opencode/firecrawl-website-urls)
- [Perplexity](https://spiral.ai/integrations/perplexity/firecrawl-website-urls)
- [Raycast](https://spiral.ai/integrations/raycast/firecrawl-website-urls)
- [VS Code](https://spiral.ai/integrations/vscode/firecrawl-website-urls)
- [Windsurf](https://spiral.ai/integrations/windsurf/firecrawl-website-urls)
- [Zapier](https://spiral.ai/integrations/zapier/firecrawl-website-urls)

## More Instinct guides

- [Firecrawl web search](https://spiral.ai/integrations/instinct/firecrawl-web-search)
- [Firecrawl website content](https://spiral.ai/integrations/instinct/firecrawl-website-content)
- [Firecrawl web research](https://spiral.ai/integrations/instinct/firecrawl-web-research)
- [Google search results](https://spiral.ai/integrations/instinct/google-search-results)
- [Firecrawl web page content](https://spiral.ai/integrations/instinct/firecrawl-web-page-content)
- [Wikidata item details](https://spiral.ai/integrations/instinct/wikidata-item-details)

## Questions about Firecrawl website URLs in Instinct

**Can Instinct use Firecrawl website URLs?**

Yes. Add Spiral to Instinct as an MCP server, and Instinct can find and run Firecrawl website URLs like any built-in tool.

**How much does Firecrawl website URLs cost in Instinct?**

$0.0017459 per run. You pay from your Spiral balance as you go, with no subscription, and a failed run is not charged. Your Instinct plan is separate.

**Do I need a Firecrawl account to use Firecrawl website URLs in Instinct?**

You need no Firecrawl account. Instinct uses one Spiral API key for every tool, and Spiral runs Firecrawl website URLs on its own Firecrawl account.

**How long does it take to set up Firecrawl website URLs in Instinct?**

About two minutes: create an API key, then send Instinct this message, then give Instinct the key.

**Does Instinct wait while Firecrawl website URLs runs?**

Firecrawl website URLs runs in the background. Instinct starts the run, checks on it and gives you the json when it's ready.

**Where do the URLs come from, and is coverage complete?**

Firecrawl primarily uses the website's sitemap, supplemented by search engine results and previously crawled pages. Map prioritizes speed and can miss links, so the result is a discovered URL inventory rather than a complete site audit.

**How fresh is the URL list?**

Sitemap data can be cached for up to seven days. Bypassing the sitemap cache helps when a sitemap has recently changed, but discovery also draws on previously crawled pages and search engine results.

**How many URLs can one call return?**

Firecrawl Map supports up to 100,000 returned links per call. That ceiling does not guarantee that every URL on a large website will be discovered.

**Does the result include page content?**

Map returns discovered URLs, with titles and descriptions when available. It does not scrape the pages' contents, so use the results to select pages for further reading.

**Can I tell when a page was last updated?**

Map does not return page modification dates. Comparing repeated URL lists can identify newly discovered URLs, but it cannot establish when those pages were published or edited.

