Spiral gives your agent every tool with one account. Browse the tools →

goose · Web

How to Use Firecrawl website URLs in goose

Find website URLs with titles and descriptions when available, using sitemap, search engine, and previously crawled data. Use it to inventory a site or shortlist pages for research; discovery can miss links.

Prompts to try with Firecrawl website URLs in goose

Copy an example into goose, or ask in your own words.

Use Firecrawl website URLs on https://docs.stripe.com to find webhook-related URLs. Save a reading list in docs/stripe-webhook-sources.md with titles and descriptions where returned.

Firecrawl website URLsgoose
Try asking
  • Use Firecrawl website URLs on https://nextjs.org to find routing documentation URLs. Compare them with the Next.js links in this project's README and flag differences for review.

  • Use Firecrawl website URLs on https://docs.astro.build to inventory discovered documentation paths. Create a CSV grouped by the first two path segments.

  • Use Firecrawl website URLs on https://docs.github.com to find authentication-related URLs. Add a source shortlist to this project's integration research notes without inferring page contents.

  • Use Firecrawl website URLs on https://www.firecrawl.dev and compare the returned URLs with our saved inventory. Produce a diff labeled newly discovered and no longer returned.

How to set up Firecrawl website URLs in goose

Pick the way that suits you. Each takes about two minutes.

Use Spiral's MCP server

  1. Add the extension

    In goose Desktop, open the sidebar, click Extensions, then Add custom extension. Enter:

  2. Sign in

    When goose first connects to Spiral, your browser opens. Sign in to Spiral, pick the organization and click Authorize. No API key. If the browser does not open, goose prints the link to open yourself.

Use the Spiral skill

  1. Send goose this message

    Your agent reads Spiral's skill and sets itself up.

    set up https://spiral.ai/SKILL.md
  2. Sign in or paste a key

    Approve Spiral in the browser, or give the agent a key from the console.

Use the Spiral CLI

  1. Install and sign in

    Installs the Spiral CLI, signs you in and adds its skill to your coding agents.

    npx -y spiral-cli@latest init

Call the Spiral API

  1. Create an API key

    In the console, under API keys.

  2. Run Firecrawl website URLs

    One request; the result comes back by polling or a webhook.

    curl https://api.spiral.ai/v1/runs \
      -H "Authorization: Bearer $SPIRAL_API_KEY" \
      -H "Content-Type: application/json" \
      -H "Idempotency-Key: $(uuidgen)" \
      -d '{
        "provider": "firecrawl",
        "model": "firecrawl/map",
        "input": {
          "url": "<url>"
        }
      }'

Then paste one of the prompts above: goose finds Firecrawl website URLs on Spiral, shows the price and runs it.

Firecrawl website URLs providers and prices

Spiral runs Firecrawl website URLs through Firecrawl. Open a card for today's price, with every fee included.

FirecrawlWeb

Settings you can ask for

Ask goose for any of these in your prompt.

SettingAllowed valuesDefaultDescription
Limita whole number up to 1000005000

Maximum number of links to return

Sitemapskip, include, or onlyinclude

Sitemap mode when mapping. If you set it to skip, the sitemap won't be used to find URLs. If you set it to only, only URLs that are in the sitemap will be returned. By default (include), the sitemap and other methods will be used together to find URLs.

Timeouta whole number

Timeout in milliseconds. There is no timeout by default.

Ignore cacheon or offoff

Bypass the sitemap cache to retrieve fresh URLs. Sitemap data is cached for up to 7 days; use this parameter when your sitemap has been recently updated.

Include subdomainson or offon

Include subdomains of the website

Ignore query parameterson or offon

Do not return URLs with query parameters

Every API field of Firecrawl Map · Site URLs is on its tool page: open its card above.

What can you do with Firecrawl website URLs in goose?

Build a documentation reading list or migration checklist from Firecrawl's discovered website URLs. A coding agent can compare those URLs with project routes and documentation links to identify items for review.

Find documentation before an integration

Discover URLs related to authentication, webhooks, or refunds on a vendor's documentation site. Build a reading list before fetching the selected pages.

Prepare a website migration

Create an initial inventory of discovered URLs and organize them by path. Compare that inventory with planned routes to identify pages that need migration review.

Select sources for a knowledge base

Shortlist documentation and help-center URLs for later ingestion. Let a reviewer choose relevant sections before collecting page content.

Watch for newly discovered URLs

Compare repeated maps of a public business website to flag URLs that newly appear or disappear from results. Treat these as discovery changes requiring review.

Review a site's content structure

Group discovered URLs by directory and use available titles to outline documentation, blog, product, and support sections.

Build a public research reading list

Find topic-related URLs on a known organization website and present them with available titles and descriptions. Keep the shortlist separate from conclusions that require reading the pages.

Firecrawl website URLs success rate and speed on Spiral

Success and speed over the last 24 hours, across every run of these tools on Spiral.

Success rate
100.0%
Fastest median run time
33.6s
Runs, last 24 hours
37
ToolSuccessMedian timeRuns
Firecrawl Map · Site URLs100.0%33.6s37

Questions about Firecrawl website URLs in goose

Can goose use Firecrawl website URLs?

Yes. Add Spiral to goose as an MCP server, and goose can find and run Firecrawl website URLs like any built-in tool.

How much does Firecrawl website URLs cost in goose?

$0.0017459 per run. You pay from your Spiral balance as you go, with no subscription, and a failed run is not charged. Your goose plan is separate.

Do I need a Firecrawl account to use Firecrawl website URLs in goose?

No. Spiral runs Firecrawl website URLs on its own Firecrawl account. You sign in to Spiral once and never paste a key.

How long does it take to set up Firecrawl website URLs in goose?

About two minutes: add the extension, then sign in.

Does goose wait while Firecrawl website URLs runs?

Firecrawl website URLs runs in the background. goose starts the run, checks on it and gives you the json when it's ready.

Where do the URLs come from, and is coverage complete?

Firecrawl primarily uses the website's sitemap, supplemented by search engine results and previously crawled pages. Map prioritizes speed and can miss links, so the result is a discovered URL inventory rather than a complete site audit.

How fresh is the URL list?

Sitemap data can be cached for up to seven days. Bypassing the sitemap cache helps when a sitemap has recently changed, but discovery also draws on previously crawled pages and search engine results.

How many URLs can one call return?

Firecrawl Map supports up to 100,000 returned links per call. That ceiling does not guarantee that every URL on a large website will be discovered.

Does the result include page content?

Map returns discovered URLs, with titles and descriptions when available. It does not scrape the pages' contents, so use the results to select pages for further reading.

Can I tell when a page was last updated?

Map does not return page modification dates. Comparing repeated URL lists can identify newly discovered URLs, but it cannot establish when those pages were published or edited.

Get started

Start using Firecrawl website URLs in goose

One Spiral account for every tool your agent needs. No subscription; you pay per run.

Paste this into goose:

Set up goose

set up https://spiral.ai/SKILL.md
Are you an AI agent? See setup options