Firecrawl's Crawl Limit, Credit Pre-Check and Webhooks for AI Agents: What the Docs Say
Last fact-checked: November 17, 2026
Note on experience: we have not run Firecrawl crawls, tested its webhooks or measured its credit use. Pages were read through a web-fetch tool that summarises them, on November 17, 2026. Check the live docs before relying on any statement. "Not stated" means the page we read did not say.
An AI agent that is told to "crawl this docs site" can turn one call into thousands of billable pages. Firecrawl † documents a few controls that decide how far one crawl can go and how you find out what happened: a limit, a credit check before the crawl starts, async job status, and webhooks. This article lays out what the docs state, where two pages differ, and what they leave out. For credit prices per feature, see our earlier article on Firecrawl credits; here we only use the numbers the crawl page itself gives.
1. The one cap the docs describe: limit
The crawl page describes limit as the "maximum number of pages to crawl", and the API reference lists its default as 10000. So a crawl request that omits limit is allowed to run to 10,000 pages. The crawl page says each crawled page consumes 1 credit, with JSON mode adding 4 credits per page and PDF parsing costing 1 credit per PDF page. By those numbers, a default-limit crawl of a large site is a five-figure credit event if every page is returned.
Other parameters in the API reference narrow what gets crawled rather than what it may cost: includePaths and excludePaths (arrays of regex strings, up to 1,000 each), maxDiscoveryDepth, allowExternalLinks (default false), allowSubdomains (default false) and crawlEntireDomain (default false). The page describes maxDiscoveryDepth as hops from the root URL, not path segments; pages at the maximum depth are still scraped but their links are not followed. The default for maxDiscoveryDepth is not stated.
No credit-cap parameter is documented on the crawl pages we read. The only cost control described is limit, plus the pre-check below.
2. A credit check before the crawl starts
The crawl page says the endpoint checks that your remaining credits cover the limit before starting; if they do not, it returns a 402 (Payment Required), and it advises setting a lower limit to avoid this. The API reference lists 402 as a response code of the crawl endpoint.
What this means for an agent, as our reading rather than vendor guidance: a 402 on a crawl can come from a high limit against a low balance, even when the site itself is small. Lowering limit is the documented remedy. Whether the check reserves credits, and whether the check compares against limit alone or against limit times per-page add-ons, is not stated.
3. Async status: what to poll and what not to trust
POST /v2/crawl returns a job ID; you poll GET /v2/crawl/{id}. The crawl page lists the statuses scraping, completed, failed and cancelled; the status endpoint's field list shows three values and mentions cancelled only in the description of completedAt, so the two pages are not fully consistent. The status response includes total, completed, creditsUsed and expiresAt.
Three details matter to an agent loop:
completed == totaldoes not mean every discovered page succeeded. The status page says to use the Get Crawl Errors endpoint to find pages that were attempted but not returned.totalis described as excluding failed pages.nextis not an exit condition. Responses over 10 MB return anextURL for the following data; the crawl page warns thatnextcan appear on failed or cancelled crawls.- Results expire. The crawl page says job results are available through the API for 24 hours after completion, then remain viewable in the activity logs. Responses include
expiresAt.
4. Cancelling a crawl
The crawl page lists cancelled as a status but describes no way to cancel. The API reference has a separate cancel endpoint: DELETE /crawl/{id}, returning 200 with "status": "cancelled" on success and 404 if the job is not found. Whether credits already consumed are refunded is not stated.
5. Webhooks
- Events: for crawls,
crawl.started,crawl.pageandcrawl.completed; the API reference'seventsoption listscompleted,page,failedandstarted. Webhooks are also listed for batch scrape, extract, agent and monitor operations. The default is all events. - Configuration:
url(required, HTTPS), optionalheaders,metadata(custom data included in payloads) andevents. - Delivery: your endpoint must return a 2xx within 10 seconds. On timeout, non-2xx or network error, Firecrawl retries at 1, 5 and 15 minutes after the failure; after three failed retries the webhook is marked failed and no further attempts are made.
- Signature: the crawl page says requests carry an
X-Firecrawl-Signatureheader with an HMAC-SHA256 signature (formatsha256=...) that you should verify with a timing-safe comparison before processing, using a secret from the Advanced tab of account settings. The webhooks overview page we read says signature verification is not stated. - Payload: common fields are
success,type,id(the job ID),webhookId(unique per delivery),data,metadata, anderrorwhensuccessisfalse. Acrawl.pageevent carries the page markdown and page metadata includingstatusCode,scrapeIdandcreditsUsed.
Where the pages differ. The crawl page says a page Firecrawl could not scrape arrives with success: false and an error message. The events page we read says how failed pages appear is not stated. If your agent treats a missing crawl.page as "page skipped", check the live docs and the Get Crawl Errors endpoint first.
6. A cautious checklist (our reading, not vendor guidance)
- Always set
limitexplicitly; do not rely on the 10,000 default. - Constrain with
includePaths,excludePathsandmaxDiscoveryDepthbefore the first run. - Treat a
402on crawl start as "lower the limit or top up", not as a transient error to retry blindly. - Store the job ID, and read
creditsUsedfrom the status response rather than computing it yourself. - Fetch results inside 24 hours.
- Make webhook handlers fast (10-second limit) and idempotent, since
webhookIdidentifies each delivery and retries are documented. - Do not use
completed == totalor the absence ofnextas proof that everything succeeded.
7. What we could not confirm
- Any parameter that caps total credits for a crawl.
- Whether the pre-check reserves credits, and how it accounts for add-ons.
- Whether credits for pages already crawled are refunded on cancel.
- Whether failed pages in a crawl are charged.
- The default for
maxDiscoveryDepth. - A maximum number of URLs per batch scrape.
- Whether the signature header is documented consistently across pages.
† Referral link (Firecrawl). Related: ScraperAPI's max_cost and async jobs, retry budgets, burst limits.
Sources (checked November 17, 2026)
- Firecrawl docs: Crawl
- Webhooks overview and event types
- Firecrawl API reference: crawl start, crawl status, crawl cancel; Batch scrape page
Not sure which article you need? See the task-by-task chooser.