Python SDK

Updated Feb 2026

The official Python SDK for Firecrawl. Supports both synchronous and asynchronous usage, all v2 API features, and auto-pagination out of the box.


Package	`firecrawl-py`
PyPI	pypi.org/project/firecrawl-py
API Versions	v1, v2
Source	github.com/mendableai/firecrawl

Installation

bash

pip install firecrawl-py

Initialization

python

from firecrawl import Firecrawl

firecrawl = Firecrawl(api_key="fc-YOUR-API-KEY")

The API key can also be set via the FIRECRAWL_API_KEY environment variable, in which case the api_key parameter can be omitted.

Methods

scrape()

Scrapes a single URL and returns the content in the requested formats. See Scrape feature docs for full parameter details.

Signature:

python

firecrawl.scrape(url, formats=["markdown"], **kwargs)

Parameters:

Parameter	Type	Required	Description
`url`	`str`	Yes	The URL to scrape
`formats`	`list[str]`	No	Output formats: `"markdown"`, `"html"`, `"rawHtml"`, `"links"`, `"screenshot"`, `"json"`

Example:

python

result = firecrawl.scrape("https://firecrawl.dev", formats=["markdown", "html"])
print(result["markdown"])
print(result["html"])

search()

Performs a web search and optionally scrapes the results. See Search feature docs for full parameter details.

Signature:

python

firecrawl.search(query, limit=5, scrape_options=None, **kwargs)

Parameters:

Parameter	Type	Required	Description
`query`	`str`	Yes	Search query string
`limit`	`int`	No	Maximum number of results (default: 5)
`scrape_options`	`dict`	No	Options for scraping results, e.g., `{"formats": ["markdown"]}`
`sources`	`str`	No	Result types: `"web"`, `"news"`, `"images"`
`categories`	`str`	No	Filter: `"github"`, `"research"`, `"pdf"`
`location`	`str`	No	Geographic targeting, e.g., `"Germany"`
`tbs`	`str`	No	Time filter: `"qdr:h"` (hour), `"qdr:d"` (day), `"qdr:w"` (week), `"qdr:m"` (month)
`timeout`	`int`	No	Timeout in milliseconds

Example:

python

results = firecrawl.search(
    "firecrawl web scraping",
    limit=3,
    scrape_options={"formats": ["markdown", "links"]}
)

for result in results.get("web", []):
    print(result["title"], result["url"])

crawl()

Crawls a website starting from a URL. This is a blocking call that waits for the crawl to complete. Auto-paginates results by default. See Crawl feature docs for full parameter details.

Signature:

python

firecrawl.crawl(url, limit=100, poll_interval=5, timeout=300, scrape_options=None, sitemap=None, **kwargs)

Parameters:

Parameter	Type	Required	Description
`url`	`str`	Yes	Starting URL
`limit`	`int`	No	Maximum pages to crawl
`poll_interval`	`int`	No	Polling interval in seconds
`timeout`	`int`	No	Maximum wait time in seconds
`scrape_options`	`dict`	No	Scrape settings, e.g., `{"formats": ["markdown"]}`
`sitemap`	`str`	No	Sitemap mode: `"only"` to crawl sitemap URLs exclusively
`pagination_config`	`PaginationConfig`	No	Controls auto-pagination behavior

Example:

python

job = firecrawl.crawl(
    url="https://docs.firecrawl.dev",
    limit=5,
    poll_interval=1,
    timeout=120
)
print(job.status)
for doc in job.data:
    print(doc.metadata.source_url)

Sitemap-only crawl:

python

job = firecrawl.crawl(
    url="https://docs.firecrawl.dev",
    sitemap="only",
    limit=25
)
print(job.status, len(job.data))

start_crawl()

Initiates a non-blocking crawl and returns a job ID for status polling.

Signature:

python

firecrawl.start_crawl(url, limit=100, **kwargs)

Parameters:

Parameter	Type	Required	Description
`url`	`str`	Yes	Starting URL
`limit`	`int`	No	Maximum pages to crawl

Example:

python

job = firecrawl.start_crawl(url="https://docs.firecrawl.dev", limit=10)
print(job.id)  # Use this ID to check status

get_crawl_status()

Retrieves the current status and results of a crawl job.

Signature:

python

firecrawl.get_crawl_status(crawl_id, pagination_config=None)

Parameters:

Parameter	Type	Required	Description
`crawl_id`	`str`	Yes	Job ID from `start_crawl()`
`pagination_config`	`PaginationConfig`	No	Controls pagination behavior

Example:

python

status = firecrawl.get_crawl_status("<crawl-id>")
print(status.status)
print(status.completed, "/", status.total)

get_crawl_status_page()

Fetches the next page of results using the opaque next URL from a previous response. Used for manual pagination.

Signature:

python

firecrawl.get_crawl_status_page(next_url)

Example:

python

while status.next:
    status = firecrawl.get_crawl_status_page(status.next)
    print("Next page:", len(status.data), "docs")

cancel_crawl()

Cancels an active crawl job.

Signature:

python

firecrawl.cancel_crawl(crawl_id)

Example:

python

ok = firecrawl.cancel_crawl("<crawl-id>")
print("Cancelled:", ok)

map()

Discovers all URLs on a website without scraping their content. See Map feature docs for full parameter details.

Signature:

python

firecrawl.map(url, limit=None)

Parameters:

Parameter	Type	Required	Description
`url`	`str`	Yes	Target URL
`limit`	`int`	No	Maximum URLs to discover

Example:

python

res = firecrawl.map(url="https://firecrawl.dev", limit=10)
for link in res.links:
    print(link)

batch_scrape()

Scrapes multiple URLs in a single batch operation. Blocking -- waits for all URLs to complete. Auto-paginates results by default.

Signature:

python

firecrawl.batch_scrape(urls, formats=["markdown"], poll_interval=1, timeout=60, pagination_config=None)

Parameters:

Parameter	Type	Required	Description
`urls`	`list[str]`	Yes	URLs to scrape
`formats`	`list[str]`	No	Output formats
`poll_interval`	`int`	No	Polling interval in seconds
`timeout`	`int`	No	Maximum wait time in seconds
`pagination_config`	`PaginationConfig`	No	Controls pagination

Example:

python

job = firecrawl.batch_scrape(
    [
        "https://firecrawl.dev",
        "https://docs.firecrawl.dev",
    ],
    formats=["markdown"],
    poll_interval=1,
    timeout=60
)
print(job.status, job.completed, job.total)

start_batch_scrape()

Initiates a non-blocking batch scrape and returns a job ID.

Signature:

python

firecrawl.start_batch_scrape(urls, **kwargs)

Example:

python

batch_job = firecrawl.start_batch_scrape([
    "https://firecrawl.dev",
    "https://docs.firecrawl.dev",
])
print(batch_job.id)

get_batch_scrape_status()

Checks the status of a batch scrape job.

Signature:

python

firecrawl.get_batch_scrape_status(job_id, pagination_config=None)

Example:

python

from firecrawl.v2.types import PaginationConfig

status = firecrawl.get_batch_scrape_status(
    batch_job.id,
    pagination_config=PaginationConfig(auto_paginate=False)
)
print(status.status, status.completed, status.total)

extract()

Extracts structured data from one or more URLs using a schema or natural language prompt. Supports wildcard URL patterns. See Extract feature docs for full parameter details.

Signature:

python

firecrawl.extract(urls, prompt=None, schema=None, enable_web_search=False)

Parameters:

Parameter	Type	Required	Description
`urls`	`list[str]`	Yes	URLs to extract from (supports wildcards like `example.com/*`)
`prompt`	`str`	No	Natural language description of the data to extract
`schema`	`dict`	No	JSON Schema defining the output structure
`enable_web_search`	`bool`	No	Allow extraction to follow external links

Example:

python

result = firecrawl.extract(
    urls=["https://firecrawl.dev/*"],
    prompt="Extract the company name, description, and pricing plans",
    schema={
        "type": "object",
        "properties": {
            "company": {"type": "string"},
            "description": {"type": "string"},
            "pricing_plans": {
                "type": "array",
                "items": {
                    "type": "object",
                    "properties": {
                        "name": {"type": "string"},
                        "price": {"type": "string"}
                    }
                }
            }
        }
    }
)
print(result)

agent()

Autonomous web data gathering agent that searches, navigates, and extracts data without requiring specific URLs. See Agent feature docs for full parameter details.

Signature:

python

firecrawl.agent(prompt, model="spark-1-mini", urls=None, schema=None, max_credits=2500)

Parameters:

Parameter	Type	Required	Description
`prompt`	`str`	Yes	Natural language description of the data to gather (max 10,000 chars)
`model`	`str`	No	`"spark-1-mini"` (default, 60% cheaper) or `"spark-1-pro"`
`urls`	`list[str]`	No	Optional URLs to focus extraction scope
`schema`	`BaseModel` or `dict`	No	Pydantic model or JSON Schema for structured output
`max_credits`	`int`	No	Credit spending limit (default: 2,500)

Example with Pydantic schema:

python

from pydantic import BaseModel, Field
from typing import List, Optional

class Founder(BaseModel):
    name: str = Field(description="Full name of the founder")
    role: Optional[str] = Field(None, description="Role or position")

class FoundersSchema(BaseModel):
    founders: List[Founder]

result = firecrawl.agent(
    prompt="Find the founders of Firecrawl",
    schema=FoundersSchema,
    model="spark-1-mini",
    max_credits=100
)
print(result)

watcher()

Subscribes to real-time crawl updates via an async generator. Must be used with AsyncFirecrawl.

Signature:

python

async for snapshot in firecrawl.watcher(job_id, kind="crawl", poll_interval=2, timeout=120):
    ...

Parameters:

Parameter	Type	Required	Description
`job_id`	`str`	Yes	Crawl job ID
`kind`	`str`	No	Job type, e.g., `"crawl"`
`poll_interval`	`int`	No	Polling interval in seconds
`timeout`	`int`	No	Maximum wait time in seconds

Example:

python

async for snapshot in firecrawl.watcher(started.id, kind="crawl", poll_interval=2, timeout=120):
    if snapshot.status == "completed":
        print("DONE", snapshot.status)
        for doc in snapshot.data:
            print("DOC", doc.metadata.source_url if doc.metadata else None)

Browser Methods

Firecrawl provides cloud browser sessions that you can control programmatically. See Browser feature docs for full details.

browser()

Creates a cloud browser session.

Parameters:

Parameter	Type	Required	Description
`ttl`	`int`	No	Session time-to-live in seconds
`profile`	`dict`	No	Browser profile with `name` and `save_changes` keys

Example:

python

session = firecrawl.browser(
    ttl=300,
    profile={
        "name": "my-profile",
        "save_changes": True,
    }
)
print(session.id)
print(session.cdp_url)
print(session.live_view_url)

browser_execute()

Executes code within a browser session.

Parameters:

Parameter	Type	Required	Description
`session_id`	`str`	Yes	Session ID
`code`	`str`	Yes	Code to execute
`language`	`str`	No	`"python"` or `"node"`

Python example:

python

result = firecrawl.browser_execute(
    session.id,
    code='await page.goto("https://news.ycombinator.com")\ntitle = await page.title()\nprint(title)',
    language="python",
)
print(result.result)  # "Hacker News"

JavaScript example:

python

result = firecrawl.browser_execute(
    session.id,
    code='await page.goto("https://example.com"); const t = await page.title(); console.log(t);',
    language="node",
)

list_browsers()

Lists active browser sessions.

python

sessions = firecrawl.list_browsers(status="active")
for s in sessions.sessions:
    print(s.id, s.status, s.created_at)

delete_browser()

Closes a browser session.

python

firecrawl.delete_browser(session.id)

Playwright CDP Integration

You can connect to a Firecrawl browser session using Playwright's CDP support:

python

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.connect_over_cdp(session.cdp_url)
    context = browser.contexts[0]
    page = context.pages[0] if context.pages else context.new_page()
    page.goto("https://example.com")
    print(page.title())
    browser.close()

PaginationConfig

Controls how auto-pagination works for crawl and batch scrape operations.

Import:

python

from firecrawl.v2.types import PaginationConfig

Options:

Option	Type	Default	Description
`auto_paginate`	`bool`	`True`	Automatically fetch all pages and aggregate results
`max_pages`	`int`	`None`	Stop after this many pages (requires `auto_paginate=True`)
`max_results`	`int`	`None`	Stop after collecting this many documents
`max_wait_time`	`int`	`None`	Stop after this many seconds

Example:

python

status = firecrawl.get_crawl_status(
    crawl_job.id,
    pagination_config=PaginationConfig(
        max_pages=2,
        max_results=50,
        max_wait_time=15
    )
)

Async Support

The AsyncFirecrawl class mirrors the synchronous Firecrawl class with non-blocking async methods. All methods return awaitables.

Import and initialization:

python

import asyncio
from firecrawl import AsyncFirecrawl

async def main():
    firecrawl = AsyncFirecrawl(api_key="fc-YOUR-API-KEY")

    # Scrape
    doc = await firecrawl.scrape("https://firecrawl.dev", formats=["markdown"])
    print(doc.get("markdown"))

    # Search
    results = await firecrawl.search("firecrawl", limit=2)
    print(results.get("web", []))

    # Crawl
    started = await firecrawl.start_crawl("https://docs.firecrawl.dev", limit=3)
    status = await firecrawl.get_crawl_status(started.id)
    print(status.status)

    # Batch scrape
    job = await firecrawl.batch_scrape(
        ["https://firecrawl.dev", "https://docs.firecrawl.dev"],
        formats=["markdown"],
        poll_interval=1,
        timeout=60
    )
    print(job.status)

asyncio.run(main())

Async Watcher

The watcher() method works as an async generator for real-time updates:

python

async for snapshot in firecrawl.watcher(started.id, kind="crawl", poll_interval=2, timeout=120):
    if snapshot.status == "completed":
        for doc in snapshot.data:
            print(doc.metadata.source_url)
        break

Error Handling

The SDK raises descriptive exceptions for API errors. Wrap calls in try/except blocks to handle failures gracefully:

python

try:
    result = firecrawl.scrape("https://example.com", formats=["markdown"])
except Exception as e:
    print(f"Firecrawl error: {e}")

Common error scenarios:

Scenario	Cause
Invalid API key	Key is missing, expired, or malformed
Rate limit exceeded	Too many requests in a short period
URL not reachable	Target URL is down or blocked
Timeout	Operation exceeded the configured timeout
Credit limit reached	Account or job credit limit exceeded

Response Structures

Scrape Response

A dictionary with keys matching the requested formats:

python

{
    "markdown": "# Page Title\n...",
    "html": "<h1>Page Title</h1>...",
    "metadata": {
        "title": "Page Title",
        "source_url": "https://example.com",
        ...
    }
}

Crawl Job Response

An object with the following attributes:

Attribute	Type	Description
`status`	`str`	Job status: `"completed"`, `"failed"`, `"scraping"`
`data`	`list`	List of scraped documents
`completed`	`int`	Number of pages completed
`total`	`int`	Total pages to crawl
`next`	`str`	Pagination URL (if more data available)

Browser Session Response

Attribute	Type	Description
`id`	`str`	Session identifier
`cdp_url`	`str`	Chrome DevTools Protocol WebSocket URL
`live_view_url`	`str`	URL for live monitoring

SDKs Overview -- Compare all available SDKs
Node.js SDK -- JavaScript/TypeScript SDK
CLI Reference -- Command-line interface
Scrape -- Single-page extraction details
Crawl -- Multi-page crawling details
Search -- Web search details
Extract -- Structured data extraction
Agent -- Autonomous data gathering

Python SDK ​

Installation ​

Initialization ​

Methods ​

scrape() ​

search() ​

crawl() ​

start_crawl() ​

get_crawl_status() ​

get_crawl_status_page() ​

cancel_crawl() ​

map() ​

batch_scrape() ​

start_batch_scrape() ​

get_batch_scrape_status() ​

extract() ​

agent() ​

watcher() ​

Browser Methods ​

browser() ​

browser_execute() ​

list_browsers() ​

delete_browser() ​

Playwright CDP Integration ​

PaginationConfig ​

Async Support ​

Async Watcher ​

Error Handling ​

Response Structures ​

Scrape Response ​

Crawl Job Response ​

Browser Session Response ​

Related Pages ​

Python SDK

Installation

Initialization

Methods

scrape()

search()

crawl()

start_crawl()

get_crawl_status()

get_crawl_status_page()

cancel_crawl()

map()

batch_scrape()

start_batch_scrape()

get_batch_scrape_status()

extract()

agent()

watcher()

Browser Methods

browser()

browser_execute()

list_browsers()

delete_browser()

Playwright CDP Integration

PaginationConfig

Async Support

Async Watcher

Error Handling

Response Structures

Scrape Response

Crawl Job Response

Browser Session Response

Related Pages