Firecrawl API SOP

Appearance

Firecrawl + Google Agent Development Kit (ADK) ​

Updated Feb 2026

Integrate Firecrawl with Google's Agent Development Kit (ADK) to build AI agents with web scraping capabilities through the Model Context Protocol (MCP).

Source: docs.firecrawl.dev/developer-guides/llm-sdks-and-frameworks/google-adk

Overview ​

Firecrawl provides an MCP server that integrates with Google's ADK, enabling agents to scrape, crawl, and extract structured data from any website. The integration supports both cloud-based and self-hosted Firecrawl instances with streamable HTTP.

Features ​

  • Web scraping, crawling, and content discovery from any website
  • Advanced search capabilities and intelligent content extraction
  • Deep research and high-volume batch scraping
  • Flexible deployment (cloud or self-hosted)
  • Streamable HTTP support for optimal performance

Prerequisites ​

Setup ​

python
from google.adk.agents.llm_agent import Agent
from google.adk.tools.mcp_tool.mcp_session_manager import StreamableHTTPServerParams
from google.adk.tools.mcp_tool.mcp_toolset import MCPToolset

FIRECRAWL_API_KEY = "YOUR-API-KEY"

root_agent = Agent(
    model="gemini-2.5-pro",
    name="firecrawl_agent",
    description='A helpful assistant for scraping websites with Firecrawl',
    instruction='Help the user search for website content',
    tools=[
        MCPToolset(
            connection_params=StreamableHTTPServerParams(
                url=f"https://mcp.firecrawl.dev/{FIRECRAWL_API_KEY}/v2/mcp",
            ),
        )
    ],
)

Local MCP Server ​

If self-hosting, replace the URL with your local Firecrawl MCP server endpoint.

Available Tools ​

ToolNameDescription
Scrapefirecrawl_scrapeScrape content from a single URL
Batch Scrapefirecrawl_batch_scrapeScrape multiple URLs with rate limiting and parallel processing
Check Batchfirecrawl_check_batch_statusCheck the status of a batch operation
Mapfirecrawl_mapDiscover all indexed URLs on a site
Searchfirecrawl_searchSearch the web and optionally extract content
Crawlfirecrawl_crawlStart an async crawl with advanced options
Check Crawlfirecrawl_check_crawl_statusCheck the status of a crawl job
Extractfirecrawl_extractExtract structured data from web pages using LLM

Configuration ​

Required ​

VariableDescription
FIRECRAWL_API_KEYYour Firecrawl API key (required for cloud, optional for self-hosted)

Optional ​

VariableDefaultDescription
FIRECRAWL_API_URLCloud APICustom API endpoint for self-hosted instances
FIRECRAWL_RETRY_MAX_ATTEMPTS3Maximum retry attempts
FIRECRAWL_RETRY_INITIAL_DELAY1000Initial delay in ms
FIRECRAWL_RETRY_MAX_DELAY10000Maximum delay in ms
FIRECRAWL_RETRY_BACKOFF_FACTOR2Exponential backoff multiplier
FIRECRAWL_CREDIT_WARNING_THRESHOLD1000Credit warning threshold
FIRECRAWL_CREDIT_CRITICAL_THRESHOLD100Credit critical threshold

Example: Web Research Agent ​

python
from google.adk.agents.llm_agent import Agent
from google.adk.tools.mcp_tool.mcp_session_manager import StreamableHTTPServerParams
from google.adk.tools.mcp_tool.mcp_toolset import MCPToolset

FIRECRAWL_API_KEY = "YOUR-API-KEY"

research_agent = Agent(
    model="gemini-2.5-pro",
    name="research_agent",
    description='An AI agent that researches topics by scraping and analyzing web content',
    instruction='''You are a research assistant. When given a topic or question:
    1. Use the search tool to find relevant websites
    2. Scrape the most relevant pages for detailed information
    3. Extract structured data when needed
    4. Provide comprehensive, well-sourced answers''',
    tools=[
        MCPToolset(
            connection_params=StreamableHTTPServerParams(
                url=f"https://mcp.firecrawl.dev/{FIRECRAWL_API_KEY}/v2/mcp",
            ),
        )
    ],
)

response = research_agent.run("What are the latest features in Python 3.13?")
print(response)

Best Practices ​

  1. Use the right tool for the job:
    • firecrawl_search -- find relevant pages first
    • firecrawl_scrape -- single pages
    • firecrawl_batch_scrape -- multiple known URLs
    • firecrawl_crawl -- discover and scrape entire sites
  2. Monitor your usage: Configure credit thresholds to avoid unexpected costs
  3. Handle errors gracefully: Configure retry settings based on your use case
  4. Optimize performance: Use batch operations for multiple URLs

Further Reading ​