Fast Scraping Mode

Updated Feb 2026

Firecrawl's caching system delivers previously scraped pages instantly, providing up to 5x faster response times. Control cache freshness with the maxAge parameter.

Overview

Every page Firecrawl scrapes is cached. When you request a page that was recently scraped, the cached version is returned immediately instead of running the full scraping pipeline. This is controlled by the maxAge parameter.

Default Behavior

• Default cache window: 2 days (172,800,000 milliseconds)
• Credit cost: 1 credit per page regardless of cache status
• Speed improvement: Up to 500% faster for cached pages

If a cached version exists and falls within the maxAge window, it is returned immediately. If no cache exists or the cache is older than maxAge, a fresh scrape runs and the cache updates.

Configuration

Parameters

Parameter	Type	Default	Description
maxAge	number	172800000 (2 days)	Cache freshness window in milliseconds. Set to 0 to bypass cache.
storeInCache	boolean	true	Whether to store results in cache. Set false to skip caching.

Common maxAge Values

Freshness	Milliseconds	Constant
5 minutes	300000	--
1 hour	3600000	--
1 day	86400000	--
2 days (default)	172800000	--
1 week	604800000	--
Always fresh	0	Bypass cache

Usage Examples

Python

from firecrawl import Firecrawl

firecrawl = Firecrawl(api_key="fc-YOUR-API-KEY")

# Use default cache (2 days)
result = firecrawl.scrape("https://example.com")

# Accept cache if less than 10 minutes old
result = firecrawl.scrape(
    "https://example.com",
    max_age=600000  # 10 minutes in ms
)

# Accept cache if less than 1 hour old
result = firecrawl.scrape(
    "https://example.com",
    max_age=3600000  # 1 hour in ms
)

# Always get fresh content (bypass cache)
result = firecrawl.scrape(
    "https://example.com",
    max_age=0
)

# Scrape but don't store in cache
result = firecrawl.scrape(
    "https://example.com",
    store_in_cache=False
)

Node.js

import Firecrawl from '@mendable/firecrawl-js';

const firecrawl = new Firecrawl({ apiKey: "fc-YOUR-API-KEY" });

// Use default cache (2 days)
const doc = await firecrawl.scrape("https://example.com");

// Accept cache if less than 1 hour old
const doc = await firecrawl.scrape("https://example.com", {
  formats: ["markdown"],
  maxAge: 3600000
});

// Always fresh
const doc = await firecrawl.scrape("https://example.com", {
  formats: ["markdown"],
  maxAge: 0
});

// Don't store in cache
const doc = await firecrawl.scrape("https://example.com", {
  formats: ["markdown"],
  storeInCache: false
});

cURL

# Cache up to 1 hour
curl -X POST https://api.firecrawl.dev/v2/scrape \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer fc-YOUR-API-KEY' \
  -d '{
    "url": "https://example.com",
    "formats": ["markdown"],
    "maxAge": 3600000
  }'

# Bypass cache entirely
curl -X POST https://api.firecrawl.dev/v2/scrape \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer fc-YOUR-API-KEY' \
  -d '{
    "url": "https://example.com",
    "formats": ["markdown"],
    "maxAge": 0
  }'

Using with Crawl

Apply caching to recursive crawls via scrapeOptions:

results = firecrawl.crawl(
    "https://docs.example.com",
    limit=100,
    scrape_options={
        "formats": ["markdown"],
        "maxAge": 86400000  # 1 day cache for docs
    }
)

const results = await firecrawl.crawl("https://docs.example.com", {
  limit: 100,
  scrapeOptions: {
    formats: ["markdown"],
    maxAge: 86400000
  }
});

When to Use Fast Scraping

Good Candidates (Use Cache)

Content Type	Suggested maxAge	Why
Documentation	1 week (604800000)	Rarely changes
Blog articles	1 day (86400000)	Updated infrequently
Product pages	1 hour (3600000)	Moderate update frequency
Company info pages	1 week (604800000)	Rarely changes
Knowledge bases	1 day (86400000)	Periodic updates
Development/testing	1 week (604800000)	Content irrelevant, speed matters

Bad Candidates (Bypass Cache)

Content Type	Use	Why
Stock prices	maxAge: 0	Real-time data required
Live scores	maxAge: 0	Updates every few seconds
Breaking news	maxAge: 0 or 300000	Must be current
Auction pages	maxAge: 0	Prices change rapidly
Social media feeds	maxAge: 0	Constantly updating

Performance Tips

Bulk Processing

For large batch operations on relatively static content, caching provides the biggest benefit:

# Scrape 500 documentation pages — cache for 1 week
urls = [f"https://docs.example.com/page/{i}" for i in range(500)]

results = firecrawl.batch_scrape(
    urls,
    formats=["markdown"],
    maxAge=604800000  # 1 week
)

If you previously scraped these pages within the week, most will return instantly from cache.

Change Tracking with Cache

When using change tracking, caching interacts with it:

• Cache returns the stored version (no new scrape occurs)
• Change tracking compares against the previous version regardless of cache
• Set maxAge: 0 when you specifically need fresh content for comparison

# Fresh scrape for change tracking comparison
result = firecrawl.scrape(
    "https://competitor.com/pricing",
    formats=["markdown", "changeTracking"],
    max_age=0  # Always fresh for accurate comparison
)

Cost

Caching does not reduce credit cost -- each request still costs 1 credit per page regardless of whether cached content is returned. The benefit is speed, not cost savings.

Scenario	Credits	Speed
Fresh scrape	1	Standard (2-10s)
Cached return	1	Instant (<1s)
Cache bypass (maxAge: 0)	1	Standard (2-10s)

When maxAge=0 Makes Sense

Setting maxAge=0 forces a fresh scrape every time. This is slower and more likely to encounter rate limits. Only use it when you genuinely need the latest content.

Related Pages

• Scrape -- All scrape parameters and format options
• Batch Scrape -- Bulk operations with caching
• Crawl -- Recursive crawling
• Change Tracking -- Monitor content changes over time
• Pricing -- Credit cost breakdown

---

Next: LLM Extract -->