Firecrawl API SOP

Appearance

SOP 001: Basic Web Scraping ​

Fresh 🌱

Extract clean markdown content from any single URL.

Overview ​

Prerequisites ​

  • ✓ Firecrawl API key (get one)
  • ✓ Python 3.8+ or Node.js 16+
  • ✓ Internet connection

Step-by-Step ​

Step 1: Install SDK ​

Python:

bash
pip install firecrawl-py

Node.js:

bash
npm install @mendable/firecrawl-js

Step 2: Initialize Client ​

Python:

python
from firecrawl import Firecrawl

firecrawl = Firecrawl(api_key="fc-YOUR-API-KEY")

Node.js:

javascript
import Firecrawl from '@mendable/firecrawl-js';

const firecrawl = new Firecrawl({ apiKey: 'fc-YOUR-API-KEY' });

Step 3: Scrape URL ​

Python:

python
result = firecrawl.scrape("https://example.com")
print(result['markdown'])

Node.js:

javascript
const result = await firecrawl.scrape('https://example.com');
console.log(result.markdown);

Step 4: Handle Response ​

python
if result['success']:
    markdown_content = result['markdown']
    title = result['metadata']['title']
    source_url = result['metadata']['sourceURL']

    print(f"Title: {title}")
    print(f"Content length: {len(markdown_content)} chars")
else:
    print("Scrape failed")

Step 5: Save Results ​

Python:

python
with open("output.md", "w") as f:
    f.write(result['markdown'])
print("Saved to output.md")

Verification Checklist ​

  • [ ] API key is valid
  • [ ] URL is accessible
  • [ ] Result contains markdown
  • [ ] Metadata extracted correctly
  • [ ] File saved successfully

Troubleshooting ​

Error: "Invalid API key" ​

  • Check API key is correct (starts with fc-)
  • Verify key hasn't been revoked
  • Get new key from dashboard

Error: "HTTP 404" ​

  • Verify URL is correct and accessible
  • Check URL is public (not behind login)
  • Try in browser first

Error: "Timeout" ​

  • URL might be slow to load
  • Try setting timeout: firecrawl.scrape(url, timeout=30000)

Empty markdown ​

  • Page might have no main content
  • Try with onlyMainContent=False

Next Steps ​

Cost: 1 credit per scrape