Firecrawl API SOP

Appearance

Workflow: AI Integration ​

Fresh 🌱

Feed website content to LLMs for analysis and processing.

Architecture ​

Implementation ​

1. Scrape Content ​

python
from firecrawl import Firecrawl

firecrawl = Firecrawl(api_key="fc-YOUR-API-KEY")

result = firecrawl.scrape("https://example.com")
content = result['markdown']

2. Prepare for LLM ​

python
# Clean up content
def prepare_for_llm(markdown):
    # Remove extra whitespace
    lines = [line.strip() for line in markdown.split('\n')]
    # Remove empty lines
    lines = [line for line in lines if line]
    # Rejoin
    return '\n'.join(lines[:1000])  # Limit to first 1000 lines

prepared_content = prepare_for_llm(content)

3. Process with LLM ​

python
from anthropic import Anthropic

client = Anthropic()

response = client.messages.create(
    model="claude-opus-4-5-20251101",
    max_tokens=1024,
    messages=[
        {
            "role": "user",
            "content": f"Summarize this content in 3 bullet points:\n\n{prepared_content}"
        }
    ]
)

summary = response.content[0].text

4. Store Results ​

python
import json
from datetime import datetime

result_data = {
    "url": "https://example.com",
    "original_content_length": len(content),
    "summary": summary,
    "processed_at": datetime.now().isoformat()
}

with open("summary.json", "w") as f:
    json.dump(result_data, f, indent=2)

Full Example ​

python
from firecrawl import Firecrawl
from anthropic import Anthropic
import json

firecrawl = Firecrawl(api_key="fc-YOUR-API-KEY")
client = Anthropic()

# Scrape content
result = firecrawl.scrape("https://example.com")
content = result['markdown'][:5000]  # Limit size

# Analyze with LLM
response = client.messages.create(
    model="claude-opus-4-5-20251101",
    max_tokens=512,
    system="You are a helpful content analyst. Analyze web content and extract key insights.",
    messages=[
        {
            "role": "user",
            "content": f"Please analyze this content:\n\n{content}"
        }
    ]
)

analysis = response.content[0].text

# Store results
output = {
    "source_url": result['metadata']['sourceURL'],
    "title": result['metadata']['title'],
    "analysis": analysis
}

print(json.dumps(output, indent=2))

Use Cases ​

1. Content Summarization ​

python
"Summarize this article in 3-5 bullet points"

2. Sentiment Analysis ​

python
"Analyze the sentiment and tone of this text"

3. Key Information Extraction ​

python
"Extract the following from this text: pricing, features, target audience"

4. Comparison ​

python
"Compare these two products based on features and pricing"

Cost Considerations ​

  • Firecrawl: 1 credit per URL
  • LLM: Pay per tokens used
  • Optimization: Limit content size, cache results

← Back to Workflows