Firecrawl API SOP

Appearance

Rails ​

Use Firecrawl with Ruby on Rails to search, scrape, and interact with web data using the REST API.

Prerequisites ​

  • Ruby 3.0+ and Rails 7+
  • A Firecrawl API key, get one free

Configuration ​

Add your API key to your Rails credentials or environment:

bash
export FIRECRAWL_API_KEY=fc-YOUR-API-KEY

Create a service ​

Create app/services/firecrawl_service.rb:

ruby
require "net/http"
require "json"
require "uri"

class FirecrawlService
BASE_URL = "https://api.firecrawl.dev/v2"

def initialize(api_key: ENV.fetch("FIRECRAWL_API_KEY"))
  @api_key = api_key
end

def search(query, limit: 5)
  post("/search", { query: query, limit: limit })
end

def scrape(url, **options)
  post("/scrape", { url: url }.merge(options))
end

def interact(url, prompt, follow_up: nil)
  # 1. Scrape to open a browser session
  scrape_result = scrape(url, formats: ["markdown"])
  scrape_id = scrape_result.dig("data", "metadata", "scrapeId")

  # 2. Send first prompt
  post("/scrape/#{scrape_id}/interact", { prompt: prompt })

  # 3. Send follow-up prompt
  result = nil
  if follow_up
    result = post("/scrape/#{scrape_id}/interact", { prompt: follow_up })
  end

  # 4. Close the session
  delete("/scrape/#{scrape_id}/interact")

  result || scrape_result
end

private

def post(endpoint, payload)
  uri = URI("#{BASE_URL}#{endpoint}")
  request = Net::HTTP::Post.new(uri)
  request["Authorization"] = "Bearer #{@api_key}"
  request["Content-Type"] = "application/json"
  request.body = payload.to_json

  response = Net::HTTP.start(uri.hostname, uri.port, use_ssl: true) do |http|
    http.request(request)
  end

  JSON.parse(response.body)
end

def delete(endpoint)
  uri = URI("#{BASE_URL}#{endpoint}")
  request = Net::HTTP::Delete.new(uri)
  request["Authorization"] = "Bearer #{@api_key}"

  Net::HTTP.start(uri.hostname, uri.port, use_ssl: true) do |http|
    http.request(request)
  end
end
end

Create a controller ​

Generate a controller:

bash
rails generate controller Firecrawl search scrape interact --skip-routes

Edit app/controllers/firecrawl_controller.rb:

ruby
class FirecrawlController < ApplicationController
skip_before_action :verify_authenticity_token

def search
  service = FirecrawlService.new
  result = service.search(params.require(:query), limit: params.fetch(:limit, 5).to_i)
  render json: result
end

def scrape
  service = FirecrawlService.new
  result = service.scrape(params.require(:url))
  render json: result
end

def interact
  service = FirecrawlService.new
  result = service.interact(
    params.require(:url),
    params.require(:prompt),
    follow_up: params[:followUp]
  )
  render json: result
end
end

Add routes ​

In config/routes.rb:

ruby
Rails.application.routes.draw do
post "api/search", to: "firecrawl#search"
post "api/scrape", to: "firecrawl#scrape"
post "api/interact", to: "firecrawl#interact"
end

Test it ​

bash
rails server

# Search the web
curl -X POST http://localhost:3000/api/search \
-H "Content-Type: application/json" \
-d '{"query": "firecrawl web scraping"}'

# Scrape a page
curl -X POST http://localhost:3000/api/scrape \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com"}'

# Interact with a page
curl -X POST http://localhost:3000/api/interact \
-H "Content-Type: application/json" \
-d '{"url": "https://www.amazon.com", "prompt": "Search for iPhone 16 Pro Max", "followUp": "Click on the first result and tell me the price"}'

Next steps ​

Search docs

Search the web and get full page content

Scrape docs

All scrape options including formats, actions, and proxies

Interact docs

Click, fill forms, and extract dynamic content

API Reference

Complete REST API documentation