AI Research Assistant Cookbook

Updated Feb 2026

Build an intelligent chat interface that autonomously decides when to scrape websites or search the web to answer user questions. This cookbook combines OpenAI's language models with Firecrawl's web processing capabilities through the Vercel AI SDK.

Architecture

The application follows a three-stage pipeline:

1. User Input Layer -- Frontend captures questions via a React/Next.js chat interface
2. Decision and Execution -- The AI model determines if web tools are needed, then executes them
3. Response Generation -- The model synthesizes tool results into natural language answers with citations

User Question
    |
    v
AI Model (tool-calling)
    |
    +---> Firecrawl Search (web search + scrape results)
    |
    +---> Firecrawl Scrape (full page content from URL)
    |
    v
AI Model (synthesis)
    |
    v
Streamed Response with Sources

Tech Stack

Component	Package
Framework	Next.js (App Router, TypeScript)
AI SDK	ai, @ai-sdk/react, @ai-sdk/openai
Web Scraping	@mendable/firecrawl-js
Validation	zod
UI Components	AI Elements (built on shadcn/ui)

Setup

Create the Project

npx create-next-app@latest ai-research-assistant
cd ai-research-assistant
npm i ai @ai-sdk/react @ai-sdk/openai zod @mendable/firecrawl-js
npx ai-elements@latest

Environment Variables

Create .env.local in the project root:

OPENAI_API_KEY=sk-your-openai-key
FIRECRAWL_API_KEY=fc-your-firecrawl-key

Tool Definitions

Create lib/tools.ts to define the web tools the AI model can call.

Scrape Website Tool

// lib/tools.ts
import { tool } from "ai";
import FirecrawlApp from "@mendable/firecrawl-js";
import { z } from "zod";

const firecrawl = new FirecrawlApp({
  apiKey: process.env.FIRECRAWL_API_KEY!,
});

export const scrapeWebsite = tool({
  description: "Scrape a website URL and return its content as markdown",
  parameters: z.object({
    url: z.string().url().describe("The URL to scrape"),
  }),
  execute: async ({ url }) => {
    const result = await firecrawl.scrapeUrl(url, {
      formats: ["markdown"],
      timeout: 30000,
    });
    return {
      content: result.markdown,
      title: result.metadata?.title,
      sourceUrl: result.metadata?.sourceURL,
    };
  },
});

Search Web Tool

export const searchWeb = tool({
  description:
    "Search the web for information and return relevant results with content",
  parameters: z.object({
    query: z.string().describe("The search query"),
    limit: z
      .number()
      .optional()
      .default(5)
      .describe("Maximum number of results"),
  }),
  execute: async ({ query, limit }) => {
    const results = await firecrawl.search(query, {
      limit,
      scrapeOptions: {
        formats: ["markdown"],
      },
    });
    return results.data.map((r) => ({
      title: r.title,
      url: r.url,
      description: r.description,
      content: r.markdown,
    }));
  },
});

API Route

Create the backend endpoint that orchestrates tool-calling and streaming.

// app/api/chat/route.ts
import { streamText, stepCountIs } from "ai";
import { openai } from "@ai-sdk/openai";
import { scrapeWebsite, searchWeb } from "@/lib/tools";

export async function POST(req: Request) {
  const { messages, model = "gpt-4o-mini", webSearch = "auto" } =
    await req.json();

  // Only register tools when web search is enabled
  const tools =
    webSearch === "auto"
      ? { scrapeWebsite, searchWeb }
      : {};

  const result = streamText({
    model: openai(model),
    system: `You are an AI research assistant with web access.
When a user asks a question that requires current information:
1. Use searchWeb to find relevant sources
2. Use scrapeWebsite to get full content from the most relevant URLs
3. Synthesize the information into a clear, cited answer

Always cite your sources with URLs. If you don't need web data, answer directly.`,
    messages,
    tools,
    // Limit execution steps to prevent runaway API costs
    stopWhen: stepCountIs(5),
  });

  return result.toDataStreamResponse();
}

Frontend Chat Interface

Build the chat UI with streaming support and tool execution visualization.

// app/page.tsx
"use client";

import { useChat } from "@ai-sdk/react";
import { useState } from "react";

export default function ResearchAssistant() {
  const [webSearch, setWebSearch] = useState<"auto" | "none">("auto");
  const [model, setModel] = useState("gpt-4o-mini");

  const { messages, input, handleInputChange, handleSubmit, isLoading } =
    useChat({
      body: { webSearch, model },
    });

  return (
    <div className="max-w-3xl mx-auto p-4">
      <h1 className="text-2xl font-bold mb-4">AI Research Assistant</h1>

      {/* Model and search controls */}
      <div className="flex gap-4 mb-4">
        <select
          value={model}
          onChange={(e) => setModel(e.target.value)}
          className="border rounded px-3 py-1"
        >
          <option value="gpt-4o-mini">GPT-4o Mini</option>
          <option value="gpt-4o">GPT-4o</option>
        </select>

        <button
          onClick={() =>
            setWebSearch(webSearch === "auto" ? "none" : "auto")
          }
          className={`px-3 py-1 rounded ${
            webSearch === "auto"
              ? "bg-blue-500 text-white"
              : "bg-gray-200"
          }`}
        >
          Web Search: {webSearch === "auto" ? "ON" : "OFF"}
        </button>
      </div>

      {/* Messages */}
      <div className="space-y-4 mb-4">
        {messages.map((m) => (
          <div
            key={m.id}
            className={`p-3 rounded ${
              m.role === "user" ? "bg-blue-50" : "bg-gray-50"
            }`}
          >
            <strong>{m.role === "user" ? "You" : "Assistant"}:</strong>
            <div className="mt-1 prose prose-sm">{m.content}</div>

            {/* Show tool invocations */}
            {m.toolInvocations?.map((tool, i) => (
              <div key={i} className="mt-2 text-xs text-gray-500 border-l-2 pl-2">
                Tool: {tool.toolName}
                {tool.state === "result" && " (completed)"}
              </div>
            ))}
          </div>
        ))}
      </div>

      {/* Input */}
      <form onSubmit={handleSubmit} className="flex gap-2">
        <input
          value={input}
          onChange={handleInputChange}
          placeholder="Ask a research question..."
          className="flex-1 border rounded px-3 py-2"
          disabled={isLoading}
        />
        <button
          type="submit"
          disabled={isLoading}
          className="bg-blue-500 text-white px-4 py-2 rounded"
        >
          {isLoading ? "Researching..." : "Send"}
        </button>
      </form>
    </div>
  );
}

Message Flow

1. User submits a question with model and search preferences
2. useChat hook transmits the message to /api/chat
3. Backend calls streamText with the registered Firecrawl tools
4. The AI model analyzes whether web tools are needed
5. If needed, Firecrawl executes search or scrape operations
6. Results stream back to the model for synthesis
7. The UI renders tool calls, their outputs, and the final response progressively

Customization

Adding More Tools

Follow the same pattern -- define a Zod schema for parameters and an execute function:

export const extractData = tool({
  description: "Extract structured data from a URL using a custom prompt",
  parameters: z.object({
    url: z.string().url(),
    prompt: z.string().describe("What data to extract"),
  }),
  execute: async ({ url, prompt }) => {
    const result = await firecrawl.scrapeUrl(url, {
      formats: ["json"],
      jsonOptions: { prompt },
    });
    return result.json;
  },
});

Swapping LLM Providers

The AI SDK supports 20+ providers. Replace OpenAI with Anthropic:

import { anthropic } from "@ai-sdk/anthropic";

const result = streamText({
  model: anthropic("claude-sonnet-4-20250514"),
  // ... rest of config
});

Cost Management

• stepCountIs(5) limits the number of tool-call rounds per request
• Use gpt-4o-mini for most queries -- switch to gpt-4o only when needed
• The web search toggle lets users disable tools when direct answers suffice
• Monitor Firecrawl credit usage in the dashboard

Best Practices

Practice	Reason
Limit tool steps	Prevents runaway API costs from recursive tool calls
Stream responses	Provides immediate feedback while tools execute
Show tool invocations in UI	Transparency about what the assistant is doing
Cache frequently accessed content	Reduces redundant scrapes for common sources
Use markdown format for scraping	Most token-efficient format for LLM consumption

TIP

Test your research assistant with factual questions that require current data -- stock prices, recent news, product specs. This validates that the tool-calling pipeline works end to end.

Resources

• AI SDK Documentation
• Firecrawl Search Docs
• Firecrawl Scrape Docs
• Full Source Code