Webhook Events

Updated Feb 2026

This page documents every webhook event type, its payload structure, and when it fires. Use this as a reference when building your webhook handler.

Payload Structure

All webhook events share this common structure:

json

{
  "success": true,
  "type": "crawl.page",
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "data": [],
  "metadata": {}
}

Common Fields

Field	Type	Description
`success`	`boolean`	Whether the operation succeeded. `false` on failure events.
`type`	`string`	The event type (e.g., `crawl.page`, `extract.completed`).
`id`	`string`	The job ID (UUID). Use this to correlate events with your original API request.
`data`	`array`	Event-specific data. Empty for lifecycle events (`started`, `completed`), populated for data events (`page`, `action`).
`metadata`	`object`	Custom metadata you provided in your webhook configuration.
`error`	`string`	Error message. Only present when `success` is `false`.

Crawl Events

`crawl.started`

Sent when the crawl job begins processing.

json

{
  "success": true,
  "type": "crawl.started",
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "data": [],
  "metadata": {}
}

`crawl.page`

Sent for each page scraped during the crawl. The data array contains the page content and metadata.

json

{
  "success": true,
  "type": "crawl.page",
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "data": [
    {
      "markdown": "# Page content...",
      "metadata": {
        "title": "Page Title",
        "description": "Page description",
        "url": "https://example.com/page",
        "statusCode": 200,
        "contentType": "text/html",
        "scrapeId": "550e8400-e29b-41d4-a716-446655440001",
        "sourceURL": "https://example.com/page",
        "proxyUsed": "basic",
        "cacheState": "hit",
        "cachedAt": "2025-09-03T21:11:25.636Z",
        "creditsUsed": 1
      }
    }
  ],
  "metadata": {}
}

Data fields for crawl.page:

Field	Type	Description
`markdown`	`string`	The scraped page content in Markdown format.
`metadata.title`	`string`	The page's `<title>` tag.
`metadata.description`	`string`	The page's meta description.
`metadata.url`	`string`	Final URL after any redirects.
`metadata.statusCode`	`number`	HTTP status code of the response.
`metadata.contentType`	`string`	The Content-Type header value.
`metadata.scrapeId`	`string`	Unique ID for this individual scrape.
`metadata.sourceURL`	`string`	The original URL requested.
`metadata.proxyUsed`	`string`	The proxy tier used (`basic`, `stealth`, etc.).
`metadata.cacheState`	`string`	Whether the result was served from cache (`hit` or `miss`).
`metadata.cachedAt`	`string`	ISO timestamp of when the cache entry was created.
`metadata.creditsUsed`	`number`	Credits consumed for this page scrape.

`crawl.completed`

Sent when the crawl job finishes and all pages have been processed.

json

{
  "success": true,
  "type": "crawl.completed",
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "data": [],
  "metadata": {}
}

Batch Scrape Events

`batch_scrape.started`

Sent when the batch scrape job begins processing.

json

{
  "success": true,
  "type": "batch_scrape.started",
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "data": [],
  "metadata": {}
}

`batch_scrape.page`

Sent for each URL scraped in the batch. The data array contains the page content and metadata.

json

{
  "success": true,
  "type": "batch_scrape.page",
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "data": [
    {
      "markdown": "# Page content...",
      "metadata": {
        "title": "Page Title",
        "description": "Page description",
        "url": "https://example.com",
        "statusCode": 200,
        "contentType": "text/html",
        "scrapeId": "550e8400-e29b-41d4-a716-446655440001",
        "sourceURL": "https://example.com",
        "proxyUsed": "basic",
        "cacheState": "miss",
        "cachedAt": "2025-09-03T23:30:53.434Z",
        "creditsUsed": 1
      }
    }
  ],
  "metadata": {}
}

Data Fields

The data array for batch_scrape.page uses the same structure as crawl.page. See the crawl.page field reference above.

`batch_scrape.completed`

Sent when all URLs in the batch have been processed.

json

{
  "success": true,
  "type": "batch_scrape.completed",
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "data": [],
  "metadata": {}
}

Extract Events

`extract.started`

Sent when the extract job begins processing.

json

{
  "success": true,
  "type": "extract.started",
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "data": [],
  "metadata": {}
}

`extract.completed`

Sent when extraction finishes successfully. The data array contains the extracted structured data, usage info, and source attribution.

json

{
  "success": true,
  "type": "extract.completed",
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "data": [
    {
      "success": true,
      "data": {
        "siteName": "Example Site",
        "category": "Technology"
      },
      "extractId": "550e8400-e29b-41d4-a716-446655440000",
      "llmUsage": 0.0020118,
      "totalUrlsScraped": 1,
      "sources": {
        "siteName": ["https://example.com"],
        "category": ["https://example.com"]
      }
    }
  ],
  "metadata": {}
}

Data fields for extract.completed:

Field	Type	Description
`data.success`	`boolean`	Whether the extraction itself succeeded.
`data.data`	`object`	The extracted structured data matching your schema.
`data.extractId`	`string`	Unique ID for this extraction.
`data.llmUsage`	`number`	LLM token cost for this extraction.
`data.totalUrlsScraped`	`number`	Number of URLs that were scraped to produce the extraction.
`data.sources`	`object`	Maps each extracted field name to the source URL(s) it came from.

`extract.failed`

Sent when extraction fails. The error field contains the failure reason.

json

{
  "success": false,
  "type": "extract.failed",
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "data": [],
  "error": "Failed to extract data: timeout exceeded",
  "metadata": {}
}

Agent Events

`agent.started`

Sent when the agent job begins processing.

json

{
  "success": true,
  "type": "agent.started",
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "data": [],
  "metadata": {}
}

`agent.action`

Sent after each tool execution (scrape, search, etc.) the agent performs.

json

{
  "success": true,
  "type": "agent.action",
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "data": [
    {
      "creditsUsed": 5,
      "action": "mcp__tools__scrape",
      "input": {
        "url": "https://example.com"
      }
    }
  ],
  "metadata": {}
}

Credits in Action Events

The creditsUsed value in agent.action events is an estimate of the total credits used so far. The final accurate credit count is only available in the completed, failed, or cancelled events.

Data fields for agent.action:

Field	Type	Description
`creditsUsed`	`number`	Estimated total credits consumed so far (cumulative).
`action`	`string`	The tool/action that was executed (e.g., `mcp__tools__scrape`).
`input`	`object`	The input parameters passed to the tool.

`agent.completed`

Sent when the agent finishes successfully. Contains the final extracted data and accurate total credits used.

json

{
  "success": true,
  "type": "agent.completed",
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "data": [
    {
      "creditsUsed": 15,
      "data": {
        "company": "Example Corp",
        "industry": "Technology",
        "founded": 2020
      }
    }
  ],
  "metadata": {}
}

`agent.failed`

Sent when the agent encounters an error. The error field contains the failure reason.

json

{
  "success": false,
  "type": "agent.failed",
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "data": [
    {
      "creditsUsed": 8
    }
  ],
  "error": "Max credits exceeded",
  "metadata": {}
}

Credits on Failure

Even when an agent fails, credits consumed up to the point of failure are still charged. The creditsUsed field in the data array reflects the final credit count.

`agent.cancelled`

Sent when the agent job is cancelled by the user.

json

{
  "success": false,
  "type": "agent.cancelled",
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "data": [
    {
      "creditsUsed": 3
    }
  ],
  "metadata": {}
}

Event Filtering

By default, you receive all events for the operation type. To subscribe to specific events only, use the events array in your webhook configuration:

json

{
  "url": "https://your-app.com/webhook",
  "events": ["completed", "failed"]
}

This is useful when you only care about job completion and do not need per-page updates. For example, a crawl job that processes 500 pages would send 500+ crawl.page events. If you only need to know when it finishes, filter to ["completed"].

Event Name Quick Reference

Event Name	Operations	When It Fires
`started`	Crawl, Batch Scrape, Extract, Agent	Job begins processing
`page`	Crawl, Batch Scrape	Each page/URL is scraped
`completed`	Crawl, Batch Scrape, Extract, Agent	Job finishes successfully
`failed`	Extract, Agent	Job encounters an error
`action`	Agent	After each tool execution
`cancelled`	Agent	Job cancelled by user

Webhooks Overview -- Configuration and setup
Webhook Security -- HMAC-SHA256 signature verification
Testing Webhooks -- Local development and debugging

Webhook Events ​

Payload Structure ​

Common Fields ​

Crawl Events ​

crawl.started ​

crawl.page ​

crawl.completed ​

Batch Scrape Events ​

batch_scrape.started ​

batch_scrape.page ​

batch_scrape.completed ​

Extract Events ​

extract.started ​

extract.completed ​

extract.failed ​

Agent Events ​

agent.started ​

agent.action ​

agent.completed ​

agent.failed ​

agent.cancelled ​

Event Filtering ​

Event Name Quick Reference ​

Related Pages ​

Webhook Events

Payload Structure

Common Fields

Crawl Events

`crawl.started`

`crawl.page`

`crawl.completed`

Batch Scrape Events

`batch_scrape.started`

`batch_scrape.page`

`batch_scrape.completed`

Extract Events

`extract.started`

`extract.completed`

`extract.failed`

Agent Events

`agent.started`

`agent.action`

`agent.completed`

`agent.failed`

`agent.cancelled`

Event Filtering

Event Name Quick Reference

Related Pages