A few weeks ago I wrote about connecting an AI agent to an SEO MCP server. MCP is the right interface when a model decides which tool to call. When you write the automation yourself, say a nightly job that pulls audit findings into a dashboard, or a script that adds keywords from your CMS, you want a plain REST API with a schema you can generate types from.
This post walks through the Crawl Foundry SEO REST API from TypeScript: authentication, the response envelope, reading stored site audit data, and the two things that make automation against a paid API safe, which are idempotency and knowing what costs money.
The basics
- Base URL:
https://app.crawlfoundry.com/api/v1 - Reference:
https://app.crawlfoundry.com/api/v1/docs, with the OpenAPI document at/api/v1/openapi.json - Auth: a bearer API key, created by an organization admin in the Agent Platform area of the app
Every API key is bound to exactly one workspace and its permissions. The secret is shown once when the key is created; after that it can only be rotated or revoked. There is no anonymous access, and a key can't enumerate other workspaces in the organization: GET /workspaces returns at most the one workspace the key belongs to.
Because the OpenAPI document is public, you can generate types instead of writing them by hand:
npx openapi-typescript https://app.crawlfoundry.com/api/v1/openapi.json -o crawlfoundry.d.ts A minimal client
Every response uses the same envelope. On success you get success: true, a machine-readable code, a message, the payload in data and request metadata in meta. On failure you get success: false and an error object that tells you whether the request is retryable.
const BASE = "https://app.crawlfoundry.com/api/v1"; const KEY = process.env.CRAWLFOUNDRY_API_KEY!; type Envelope<T> = | { success: true; code: string; message: string; data: T; meta: Record<string, unknown> } | { success: false; code: string; message: string; error: { code: string; retryable: boolean } }; export async function cf<T>(path: string, body?: unknown): Promise<T> { const res = await fetch(`${BASE}${path}`, { method: body === undefined ? "GET" : "POST", headers: { Authorization: `Bearer ${KEY}`, "Content-Type": "application/json", }, body: body === undefined ? undefined : JSON.stringify(body), }); const json = (await res.json()) as Envelope<T>; if (!json.success) { throw new Error(`${res.status} ${json.code}: ${json.message} (retryable: ${json.error.retryable})`); } return json.data; } The code values are stable strings such as CTX-AGP-401-AUTH_REQUIRED, which are much easier to branch on than message text. Each response also carries an X-Request-Id header; log it, because it is what support will ask for.
Finding your workspace
const workspaces = await cf<unknown>("/workspaces"); console.log(workspaces); The workspace ID (it starts with ws_) goes into every workspace-scoped path.
Reading stored audit data
Most SEO functions are exposed as job endpoints under /workspaces/{workspaceId}/jobs/{tool}, mirroring the tools the MCP server offers. For site audits, seo_analyze_site has a stored mode that only reads what is already saved and never starts a crawl:
const ws = "ws_your_workspace_id"; const sites = await cf<any>(`/workspaces/${ws}/jobs/seo_analyze_site`, { mode: "stored", request: { section: "sites", limit: 20 }, }); Once you have a site ID, the overview mode combines several stored views in one call:
const overview = await cf<any>(`/workspaces/${ws}/jobs/seo_analyze_site`, { mode: "overview", request: { siteId: "your_site_id", modules: ["run", "issueGroups", "aiReadiness"], limit: 10, }, }); Two fields deserve attention in these responses. data.dataGaps and data.coverage tell you what is missing or incomplete, for example no completed run yet. And meta.billing.classification tells you how the call was billed; for stored reads it is a free classification. If your dashboard shows an empty chart, check the gaps before you assume the site has no issues.
What costs money
Reads of stored data are free. Anything that fetches fresh data or generates content uses the organization's EUR balance, with the same cost classes as the MCP server: pay per use for fresh data, price first for larger actions, and approval only for publishing. Paid actions go through a quote, approval and execution lifecycle, and long-running work returns an operation you can poll:
const op = await cf<any>(`/workspaces/${ws}/jobs/seo_get_operation`, { operationId: "aop_...", }); console.log(op.status); If the balance can't cover an action, the API answers with HTTP 402 and CREDITS_INSUFFICIENT instead of running it.
Idempotency, or: retries that don't double-charge
Network calls fail in the middle. For writes, that raises the question of whether the first attempt went through. The API answers it with idempotency keys. Job operations that change something take an idempotencyKey in the body, for example when configuring a site for audits:
await cf(`/workspaces/${ws}/jobs/seo_manage_site_audit`, { operation: "configureSite", input: { baseUrl: "https://example.com/" }, idempotencyKey: "configure-example-com-2026-11", }); Send the same key with the same request again and you get the original result rather than a second execution. Send the same key with a different request and you get a 409 IDEMPOTENCY_CONFLICT. While the first attempt is still running, a retry gets a 409 IDEMPOTENCY_IN_PROGRESS with a Retry-After header. Some write endpoints outside the job routes take the key as an Idempotency-Key header instead; the reference lists which.
Derive keys from the business intent ("configure this site this month"), not from a random UUID per attempt, or the protection disappears on retry.
Where to go from here
The same API covers keyword lists and tags, rank tracking schedules, site audit schedules and alert rules, content projects and SEO work items. The SEO MCP server page explains how REST and MCP relate and what the cost classes mean, and the full reference is at the API docs link above. Crawl Foundry runs pay-per-use without a subscription obligation, and API access uses the same balance as everything else, without a separate API plan.
