Smaller scheduled batches can raise your event budget

Your row budget stayed flat. Your scheduled starts multiplied.

You split a weekly company-directory refresh into smaller jobs, hoping to make failures less annoying. Then the event budget grows. The row allowance still looks innocent.

Splitting the same company-record allowance across more runs increases start-event fees. For Nimbryxo-studio's directory Actor, starts are memory-dependent, result events follow dataset items, and platform usage is charged separately.

Make the input part of the budget

Start with a restricted input your estimator actually understands. Supporting every mode would widen the billing assumptions before you need them.

Example: this input selects a bulk listing path, with enrichment explicitly disabled.

import assert from 'node:assert/strict'; import { z } from 'zod'; import pg from 'pg'; const BrowseInput = z.object({ mode: z.literal('browse'), location: z.literal('chicago'), includeDetails: z.boolean(), includeProfile: z.boolean(), includeBenefits: z.boolean(), maxItems: z.number().int().min(1).max(10000), }).strict(); const run_input = BrowseInput.parse({ mode: 'browse', location: 'chicago', includeDetails: false, includeProfile: false, includeBenefits: false, maxItems: 100, }); 

This schema deliberately accepts only a subset of the Actor's inputs. Changes outside that subset require another budgeting review.

The explicit flags matter. A saved task shouldn't quietly inherit enrichment behavior your cost model never considered.

What actually creates a charge?

The Store prices describe separate events with different triggers.

The start charge uses one event per GB, with a minimum of one event. A run boundary therefore has a price even when your total record allowance stays unchanged.

That explains why smaller batches can cost more in event fees. You're buying additional starts, alongside the records.

Memory belongs in the scheduling configuration. If it changes, invalidate the old estimate rather than carrying its start-event allowance forward.

Enrichment changes work before it changes rows

maxItems caps company records returned. It doesn't describe all the requests needed to produce them.

includeProfile and includeBenefits each add one request per company. includeDetails fetches detail records and can fall back to profile pages when details are missing or lack a tech stack.

Large runs with details enabled are slow. Use includeDetails: false for bulk listing when those fields aren't needed.

For a recurring directory refresh, separate discovery from selective enrichment. Refresh the listing broadly, then enrich companies whose additional fields support an actual decision.

That separation makes spend easier to explain. It doesn't make enrichment free, and the published event rates can't bound its platform usage.

Calculate an event-fee ceiling before launch

Use integer money internally. Floating-point rounding is an unnecessary guest in a budget ledger.

This function prices the restricted input, requires an explicit start-event allowance, and checks the published arithmetic without printing output. Its allowance arguments belong to your scheduler, not the Actor input.

const ratesMicroUsd = { FREE: { item: 990, start: 100000 }, GOLD: { item: 490, start: 30000 }, } as const; type Tier = keyof typeof ratesMicroUsd; const PositiveInteger = z.number().int().min(1) .max(Number.MAX_SAFE_INTEGER); function eventCeilingMicroUsd( input: unknown, tier: Tier, plannedStarts: unknown, startEventAllowance: unknown, ): number { const parsed = BrowseInput.parse(input); const starts = PositiveInteger.parse(plannedStarts); const startEvents = PositiveInteger.parse(startEventAllowance); const rate = ratesMicroUsd[tier]; const perRun = parsed.maxItems * rate.item + startEvents * rate.start; const total = perRun * starts; if (!Number.isSafeInteger(perRun) || !Number.isSafeInteger(total)) { throw new Error('Event reservation exceeds safe integer precision'); } return total; } function usdString(microUsd: number): string { return `${Math.trunc(microUsd / 1_000_000)}.${ String(microUsd % 1_000_000).padStart(6, '0') }`; } for (const [maxItems, tier, expectedUsd] of [ [100, 'FREE', 0.199], [100, 'GOLD', 0.079], [1000, 'FREE', 1.09], [1000, 'GOLD', 0.52], ] as const) { assert.equal( eventCeilingMicroUsd({ ...run_input, maxItems }, tier, 1, 1) / 1_000_000, expectedUsd, ); } assert.throws(() => eventCeilingMicroUsd(run_input, 'FREE', 1, undefined), ); 

Treat the result as a conditional event-fee ceiling. It assumes the configured memory fits the supplied start-event allowance and the submitted input follows this company-record path.

It isn't a whole-run spending ceiling. Adding an enrichment flag leaves this event formula unchanged while adding work that consumes platform usage.

The function can't verify your memory configuration. Keep that configuration and its verified event allowance together, and reject scheduling changes that update only one.

Budget attempts, including scheduled retries

A recurring budget needs a finite planning window. “Weekly” describes timing; it doesn't specify how many launches you authorize.

Keep planned starts and retry starts separate. Otherwise, retries become the place where the budget quietly acquires imagination.

Example: these plans compare equal scheduled record allowances, then reserve additional attempts explicitly. The start-event allowance is illustrative; replace it with an allowance verified for your configured memory.

const schedulePlans = { bulk: { input: BrowseInput.parse({ ...run_input, maxItems: 1000 }), plannedStarts: 1, retryStarts: 2, }, split: { input: run_input, plannedStarts: 10, retryStarts: 2, }, }; const configuredStartEventsPerRun = 1; const comparison = Object.fromEntries( Object.entries(schedulePlans).map(([name, plan]) => [ name, { scheduledRecords: plan.input.maxItems * plan.plannedStarts, scheduledEventMicroUsd: eventCeilingMicroUsd( plan.input, 'FREE', plan.plannedStarts, configuredStartEventsPerRun, ), reservedEventMicroUsd: eventCeilingMicroUsd( plan.input, 'FREE', plan.plannedStarts + plan.retryStarts, configuredStartEventsPerRun, ), }, ]), ); assert.equal( comparison.bulk.scheduledRecords, comparison.split.scheduledRecords, ); assert.ok( comparison.split.scheduledEventMicroUsd > comparison.bulk.scheduledEventMicroUsd, ); 

The comparison separates scheduled work from permission to repeat it. Equal scheduled allowances don't imply equal retry allowances: repeating a larger batch authorizes more records.

Retries can emit records already collected by an earlier attempt. Deduplicating those records downstream doesn't undo their event charges.

Choose smaller batches for freshness or failure isolation when those benefits justify repeated starts. A row allowance also doesn't guarantee progress through the directory; batch boundaries need their own coverage design.

Reserve locally, cap remotely

Preflight admission answers: can this planning window afford another attempt?

A platform-side cap answers a different question: how much may this particular pay-per-event run charge?

Use both. A local estimate can reject work before it starts and coordinate concurrent workers. It can't enforce spending inside an already running process.

The following functions create your application's accounting tables and reserve an attempt atomically. The table columns are local bookkeeping, separate from the Actor's input and output.

async function initializeEventBudget( pool: pg.Pool, period: string, periodMicroUsd: number, ): Promise<void> { PositiveInteger.parse(periodMicroUsd); await pool.query(` CREATE TABLE IF NOT EXISTS directory_event_budget ( period text PRIMARY KEY, remaining_micro_usd bigint NOT NULL CHECK (remaining_micro_usd >= 0) ) `); await pool.query(` CREATE TABLE IF NOT EXISTS directory_event_attempt ( period text NOT NULL REFERENCES directory_event_budget(period), attempt_key text NOT NULL, amount_micro_usd bigint NOT NULL CHECK (amount_micro_usd > 0), submitted_input jsonb NOT NULL, submission_status integer, submission_body text, PRIMARY KEY (period, attempt_key) ) `); await pool.query(` INSERT INTO directory_event_budget (period, remaining_micro_usd) VALUES ($1, $2) ON CONFLICT (period) DO NOTHING `, [period, periodMicroUsd]); } async function reserveAndStart( pool: pg.Pool, period: string, attemptKey: string, input: unknown, tier: Tier, startEventAllowance: unknown, ): Promise<Response> { const token = process.env.APIFY_TOKEN; if (!token) throw new Error('APIFY_TOKEN is required'); if (!attemptKey) throw new Error('An attempt key is required'); const parsed = BrowseInput.parse(input); const reserved = eventCeilingMicroUsd( parsed, tier, 1, startEventAllowance, ); const url = new URL( 'https://api.apify.com/v2/acts/' + 'nimbryxo-studio~builtin-tech-company-directory-and-hiring-signal/runs', ); url.searchParams.set('maxTotalChargeUsd', usdString(reserved)); const admission = await pool.query(` WITH debited AS ( UPDATE directory_event_budget SET remaining_micro_usd = remaining_micro_usd - $2::bigint WHERE period = $1 AND remaining_micro_usd >= $2::bigint RETURNING period ) INSERT INTO directory_event_attempt ( period, attempt_key, amount_micro_usd, submitted_input ) SELECT period, $3::text, $2::bigint, $4::jsonb FROM debited RETURNING attempt_key `, [period, reserved, attemptKey, JSON.stringify(parsed)]); if (admission.rowCount !== 1) { throw new Error('Budget unavailable'); } const response = await fetch(url, { method: 'POST', headers: { Authorization: `Bearer ${token}`, 'Content-Type': 'application/json', }, body: JSON.stringify(parsed), }); await pool.query(` UPDATE directory_event_attempt SET submission_status = $3::integer, submission_body = $4::text WHERE period = $1 AND attempt_key = $2 `, [ period, attemptKey, response.status, await response.clone().text(), ]); if (!response.ok) throw new Error('Run submission needs review'); return response; } 

Initialize the budget once per planning period. The conflict clause deliberately prevents a worker restart from replenishing an existing allowance.

The atomic statement prevents concurrent workers from spending the same remaining balance. A duplicate attempt key raises a database error and rolls back that statement, including its debit.

Use a stable key for each authorized attempt. A worker restart reuses it; an explicitly authorized retry gets a fresh key.

A lost response creates uncertainty: the request may have started a run. Keep that reservation until you reconcile it. Automatically refunding it would turn a transport failure into permission to spend again.

Persist the returned run identifier in your worker's job record. Reconcile each reservation against actual event charges afterward, releasing unused allowance only once.

This code handles admission and submission, not that reconciliation workflow. Keep platform usage in its separate ledger, and don't treat maxTotalChargeUsd as a ceiling for that component.

A capped extraction needs a publication gate

The worked example is BuiltIn Tech Company Directory & Hiring Signal Scraper. The budgeting pattern applies whenever starts, results, and execution resources have different charging rules.

The operational catch arrives downstream: a stopped run may leave a partial dataset. Treat retained rows as observations, not proof that a directory refresh completed.

Keep the flow explicit.

reserve attempt -> submit capped run -> retain run evidence | v stage dataset | publication gate | current snapshot 

Retain the submitted input, run outcome, warnings, and dataset reference together. Stage all dataset pages before deciding whether the result can replace your current snapshot.

A successful status alone can't establish coverage. The Actor can warn when its page safety cap stops browsing before maxItems. Conversely, returning fewer records than the limit doesn't prove a safety stop.

Your publication gate needs those distinctions. A failed, interrupted, or early-stopped extraction shouldn't erase companies from the previous snapshot merely because they weren't observed.

Even reaching the requested limit establishes a bounded observation, not directory completeness. Your downstream deletion policy needs stronger evidence than “the job returned rows.”

For a bounded refresh, updating observed companies can be reasonable while withholding deletion decisions. Keep the coverage state visible to whatever consumes the snapshot.

Before the next schedule fires

• Fix the planning window, planned starts, and retry allowance.
• Supply a start-event allowance tied to the configured memory.
• Reserve event fees atomically before submitting each attempt.
• Submit the validated maxItems and remote charge cap together.
• Keep uncertain submissions reserved until reconciliation.
• Budget platform usage separately.
• Publish staged records only after checking outcome, warnings, and coverage.

A cheaper-looking row target isn't a schedule budget. Count the attempts you're authorizing.