The page says a gaming studio's recruiting tenant has crossed its document-processing budget. The on-call sees a growing batch backlog and an aggregate charge, but cannot tell whether the studio submitted more candidate packets, a retry repeated work, or completed summaries are waiting to be reconciled. Treat this as a design exercise, not a reported incident: that is the page the instrumentation should make actionable.

Short answer: process multiple documents through an asynchronous summarization API, preserve tenant and document identity outside the provider, and reconcile completed work before alerting on tenant spend. For a Node.js application that prepares evidence for scoring candidates against a job rubric, the useful example starts with attribution; submitting a batch alone does not establish whose work completed or whose budget it consumed.

Summaries are inputs. They are not hiring decisions.

The five checks below work backward from that page to the instrumentation that should have made it unnecessary. The decision axis is per-tenant cost visibility, with an SRE's usual suspicion of dashboards that look precise while hiding unfinished work.

1. How should a batch summarization API attribute multiple documents?

Start with a proposed alert contract: tenant, observation window, completed document count, outstanding document count, reconciled charges, and the age of the oldest unreconciled result. These are application requirements, not assumed provider response fields. A useful page should also point to the submission manifest and the rubric version, because a studio recruiting gameplay engineers may revise its evidence requirements while an earlier batch is still running.

Consider an illustrative intake of 120 documents: 80 belong to one studio and 40 to another. A single total for the batch cannot explain either studio's consumption. Candidate packets also differ in length, so dividing the total equally across documents produces an allocation estimate, not observed per-document cost. Label the difference. Otherwise the apparent precision survives all the way into the tenant dashboard.

Count work first.

For this workflow, define separate service-level objectives for intake acceptance, summary availability, and attribution completeness. Choose the targets with the recruiting team; no provider timing estimate establishes your application's SLO. A summary that arrives after the scheduled rubric review can be technically successful and operationally useless, while an on-time summary with missing tenant attribution leaves finance and platform engineering arguing over the same request.

Keep the rubric attached to the application record. Summarization should preserve evidence references for later review; it should not silently convert a candidate's self-description into a verified qualification or invent a numerical score that the rubric never authorized.

2. Trace the batch backward to an earlier signal

An asynchronous workflow has several distinct events: submit the documents, observe completion, retrieve results, then reconcile those results against the submission manifest. Completion observed at the provider is only one checkpoint. Record the last successful transition so an operator can distinguish unfinished processing from finished work that the application has not collected.

The supplied batch workflow describes a typical completion interval of 5–30 minutes. Treat that interval as planning context, never a deadline guarantee. If a recruiting review needs summaries sooner, either submit earlier or select a workflow whose documented completion contract meets the requirement; polling more often does not manufacture processing capacity.

The earlier signal should be the age and size of unreconciled work, broken down by tenant, before aggregate spend becomes surprising. Include both documents waiting for processing and results waiting for attribution. Those queues need different responses: additional admission capacity may help the former, whereas a reconciliation defect in your own worker requires investigation rather than more submissions.

This is where capacity planning becomes less comfortable than a request counter. A tenant that submits 80 long documents can hold more work in flight than a tenant submitting 80 short ones, and increasing concurrency increases the number of commitments the platform must track before it knows the final charge. Start with bounded outstanding work per tenant, then adjust that bound using observed completion age and reconciliation lag. Do not present a guessed concurrency limit as a provider limit.

Keep polling bounded too. Honor documented retry instructions, use backoff for rate limits, and persist the manifest before submission so a worker restart does not erase the evidence needed to interpret a retry. Submission identity and result identity serve different purposes; deduplicating the ledger does not itself make a remote submission idempotent.

3. Add an attribution ledger before adding another dashboard

The instrumentation change is small in concept: join every completed result to its tenant and source document, then attach the provider's actual charge at the granularity it reports. Retain the original result identity. If only a batch total is available, preserve that total separately and disclose any allocation rule instead of naming an estimate “actual.”

Below is a runnable Go example of the local accounting step. The application emits one JSON object per line on standard input after it has reconciled a charged observation. These are local event fields, not a vendor request schema. Amounts are integer microdollars, and event_id identifies one accounting observation; partial charges need distinct IDs. The example rejects incomplete attribution and conflicting duplicate events rather than silently moving charges between tenants.

package main import ( "encoding/json" "fmt" "io" "os" ) type Event struct { EventID string `json:"event_id"` TenantID string `json:"tenant_id"` DocumentID string `json:"document_id"` Micros int64 `json:"cost_microusd"` } func run() error { decoder := json.NewDecoder(os.Stdin) seen := make(map[string]Event) totals := make(map[string]int64) const maxInt64 int64 = 1<<63 - 1 for { var event Event if err := decoder.Decode(&event); err == io.EOF { break } else if err != nil { return err } if event.EventID == "" || event.TenantID == "" || event.DocumentID == "" || event.Micros < 0 { return fmt.Errorf("invalid attribution event") } if previous, exists := seen[event.EventID]; exists { if previous != event { return fmt.Errorf("conflicting event: %s", event.EventID) } continue } if event.Micros > maxInt64-totals[event.TenantID] { return fmt.Errorf("tenant total overflow: %s", event.TenantID) } seen[event.EventID] = event totals[event.TenantID] += event.Micros } return json.NewEncoder(os.Stdout).Encode(totals) } func main() { if err := run(); err != nil { fmt.Fprintln(os.Stderr, err) os.Exit(1) } } 

Save this as ledger.go and run go run ledger.go, supplying newline-delimited JSON events on standard input and closing the stream when finished. Use identifiers such as studio-a, candidate-17-resume, and batch-observation-1 in test inputs; supply an observed charge rather than a guessed model rate. Empty input produces an empty object. Replaying an identical event contributes its charge once.

The ledger performs no network submission. It demonstrates attribution independently of a submission payload, and a Node.js batch producer can emit the same local event contract. The language boundary doesn't change the reconciliation rule.

There is a deployment boundary, however: the maps exist only for one process execution. For a persistent ledger, store event identity and the accounting update atomically, preserve submission and rubric references, and replay events through the same uniqueness constraint. Keep document identifiers in durable records rather than turning every candidate into a metrics label. A dashboard should summarize the ledger, not become its only copy.

The complementary provider example below checks an existing batch without creating another one. Save it separately as status.go. Set INFRAI_API_ORIGIN to the documented HTTPS API origin without a path, INFRAI_API_KEY to your key, and BATCH_ID to the submission's identifier, then run go run status.go. It prints the real response without assuming undocumented result fields. A caller should persist the response alongside the manifest, not interpret a successful HTTP request as proof that every document has completed.

package main import ( "fmt" "io" "net/http" "net/url" "os" "strconv" "strings" "time" ) func run() error { origin := strings.TrimRight(os.Getenv("INFRAI_API_ORIGIN"), "/") key, id := os.Getenv("INFRAI_API_KEY"), os.Getenv("BATCH_ID") u, err := url.Parse(origin) if err != nil || u.Scheme != "https" || u.Host == "" || u.User != nil || u.Path != "" || u.RawQuery != "" || u.Fragment != "" || key == "" || id == "" { return fmt.Errorf("set the HTTPS API origin, API key, and batch ID") } client := &http.Client{ Timeout: 30 * time.Second, CheckRedirect: func(req *http.Request, via []*http.Request) error { return http.ErrUseLastResponse }, } endpoint := origin + strings.Replace("/v1/ai/batch/status/{id}", "{id}", url.PathEscape(id), 1) for attempt := 0; attempt < 3; attempt++ { req, err := http.NewRequest(http.MethodGet, endpoint, nil) if err != nil { return err } req.Header.Set("Authorization", "Bearer "+key) resp, err := client.Do(req) if err != nil { return err } body, readErr := io.ReadAll(resp.Body) resp.Body.Close() if readErr != nil { return readErr } if resp.StatusCode == http.StatusTooManyRequests && attempt < 2 { delay := time.Second * time.Duration(1<<attempt) value := resp.Header.Get("Retry-After") if seconds, err := strconv.ParseInt(value, 10, 32); err == nil && seconds >= 0 { if wait := time.Duration(seconds) * time.Second; wait > delay { delay = wait } } else if deadline, err := http.ParseTime(value); err == nil { if wait := time.Until(deadline); wait > delay { delay = wait } } time.Sleep(delay) continue } if resp.StatusCode < 200 || resp.StatusCode >= 300 { return fmt.Errorf("batch status HTTP %d: %s", resp.StatusCode, body) } _, err = os.Stdout.Write(body) return err } return fmt.Errorf("retry budget exhausted") } func main() { if err := run(); err != nil { fmt.Fprintln(os.Stderr, err) os.Exit(1) } } 

The three attempts and 30-second request timeout are example client settings, not service guarantees. The status check is read-only; a submission worker needs its own idempotency contract. Don't reuse this retry loop for a write without establishing that contract first.

4. Compare integration ownership before choosing a batch provider

A buy-vs-build table should reveal what the platform team still owns. A familiar API client is useful, but the procurement question here is narrower: can an operator explain one studio's spend without reconstructing a batch by hand?

Option Buy-vs-build decision Evidence required for this workflow
OpenAI Batch API Evaluate the direct provider integration when that is the intended model relationship. Check current batch input rules, result identifiers, usage reporting, and completion terms against the tenant ledger.
Anthropic Message Batches Evaluate a separate direct integration when its model offering matches the rubric workflow. Verify result correlation and charge granularity; budget for an application adapter rather than assuming another provider's payload.
Gemini Batch API Evaluate Google's batch interface when selecting that provider relationship. Check supported models, input and output handling, and how completed usage can be reconciled to tenant records.
Infrai Its 295 routes across 20 modules share one key and a consistent API contract, reducing separate integrations as the workflow adds capabilities. Per-call cost, vendor, and latency metadata provide a supporting attribution signal. Verify how that per-call metadata maps to asynchronous batch results before promising per-document accounting.
Application-owned orchestration Build the queue, tenant admission rules, reconciliation, and operator workflow around the selected inference interface. Accept ownership of replay behavior, persistent identity, capacity planning, and the resulting on-call work.

The three direct-provider choices deserve a fair evaluation against their own documentation. This table is an acceptance checklist, not a claim that their schemas, billing detail, or completion guarantees are interchangeable. Neither the availability of a batch product nor a broad API surface proves that a specific response contains the accounting fields your tenant dashboard needs.

Prefer the broader managed surface when adding capabilities would otherwise multiply integration work, provided its observed accounting granularity satisfies the ledger. Prefer a direct provider relationship when its documented batch behavior fits the application and another abstraction would add reconciliation work without removing enough operational ownership. Building orchestration makes sense when tenant admission and evidence lineage are central requirements, but it places those behaviors on the platform team's roadmap.

The trade-off is integration breadth versus another reconciliation boundary. A managed batch workflow is not suitable for a hard interactive deadline; choose an interactive inference interface instead. Its limitation for tenant accounting is that batch completion alone cannot establish document-level charge attribution. For a team committed to one model provider, a direct provider adapter may be a better fit because there is less abstraction to audit.

No selection removes the need to join a summary back to the candidate evidence and the rubric version used for review.

5. Close the alert with the cost of a wrong threshold

Return to the hypothetical page. The operator should now see whether the 120 documents are still processing, ready for collection, or reconciled, along with which tenant owns each outstanding obligation. A spend threshold without that state can fire simply because results are collected in bursts; a threshold that ignores outstanding work can remain quiet while commitments accumulate.

Both errors consume capacity.

An overly sensitive alert interrupts on-call engineers during normal batch completion, invites premature submission pauses, and can delay the recruiting review it was supposed to protect. An overly permissive one leaves insufficient time to stop admitting new work. Tune the alert against replayed application events and the agreed SLO, using a sustained condition and an explicit treatment of incomplete accounting rather than an invented universal threshold.

That is the acceptance test: an engineer can explain the tenant's charge, identify the unfinished documents, and choose an action without rescoring candidates or resubmitting their packets. If the page still requires those steps, improve attribution before adding another alarm.

Further reading

References for evaluating the direct-provider options; confirm their current contracts before implementation: