For long document text-to-JSON extraction, use bounded review jobs and preserve an explicit incomplete state after a timeout. Chunking must respect the token limit without losing required evidence. A parseable response is insufficient evidence that the review finished.

Short answer: For an edtech service reviewing code changes and returning structured JSON findings, separate request deadlines, token budgets, and evidence coverage. Chunk along source boundaries, account for every required chunk, and validate findings before publishing them. Keep provider adapters behind that application contract so switching providers cannot quietly redefine what a completed review means.

Start the postmortem with one question: what page fired? A timeout alarm tells the responder that a request exceeded a boundary; it doesn't establish whether the document exceeded a context window, retrieval omitted a relevant change, or a response arrived too late to be useful. A green dashboard that counts only successful JSON parses leaves those distinctions unresolved.

How should long document JSON extraction handle a timeout?

Consider a proposed review fixture: a change modifies an assignment grader, its authorization helper, and the tests around student submissions. The desired output is a collection of findings tied to changed files and lines. This is an illustrative workload, not an incident report or a claim about measured performance.

Keep the original failure evidence. Record the review revision, job identifier, chunk identifier, adapter configuration, elapsed time, and outcome category. Avoid recording raw student submissions or source text merely to make an operational chart easier to inspect; stable identifiers can connect the responder to access-controlled evidence.

Then separate the failure domains. A local deadline means the caller stopped waiting. A transport error means the attempt failed at that boundary. A context-budget rejection concerns the input and requested output allowance. Invalid JSON, an unexpected field, and a finding pointing outside the supplied source are application validation failures. Those outcomes deserve different counters because their corrective actions differ, even if the user-facing review status is incomplete in every case.

Don't retry blindly.

RFC 9110 distinguishes idempotent request semantics and restricts automatic retries of non-idempotent requests when the client cannot establish that retrying is safe. A deadline alone doesn't prove that the remote operation never ran. Keep extraction attempts separate from publishing review comments, assign stable application job identities, and make the publication step deduplicate by review revision and finding identity. That is an application design requirement; an HTTP header cannot establish it by itself.

Budget the review before selecting evidence

A long document is a scheduling problem before it is a prompting problem. Reserve space for instructions, the findings contract, and output, then fit source chunks into the remaining input budget using the selected adapter's tokenizer and documented limits. Character count is a rough size signal, not an exact token count. Don't transfer one provider's token estimates unchanged to another.

For the grader fixture, prefer chunks that retain a changed function and its nearby context, with stable file paths and original line numbers attached outside the model-generated text. If a function must be split, retain the relationship between its fragments. Overlap can preserve a boundary-crossing clue, but it also repeats evidence and can produce duplicate findings; reconciliation needs source coordinates and the review revision rather than string equality alone. Embeddings and reranking belong to evidence selection. Neither is proof that all required changes were reviewed. A useful policy is to make every changed region mandatory and use retrieval to select supporting context, such as a related helper or test, within the remaining budget. Rerank those optional candidates if that improves the chosen evaluation fixtures. Record which candidates survived, and retain enough information to distinguish an intentionally excluded candidate from a required chunk that never ran. Otherwise the postmortem must reconstruct the review scope from transient request logs, precisely when the responder needs a durable answer about what was actually inspected.

Missing evidence matters.

An empty findings array says nothing about a helper the extraction never received.

Suppose the selected context includes the grader but omits the authorization helper. An empty findings array cannot establish that access checks are correct. The coordinator should retain a coverage ledger containing the expected chunks, their terminal states, and the selected supporting evidence. Only the coordinator can decide whether the required work completed; a model response should never be allowed to declare an unseen chunk reviewed.

The trade-off is additional orchestration and potentially more source processing than a retrieval-only approach. The benefit is a review status that survives a provider change: completed means the same required evidence passed through the same acceptance rules. Choose a narrower review scope explicitly if that cost is unacceptable, and show the scope to the developer.

Put a small contract around each attempt

Use a language-neutral findings contract even if the caller is a Node.js service. The following Go boundary illustrates the validation step behind that service; it uses an abstract adapter, with no implied endpoint or provider-specific request shape. A real adapter must honor the context and apply its own transport and response-size limits.

The example permits only supplied file paths and line ranges, rejects unknown JSON fields and trailing values, and caps the returned payload. Its 64 KiB (65,536 bytes) cap is an illustrative local policy, not a model limit. The coordinator still owns coverage, cancellation scheduling, retries, and publication.

package review import ( "bytes" "context" "encoding/json" "errors" "io" ) type Finding struct { File string `json:"file"` Line int `json:"line"` Reason string `json:"reason"` } type Result struct { Findings []Finding `json:"findings"` } type SourceRange struct { File string Start, End int } type Adapter interface { Extract(context.Context, string) ([]byte, error) } func Extract(ctx context.Context, a Adapter, input string, allowed []SourceRange) (Result, error) { var out Result raw, err := a.Extract(ctx, input) if err != nil { return out, err } if err := ctx.Err(); err != nil { return out, err } if len(raw) > 64*1024 { return out, errors.New("findings payload exceeds local cap") } dec := json.NewDecoder(bytes.NewReader(raw)) dec.DisallowUnknownFields() if err := dec.Decode(&out); err != nil { return Result{}, err } var extra any if err := dec.Decode(&extra); err != io.EOF { return Result{}, errors.New("unexpected trailing JSON") } if out.Findings == nil { return Result{}, errors.New("findings array is required") } for _, f := range out.Findings { valid := false for _, r := range allowed { if f.File == r.File && f.Line >= r.Start && f.Line <= r.End { valid = true break } } if !valid || f.Reason == "" { return Result{}, errors.New("invalid finding evidence") } } return out, nil }

Treat this as a minimum acceptance boundary. Its limitation is that structure and source coordinates cannot establish that a finding's reasoning is correct. The standard JSON decoder also has documented behavior around duplicate object keys, so this example should not be described as a duplicate-key rejection mechanism. If that is part of the contract, add and verify a separate check.

The context check after extraction rejects an already-expired attempt, but it cannot force an adapter that ignores cancellation to stop doing work. Test that behavior at the adapter boundary. A local byte check after the response arrives likewise cannot protect against an unbounded transport read, which is why response limits must also exist inside the adapter.

Verify the failure paths before changing traffic

Build a fixed set of review fixtures around the acceptance contract. Include an empty findings array for a completed clean chunk, a missing findings field, a valid finding outside the supplied line ranges, malformed JSON, a second JSON value, and an adapter that returns after cancellation. Include a multi-chunk change whose relevant helper falls outside the initial retrieval candidates. These are proposed tests, not reported results.

Check the final review state as well as each attempt. One completed chunk and one timed-out chunk must remain an incomplete review. Retrying a completed extraction must not publish the same comment twice. Replaying a result against a newer revision must fail revision validation, even when its file and line still exist.

The acceptance test ends at the published review, not at the adapter's return value.

For portability, run the same fixtures through each candidate adapter and compare accepted findings, rejected payloads, coverage, and deadline behavior. A shared request shape is useful, but it doesn't demonstrate identical token accounting or output behavior. Keep adapter-specific translation out of the coordinator's definition of completion.

Bound retries by the remaining job deadline and a maximum attempt count. Queue waiting consumes the review's overall time allowance too; restarting a full timeout for every attempt can leave a developer waiting far longer than the original budget suggests. Track attempted work and token usage where available to understand the operational cost of repeated extraction, without turning a provider's unit price into an architecture argument.

Roll back without publishing a false clean review

Keep chunking rules, retrieval settings, findings-contract version, and adapter configuration attached to the job. Change one of those at a time during a limited rollout, then inspect incomplete-review rates, validation failures, and evidence coverage alongside elapsed time. A faster extraction with less required evidence is a different review scope.

If the rollout breaks the acceptance contract, stop assigning new jobs to that configuration and restore the previously validated configuration. Quarantine unfinished results, retain their revision identities, and reschedule only within the original review policy. Never convert abandoned work into an empty successful review to clear an alert.

The operational recommendation is straightforward: publish a clean review only after all required chunks reach accepted completion. Keep failure categories and evidence accessible enough that the next responder can explain which boundary failed, what work remains, and whether replay is safe. That is the contract worth carrying between providers.

References