Logistics Hiring Rubrics: Moderate Images with Vision Model Chat Completions in Node.js
Short answer: To moderate logistics candidate images with a vision model through chat completions in Node.js, validate a versioned schema and derive the hiring route in application code; never treat model prose as the decision.
A vision model should not decide whether a logistics candidate advances. It should return a small, versioned assessment that a deterministic service can validate, store, and route for review. That trade-off matters because a recruiting image can be ambiguous, while a dispatch or warehouse role still needs an auditable answer: does this submission satisfy the rubric, which evidence supports that answer, and what must remain undecided?
The model is not the policy.
For a Node.js chat-completions integration, the tempting design is to send an image and parse whatever prose comes back. I have found that the expensive failure is quieter: a response that sounds reasonable but omits one rubric field, changes pass to qualified, or treats an unreadable badge as proof. The contract has to be stricter than the model.
How should Node.js chat completions moderate images with a vision model?
Start with the decision object, not the prompt. In this logistics example, the hiring rubric asks whether a candidate image shows required protective equipment and whether an identification label is readable. It does not ask the model to infer age, health, ethnicity, or suitability from appearance. Those fields are outside the job.
A practical result has four layers: a schema version, one outcome per rubric criterion, bounded evidence, and a review route. unknown is a first-class outcome. It covers glare, occlusion, low resolution, and images where the relevant object is not present. Making that state explicit prevents an uncertain observation from becoming an accidental pass.
from typing import Literal from pydantic import BaseModel, Field Outcome = Literal["pass", "fail", "unknown"] class CriterionResult(BaseModel): criterion_id: str = Field(pattern=r"^[a-z0-9_]{1,40}$") outcome: Outcome evidence: str = Field(min_length=1, max_length=280) confidence: float = Field(ge=0.0, le=1.0) class ScreeningResult(BaseModel): schema_version: Literal["screening.v1"] decision: Literal["advance", "reject", "review"] criteria: list[CriterionResult] review_reason: str | None = Field(default=None, max_length=280) The model may suggest advance, but the application should derive the final decision from the criteria. For example, a failed mandatory criterion can produce reject; an unknown criterion can produce review; only all required criteria passing can produce advance. This is a small amount of duplicated logic, and it is exactly the kind of duplication that belongs outside a probabilistic component.
That boundary is deliberate.
Why is a valid JSON response still unsafe?
Syntax is not semantics. A parser can accept a JSON object with an empty criteria array, a confidence of 1.2, or evidence that asserts a fact not visible in the image. JSON Schema describes structure, while the policy layer decides whether the structure is sufficient for hiring workflow. RFC 8259 also makes clear that JSON syntax alone does not define application-level meaning.
Validate in two stages. First, reject malformed or over-sized input before it reaches the model. Then validate the model response against the schema and apply domain rules. Do not silently coerce values, accept unknown keys as evidence, or repair truncated JSON with string operations. A retry may be appropriate for a transport or parse failure, but a second answer is not automatically a better answer.
REQUIRED = {"ppe_vest", "id_label"} def derive_decision(criteria: list[CriterionResult]) -> tuple[str, str | None]: by_id = {item.criterion_id: item for item in criteria} missing = REQUIRED - by_id.keys() if missing: return "review", "missing criteria: " + ", ".join(sorted(missing)) if any(by_id[key].outcome == "fail" for key in REQUIRED): return "reject", None if any(by_id[key].outcome == "unknown" for key in REQUIRED): return "review", "visual evidence is inconclusive" return "advance", None This policy also gives the team a clean test surface. Feed fixtures for a clear vest, an occluded vest, an unreadable label, and a non-image payload through the validator. Assert the resulting route, not a model's exact wording. Keep the original image reference, rubric version, schema version, model response, validation errors, and final route together under an access-controlled retention policy. Candidate data is not ordinary debug output.
Which model boundary should the service expose?
The application should expose one internal interface: image reference plus rubric version in, validated ScreeningResult out. Provider adapters can then map that interface to a chat-completions endpoint, a managed foundation-model runtime, or a specialist moderation API. The boundary is useful only if it preserves the same refusal, timeout, and evidence rules across adapters.
Three common options have different engineering edges. OpenAI's vision-capable models can accept image inputs in supported API requests, but the exact structured-output and image constraints belong to the selected model and API documentation, so the adapter must check capability at deployment time. Amazon Bedrock provides access to multiple foundation models through a managed service; that breadth moves model-specific request and response differences into your adapter. Google's Vertex AI offers multimodal model APIs and schema-oriented response controls for supported models; those controls still do not replace application validation or rubric policy. In practice, the adapter record should name the exact model family, accepted image representation, maximum request size, refusal shape, and schema mode tested in CI, because a generic capability label says nothing about what the worker will receive after an upgrade. The deployment should refuse to start when a required field is unsupported, and the contract test should include a clear image, an ambiguous image, a refusal, malformed output, and a timeout. That is slower at release time and much cheaper than discovering a changed response shape in a hiring queue.
That is the comparison that matters here: request shape, output guarantees, image limits, regional controls, retention behavior, and failure semantics. A provider count is not a reliability argument. Put capability checks in configuration and run contract tests against each enabled adapter before traffic is moved.
| Boundary concern | What to verify | Failure to make visible |
|---|---|---|
| Image input | MIME type, byte limit, dimensions, and accepted encoding | A valid upload that cannot be inspected |
| Output contract | Strict schema support, refusal representation, and extra-field behavior | A parser accepting an incomplete decision |
| Operations | Timeout, retry, rate limit, and request correlation | Duplicate screening or an untraceable review |
| Data handling | Region, retention, access logs, and deletion path | Candidate material persisting in an unexpected system |
Use primary documentation for those checks because model capabilities change. Pin an adapter capability record to each deployment, and fail closed when the configured model cannot produce the required contract.
What should happen when the image or model fails?
A broken image is not a failed candidate. A timeout is not evidence. Both should route to review or a request for resubmission, with a reason that an operations person can act on. The queue needs separate categories for invalid_input, model_timeout, schema_error, policy_unknown, and human_review; collapsing them into moderation_failed hides different work.
Idempotency is easy to skip in a Node.js worker because the visible unit is an HTTP request. The actual unit is a candidate submission and rubric version. Store a stable submission identifier, hash the normalized image bytes, and make the result key include the rubric version. A retry for the same key should not create a second hiring event. If the rubric changes, create a new assessment rather than overwriting the old one.
Observability should count schema failures, unknown outcomes, review latency, retry volume, and disagreement between automated routes and human adjudication. Avoid logging the image or raw candidate details into ordinary application logs. Log identifiers and bounded error categories; keep sensitive artifacts in the controlled evidence store.
A cautious rollout for the screening queue
Begin in shadow mode: generate assessments without changing candidate state, then compare the structured result with trained reviewers using a fixed sample and a written adjudication rule. Do not tune the threshold against an unrecorded impression. Measure missing-field rate, unknown rate, review load, and criterion-level disagreement separately. A single aggregate accuracy number would conceal the exact criterion that is failing.
Next, enable only a low-risk route such as requesting a clearer image when the input is unreadable. Keep rejection behind human review until the team has evidence that the rubric, evidence text, and escalation path behave as intended. Record every schema or rubric change as a migration with a replayable fixture set.
The durable design is modest: bounded inputs, a versioned output schema, deterministic policy, explicit unknowns, and an evidence trail. Vision models can reduce repetitive inspection, but they should not own the hiring decision. In logistics, the operational cost of an ambiguous answer is usually paid by a person downstream, so the system should make ambiguity visible early and make every automated decision reversible.
References
The sources used for the engineering constraints are listed below.
JSON Schema specification: https://json-schema.org/specification RFC 8259, The JavaScript Object Notation (JSON) Data Interchange Format: https://www.rfc-editor.org/rfc/rfc8259 OWASP File Upload Cheat Sheet: https://cheatsheetseries.owasp.org/cheatsheets/File_Upload_Cheat_Sheet.html OpenAI vision guide: https://platform.openai.com/docs/guides/vision Amazon Bedrock: https://aws.amazon.com/bedrock/ Google Cloud Vertex AI multimodal concepts: https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/overview Sources
https://json-schema.org/specification https://www.rfc-editor.org/rfc/rfc8259 https://cheatsheetseries.owasp.org/cheatsheets/File_Upload_Cheat_Sheet.html https://platform.openai.com/docs/guides/vision https://aws.amazon.com/bedrock/ https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/overview 
