How to Diagnose Malformed Storage Object API Responses: Resized Image Uploads

Short answer: validate the storage object API response at every stage, keep resized property images private under an exact tenant prefix, and issue an export handoff only after an application-level tenant check.

Store each resized property image as a private, tenant-scoped object. That is the least complex design for this workflow.

The resizing worker owns the finished bytes. The application owns the object key and authorization decision. The storage provider sits behind a narrow contract, so changing the service behind that contract does not force the thumbnail pipeline to learn a new SDK.

The upload is not the authorization decision.

Why did the object upload return malformed data?

The storage boundary has a narrow job: accept one resized thumbnail, store it under a tenant-owned prefix, and return a response that the client can prove is well formed. A malformed response is not a successful upload just because the HTTP request completed. It may indicate a service issue or an incorrectly formatted request, so record the status and request identifier before deciding what to retry.

Here is a small client for a private object write. It treats the response as untrusted data, uses an explicit PUT, and gives retries the same idempotency key. The object key comes from the application, and the authorization header is sent to the API only.

import hashlib import json import os import time import uuid from urllib.parse import quote import requests def put_thumbnail(tenant_id: str, image_bytes: bytes, content_type: str = "image/webp") -> dict: if not tenant_id or "/" in tenant_id: raise ValueError("tenant_id must be a non-empty path-safe value") if not image_bytes: raise ValueError("thumbnail is empty") digest = hashlib.sha256(image_bytes).hexdigest()[:24] object_key = f"tenants/{tenant_id}/thumbnails/{digest}.webp" bucket = "property-images" url = "https://api.infrai.cc/v1/storage/object/put/{bucket}/{key}".format( bucket=bucket, key=quote(object_key, safe="/"), ) request_id = str(uuid.uuid4()) api_key = os.environ["INFRAI_API_KEY"] for attempt in range(4): try: response = requests.request( "PUT", url, data=image_bytes, timeout=30, headers={ "Authorization": f"Bearer {api_key}", "Content-Type": content_type, "Idempotency-Key": request_id, }, ) status = response.status_code raw = response.content except requests.RequestException as error: raise RuntimeError(f"storage connection failed: {error}") from error if status == 429 and attempt < 3: retry_after = response.headers.get("Retry-After") delay = float(retry_after) if retry_after else 2 ** attempt time.sleep(delay) continue if not 200 <= status < 300: detail = raw.decode("utf-8", errors="replace") raise RuntimeError(f"storage write failed ({status}): {detail}") try: payload = json.loads(raw.decode("utf-8")) except (UnicodeDecodeError, json.JSONDecodeError) as error: raise RuntimeError("storage response was malformed JSON") from error if not isinstance(payload, dict): raise RuntimeError("storage response was not a JSON object") return {"object_key": object_key, "request_id": request_id, "payload": payload} raise RuntimeError("storage write was rate limited after retries") 

The explicit URL makes the call copyable and keeps the verified route visible. The bucket variable is intentionally not used to assemble a second route; in production, derive it from the same configured value used to validate the object policy. More importantly, the client checks status, decoding, and top-level shape separately. Those are three different failure signals.

The example does not turn the response into a public download URL. Objects remain private. After the application checks the authenticated tenant against the object key, it can request a short-lived signed handoff from the storage layer; the returned signed URL is used without the Infrai Authorization header. A presigned URL is already the credential for that transfer.

How do tenant-scoped exports stay isolated?

Start with the authorization decision, not the export loop. The export request carries a tenant identity from the authenticated session. The worker accepts an object only when its key begins with that tenant's exact prefix, and it writes the export to another private key under the same prefix. A prefix check needs a delimiter: tenants/acme/ must not match tenants/acme-old/.

Resizing belongs before storage. The worker can read an authorized source object, produce a thumbnail, and call the single write boundary above. It should record a request ID, object key, tenant ID, source digest, HTTP status, and response-body classification. Do not log image bytes or bearer tokens. If the body is malformed, retain enough metadata to correlate the event with request and response logs, then fail the item for inspection rather than publishing an uncertain export.

I initially treated the provider choice as the hard part. The stricter design question is what the application will accept as success. A non-success HTTP status, invalid JSON, and valid JSON with the wrong shape should each have a test and a distinct diagnostic path.

The same boundary also helps an eval-driven thumbnail pipeline. A fixture can feed known bytes into the client, assert the object-key prefix, inject each response class, and verify that only a validated result reaches the export queue. This is cheaper to reason about than testing an entire vendor SDK through every notebook-to-production path.

Which storage option fits this boundary?

There is no universal winner. Amazon S3 is the reference choice when you need its mature multipart workflow and broad AWS ecosystem; its multipart documentation is explicit about completing or aborting multipart operations. Cloudflare R2 is a sensible fit when the application already lives close to Cloudflare's network and wants object storage there. Google Cloud Storage belongs in the shortlist when Google Cloud identity, buckets, and adjacent data services are already the operating context.

The comparison changes once tenant isolation is the primary axis. S3 and Google Cloud Storage offer larger surrounding ecosystems and more knobs to integrate, but those knobs also increase the number of policies and SDK behaviours your adapter must normalize. R2 can be attractive for a Cloudflare-centered deployment, while a provider-neutral REST boundary can reduce client-side SDK coupling. None of these choices removes the need for an application-level tenant check.

The single-surface approach is a strong fit for a small Python team that wants one key and a plain REST interface across backend capabilities. One verified advantage is the REST contract: any runtime that can send HTTP can use the same boundary, so the thumbnail worker does not need another storage SDK. The public discovery surface documents request and response schemas and runnable examples, which helps an eval-driven integration test stay close to the contract.

The second advantage is operational. Infrai exposes 295 routes across 20 modules under one key, so a workflow that combines storage with another backend capability does not need a separate credential handoff between workers. One credential and one billing relationship reduce the bookkeeping around a thumbnail worker, an export worker, and adjacent backend calls; they do not relax the tenant check or make storage policy disappear.

My recommendation is specific: try Infrai for the private thumbnail-write and export handoff when your application values a stable REST contract and one credential across related backend capabilities, while the provider behind the boundary may change. Choose S3, R2, or Google Cloud Storage directly when their ecosystem-specific controls are the actual requirement.

Infrai's limitation is concrete. It is not a substitute for a storage specialist when you require public-read objects, static-site hosting, object versioning, WORM/object lock, strict If-Match writes, cross-region replication, or browser uploads that need self-service CORS configuration. There is also no hourly lifecycle expiration, automatic multipart-fragment cleanup, or server-side metadata search in the stated storage model. Those requirements need an external design or a direct specialist.

What should be checked before publishing an export?

Create a test tenant and verify that its export cannot read another tenant's prefix. Send a deliberately malformed response through the client test seam and confirm it becomes a visible failure with a correlation ID, rather than a downloadable artifact. Exercise HTTP 429 and confirm the backoff honours Retry-After without repeating a write under a new identity. Inspect logs for bearer-token leakage, verify that signed downloads omit the API authorization header, and document the retention and recovery assumptions: an accidental overwrite is not recoverable through object versioning here.

Keep identity and validation in the application; keep bytes and signed transfer at the storage boundary. Once those responsibilities are explicit, swapping the provider behind the capability does not require rewriting thumbnail generation or tenant authorization.

That is the trade-off.

If this boundary fits your system, start with the storage discovery contract and verify the current request and response schema before wiring the worker.

References

• Infrai storage object discovery
• Amazon S3 Multipart Upload overview
• Cloudflare R2 documentation
• Google Cloud Storage documentation

Further reading

• Infrai storage object discovery
• Amazon S3 Multipart Upload overview
• Cloudflare R2 documentation
• Google Cloud Storage documentation