Use a new immutable object key for every avatar upload, then conditionally update the profile record that selects the current file. This avoids turning an object-storage overwrite race condition into the product's definition of which user action wins; the existing avatar remains readable until a later, idempotent cleanup pass can prove it is no longer referenced.

The UI calls this “replace photo.” The system has two separate jobs: accept bytes and publish a chosen revision. Treating them as one PUT to avatars/42/current.jpg makes the completion order of distributed requests decide the visible result. That is a poor policy for a user action, and it is difficult to observe once CDN caching and client retries join the path.

Order matters.

How should an avatar upload safely replace an existing file in object storage?

Start by assigning each proposed image a fresh, unpredictable key such as avatars/<account-id>/<random-token>. Upload to that location, then perform a single conditional update of the profile row: change avatar_key only when its current revision equals the revision the client read. The database update, rather than a stable object key, is the authority for publication.

Consider a bounded production scenario. A user has two browser tabs open. Upload A starts first, upload B starts later, B reaches storage first, and A completes after B. With a fixed key, A's late overwrite can replace the bytes selected by the user's later action. With an immutable key and a compare-and-swap on the profile revision, B can publish its key and A loses the conditional update; A's object is harmlessly unreferenced. The storage operation may have succeeded in both cases. The application outcome is different.

This distinction matters during retry handling. A client can lose a response after a successful upload and resend the request, while a queue can deliver a command after a newer command has already committed. HTTP defines PUT as replacing the state of its target resource, and conditional requests provide preconditions, but neither rule supplies a transaction across an object namespace, a database row, and a cache. A storage write can therefore be durable while the profile row still selects the old key; a database commit can select the new key while a cache retains a prior representation under a stable URL; and a retry can recreate an already-settled intent unless the application records an idempotency result. Those are independent state machines, so don't use a transport success as the business success criterion.

Small invariant, big payoff: a profile has one current revision; published bytes are immutable; an older request cannot move the current pointer; and deletion happens only after the object has no live reference. The invariant gives an on-call team something concrete to test and page on, instead of treating a 2xx storage response as proof that the avatar changed.

The incident lesson: completion order is not user intent

The failure mode is usually quiet. A request log reports a successful upload, an image exists at the expected object location, and the support report arrives much later because the page shows a different profile photo than the user chose. No storage durability claim has been violated. The policy was underspecified.

There are three common ways to define that policy:

Design Authority for “current” What can go wrong Operational cost
Stable key and overwrite Last completed object write A stale completion can win; caches can serve an older representation Cache and conditional-write analysis
Immutable keys plus conditional pointer Profile revision in the database Orphaned objects need collection Revision logic and a sweeper
Per-user serialized command stream Ordered command consumer Backlog, replay, and partition handling become correctness concerns Queue operations and lag SLOs

For user-controlled profile images, the second design generally makes the failure boundary easiest to state: the request either advances the revision or it does not. A 409 Conflict is a useful result when the client attempted to publish against an obsolete revision; return the current revision so the client can reload before presenting another choice. Do not silently retry the pointer swap with a newer expected revision, because that would change the user's intent into “last arrival wins.”

The object can be written before the pointer transaction because an unreferenced object is recoverable state. Deleting the old object before the pointer transaction is unsafe: a failed transaction can leave the live profile pointing to removed bytes. This ordering is the preventative path. It also keeps the request critical path short.

Keep the pointer authoritative.

A Go write path with an explicit publication boundary

The interface below deliberately separates storage from the repository. SwapAvatar must be one conditional database update, such as UPDATE profiles SET avatar_key = ?, avatar_revision = avatar_revision + 1 WHERE account_id = ? AND avatar_revision = ?; its boolean says whether that update selected a row. The code does not delete the prior object.

package avatar import ( "context" "crypto/rand" "encoding/hex" "errors" "fmt" "io" ) var ErrRevisionConflict = errors.New("avatar revision conflict") type ObjectStore interface { Put(ctx context.Context, key string, body io.Reader) error } type AvatarRepository interface { SwapAvatar(ctx context.Context, accountID, key string, expectedRevision int64) (revision int64, swapped bool, err error) } type Service struct { Objects ObjectStore Profiles AvatarRepository } func (s Service) Replace(ctx context.Context, accountID string, expectedRevision int64, body io.Reader) (string, int64, error) { token := make([]byte, 16) if _, err := rand.Read(token); err != nil { return "", 0, fmt.Errorf("generate object key: %w", err) } key := "avatars/" + accountID + "/" + hex.EncodeToString(token) if err := s.Objects.Put(ctx, key, body); err != nil { return "", 0, fmt.Errorf("store avatar: %w", err) } revision, swapped, err := s.Profiles.SwapAvatar(ctx, accountID, key, expectedRevision) if err != nil { return "", 0, fmt.Errorf("publish avatar pointer: %w", err) } if !swapped { return "", 0, ErrRevisionConflict } return key, revision, nil }

In the handler, bind an idempotency key to the account, the intended request, and its outcome. A retry then retrieves its prior result rather than producing another candidate object. Validate media type and size before the write, store no image body in logs, and emit one structured event containing the request identifier, expected revision, resulting revision, byte count, object key, and outcome. Access-controlled object URLs deserve the same care as credentials in logs.

Cache behavior belongs in the design review. A URL derived from an immutable key can be cached aggressively because its bytes do not change. A stable URL needs a revalidation policy that is correct for the reader's cache, the CDN, and any image proxy. The former makes rollout and rollback clearer: publishing a new pointer changes future URLs, while prior URLs retain a consistent representation for their configured lifetime.

Capacity, cleanup, and the conditions that change the choice

The catch is extra inventory. An immutable-key strategy retains the current image, recently superseded images, and objects created by requests that lost the revision race. Garbage collection is therefore part of the architecture, not housekeeping. A collector should run with a minimum object age, pagination, rate limits, a final reference check immediately before deletion, and a dry-run mode. It must be idempotent: repeating a deletion candidate should converge on the same desired state.

Set the minimum age from measured bounds: maximum client retry window, transaction duration, asynchronous queue delay, and any retention or cache rule that requires the old object to remain reachable. Then add margin and model the resulting byte inventory at peak upload rate. A capacity plan should include retained versions, incomplete uploads where the chosen protocol creates them, collector lag, and the growth rate of unreferenced bytes. A low request-error rate does not protect the storage budget.

Useful service-level indicators follow the business transition rather than the transport call:

  • accepted pointer swaps divided by valid upload attempts;
  • conflict rate, separately from validation and storage failures;
  • elapsed time from accepted publication to a readable current object;
  • age and byte total of unreferenced objects; and
  • reconciliation count for profile pointers whose referenced object is absent.

Reconciliation is worth scheduling even in a small service. Scan old candidates, compare them with live profile references, and test restoration from the backups or replication mechanism the team actually owns. A retention setting is not a recovery exercise.

This approach is not suitable when the requirement is synchronous physical erasure at the end of the request and the wider deletion design cannot prove that guarantee. Use a conditional overwrite when retained overlap is unacceptable, the storage system's preconditions are well specified, every cache is correctly revalidated or bypassed, and last-writer behavior matches the product. Use per-account serialization when the product requires command order rather than conflict detection. Those choices move the coordination authority; they do not eliminate it.

What should a storage selection review require?

Ask the same questions of a managed service, a self-hosted deployment, or a generic API abstraction. Which component owns the current-version decision? Can it perform a conditional write or a transaction with the profile record? What is the documented consistency and cache behavior for reads after a write? How are retries deduplicated? Who operates lifecycle cleanup, audit retention, access control, and restoration drills?

The buy-versus-build decision should include the on-call contract, not just an upload API. A managed object service can remove hardware and durability operations while leaving publication semantics, database ownership, and application-level garbage collection with the team. A self-hosted service can meet locality or control constraints but adds upgrades, capacity forecasting, replication verification, and paging responsibility. Choose the model whose failure modes fit the team's error budget and operating maturity.

For most avatar systems, a fresh object key plus a conditional profile-pointer update is the narrowest path to a clear rule: the record selects a revision, immutable bytes make caching predictable, and delayed collection handles losing attempts. The implementation is modest; the discipline is in measuring the states that accumulate around it.

References