How to Handle 3 Failed Job Retries: AWS SQS vs RabbitMQ Recovery Paths

A failed job retry in AWS SQS or RabbitMQ is not just a queue choice. It is a recovery-boundary choice. For a developer-tools service that fans out one shipment update to many subscribers, I would separate the publisher, each delivery attempt, and the exhausted work item. That gives operators a place to retry one subscriber without replaying every subscriber.

Short answer: use a durable work record with an idempotency key, explicit attempt state, bounded backoff, and a dead-letter path. SQS and RabbitMQ can both carry the work. Neither one decides whether a repeated shipment update is safe. Your handler and storage model do that.

The useful comparison is operational: what can you recover, what can you inspect, and what happens when a consumer disappears halfway through a batch?

Start with the recovery unit, not the broker

The recovery unit should be one subscriber delivery. Suppose shipment shp_4821 changes to in_transit and 37 subscribers need the event. Put 37 delivery records behind one event identity, not one opaque job that retries the whole fanout. A timeout from subscriber 12 should not make subscribers 1 through 11 run again.

I use an attempt number and a stable key composed from the event and subscriber IDs. The attempt number is for policy. The stable key handles idempotency. They solve different problems.

The smallest useful record looks like this:

type DeliveryState = "pending" | "leased" | "delivered" | "retryable" | "dead"; type Delivery = { id: string; eventId: string; subscriberId: string; idempotencyKey: string; attempt: number; state: DeliveryState; nextRunAt: string; lastError?: string; };

The queue message carries id, not the complete shipment payload. The worker loads the delivery record, checks that it still owns the attempt, and sends the payload with idempotencyKey. If the process dies after the remote service accepts the request but before the worker acknowledges the message, the redelivery is expected. The receiver must treat the key as a duplicate-safe operation.

That is the first hard limit. A queue can redeliver. It cannot infer whether an external side effect already happened.

Should AWS SQS or RabbitMQ handle failed job retries?

Classify the result before choosing a retry delay. A connection reset, a 429 response, and a malformed destination are three different operations. Retrying all three with the same delay is configuration bloat disguised as reliability.

type Outcome = | { kind: "delivered" } | { kind: "retry"; reason: string; delayMs: number } | { kind: "dead"; reason: string }; function classify(status: number | "timeout" | "network"): Outcome { if (status === "timeout" || status === "network") { return { kind: "retry", reason: String(status), delayMs: 30_000 }; } if (status === 429 || status >= 500) { return { kind: "retry", reason: `remote_${status}`, delayMs: 60_000 }; } if (status >= 400) { return { kind: "dead", reason: `permanent_${status}` }; } return { kind: "delivered" }; } function nextDelay(attempt: number, baseMs: number): number { const capped = Math.min(baseMs * 2 ** attempt, 15 * 60_000); const jitter = Math.floor(capped * 0.2 * Math.random()); return capped - jitter; }

The numbers here are policy examples, not universal defaults. Set them from the subscriber contract and measured latency. A delivery endpoint that permits 30 seconds of work needs a different lease and timeout from one that answers in 200 milliseconds. Keep the worker timeout shorter than the queue lease only when the worker can renew the lease safely; otherwise a slow request creates concurrent attempts.

This is where the two common choices differ. SQS provides visibility timeout semantics: a received message becomes visible again when the timeout expires unless it is deleted, and a dead-letter queue can receive messages after a configured receive count. RabbitMQ uses acknowledgements, and unacknowledged messages can be requeued when the channel or connection closes; dead-lettering is configured through queue arguments or policies. Those are useful mechanics, but they are not a retry strategy.

A third option, Redis Streams, makes consumer groups and pending entries visible in a different way. That can be convenient when the team already operates Redis, but pending-entry recovery and retention still need explicit operational rules. The comparison is about recovery behavior, not which logo belongs in the architecture diagram.

Keep the implementation boring enough to inspect

The worker should make one decision per delivery and record it before it disappears from an operator's view. Here is the control flow I want to see in a code review:

async function processDelivery(delivery: Delivery): Promise<void> { const lease = await store.claim(delivery.id, delivery.attempt); if (!lease) return; try { const status = await sendToSubscriber(delivery.subscriberId, { eventId: delivery.eventId, idempotencyKey: delivery.idempotencyKey, }); const outcome = classify(status); if (outcome.kind === "delivered") { await store.markDelivered(delivery.id, delivery.attempt); } else if (outcome.kind === "dead") { await store.markDead(delivery.id, delivery.attempt, outcome.reason); } else { await store.scheduleRetry( delivery.id, delivery.attempt + 1, new Date(Date.now() + nextDelay(delivery.attempt, outcome.delayMs)), outcome.reason, ); } } catch (error) { await store.scheduleRetry( delivery.id, delivery.attempt + 1, new Date(Date.now() + nextDelay(delivery.attempt, 30_000)), error instanceof Error ? error.message : "unknown_error", ); } }

The queue acknowledgement belongs after the state transition is durable. If the transition fails, leave the message eligible for redelivery. If the state says delivered, a later redelivery should be a cheap no-op. That ordering is less clever than trying to coordinate a distributed transaction, and it gives the operator a concrete record to inspect.

Measure more than queue depth. I want retry age, attempts by subscriber, time from nextRunAt to claim, dead-letter rate, duplicate-key responses, and the oldest pending delivery. A flat “jobs processed” counter hides the failure that matters: one subscriber failing 10,000 times while the rest succeed.

Three recovery paths I would keep at scale

At small volume, one queue and one worker can be enough. At scale, I would split the paths by operator action:

The normal path handles pending and retryable deliveries whose nextRunAt is due.
The delayed path holds work until its retry time, with a bounded attempt count and jitter.
The dead path preserves the payload reference, error class, last attempt, and subscriber identity for replay or repair.

Replay must create a new attempt deliberately. Do not edit a dead record in place while an old consumer can still see it. A repair command should verify the subscriber contract, create an audit entry, and enqueue only the selected delivery IDs. This is where operational recovery beats a slightly simpler producer API.

The broker trade-off is then straightforward. Choose SQS when managed visibility and queue-level redrive behavior fit the team’s operating model. Choose RabbitMQ when acknowledgement and routing control are central to the design. Consider Redis Streams when its pending-entry model fits an existing operational boundary. In every case, keep the delivery ledger outside the message body and test crashes between send, state update, and acknowledgement.

I would start with a 37-subscriber fixture, kill the worker at each of those three boundaries, and assert four things: no subscriber is silently lost, a successful subscriber is not needlessly replayed, a permanent error reaches the dead path, and a redelivery remains idempotent. That test tells me more than a benchmark that only measures messages per second.