Dead Letter Queue vs Retry: When to Use Each (and When to Use Both)

Quick answer

• Retry is for transient failures — network blips, timeouts, temporary overload. The fix is trying again with a delay.
• Dead letter queue (DLQ) is for persistent failures — bad data, missing resources, bugs. The fix is human inspection and remediation.
• In practice, you need both: retry N times with backoff, then route to the DLQ.

The decision framework

Every message-processing system faces this question: a message failed. What now?

The answer depends on why it failed:

The key insight: you don’t pick one pattern and ignore the other. You chain them. Retry handles the transient failures. The DLQ catches what retries can’t fix.

How retry works (and how it fails)

Retry means: wait, then try again. Simple. But naive retry creates problems:

# Bad: retry immediately, forever
while True:
    try:
        process(message)
        break
    except Exception:
        pass  # infinite tight loop

Three problems with this:

1. Tight loop — hammers the downstream service, making the outage worse
2. Infinite — a message with bad data spins forever, blocking the queue
3. No visibility — nobody knows this message is stuck

Retry done right: exponential backoff

import time

max_retries = 3
for attempt in range(max_retries):
    try:
        process(message)
        break
    except TransientError:
        if attempt == max_retries - 1:
            raise  # exhausted retries
        wait = 2 ** attempt  # 1s, 2s, 4s
        time.sleep(wait)

With exponential backoff:

• Attempt 1: immediate (or after 1s)
• Attempt 2: wait 2s
• Attempt 3: wait 4s
• After that: give up and send to DLQ

Total wait time: ~7 seconds. If the downstream service recovers within that window, the message goes through. If not, it goes to the DLQ instead of blocking the queue forever.

How a DLQ works (and when you need it)

A dead letter queue is exactly what it sounds like: a queue for messages that couldn’t be processed. When a message fails after all retries are exhausted, you move it to the DLQ instead of dropping it.

The DLQ gives you:

• Visibility — you can inspect failed messages and see what’s breaking
• Non-blocking — healthy messages continue processing while bad ones sit in the DLQ
• Recovery path — after fixing the root cause, you can replay messages from the DLQ

def process_with_dlq(message):
    try:
        process(message)
    except TransientError:
        retry_with_backoff(message)      # try again
    except PersistentError:
        dlq.send(message)                # inspect later
    except Exception:
        retry_with_backoff(message)      # unknown: try first
        if retries_exhausted(message):
            dlq.send(message)            # then DLQ

The combined pattern

In production, you almost always use both:

Message arrives
  │
  ▼
Try to process ──► Success ──► Done
  │
  ▼ (transient error)
Retry #1 (1s delay)
  │
  ▼ (still fails)
Retry #2 (2s delay)
  │
  ▼ (still fails)
Retry #3 (4s delay)
  │
  ▼ (still fails)
Send to DLQ ──► Alert on-call ──► Human inspects

The DLQ is your last line of defense. It preserves the message body, the error, and the retry count so you can debug.

How AI assistants get this wrong

AI-generated message processing code almost never includes both patterns:

# What AI generates:
@app.post("/webhook")
def handle_event(event: dict):
    process(event)  # no error handling at all
    return {"status": "ok"}

# What it should generate:
@app.post("/webhook")
def handle_event(event: dict):
    try:
        process(event)
    except TransientError:
        retry_with_backoff(event, max_retries=3)
    except PersistentError:
        dlq.send(event, error=str(e))
        alert("message sent to DLQ", event_id=event["id"])
    return {"status": "accepted"}

The assistant skips error handling entirely — it assumes every message will succeed. This is the happy-path problem: the AI writes code that works when nothing goes wrong, and production is where things go wrong.

When to use only retry

• API calls during deployment — the new instance takes 30 seconds to start. Retry for 60 seconds, don’t DLQ.
• DNS or network hiccups — resolves within seconds.
• Rate limiting — the 429 response includes a Retry-After header. Respect it.

When to use only DLQ (skip retry)

• Invalid message schema — if the payload is malformed, retrying won’t fix it. DLQ immediately and alert.
• Authorization failures — a message from a revoked API key won’t become valid.
• Missing entity — processing a message for a deleted user. DLQ and log.

When you need both

Everything else. Almost every production system chains retry → DLQ. The retry count (3-5) and backoff (exponential) are tuned to your SLA, but the pattern is universal.

Where this bites vibecoders

The AI writes process(message) and stops. No retry, no DLQ, no error handling. The first time a downstream service blips, messages start dropping silently. The fix is retrofitting retry + DLQ into code that was never structured for it — much harder than building it in from the start. The habit: every AI-generated message handler should include retry with backoff and a DLQ fallback before it ships.

Checklist

• Every message handler has error handling (not just the happy path)
• Retry with exponential backoff for transient failures
• DLQ as fallback after retries are exhausted
• Alert when a message hits the DLQ
• DLQ messages are inspectable (preserve body, error, timestamp)
• Replay mechanism exists to reprocess DLQ messages after fixing the root cause

FAQ

Can’t I just retry forever until it works?

No — infinite retries hide problems and build backpressure. A message that fails because of bad data will fail every time. That’s when you need a DLQ: move the poison message out of the way so healthy messages keep flowing, and inspect it separately.

How many retries before the DLQ?

Start with 3, with exponential backoff between attempts. If it still fails after 3 tries, the problem is likely persistent (bad data, missing dependency, permission error) and belongs in the DLQ. Adjust the number based on your SLA — payment processing might want more retries than a marketing email.

What’s the difference between a DLQ and a retry queue?

A retry queue holds messages temporarily while waiting for the next attempt. A DLQ holds messages that have exhausted all retries and require human intervention. The retry queue is a holding pattern; the DLQ is the failure archive.

Related topics

• What Is a Dead Letter Queue?
• What Is Exponential Backoff?
• How to Add Retry Logic to API Calls
• How to Build a Background Job Queue
• What Is Idempotency (and Why Does It Matter for APIs)?

Sources

• AWS: Dead Letter Queues
• RabbitMQ: Dead Letter Exchanges
• Google Cloud: Retrying requests