Symfony Messenger: Retry Strategies and Failure Transports for Async Resilience

When an asynchronous Symfony Messenger handler throws an exception, the message does not simply disappear by default—but without explicit configuration it can still be lost, block the queue, or retry forever. The practical fix is to attach a retry_strategy to the transport and define a global failure_transport. Exhausted messages are then moved to a dedicated failure transport, usually a Doctrine table, where they can be inspected and retried manually without blocking the main queue.

How retry and failure transports work together

Messenger wraps message handling in a chain of middleware. When a handler throws, the retry middleware catches the exception and re-queues the message on the same transport after a configured delay. It attaches a retry-count stamp to the envelope so subsequent attempts can be tracked. Once the number of redeliveries reaches max_retries, the middleware stops retrying and routes the message to the transport named by framework.messenger.failure_transport.

This separation matters: the main async transport keeps flowing, while failed messages accumulate in a predictable place. A failure transport is just another Messenger transport, commonly backed by Doctrine, that stores the serialized message and its metadata so you can examine it later.

A worked configuration example

Place the following in config/packages/messenger.yaml. You need write access to that file and a configured Doctrine connection if you use the Doctrine transport.

framework:
    messenger:
        # Global fallback when retries are exhausted
        failure_transport: failed

        transports:
            async:
                dsn: 'doctrine://default'
                retry_strategy:
                    max_retries: 3
                    delay: 1000        # milliseconds before first retry
                    multiplier: 2      # exponential backoff factor
                    max_delay: 60000   # cap at 60 seconds
            failed:
                dsn: 'doctrine://default?queue_name=failed'

Here, the async transport uses Doctrine storage. The retry_strategy waits one second before the first retry, then doubles the delay for each subsequent attempt, capping at sixty seconds. After three redeliveries, the message is sent to the failed transport, which uses the same Doctrine connection but a separate queue name.

Version note: These keys follow the standard Messenger component layout. Because option names and default middleware behavior can shift between Symfony minor versions, confirm the exact spelling and defaults against the reference documentation for your installed Symfony version before deploying.

Running workers and verifying behavior

Start a worker from your project root with:

bin/console messenger:consume async -vv

With a deliberately failing handler, the verbose output should show the retry middleware acting: the envelope gains retry stamps and the delay between attempts grows according to the multiplier. When the retry count reaches max_retries, the message should no longer appear on the async queue.

To confirm the failure transport is wired correctly, list stored failures:

bin/console messenger:failed:show

Then requeue a single failure by its ID:

bin/console messenger:failed:retry <id>

Risk: messenger:failed:retry removes the message from the failure transport and re-dispatches it. Only run it after the underlying handler bug or external dependency issue has been resolved.

Limits and common mistakes

• Transport support: Retries require a transport that can delay and redeliver messages, such as Doctrine, Redis, or AMQP. The Sync and in-memory transports cannot retry across separate processes.
• Idempotency: Retries do not guarantee that a handler runs exactly once. If a handler sends an email or charges a payment, it must tolerate duplicate execution or use an idempotency key.
• Worker limits: A long max_delay can exceed a worker's --time-limit or --memory-limit. Tune worker lifecycle options independently from retry delays.
• Swallowed exceptions: Catching exceptions inside a handler prevents the retry middleware from ever seeing the failure. Let exceptions propagate unless you have a specific reason to handle them.
• Missing storage table: If the failure transport uses Doctrine, the messenger storage table must already exist. Failed messages accumulate there, so monitor its size.
• Retry semantics: max_retries counts transport redeliveries, not logical business attempts. A message that processes a batch and throws will be redelivered in full.

A conservative pre-production check

Before relying on this in production, deploy a test handler that always throws. Consume the transport with -vv, confirm the retry count increases and the message eventually lands in the failure transport, then use messenger:failed:show and messenger:failed:retry to prove recovery. Inspect the Doctrine messenger_messages table (or your chosen transport storage) to verify that delayed and failed records are persisted with the expected headers. Because exact command flags and table schemas can vary by Symfony version, validate each step against your installed release.