Decoupling Slow Work in Symfony with Messenger and Async Transports

Problem: Long‑running work blocks the HTTP request

In a typical Symfony controller you might call a service that generates a PDF, sends an email, or performs a database migration. Those operations can take several seconds, and the whole HTTP request stays open until they finish. The user sees a slow page, and the server wastes resources while waiting. If the operation fails, the user receives an error only after the request times out, which is poor UX. Returning a response only after the work completes also makes it difficult to show progress or cancel the operation.

Thesis: Messenger decouples work via async transports

Symfony Messenger provides a command and event bus pattern that lets you fire a message from the controller and return immediately. The message is placed on an async transport (for example, a RabbitMQ queue) and processed by a separate worker process. The HTTP response is sent right away, improving perceived performance and freeing the web request for other traffic. By offloading the work, you also reduce memory pressure on the web server and make the system more resilient to traffic spikes.

Core concepts

A MessageBusInterface is the entry point for dispatching messages. It receives a plain PHP object (the message) and decides where to send it. A MessageHandlerInterface defines how a handler processes a specific message class. An envelope wraps the message and carries stamps such as SentStamp, ReceivedStamp and HandledStamp. Middleware can be attached to the envelope to add logging, validation or retry logic before the message reaches a handler. The envelope also preserves the message class name, which is needed for routing. The bus can operate synchronously (handlers run in the same request) or asynchronously (handlers run in a worker).

Async transports

The transport is the mechanism that moves the envelope from the web process to a worker. Symfony ships with three built‑in transports:

• doctrine:// – stores messages in a database table. Good for local development or small workloads. The Doctrine transport uses a polling loop to read new rows, which is simple but not optimal for high‑throughput scenarios.
• amqp:// – uses RabbitMQ or any AMQP‑compatible broker. Suited for medium to high throughput and offers true push‑based delivery.
• redis:// – uses Redis as a lightweight broker. Works well when you already run Redis and need a fast, in‑memory queue.

Each transport has a DSN string in messenger.yaml that tells Symfony where to send the message. The DSN includes the broker URL and any required credentials.

Worked example: dispatching a PDF generation message

Below is a minimal, fully functional example that you can copy into a fresh Symfony project.

# Message class
class GeneratePdfMessage
{
    public function __construct(public string $orderId) {}
}

// Handler implementation
class GeneratePdfHandler implements \Symfony\Component\Messenger\Handler\MessageHandlerInterface
{
    public function __invoke(GeneratePdfMessage $msg)
    {
        // Simulate a slow operation
        sleep(5);
        // In real code you would generate the PDF and store it
        // file_put_contents("pdf/{$msg->orderId}.pdf", $content);
    }
}

// messenger.yaml configuration
framework:
    messenger:
        transports:
            doctrine: 'doctrine://'
        routing:
            App\Message\GeneratePdfMessage::class: doctrine
        buses:
            async: '@doctrine_transport'

# Controller method
public function pdfAction(Order $order, MessageBusInterface $bus): Response
{
    // Fire‑and‑forget the work
    $bus->dispatch(new GeneratePdfMessage($order->getId()));
    return new Response('PDF generation started');
}

// Run a consumer that processes up to 10 messages at a time
php bin/console messenger:consume async --limit=10

When you call pdfAction, Symfony immediately returns the HTTP response. The GeneratePdfMessage is placed on the Doctrine transport, which writes it to a messenger_messages table. A separate worker process runs messenger:consume async, reads the table, creates an envelope, and invokes GeneratePdfHandler. After five seconds the handler finishes and the message is marked as HandledStamp. If the handler throws an exception, the message is retried according to the transport’s retry policy, and after the maximum attempts it is moved to a failure transport table for later inspection.

Trade‑off: eventual consistency and idempotency

Because the HTTP request ends before the handler runs, the UI may need to poll for completion or use websockets to show progress. This introduces eventual consistency: the data the user sees may be stale until the background job finishes. Handlers should be idempotent because a message can be redelivered if a worker crashes or a retry occurs. Also, configure a failure transport and set retry limits so that messages that keep failing move to a dead‑letter queue instead of disappearing silently. In practice you may need to store intermediate state (for example, a “job in progress” flag) or expose a status endpoint that reads the messenger table.

Actionable closing: get it running safely

• Run php bin/console debug:messenger to verify that the bus, handler, and transport are correctly wired in the application container.
• Check the DSN in config/packages/messenger.yaml; the scheme (doctrine, amqp, redis) must match the broker you intend to use.
• Dispatch a test message from a controller or a command, then execute php bin/console messenger:consume async --limit=10 and observe the handler log output.
• Intentionally throw an exception in the handler to confirm that the message is retried according to your configured strategy and eventually lands in the failure transport table.
• Run worker processes under a process manager such as systemd, Kubernetes, or a managed runner like symfony/messenger:consume in a separate container, keeping them independent from the HTTP server.
• Monitor the failure transport table (for Doctrine) or the broker’s dead‑letter queue to ensure that problematic messages are not lost.