Redis vs Database Queue Driver for Sub‑100 ms Job Latency in Lumen

Redis can reliably keep Lumen job processing under 100 ms during peak traffic when configured with in-memory operation and AOF persistence enabled; a relational database driver typically adds disk sync or transaction commit latency that makes sub‑100 ms difficult without aggressive tuning.

Explanation

The analysis shows that Redis’s in-memory operation avoids disk I/O, and with AOF persistence it retains jobs across restarts, satisfying durability while meeting latency requirements. Most Redis queue implementations (e.g., Laravel Horizon, Bull) also include automatic retry logic for failed jobs, whereas a plain database driver would need custom retry handling or an external retry system.

Confirmed facts

• Redis latency stays low as long as enough RAM is allocated and appendonly yes is configured in redis.conf.
• Database drivers add commit overhead; even on SSD the extra fsync can push latency beyond 100 ms under load.
• Automatic retries are built-in for Redis-based queues but not for generic database queues.
• Changing the queue connection can be applied without restarting workers if workers re-open the connection on reconnection events; otherwise a worker restart may be required.

Steps needed

1. Verify Redis config: appendonly yes and confirm sufficient RAM allocation.
2. Run a load test on Lumen with the queue driver under peak traffic and measure processing latency to confirm it stays below 100 ms.
3. Check worker logs for connection re-establishment events after a configuration change to verify no restart is required.

Missing diagnostic

Do the workers automatically re-open the queue connection when the connection string is updated, or do they require a restart to pick up the new DSN?