Running Web and Worker Processes Separately on Scalingo

Problem and takeaway

A single Scalingo application that handles HTTP requests and background jobs in the same process type cannot scale them independently. A traffic spike forces you to add web containers that also execute job code, while a long-running job can consume CPU and memory intended for serving requests.

Use a Procfile to define two process types: web and worker. Scalingo routes external HTTP traffic only to web containers. Worker containers stay off the HTTP router and pull jobs from a Redis-backed queue. PostgreSQL remains the durable system of record. Both process types run the same codebase and environment, but you can scale each one separately.

Requirements

You need a Scalingo application, permission to deploy to it, and access to provision PostgreSQL and Redis add-ons in your region. Add-on plans, connection-pooler features, and autoscaling availability vary by region and subscription tier, so verify current Scalingo documentation before production use. The application code must be able to start different commands depending on the process type, typically through the Procfile command itself.

Smallest suitable design

Start with one standard web container and one standard worker container. Create a Procfile in the repository root:

web: bundle exec puma -C config/puma.rb
worker: bundle exec sidekiq -c 5

Replace the commands with your stack's equivalents. The web command starts the HTTP server; the worker command starts the background job processor. Procfile process names and commands are case-sensitive, and typos produce a missing process type rather than an obvious push error.

Commit and push the change:

git add Procfile
git commit -m "Separate web and worker process types"
git push scalingo main

Run this from a terminal with the Scalingo remote configured and permission to deploy to the target app. After deploy, run scalingo ps from a terminal authenticated to the Scalingo CLI with read access to the app. Expected result: both web and worker appear in the process list. If only one appears, check Procfile spelling and deploy logs.

Scale each type independently, for example:

scalingo scale web=2 worker=3

Confirm the exact CLI syntax for your Scalingo CLI version; it may differ. Increasing web containers without reducing per-container database pool size can exhaust the PostgreSQL plan's connection limit.

Trust and data boundaries

The trust boundary is the platform-managed private network. Web containers are internet-facing through the Scalingo HTTP router. Worker containers, PostgreSQL, and Redis are not directly exposed to the internet; applications reach them using injected credentials and platform-provided connection strings. Do not open database or Redis ports publicly.

PostgreSQL is the durable system of record. Redis is the queue backend and should be treated as ephemeral; size the Redis plan for the expected working set to avoid eviction. Do not use Redis as a durable database.

Scalingo injects add-on credentials as environment variables, commonly DATABASE_URL for PostgreSQL and a Redis URL for the queue. Read these at runtime and never hardcode connection strings in the repository. Inspect environment variables after provisioning to confirm credentials are present; avoid printing secret values to logs.

Operational checks

• Run scalingo ps after every deploy to confirm both web and worker containers are listed and healthy.
• Confirm the PostgreSQL and Redis add-on environment variables are present and not hardcoded in application code.
• Check application logs during deploy to verify database migrations complete before new web containers receive traffic.
• Monitor the database add-on connection metric against the plan limit while increasing web container count.
• Monitor queue depth using your queue library's tools; a stuck or growing queue often indicates a worker that is not running or cannot connect to Redis.

A practical way to verify the result is to compare the scalingo ps output before and after the Procfile change. Both process types should appear with a healthy state. If a process type is missing, the most likely cause is a Procfile typo or a failed container start visible in the deploy logs.

Failure modes and conditions that change the design

Failure mode	Symptom	Check	Adjustment
PostgreSQL connection exhaustion	Database rejects new connections; errors appear in web container logs	Multiply web container count by per-container pool size and compare with the plan limit	Lower per-container pool size, enable connection pooling if available on your plan, or reduce web container count
Worker crash during deploy	In-progress jobs interrupted; queue depth stops decreasing	Inspect worker logs around the deploy and confirm unacknowledged job handling in the queue library	Use a queue library with durable acknowledgments and schedule deploys during low queue depth
Redis eviction under load	Missing jobs or queue-related errors	Monitor Redis memory usage and eviction metrics	Size the Redis plan for the working set; consider a durable queue backend if jobs must survive restarts

Change the design when any of the following apply:

• Throughput requires autoscaling beyond manual container counts.
• Compliance mandates a specific region or private networking not covered by the default app network.
• Long-running jobs need different container size or resource isolation from web containers.
• Job durability requirements make an ephemeral Redis queue unacceptable.

Limitations

This note covers a single Scalingo app with web and worker process types. It does not cover multi-app topologies, private networking details, or autoscaling configuration. Specific add-on plans, connection-pooler features, and availability vary by region and subscription tier. Redis durability depends on the client library and configuration; verify current behavior in Scalingo and library documentation before relying on it for production jobs.

Rollback

If the split causes problems, revert the Procfile commit and redeploy, or scale back to the minimal topology:

scalingo scale web=1 worker=1

This restarts the previous arrangement. Confirm the exact scaling command syntax in current Scalingo documentation.

Verification required: confirm add-on plan names, environment variable keys, CLI syntax, and autoscaling availability against current Scalingo documentation for your region and subscription tier before applying this design in production.