Scalingo Review Apps: Designing a Safe, Resource‑Efficient Preview Pipeline

Problem Statement

When a team opens a pull request (PR), developers need a realistic, isolated environment that mirrors production to run integration tests, visual reviews, and manual QA. Scalingo’s Review Apps automate this by spinning up a temporary app instance for each PR. However, naive use can quickly exhaust the plan’s resource quota, leak data, or leave stale environments. This article outlines a minimal, secure design that keeps resource usage predictable, enforces data isolation where required, and provides operational checks to keep the preview pipeline healthy.

Requirements

• Automatic creation of a new app per PR with identical code, buildpacks, and environment variables.
• Automatic teardown when the PR is closed or merged.
• Resource limits: number of concurrent Review Apps and maximum lifetime.
• Optional data isolation: separate databases or temporary schemas for each Review App.
• Visibility: list active Review Apps in the dashboard and via CLI.
• Minimal operational overhead: no manual cleanup or complex scripting.

Smallest Suitable Design

The core of the design is a single Scalingo app (the “parent”) with Review Apps enabled. Scalingo handles the lifecycle: on PR open, it clones the parent’s configuration, spins up a new app, assigns a unique sub‑domain (e.g., pr-123.myapp.example.scalingo.io), and deletes it on PR close. This approach requires no additional services or orchestration.

Key implementation steps:

1. Enable Review Apps for the repository in the Scalingo dashboard or via CLI.
2. Set concurrency and lifetime limits to match the plan’s quota.
3. Configure add‑on plugins that support branching or use a dedicated database per Review App if data isolation is critical.
4. Add a post‑deploy hook to clean up temporary data if the Review App persists beyond the PR lifecycle.

Trust & Data Boundaries

By default, Review Apps inherit all environment variables and add‑ons from the parent app. This means they share the same database instance, which may expose production data to the PR environment. To mitigate:

• Database Isolation: Use a database add‑on that supports per‑app databases (e.g., PostgreSQL with scalingo db:create per Review App) or a schema‑per‑app strategy. If the add‑on cannot be branched, create a temporary database in a pre‑deploy hook and set the DATABASE_URL accordingly.
• Secrets Management: For sensitive keys that should not be available to PR reviewers, use a separate set of environment variables in the Review App’s configuration. Scalingo allows scalingo env:set --app pr-123 --key API_KEY --value "" to override the parent value.
• Network Isolation: Review Apps run in the same data center as the parent app. For high‑security workloads, consider deploying the parent app in a private network and restricting Review Apps to a read‑only subnet.

Operational Checks

To ensure the preview pipeline stays within resource limits and behaves predictably, implement the following checks:

• Dashboard Monitoring: In the Scalingo dashboard, navigate to the parent app → Review Apps tab. Verify the Concurrent and Lifetime limits match the configured values.
• CLI Verification: Run scalingo --app myapp review-apps list to list active Review Apps. Each item should include the PR number, app name, and URL.
• Build Log Validation: Inspect the build logs for a Review App to confirm that the same buildpacks and environment variables were used as the parent. Look for lines like Using buildpack: heroku/nodejs and Setting environment variable: NODE_ENV=production.
• Resource Quota Alerting: Enable Scalingo’s built‑in alerts for CPU, memory, or add‑on usage. If a Review App spikes resource usage, investigate whether the PR includes heavy test suites or data‑intensive operations.
• Cleanup Confirmation: After merging or closing a PR, run scalingo review-apps list again. The app should no longer appear, and the DNS record should be removed within a few minutes.

Failure Modes & Conditions That Require Design Change

Although Scalingo’s Review Apps are robust, certain scenarios expose limitations:

• Quota Exhaustion: Rapidly opening many PRs can exceed the plan’s allowed number of concurrent apps. If this occurs, increase the concurrency limit or move to a higher tier.
• Shared Data Leakage: When Review Apps share a database, tests that mutate data can affect the parent app or other Review Apps. If tests require a clean slate, switch to per‑app databases or use migration scripts that reset state.
• Long‑Running PRs: Some PRs may take days to review. The default maximum lifetime (e.g., 7 days) may be insufficient. Adjust the lifetime or implement a manual “keep alive” hook that extends the Review App’s TTL.
• Add‑on Compatibility: Certain add‑ons (e.g., Redis, Elasticsearch) do not support per‑app instances. If isolation is critical, provision a separate add‑on per Review App or use a shared instance with key prefixes to avoid collisions.
• Compliance Constraints: For regulated workloads, storing production data in a PR environment may violate policies. In such cases, disable Review Apps for that repository or enforce a custom hook that wipes data on creation.

When to Change the Design

Consider the following triggers for redesigning the Review App strategy:

• Consistent resource over‑usage reported by Scalingo alerts.
• Security reviews flag data leakage risks in PR environments.
• CI/CD pipelines fail due to stale data or missing test fixtures.
• Team growth leads to a higher PR volume that the current plan cannot support.
• New Scalingo features (e.g., per‑app add‑on provisioning) become available and offer a more efficient workflow.

Concrete Example: Enabling Review Apps via CLI

# Enable Review Apps for the parent app
scalingo --app myapp review-apps enable

# Set concurrency and lifetime limits
scalingo --app myapp review-apps set-concurrency 5
scalingo --app myapp review-apps set-lifetime 7d

# List active Review Apps
scalingo --app myapp review-apps list

# View logs for a specific Review App
scalingo --app pr-123-myapp logs

Replace myapp with your app name. The set-concurrency command caps the number of Review Apps that can run simultaneously, preventing quota exhaustion. The set-lifetime command ensures that stray Review Apps are automatically destroyed after the specified period.

Limitations & Practical Verification

• Review Apps consume the same CPU and memory limits as regular apps; if the parent app is on a low‑tier plan, Review Apps may hit limits quickly. Verify by monitoring the CPU and Memory metrics in the Scalingo dashboard.
• Data isolation is not guaranteed by default. If your tests rely on a clean database, you must explicitly provision separate databases or use a migration hook.
• Scalingo’s automatic teardown occurs within a few minutes of PR closure. If you need immediate cleanup, use the scalingo delete --app pr-123-myapp command.
• Review Apps are not automatically load‑balanced; each instance receives traffic only to its unique sub‑domain. If you need to test multi‑region deployments, consider deploying the parent app in multiple regions and enable Review Apps per region.

Conclusion

Scalingo Review Apps provide a lightweight, automated way to preview PR changes. By setting clear resource limits, enforcing data isolation, and monitoring operational metrics, teams can avoid common pitfalls such as quota exhaustion and data leakage. The minimal design—enable, configure limits, and rely on Scalingo’s lifecycle management—offers the best balance of simplicity and safety for most teams. When workload or compliance demands grow, the architecture can be extended with per‑app add‑ons, custom hooks, or higher‑tier plans, ensuring the preview pipeline remains reliable and secure.