Review Documentation Changes in Pull Requests with Read the Docs Preview Builds

Why diffs aren’t enough for documentation

When a pull request touches documentation, reviewers often look at the raw diff to spot typos or wording changes. That approach misses problems that only appear after the documentation is built: broken cross‑references, missing images, altered anchor links, or formatting glitches caused by a new directive or theme tweak. Catching those issues after merge forces a re‑run of the documentation pipeline and can delay releases.

Thesis: let Read the Docs build the PR and show the rendered result

Read the Docs can automatically build documentation for every pull request when the repository is connected via a GitHub or GitLab webhook. Each PR gets a temporary version (slug like pr-123) with its own preview URL, and a commit status/check links reviewers directly to the rendered output. Reviewing the rendered page gives the same confidence as running CI on code before merge.

How to enable PR preview builds

1. In the Read the Docs project dashboard, open the project’s settings and locate the “Pull Request Builds” option (the exact label may vary; look under Build & Deploy or Automation). Enable it.
2. Optionally add automation rules to limit which PRs trigger builds (e.g., only from users with write access or matching specific labels).
3. Add a .readthedocs.yaml file to the repository root (version 2) to control the build environment. A minimal example for a Sphinx project:

version: 2

build:
  os: ubuntu-22.04
  tools:
    python: "3.11"

python:
  install:
    - requirements: docs/requirements.txt

sphinx:
  configuration: docs/conf.py

# If you use MkDocs instead, replace the sphinx section with:
# mkdocs:
#   configuration: mkdocs.yml

The build.os key is required in v2; omitting it will cause the build to fall back to a default image or warn. The python.install section points to a requirements file that contains only the packages needed to build the docs, keeping preview builds fast and reproducible.

Worked example: from commit to preview URL

Assume a public repository myorg/my-docs with a docs/ folder containing a simple Sphinx project.

1. Push the repository to GitHub and connect it in Read the Docs (project → Import a Repository).
2. Enable Pull Request Builds as described above.
3. Create a feature branch, edit docs/index.rst to add a new section, and open a pull request.
4. Read the Docs receives the webhook, starts a build for the PR, and creates a temporary version pr-42.
5. The build logs show the Sphinx invocation; when finished, a preview URL appears, typically resembling https://my-docs-42.readthedocs.io/ (the exact pattern depends on the project slug and hosting environment).
6. On the GitHub PR page, a new status check labeled “Read the Docs” appears with a link to that preview URL. Clicking it opens the fully rendered documentation, letting you verify links, images, and layout.

If the build fails, the status check shows an error and the logs point to the problem (e.g., a missing module or a Sphinx configuration mistake), allowing you to fix it before reviewers spend time on the diff.

Trade‑offs and practical limits

• Build quota: Each open PR consumes a build slot on the Read the Docs service. On the free community host (readthedocs.org) this counts toward your monthly build credits; busy repositories may need to monitor usage or upgrade to a commercial plan.
• Public preview URLs: PR previews on readthedocs.org are publicly reachable. If your documentation contains unreleased or sensitive information, consider using the commercial offering (readthedocs.com) where visibility can be restricted, or accept the exposure.
• Stale versions: After a PR is merged or closed, the pr‑NN version remains in the project’s Versions list unless removed. Over time, many stale previews can clutter the list. Teams should institute a cleanup habit—manual deletion from the Versions page or an automation rule that removes versions older than a set number of days.

Checking the result and cleaning up

After a PR is merged:

1. Go to the project’s Versions list in the dashboard.
2. Locate the pr‑NN entry; note whether it persists.
3. If cleanup is desired, select the version and choose “Delete”.
4. Optionally verify that the status check on the PR disappears (or shows as completed) and that the preview URL no longer resolves.

This quick check confirms that preview builds are not leaving permanent artifacts unless you intentionally retain them.

Actionable closing

Start small: enable PR preview builds on a low‑traffic repository, add the minimal .readthedocs.yaml snippet, and open a test PR. Verify that a status check appears and that the rendered preview catches at least one documentation‑only issue that a diff would miss. Once comfortable, extend the setting to your main projects, adjust automation rules to fit your workflow, and schedule a regular review of the Versions list to keep preview versions from accumulating.