Writing Incident Runbooks in Gitbook That Actually Help During an Outage

Problem: Flat Command Lists Fail During an Incident

The useful takeaway is that a Gitbook page for incident response should read like a decision tree, not a recipe. When a database is slow or a deployment is failing, an engineer needs to know which precondition applies, which command to run, and how to verify the result before moving on.

Many teams copy a shell command sequence into a Markdown page and call it a runbook. That works in a calm review meeting, but under pressure the reader misses context: does this command require superuser access? What happens if the replica is already behind? Without branches and checks, the page becomes a liability.

Structure the Page Around Preconditions and Checks

Treat each procedure as a sequence of guarded steps. Use Markdown headers for the goal, a short precondition line, the command block, and an explicit verification step. This works in Gitbook because it renders standard Markdown consistently across the legacy GitBook CLI and the modern GitBook cloud service; behavior depends on your installed version, so verify rendering after you publish.

Worked Example: Rolling Back a PostgreSQL Major Upgrade

Suppose your team upgraded PostgreSQL from 14 to 16 and the application is now timing out on queries that worked before. The rollback page should not start with pg_dumpall. It should start with a check.

# Rollback: PostgreSQL 16 to 14

## Goal
Restore the previous database version and confirm application connectivity.

## Precondition
- You have a base backup taken before the upgrade.
- The application is in maintenance mode or can tolerate read-only errors.

## Step 1: Stop the application and database
Run this on the database host as the `postgres` system user:

```bash
sudo systemctl stop app-server
sudo systemctl stop postgresql
```

## Step 2: Verify the backup exists
```bash
ls -lh /var/backups/postgres/pre-upgrade-2026-11-01.base
```
Expected: a non-empty file owned by `postgres`. If missing, stop and page the on-call DBA.

## Step 3: Restore the data directory
```bash
mv /var/lib/postgresql/16/main /var/lib/postgresql/16/main.failed
cp -a /var/backups/postgres/pre-upgrade-2026-11-01.base /var/lib/postgresql/14/main
chown -R postgres:postgres /var/lib/postgresql/14/main
```

## Verification
Start PostgreSQL 14 and run:
```bash
sudo -u postgres psql -c "SELECT version();"
```
Expected output begins with `PostgreSQL 14`. Then start `app-server` and confirm health checks return `200 OK`.

## Rollback of the Rollback
If the restored cluster will not start, do not edit files in place. Restore from the base backup again and escalate.

Placeholders such as `/var/backups/postgres/pre-upgrade-2026-11-01.base` and `app-server` must match your environment. Run system-level commands with root or sudo privileges; database commands typically run as the `postgres` OS user. A wrong `mv` or `cp` can destroy the upgraded cluster, so verify the backup path before overwriting anything.

Comparison: Linear vs. Guarded Steps

Linear runbook	Guarded Gitbook page
Lists commands in order	States preconditions before each command
Assumes the reader knows the system state	Requires a verification check before proceeding
Hard to audit during an incident	Each step has an expected output to compare against

Limitation: Gitbook Version and Formatting

Gitbook supports standard Markdown code blocks and tables, but advanced features such as collapsible sections, live terminal blocks, or custom callout styling depend on the Gitbook version or theme you are using. Do not rely on interactive elements in an emergency page. Keep the runbook to plain Markdown, headers, lists, and code blocks so it renders the same in the Gitbook editor, in a PDF export, and in a cached browser tab.

Actionable Next Step

1. Pick one existing runbook page in your Gitbook space.
2. Add a "Precondition" section before the first command.
3. Add an "Expected output" line after every command that changes state.
4. Build or preview the Gitbook space and confirm the code blocks render with syntax highlighting and the table displays correctly.

If the preview looks correct, the page is ready to use. If a code block disappears or a table collapses, simplify the Markdown until it renders in your installed Gitbook version before trusting it during an incident.