Using Mattermost Playbooks to Standardize On‑Call Incident Response

The problem: scattered runbooks and chat

When an alert fires, engineers often open a wiki page, a chat channel, and a ticketing system in separate tabs. Switching between them wastes time and can cause steps to be missed. Mattermost Playbooks aims to keep the checklist, the conversation, and the run timeline inside a single, auto‑created channel.

What Mattermost Playbooks provides

Playbooks is a built‑in workflow tool that lets you define a repeatable checklist, assign owners, set timed status reminders, and generate a retrospective when the run finishes. Each run gets its own channel, so all discussion, checklist updates, and timestamps stay together. The feature works inside the Mattermost server; no external orchestrator is required.

Worked example: an on‑call incident playbook

Defining the playbook

In the Mattermost UI, navigate to Playbooks → New Playbook and fill in:

• Title: On‑Call Incident Response
• Channel name template: incident-{date}-{title}
• Checklist items:
  Acknowledge alert → Assess impact and set severity → Page secondary if needed → Identify root cause → Apply mitigation → Verify service restored → Draft retrospective
• Status‑update cadence: every 20 minutes, post a reminder to the run channel
• Welcome message: a short template that renders the checklist and tells participants to use /playbook update to add a status summary
• Retrospective template: sections for timeline, what went well, what could improve, and action items, auto‑populated from the run’s activity

Running the playbook

You can start a run manually with the slash command /playbook run or automatically via an incoming webhook. The webhook URL is shown in the playbook’s trigger settings after you save it.

Status reminders and checklist

When the run begins, Playbooks creates the channel, invites the on‑call group (configured under member invitation rules), posts the welcome message, and starts the reminder timer. As each checklist item is ticked off, the channel shows real‑time progress. The timer posts a reminder every 20 minutes, prompting the team to update the checklist or add a note.

Retrospective and outgoing webhook

When the run is marked complete, Playbooks generates a retrospective from the recorded activity and posts it in the channel. You can also configure an outgoing webhook to fire on run end, sending a JSON payload to an external system (for example, to create a ticket in Jira). The payload includes the run title, start/end times, checklist state, and a link to the channel.

Integration considerations

Starting from Alertmanager

To have Alertmanager trigger a playbook, add a webhook receiver that posts to the playbook’s incoming webhook URL. A minimal snippet looks like:

receivers:\n  - name: 'mattermost-playbook'\n    webhook_configs:\n      - url: 'https://mattermost.example.com/plugins/playbooks/api/v0/runs/webhook/WEBHOOK_ID'

Run this on the Alertmanager server; ensure the server can reach the Mattermost host and that the webhook secret (if configured) matches. Test with amtool alert add test severity=critical against a staging instance and verify that a channel appears with the expected title.

Post‑run ticketing

Configure the outgoing webhook under the playbook’s actions. Point it at a request‑capture service (such as webhook.site) first to inspect the payload shape. Because Playbooks does not transform fields, you may need to adjust your ticketing API to accept the incoming JSON or use a middleware to map fields.

Trade‑offs and limits

• Licensing and packaging: Some automation and reporting capabilities have been tied to Professional or Enterprise editions in certain releases, and the feature has moved between a bundled plugin and core server code. Check System Console → License and the Playbooks settings for your exact version to confirm what is available.
• Not a paging system: Playbooks does not manage on‑call rotations, escalation policies, or SMS/voice paging. You still need a separate tool (e.g., PagerDuty, Opsgenie) to alert the engineer; Playbooks starts after the human is already notified.
• API drift: REST endpoints and webhook payloads have changed across major Mattermost releases. Validate any automation against a staging instance that mirrors your production version, and re‑test after upgrades.
• Overhead for simple checklists: For lightweight, recurring tasks (like a weekly deploy checklist), slash commands with channel templates or the Boards feature may be simpler. Playbooks adds value when you need run history, timers, and a retrospective.

Verification checklist

1. Deploy a test Mattermost instance matching your target version and license (Docker Compose is a quick way for self‑hosted).
2. Create the example playbook, run it via slash command, complete a few checklist items, observe a reminder post, finish the run, and check that the retrospective appears.
3. Trigger a run using curl against the incoming webhook: curl -X POST -H \"Content-Type: application/json\" -d '{\"title\":\"Test\",\"description\":\"from curl\"}' WEBHOOK_URL and confirm the channel is created.
4. Point the outgoing webhook at a request‑capture endpoint, run the playbook to completion, and inspect the JSON payload to ensure it contains the fields you expect.
5. Review the changelog and documentation for your target version to verify feature names, limits, and any licensing notes.

Closing

Mattermost Playbooks works best when your incident response already lives in Mattermost and you want the checklist, discussion, and retrospective in a single channel. It removes the need to juggle multiple tabs but does not replace your pager, ticketing system, or runbook authoring process. Treat it as the glue layer: test the automation paths on your exact version, keep paging outside, and adopt it for workflows where the added structure saves more time than it costs.