A runbook is a short, step-by-step guide for handling one specific problem, written so the person on call can fix it without waking an expert. A good runbook template has six parts: the trigger or symptom, prerequisites, diagnostic steps, the fix, escalation, and rollback. Keep each runbook to one failure mode, write it in plain imperative steps, and link it straight from the alert that fires so it is one click away at 3am.
The value of a runbook is that it moves knowledge out of one senior engineer's head and into a document anyone on the rotation can follow. Without runbooks, every incident routes to the same two people, on-call is miserable, and recovery waits for whoever happens to know the system. With them, a first-year engineer can handle a familiar failure at 3am and only escalate when the script runs out.
What is a runbook?
A runbook is a documented procedure for a single, recurring operational task or failure: restarting a stuck worker, clearing a full disk, failing over a database, responding to a spike of 500 errors. It describes exactly what to check and what to do, in order, so the outcome does not depend on who is holding the pager. Think of it as the difference between "the API is down, good luck" and a checklist that walks you from symptom to fix.
What should a runbook template include?
Every runbook, whatever it covers, benefits from the same skeleton. Fill in these six sections and you have a document a tired responder can actually follow.
| Section | What goes in it |
|---|---|
| 1. Trigger and symptoms | The alert or observation that sends someone here, and how to confirm this is the right runbook |
| 2. Prerequisites | Access, tools and permissions needed before starting, so nobody stalls halfway |
| 3. Diagnostic steps | The checks to run, in order, to pinpoint the cause, with the commands or dashboards to use |
| 4. The fix | The exact steps to resolve it, as plain imperative instructions |
| 5. Escalation | Who to page and when, if the fix does not work or the situation is worse than expected |
| 6. Rollback and verification | How to undo the change if it makes things worse, and how to confirm the service is truly healthy |
Here is the shape filled in for a common case, an API returning a burst of 500 errors:
Runbook: API returning 500s
Trigger: down alert on the /health endpoint, or error rate above 5%.
Prereq: access to the app dashboard and deploy console.
Diagnose: 1) check recent deploys, 2) check database connections, 3) check upstream dependency status.
Fix: if a deploy in the last 30 min lines up, roll it back; if connections are exhausted, restart the pool.
Escalate: if error rate is still above 5% after 10 minutes, page the service owner.
Rollback and verify: confirm error rate under 1% and /health green for 5 minutes before closing.
Notice the steps are specific and ordered. "Investigate the errors" is not a runbook step; "check recent deploys, then database connections, then upstream status" is. The more decisions you make in advance, on a calm afternoon, the fewer a stressed responder has to make live.
alertping
A runbook is only useful if the alert reaches someone
AlertPing checks every 30 seconds, confirms from three regions, and pages the right person by SMS, email, Slack or webhook, so your runbook gets opened in time to matter. SMS on every plan, flat from $19 a month.
What is the difference between a runbook and a playbook?
A runbook handles one specific, well-understood task with a known fix: the steps are deterministic and you could almost automate them. A playbook is broader and covers a class of situations that need judgment, like "how we run a major incident," including roles, communication and decision points. Runbooks are the detailed procedures; playbooks are the strategy that decides which runbook to reach for and how to coordinate when several are in play.
In practice you want both. The playbook tells the incident commander how to run the response, and it references the runbooks that resolve the individual failures underneath it. Our incident postmortem template is the after-the-fact companion: the runbook fixes it, the postmortem makes sure it does not happen again.
How do you keep runbooks up to date?
Stale runbooks are worse than none, because they send a responder confidently down a dead end. Keep them current with two habits. First, update the runbook as part of the incident it was used in: if a step was wrong or missing, fix it before you close the incident, while the detail is fresh. Second, review runbooks on a schedule, and delete ones for systems that no longer exist. A runbook that references a server you decommissioned last quarter is a trap.
The escalation section needs the most maintenance, because people and ownership change. An escalation step should route the page to whoever owns that service now, not to the engineer who owned it two reorgs ago. Tie escalation to a role or a rotation rather than a named person, and it stays correct as the team changes.
Where should runbooks live?
Runbooks belong wherever the responder already is during an incident, one click from the alert. Linking the relevant runbook directly in the alert payload, so it arrives with the page, beats a wiki nobody can find at 3am. Many teams keep runbooks in version control next to the code, so changes get reviewed and the history is clear, then surface the link through their alerting tool.
Whatever you choose, optimize for one thing: can a tired person on call find and follow this in under a minute? If yes, it will get used. If it takes a hunt through three tools, it will not, and you are back to paging the one expert who knows the system. Pair good runbooks with a healthy on-call rotation and reliable alerting, and 3am stops being the part of the job people dread.