Saw a few threads here about the latest, greatest incident management platforms. The ones that promise "AI-driven resolution" and "seamless orchestration" for the low, low price of $45 per user per month, billed annually. Spare me.
I got tired of our team's runbooks being static documents that were wrong the moment we wrote them. Also tired of the sales cycle for a "dynamic runbook" tool. So I built a Slack bot in a weekend. It's not fancy. It listens for a specific command in our incident channel, fetches the current runbook from a Google Doc, and posts the relevant section. The key part? It updates the Google Doc live if the incident commander posts a correction with a specific format. Now the source of truth updates *during* the incident, not three weeks later in a retro.
The core of it is just:
- A Slack app with the `commands` and `chat:write` scopes.
- A Google Docs API service account to read/update.
- A simple Flask server (could be anything) to map the command (`/runbook-step3`) to a doc paragraph.
- Logic to parse a correction message (e.g., "`!correction step3: actually, restart service B before A`") and patch the doc.
The ROI on this was about negative two days of my time, but it actually gets used. Unlike the $30k/year platform we demo'd that required a dedicated "runbook editor" role and a proprietary format. It works because it's where we already are: Slack and a shared doc.
My point isn't that you should use my janky script. It's that before you sign a three-year commitment for a "runbook module," ask:
* Can your team update the instructions *while the page is still active*?
* Does it integrate with where your team actually communicates, or is it another silo?
* What's the actual cost? Look beyond the per-user fee. What about the hours to migrate your existing docs into their special syntax? The training? The admin overhead?
Sometimes a duct-tape solution that fixes the core pain point is better than a "complete" solution that introduces ten new ones.
trust but verify
That's a really clever approach to the "static runbook" problem. I've seen similar issues in our manufacturing ERP data where process checklists become outdated because the update step is too cumbersome. The immediate correction during the incident is the key piece. How do you handle version control or an audit trail of those live edits to the Google Doc? I'd be worried about a well-intentioned but incorrect correction overwriting the source during a high-pressure situation without a way to review the change later.