We've been running this template for about 18 months across three engineering teams, and it's cut our mean time to meaningful retrospective by about 60%. More importantly, it forces a focus on systemic fixes over individual blame, which was a chronic issue in our early post-mortem culture. The goal is to produce a living document that feeds directly into our observability and orchestration tooling, not a PDF that gets filed away.
The template is implemented as a Markdown file in our incident command repository, with a CI job that auto-creates a Jira epic and Confluence page from the completed doc. We require completion before the incident ticket can be moved to "reviewed."
Key design principles:
* **Timeline-first:** Starts with a raw, unfiltered timeline to separate facts from interpretation.
* **Blast radius quantification:** Requires hard metrics from our monitoring stack, not guesses.
* **Action tracking:** Every action item must be tagged with an owner and a type (e.g., `detection`, `mitigation`, `prevention`). Prevention items are prioritized in quarterly planning.
* **Tooling integration:** References to specific dashboards, alert rules, and runbooks are mandatory.
Here's the core structure:
```markdown
# Incident Review: [Brief Title]
## Metadata
* **Incident ID:** [e.g., INC-2023-141]
* **Severity:** P1/P2/P3
* **Start Time (UTC):**
* **End Time (UTC):**
* **Duration:**
* **Services Impacted:** [List, with links to service catalog]
* **Detected By:** [Alert source, e.g., Prometheus AlertManager rule `x`, Customer support ticket]
## Impact
* **User Impact Description:**
* **Quantitative Blast Radius:**
* Error rate increase (%):
* Latency increase (p95/p99):
* Affected requests/sec:
* Affected users/tenants:
## Timeline (UTC)
| Time | Event | Source (Logs, Dashboard, Person) |
|------|-------|-----------------------------------|
| | | |
| | | |
* **Time to Detection (TTD):** [auto-calculated]
* **Time to Mitigation (TTM):** [auto-calculated]
* **Time to Resolution (TTR):** [auto-calculated]
## Root Cause(s)
* [Primary contributing factor. Must point to a process or system failure, not "engineer error".]
## Contributing Factors
* [e.g., inadequate runbook, missing high-cardinality label on metric, cascading failure mode not modeled in chaos testing]
## Detection Gaps
* How was this incident detected? Could it have been detected sooner? What signals were missed?
## Mitigation Gaps
* Was mitigation delayed? Were runbooks accurate? Was rollback effective and fast?
## Action Items
| Description | Type (Detection/Mitigation/Prevention) | Owner | Ticket ID |
|-------------|----------------------------------------|-------|-----------|
| | | | |
| | | | |
## Lessons Learned
* **What went well?**
* **What went poorly?**
* **Where did we get lucky?**
```
I'm particularly interested in feedback on the "Contributing Factors" vs. "Root Cause" separation—some teams find it useful, others think it's redundant. Also, how are you all handling the automated tracking of action items to completion? Our Jira integration is still clunky and requires too much manual syncing.
– A
Show me the benchmarks.