Alright, let's cut through the marketing fluff. Vendors love to blur these lines so you buy three of their modules. Here's the raw breakdown.
Think of it like a building's fire system:
* **Monitoring** is the network of smoke detectors and sensors. It's constantly checking "Is the temperature normal? Is there smoke?" It's the data collection layer. Example: Prometheus scraping metrics.
* **Alerting** is the fire alarm that goes off when a sensor detects smoke. It's a notification based on a rule. It's *supposed* to wake someone up. Example: An OpsGenie alert from a Prometheus `ALERT` rule.
* **Incident Management** is what happens *after* the alarm sounds: who grabs the extinguisher, who calls 911, how you document the damage, and how you decide to install a sprinkler system next time. It's the process. Example: Declaring an incident in Jira Service Management, running a war room in Slack, and doing a post-mortem in Confluence.
Most teams screw this up because they treat the alert as the end goal. It's not. An alert is just the starting pistol. The real work is the incident process.
Here's a concrete, oversimplified config snippet that shows the handoff from monitoring to alerting:
```yaml
# Monitoring (Prometheus rule)
groups:
- name: example
rules:
- alert: HighRequestLatency # This is the ALERT definition
expr: job:request_latency_seconds:mean5m{job="myapp"} > 0.5
for: 5m
annotations:
summary: "High latency on {{ $labels.job }}"
# Alerting (Alertmanager config)
route:
group_by: ['alertname']
receiver: 'oncall_pager'
receivers:
- name: 'oncall_pager'
opsgenie_configs:
- api_key: ''
message: '{{ .CommonAnnotations.summary }}' # This PAGES someone
```
Where does incident management start? The moment that OpsGenie notification creates a ticket and triggers an escalation policy. The tooling for that is a whole other beast.
The biggest failure pattern I see? Teams have great monitoring and loud alerting, but zero formal incident management. They just scramble in chat until the problem goes away. Then they're surprised when the same thing happens next month.
So, what's the difference? Monitoring tells you the system state. Alerting tells a human about a bad state. Incident management is what that human (or team) does about it. Don't let a vendor sell you one tool for all three.
-- bb
-- bb
Your fire system analogy is excellent for the layering concept, but I'd tighten the implementation dependency. That oversimplified config snippet you hinted at is critical because the handoff often fails in the glue code.
Teams frequently have monitoring and alerting coupled too tightly, like a Prometheus Alertmanager rule that pages directly. The real architectural separation comes when you treat the alert as a structured event stream. That event should be ingested by a dedicated incident management platform which then handles routing, state, and context enrichment - not just forwarding a notification.
For example, you might have a Terraform module that deploys a Lambda function to parse a CloudWatch Alarm JSON and transform it into a PagerDuty incident with enriched tags from your CMDB. That separation lets you change monitoring backends without rewriting your entire response playbook. The alert isn't the starting pistol, it's the race officials confirming the runner is on the correct track before firing it.
infrastructure is code