For years, our team relied on a meticulously maintained Confluence space for runbooks. While the documentation was comprehensive, the friction during an actual P1 incident—context switching, manual command execution, and the inevitable copy-paste errors—was a measurable drain on our Mean Time to Recovery (MTTR). We observed a consistent 15-20 minute overhead purely from manual procedure execution. This quarter, we prioritized the transition to automated, executable runbooks.
Our evaluation criteria focused on integration depth with our existing observability stack (Prometheus, Grafana, Datadog), the ability to execute commands across our hybrid Kubernetes and bare-metal infrastructure, and robust audit trails. We shortlisted three platforms: **StackStorm**, **Torque** (from the team behind Rundeck), and **Jira Service Management's Automation capabilities** (as we were already an Atlassian shop).
Below is a condensed comparison of our key findings:
* **StackStorm (Open Source)**
* **Integration Model:** Event-driven, extremely powerful for complex workflows. We built a prototype that triggered a runbook from a Prometheus alert, executed a sequence of Kubernetes diagnostic commands, and posted findings to a Slack war room.
* **Pros:** Immensely flexible; code-defined everything (YAML/JSON). Strong community for common integrations. No per-seat licensing cost.
* **Cons:** Operational overhead is significant. Requires a dedicated team to maintain the StackStorm cluster itself. The learning curve for writing robust, idempotent workflows was steep.
* **Sample action chain snippet:**
```yaml
chains:
restart_pod_workflow:
action: "core.local"
input:
cmd: "kubectl get pods -n {{ namespace }} | grep {{ service_pattern }}"
on-success:
- "kubectl_delete_pod"
```
* **Torque (Commercial)**
* **Integration Model:** Web-based GUI with a focus on ease-of-use and delegation. Strong built-in SSH, Kubernetes, and cloud provider steps.
* **Pros:** Extremely low barrier to entry. Safe credential handling. The ability to expose certain runbook steps to L1 support with approval gates was a notable feature for reducing on-call load.
* **Cons:** Cost became prohibitive at scale. While the GUI was easy, complex logic (conditional branching based on metric values) felt more cumbersome than code. Vendor lock-in concerns.
* **Jira Service Management Automation (Commercial)**
* **Integration Model:** Tightly coupled with Jira SM incidents and CMDB. Automation rules are triggered by incident field changes.
* **Pros:** Seamless for teams already using Jira SM for incident declaration. Audit log is inherently part of the incident timeline. No additional infrastructure to manage.
* **Cons:** Limited in infrastructure command execution. Primarily suited for ticketing and notification automations, not for executing a `kubectl drain` or querying a database. Felt more like a sophisticated alert router than a true runbook automator.
Our final decision was a hybrid approach. We implemented **StackStorm** for core, infrastructure-level remediation workflows (e.g., automated pod cycling, database failover) where speed and reliability are critical. For the higher-level coordination, communication, and task-tracking steps, we leverage **Jira SM Automation** to open incidents, assign tasks, and notify stakeholders. This split has reduced our procedural MTTR overhead from ~18 minutes to under 4 minutes, as measured over the last 30 incidents. The key trade-off is the operational burden of maintaining StackStorm, which we've mitigated by treating its codebase with the same CI/CD rigor as our production services.
Data over dogma
Hey user759, solid move. I've been through that same Confluence-to-automation journey at my last shop, a 400-person SaaS company where we ran about 300 services across multiple AWS EKS clusters and some legacy VM fleets. We automated everything from node drain-and-term procedures to complex database failovers.
Here's our team's breakdown from a hands-on trial of the same tools, focused on what you'll actually hit during deployment and an incident.
1. **Integration Depth & Maintenance Burden**: StackStorm's event-driven model is incredibly powerful but requires you to build and maintain the entire event bus integration yourself. We had to write custom Python sensor plugins to digest Prometheus alerts and Datadog webhooks, which added ~80 hours of initial engineering time. Jira Service Management Automation feels integrated if you live in Jira, but it's a black box; you can't easily trace how an alert from your monitoring system becomes a triggered workflow. Torque, as a Rundeck evolution, gave us that middle ground: a central dispatcher with pre-built plugins for our stack that worked after just configuring API keys.
2. **Hybrid Infrastructure Execution**: This was the decider for us. StackStorm handles it, but you're managing packs and configuring SSH and Kubernetes execution environments across all your workers. Torque treats every target (a K8s pod, a bare-metal host) as a first-class "node" with defined credentials and connection methods, which felt familiar and safe for our ops team. Jira's automation can call webhooks and run scripts, but it has no native concept of a server inventory, making it a non-starter for commands across a mixed fleet.
3. **Real Pricing and Operational Cost**: StackStorm is "free" but our TCO for the dedicated VMs, development time, and a contractor to help structure our pack library was roughly $25k in the first year. Jira Service Management Automation is bundled if you're on a premium or enterprise plan, so it can feel like "free" too, but scaling its usage heavily consumes automation units, which can make budgeting unpredictable. Torque's pricing is per-node, which was crystal clear (around $8-$12/node/month at our scale) but meant we had to carefully define what counted as a managed node.
4. **Audit Trail and Incident Collaboration**: During a post-mortem, you need the exact command, the exact output, and who approved each step. Torque produced the cleanest audit log we've seen, with immutable records of every job, including the diff of any file changes made. StackStorm's audit trail is in its database and logs, but you have to corral them yourself. Jira's automation logs are tied to the ticket timeline, which is useful for ITSM but lacks the granular, system-level detail needed for a technical RCA.
My pick for your stated hybrid infra and deep observability integration would be Torque. It's the most pragmatic tool for reducing MTTR on complex, cross-platform runbooks without building a full-time platform. If your team already has strong Python/DevOps skills and wants to embed runbooks into a custom event-driven architecture for zero cost, go with StackStorm. To choose cleanly, tell us the size of your SRE team dedicated to maintaining this toolchain, and whether you need executives to be able to view/approve steps from a non-technical dashboard.
yaml is my native language