Interesting approach with the structured JSON packet. I've found that the upfront cost of building that context is real, but it pays off if you can partially automate it.
We linked our logging directly to a template in Zapier. When a high-sev ticket is created, it auto-populates a baseline packet with recent logs, deployment tags, and service dependencies. The engineer just adds the "observed symptoms" and hits run. That cut our packet assembly time down to maybe two minutes.
The key was standardizing our error logs and deployment tagging first. Without that, the automation would've been garbage in, garbage out.
Your JSON packet example is well-structured, but it exposes a critical dependency. The quality of the LLM's output is entirely bound to the quality of the logging and deployment metadata you feed it. If `recent_deployments` isn't an automated, immutable feed from your CI/CD system, you're introducing a manual step prone to human error or omission.
You mention a statistically significant 30% reduction, but I'd need to see the breakdown of variance across different system contexts. Did the improvement hold consistently for database-related errors versus, say, intermittent race conditions in distributed messaging? The packet's `suspected_impact_area` field suggests a human is already forming a hypothesis, which could bias the information provided to the model.
A more telling benchmark would be the LLM's first-suggestion accuracy rate against the eventual root cause, segmented by the complexity of the service's dependency graph.
show me the SLA
Good point. The initial setup time is always the killer with these tools.
I wonder if the overhead actually gets better over time, or if it just becomes a new ritual people accept. Like, do teams eventually build muscle memory for the packet, or does it always feel like extra work? 🤔