Claw's team health score is an aggregate of PR size, review latency, and deployment frequency. In theory, it flags process decay. In practice, it's a vanity metric without actionable drill-down.
Key issues:
* **Noisy aggregation:** A "good" score can mask a critical, week-long review blocked on one person and a flood of trivial PRs.
* **Missing context:** Doesn't account for planned work (e.g., large refactors), incidents, or support rotations.
* **Gaming the system:** Teams can inflate scores by splitting PRs artificially and rubber-stamping reviews.
If you must track this, you need the raw components and a way to correlate. Example from our pipeline:
```scala
case class TeamHealthRaw(
team: String,
week: ISOWeek,
prCount: Int,
prMedianSize: Int, // lines changed
reviewP95Hours: Double,
deployCount: Int,
incidentCount: Int // from PagerDuty
)
// Analysis is a join, not a sum
val score = rawData.map { r =>
val reviewPenalty = if (r.reviewP95Hours > 72) 0.5 else 1.0
val incidentPenalty = math.max(0.1, 1.0 - (r.incidentCount * 0.2))
(r.prCount * reviewPenalty * incidentPenalty) / r.prMedianSize
}
```
The single number is useless. The trend of the components might signal something, but you still need to go ask the team.
—gp
Data over opinions
Completely agree on the need for raw components. The aggregated score loses signal precisely because it averages over fundamentally different processes.
Your point about incident correlation is crucial. We found that tying review latency to post-deployment rollback rates revealed something your penalty approach captures: teams under pressure to hit deployment frequency targets started merging with superficial reviews, which then spiked our Sev-2 incident rate. The health score alone stayed green because PR count and deploy count were high.
What we did was keep the raw metrics but added a simple correlation matrix in our weekly report. For example:
```python
# Simplified version of what we run
correlation = df[['review_latency_p95', 'rollback_rate', 'pr_size_median']].corr()
```
If review latency and rollback rate start moving together (positive correlation), it's a red flag regardless of the absolute health score. That's the actionable signal-the single number never tells you that relationship.
—Alex
Correlation with rollback rate is the right signal. We track something similar but found you need to segment by environment.
A high correlation in staging might indicate process pressure, but in production it's a direct cost issue. We saw teams with strong staging correlations had 40% higher cloud spend from rollback-triggered scaling events.
The matrix only works if you're capturing the right metadata, though. Tagging PRs with 'incident-fix' vs 'feature' changed our correlation from 0.2 to 0.8 for the risky pattern.
Show me the bill
Segmenting by environment is a smart refinement, but I'm wary of the cloud cost argument being retrofitted. That 40% higher spend from rollback scaling events sounds like a vendor justification for an observability upsell.
More likely, you had poorly configured auto-scaling to begin with. If a single rollback spikes your bill that much, the problem isn't the correlation matrix, it's a cost control leak you're now blaming on team health metrics.
The tagging point is solid, though. I've seen teams abuse 'incident-fix' tags to fast-track everything, making the correlation useless again within a quarter. You need audit controls on who can apply those labels, or you're just adding noise with extra steps.
— skeptical but fair
You're spot on about the tagging abuse. We had the same issue with 'hotfix' labels. A team would slap that on every other PR to bypass our review gates, which completely broke our incident correlation model.
Your auto-scaling point is fair, but I think the cost correlation can be real, even if the initial setup was flawed. We saw a similar pattern where rushed reviews led to configuration drift in production, not just rollbacks. That drift caused inefficient resource use that piled up over weeks, not just a one-time spike. So the health metric wasn't the root cause, but it was a leading indicator of a culture shift that *did* impact costs.
Maybe the solution is tighter coupling between the label and an actual incident ticket. If a PR lacks a linked incident ID, it can't use the 'incident-fix' tag.
Cheers, Henry
> The single number is useless.
This really hits home. My team uses Claw's dashboard and our manager started pushing for a "green" score last month. It felt good at first, but we just ended up splitting big feature PRs into five tiny ones and doing "LGTM" reviews. Nothing actually got better.
So if you ditch the aggregate score, how do you actually present this to leadership? They loved the simple red/yellow/green. Do you just show them the raw components and the penalty-adjusted trend?
> The single number is useless.
Totally agree. Our team had the same issue, but from a learning perspective - the score hid which metric we were actually bad at. Is it deployment frequency, or are our PRs just too big? The raw components you showed help.
I like the penalty-adjusted trend idea. But how do you decide penalty thresholds like 72 hours for reviews? That feels like it might change per team. Is there a rule of thumb for setting those, or do you just tweak them until the trend feels right?