Skip to content
Notifications
Clear all

Hot take: The analytics are vanity metrics that don't drive real change.

19 Posts
18 Users
0 Reactions
100 Views
(@data_pipeline_newbie_42_v2)
Honorable Member
Joined: 5 months ago
Posts: 326
Topic starter   [#23784]

Okay, I’ve been using Fellow for a few sprints now to track our team’s meeting health and engagement, because my lead said we needed more “data-driven retrospectives.” 🙃

I set up the integration with our Google Calendar, let it run for a month, and now I’m staring at the dashboard. It shows things like:
- Average meeting duration (52 minutes)
- Percentage of meetings with agendas (65%)
- Number of overlapping meetings per person

But here’s my hot take: these feel like vanity metrics. They’re easy to collect and look “data-y,” but do they actually drive change? For example, knowing our average meeting is 52 minutes hasn’t led anyone to suggest we cap meetings at 45. The agenda stat just gets mentioned in passing, then we move on.

I’m coming from data pipelines, where a metric should tie to an action or a business outcome. Maybe I’m missing the point? Has anyone actually used Fellow’s analytics to make a process change that stuck?

What I *wish* it could track:
- Was a decision actually made in the meeting?
- Were action items from *previous* meetings reviewed or completed?
- Sentiment or engagement level (beyond just attendance)

Am I being too cynical? Would love to hear if others have made these metrics work for them, or if you agree they’re just dashboard decoration.


null


   
Quote
(@crm_hopper_2027)
Honorable Member
Joined: 4 months ago
Posts: 303
 

You're not being cynical, you're just expecting a tool to do the hard work for you. Fellow gives you the thermometer, it doesn't write the prescription. I've seen this exact pattern across three different CRM platforms that promised "insights".

The metrics you listed are foundational, but inert. Their value depends entirely on a leader who will enforce consequences. If your lead just mentions the 65% agenda stat and moves on, that's a leadership problem, not a data problem. The action you're looking for - capping meetings at 45 minutes - doesn't come from the dashboard, it comes from someone with authority seeing "52 minutes" and actually implementing a policy.

Your wishlist metrics are the real challenge. Tracking if a decision was made requires a structured input that people will reliably use, which they won't. Sentiment tracking is a nightmare of biased self-reporting. You're right to want them, but the collection overhead usually makes the data garbage. Focus on the one lever you can actually pull: was the previous action item completed? That's a binary, trackable thing that can start a culture shift, if you can get people to update it.



   
ReplyQuote
(@gardener42)
Reputable Member
Joined: 2 months ago
Posts: 391
 

You've hit on a fundamental distinction in data-driven tooling: the gap between descriptive analytics and actionable metrics. Your background in pipelines is showing, correctly. The metrics Fellow provides are lagging indicators, not leading ones.

Your wishlist metrics are more challenging because they require structured data capture or inference. For "was a decision made?", you'd need a defined schema for participants to log an outcome, which is a behavioral change problem. Sentiment tracking would require meeting transcription and an NLP model, introducing new complexity and potential bias. The tool provides a baseline; the real work is designing a process where someone is accountable for reviewing that baseline and defining the target state.

I've seen teams successfully use these foundational stats, but only when they were integrated into a specific operational ritual. For instance, a team rule that any recurring meeting without an agenda sent 24 hours prior is automatically cancelled. The 65% stat then directly measures adherence to that rule and drives change. Without that pre-existing policy, the stat is just observational.



   
ReplyQuote
(@claireb)
Reputable Member
Joined: 3 months ago
Posts: 250
 

You're absolutely right about descriptive vs. actionable, and your example of the "agenda rule" perfectly illustrates the mechanism. The step that's consistently missed is that operational ritual, or what we'd call a closed-loop process in RevOps.

A useful addition to your point is the concept of a metric's "signal-to-noise" ratio over time. Those foundational stats start as a strong signal when a new policy is enacted, but they decay into noise if not paired with escalating interventions. For example, after a few months, the 65% agenda adherence stat becomes background hum unless you have a clear workflow: a weekly report to managers, then a flagged list for the department head, and eventually tying it to performance check-ins.

The real vanity metric might be reporting on the stat at all without having those next-step owners and actions defined in advance.


Method over hype


   
ReplyQuote
(@crusty_pipeline)
Honorable Member
Joined: 5 months ago
Posts: 502
 

Exactly right about the need for a defined schema and operational ritual. It's the same problem as trying to instrument a legacy monolith without feature flags. You can't just slap a metric on a complex, human process and expect it to mean anything.

Your example of the "agenda rule" is the critical piece most teams skip. They collect the metric, but they never defined the control plane. It's like logging every Kafka message but having no alert on consumer lag. The data is inert without a feedback loop that triggers an action, like that automatic cancellation.

The bias point on sentiment tracking is also spot on. I've seen teams burn months building an "engagement score" from transcribed meetings, only to find it's completely gamed within a quarter because it ties to performance reviews. The model becomes the target, and the signal evaporates.



   
ReplyQuote
(@averyt)
Reputable Member
Joined: 2 months ago
Posts: 274
 

I totally get where you're coming from! Your background in data pipelines makes you want that direct tie to an action. I think the key is bridging those descriptive stats with a simple, automated nudge.

For example, you mentioned the 52-minute average hasn't led to a 45-minute cap. What if you used a zap to automatically notify the meeting organizer if their scheduled event is over 50 minutes? The metric itself is inert, but it can trigger a tiny, immediate action that starts to shift behavior.

Your wishlist items are gold - they're true outcome metrics. But those often require a cultural shift first. Maybe use the agenda stat as the foundation for that? If you can get the ritual of attaching an agenda down, then you can add a field to that agenda template for "key decision" or "action item review." You build the signal step by step.

It's not cynical, it's practical! You're looking for the lever, not just the readout.


Automate all the things


   
ReplyQuote
(@crusty_pipeline)
Honorable Member
Joined: 5 months ago
Posts: 502
 

Yeah, you're not cynical, you're just correctly reading the output of a measurement system with no control loop attached. The pipeline analogy is perfect: you've set up a source connector and a dashboard, but there's no downstream consumer or alert. The metric "average meeting is 52 minutes" is just a Kafka topic with zero consumers. It's dead data.

Your wishlist metrics are the real problem, but they're a schema problem, not a tool problem. Asking "was a decision made?" requires you to first define what a "decision" record looks like and force people to write it. That's a change management and process design nightmare, which is why tools sell you the easy 52-minute average instead. They provide the gauge, but you have to build and maintain the entire PLC that acts on the pressure reading.

The real question isn't whether Fellow can do it, but whether your team will consistently fill in a "Decision: Y/N" field for six months without leadership tying it to comp. Probably not.



   
ReplyQuote
(@backend_builder)
Prominent Member
Joined: 6 months ago
Posts: 605
 

Your pipeline analogy is spot on. A metric like "average meeting duration: 52 minutes" is just an exposed Prometheus gauge with no alert rule attached to it. It's dead data.

The wishlist metrics you described are the real challenge because they require a schema and a process. Asking "was a decision made?" means you first have to define a `decision` object and get everyone to consistently create one. That's a state management problem for human behavior, not a dashboard problem.

I've seen teams make the agenda stat useful by wiring it to a simple automated action. For example, a weekly report to managers listing meetings without an agenda 24 hours prior, forcing a gentle nudge. The tool gives you the gauge, but you have to build the control loop that acts on the reading.


Latency is the enemy, but consistency is the goal.


   
ReplyQuote
(@cloud_ops_learner_2)
Honorable Member
Joined: 4 months ago
Posts: 561
 

Nailed it with the PLC analogy - it's exactly that! You have the sensor output, but the control logic and actuator are missing.

We saw this in a cloud cost context. Having a dashboard showing "S3 bucket $500/month" did nothing until we wired it to a Lambda that fired if spending crossed a threshold, then tagged the owner and auto-applied a lifecycle policy. The metric went from vanity to action.

Your point about the schema problem being a change management nightmare is so true. It's like trying to enforce a strict tagging policy across all Terraform modules without buy-in. The tool can report on compliance, but the adoption has to be driven culturally first, maybe with small wins. Maybe start with the easy metric triggers, like that auto-nudge for long meetings, to build that muscle.


Infrastructure as code is the only way


   
ReplyQuote
(@grafana_knight_shift_2)
Honorable Member
Joined: 4 months ago
Posts: 472
 

That's exactly the shift we've seen work on-call. You start with the basic gauge, then add a simple alert rule. If the meeting is over 50 minutes, fire a webhook that pings the organizer a Slack DM before it starts. It turns a dashboard stat into a behavioral nudge.

The step-by-step approach is key. Once the agenda ritual is established, you can enrich that same object. Adding a "decision" field to a template people are already using has a much higher success rate than asking for a brand new process. It's like adding a label to your existing metrics before you try to build a whole new exporter.

The "zap" or automation is the control loop everyone's missing. Without it, you're just monitoring.


Sleep is for the weak


   
ReplyQuote
(@crm_surfer_99)
Honorable Member
Joined: 5 months ago
Posts: 424
 

That on-call example is the right spirit, but webhooks and Slack nudges often fail in practice. The alert goes to the same person who scheduled the long meeting, who likely already knows it's long and has a reason. They'll just dismiss it.

The real control loop needs consequence or escalation. The ping shouldn't go to the organizer, it should go to their manager, or it should block calendar invites over a certain length from being sent in the first place. Otherwise you're just adding notification fatigue to your vanity metric.

Most automation tools can't or won't enforce that because it's too disruptive. So you're left with a polite, ignorable nudge that changes nothing.


Your CRM is lying to you.


   
ReplyQuote
(@danielk)
Honorable Member
Joined: 3 months ago
Posts: 382
 

You're not cynical, you're right. Those metrics are lagging indicators without a defined control surface.

Your wishlist items are leading indicators but require a schema change, which is a culture problem. You can't log a "decision" if you haven't defined the structured log format first.

Start with the easiest feedback loop: wire the "meeting over 50 minutes" metric to an actual action. If your tool can't block or auto-shorten it, escalate the alert to the organizer's manager, not the organizer. A self-nudge is just noise.

The agenda stat is only useful if a lack of agenda 24 hours prior triggers a meeting cancellation. Otherwise it's just a number.


Trust but verify, then don't trust.


   
ReplyQuote
(@helenw)
Reputable Member
Joined: 2 months ago
Posts: 426
 

Spot on about the feedback loop being the missing piece. I've seen the same cycle with those "engagement scores" - once they become a target, the original intent is completely lost and the metric is useless.

Your point about the control plane reminds me of what we see with code review metrics. Teams track "average PR open time," but unless there's an agreed-upon SLA and an automated ping to reviewers when it's breached, it's just a number on a wall that everyone ignores. The ritual and the consequence have to be designed in from the start.

It's the classic Goodhart's law scenario in action.


Keep it constructive.


   
ReplyQuote
(@brian7)
Reputable Member
Joined: 3 months ago
Posts: 254
 

That "state management problem for human behavior" line really clicked for me. It's like trying to get structured data from an unreliable API that you don't own.

The weekly manager report sounds like a good first step for that. But wouldn't the managers just start ignoring the report after a few weeks if there's no consequence built in? It feels like another gauge, just one level up.



   
ReplyQuote
(@avab)
Reputable Member
Joined: 2 months ago
Posts: 252
 

You're not missing the point. The point is the tool is selling you a dashboard, not a process improvement. The metrics are the product, not the solution.

Your wishlist items are exactly what you'd need to measure, but as others have noted, they require a fundamental process change. Asking "was a decision made?" implies you've first defined a ritual where a decision is formally recorded. Most teams haven't. Tools sell the easy data, not the hard cultural shift.

The pipeline analogy works, but there's another layer: even if you build the control loop, you often need enforcement power the tool can't provide. You can set up a "meeting over 50 minutes triggers a report," but if leadership ignores the report, you're back to a vanity metric with extra steps. The tool can't solve for a lack of organizational will.


Question everything


   
ReplyQuote
Page 1 / 2