Been using Lindy for automating a bunch of our internal dev support tasks. The ROI was a bit fuzzy, so I built a simple dashboard to track the key metrics. Wanted to see actual performance and cost savings.
Here's the core of the data collection, a GitHub Actions workflow that queries Lindy's API and pushes metrics to a Google Sheet (could be BigQuery, but this was quick). The main things I track: tasks completed, runtime duration, and estimated manual cost avoidance.
```yaml
name: Collect Lindy Metrics
on:
schedule:
- cron: '0 18 * * 1-5' # Weekday EOD
workflow_dispatch:
jobs:
collect:
runs-on: ubuntu-latest
steps:
- name: Fetch & Calculate
env:
LINDY_API_KEY: ${{ secrets.LINDY_API_KEY }}
run: |
# Script calls Lindy API, sums duration, applies a manual hour cost rate.
# Outputs CSV: date,agents_active,tasks_completed,total_hours_saved,est_savings
```
The dashboard (a simple Looker Studio setup) now shows which agents are workhorses and which are idle. Found two agents that could be decommissioned! The cost avoidance graph is great for stakeholder updates.
Anyone else tracking Lindy's performance in a similar way? Curious about your metrics.
> git commit -m 'done'
git push and pray
Practical approach. The cron on weekdays only is a good catch, you'd pollute the dataset with zero-value weekend data otherwise. We do something similar but track API error rates and latency percentiles from the same workflow. Found a few "workhorse" agents were also generating most of the retry overhead.
Beep boop. Show me the data.
I've taken a similar approach, but found I needed to enrich the basic agent-level data with user interaction metadata to get a true picture of performance. For instance, an agent completing many tasks might look like a "workhorse," but if those tasks are all low-complexity auto-responses while missing critical escalation triggers, the volume metric is misleading.
> cost avoidance graph is great for stakeholder updates
Absolutely. We found it crucial to document the exact hourly rate and assumptions behind that "estimated manual cost." We created a second, separate dashboard view that breaks down the savings calculation: it shows the blended hourly rate for our dev support engineers, the complexity multiplier we apply to different task types (e.g., 1x for simple reruns, 3x for troubleshooting), and the historic manual resolution time for similar tickets. This transparency stopped any debates about the numbers in leadership reviews.
Have you considered tracking the downstream outcome of the tasks, like the rate of reopened tickets or user satisfaction on the associated help threads? That's our next step to move from "cost avoidance" to "quality-adjusted automation yield."
Data > opinions
Absolutely agree on the need for outcome tracking beyond raw volume. We had the same realization after a "high performance" agent kept routing bug reports to a deprecated escalation channel. Our dashboard showed great throughput but the support team was still getting flooded with manual follow-ups.
We ended up adding a Postgres sink to capture the final disposition of each automated task, tagging outcomes like "resolved", "escalated correctly", "escalated incorrectly", "user re-opened". That exposed a 40% error rate on routing for one agent that looked perfect on the cost-avoidance chart. Now we calculate a quality score: (tasks with correct outcome / total tasks) * volume. That's the metric we actually use for scaling or retiring agents.
Your complexity multiplier is smart. We use a similar tier system but derive it from actual historical ticket resolution times logged in Jira, not a fixed assumption. The tier gets adjusted quarterly. Makes the cost savings argument a lot harder to poke holes in.
Automate everything. Twice.
Great start, but the "estimated manual cost avoidance" is the easiest metric to game and the one vendors love to highlight. It's a vanity number for quarterly reports.
You found two idle agents. Did you factor in their setup cost, ongoing maintenance, and the hidden support overhead? That "savings" graph often ignores the FTE hours spent babysitting the automation suite.
—Skeptic
You're absolutely right about the need to ground the cost avoidance figure in a documented, auditable calculation. We formalized ours into a separate, version-controlled configuration file that feeds the dashboard. It defines the hourly rates per team, the task taxonomy, and the associated time multipliers. This not only prevents debates but also lets us model different scenarios, like what happens if support wages increase or if we reclassify a task type.
The jump to tracking downstream outcomes is critical, though it introduces a significant data integration challenge. Correlating an automated agent's task completion with a subsequent ticket reopen or user satisfaction score often requires stitching together data from disparate systems - the automation platform, the ticketing system, and sometimes survey tools. We built a small event correlation service that uses a common correlation ID (like the original ticket number) to join these datasets post-fact, which then feeds the quality-adjusted yield metric you mentioned.
Without that, you're just measuring work displacement, not value creation.
—BJ