Skip to content
Notifications
Clear all

Showcase: Built a marketing lead-scoring agent. Here's the precision/recall vs. old rules.

2 Posts
2 Users
0 Reactions
3 Views
(@dianaf)
Estimable Member
Joined: 1 week ago
Posts: 84
Topic starter   [#14551]

Okay, I've been lurking for a bit while learning LangGraph, and I finally have something concrete to share. My team's old marketing lead-scoring system was a bunch of nested if-else rules in our CRM. It was... brittle. Scoring logic lived in a spreadsheet that only one person understood.

I built a replacement agent using LangGraph to evaluate inbound leads. It pulls data (website activity, form fields, email engagement), reasons through a scoring checklist, and assigns a priority tier. The graph structure is perfect for this—I have a router node that decides if it needs to check demo request info, review engagement scores, or move to final scoring.

The real test was comparing output against our old rules on the last 500 leads. The old system had a "high-priority" precision of about 62%. Our sales team said a lot were wastes of time.

**After tuning the agent's decision path prompts and a few cycles:**

* **Precision for high-priority leads:** **89%**
* **Recall:** We're capturing 95% of the truly good leads the old system caught, plus a bunch it missed because of rigid rules.

The cool part wasn't just the lift, but the visibility. Seeing the agent's path in the LangGraph debugger showed us *why* it scored something high—like a combination of job title *and* specific page views that our old rules didn't connect. It’s also way easier to adjust now; I just modified a prompt in one node to weight a factor differently, instead of untangling a mess of SQL-like rules.

Has anyone else moved from static rules to an agentic scorer? I'm wondering how you handled the "grey area" leads where the agent was uncertain—we added a human review node for scores in the middle range, which has been a nice hybrid approach.



   
Quote
(@finops_tracker_99)
Estimable Member
Joined: 5 months ago
Posts: 87
 

Our FinOps team at a mid-sized SaaS shop (300ish employees, heavy AWS, some Azure) runs billing anomaly detectors on a mix of cron jobs and Lambda functions, so I'm always looking at how to structure and track these workflows.

Glad you posted this, because I've been weighing LangGraph against more traditional workflow orchestrators for cost alerting agents. Here's a breakdown from my own testing:

* **Integration & plumbing effort:** LangGraph is a library, not a hosted service. Getting it into production meant building out the surrounding pipeline ourselves. At my last shop, that was about 40 hours of dev time for logging, state persistence, and metrics. If you don't have a platform team, factor that in.
* **Observability cost:** The visibility into the agent's path is great, but you pay for it. Ingesting those trace logs into something like Datadog or even CloudWatch can add $50-200/month to your bill depending on volume, because you're logging a lot more discrete steps than a simple function.
* **Cold-start performance:** For a lead scoring agent likely running as an HTTP endpoint, the cold-start on a serverless deployment was noticeable. Our similar agent on Lambda (Python) saw 3-4x latency on cold starts, which we had to mitigate with provisioned concurrency, adding about $150/month to the run cost.
* **Scaling and batching:** LangGraph works on a single "thread" or unit of work. If you suddenly need to score 10,000 leads from a backlog, you'll be spinning up 10,000 graph executions. It doesn't do out-of-the-box batching across leads, so throughput costs can spike. We built a batch wrapper, but it was extra work.

Given what you've built, I'd lean LangGraph for your use case because the reasoning paths are non-linear and you clearly benefit from the graph structure. If your main constraint was keeping infra costs under $200/month and you had a high, sporadic volume, I'd point you to a simpler queue-worker model. Tell us your approximate leads-per-hour and if this runs in your own VPC or a fully managed serverless platform.



   
ReplyQuote