Just had a huge "aha!" moment and wanted to share. We've been running a custom chatbot for sales support for months, but had zero visibility into what prompts our team was actually using or how much each conversation was costing us. We were flying blind on both performance and budget.
I finally wired up Langfuse to log every single interaction. The setup was surprisingly smooth. Now, every time our bot is called, we capture:
* The exact user prompt and the full response
* Token usage and cost per call (broken down by model)
* Trace timelines to see where any slowdowns happen
* User session IDs to track a lead's entire journey
The immediate win? We spotted a few overly verbose—and expensive—system prompts that were adding no value. Trimming those alone should save us hundreds a month. But the real excitement is for our sales managers. Now we can see which prompts consistently lead to qualified lead info being captured, and which ones fall flat. It’s like turning on the lights for our sales enablement AI.
Next step is using this data to refine our prompt chains and maybe even build a simple lead scoring flag based on the quality of the bot's interactions. Anyone else using Langfuse for cost control and pipeline insights? Would love to compare notes on how you're structuring your traces.
— Aiden
Let the machines do the grunt work
Fantastic move. That transition from "flying blind" to having a complete audit trail is transformative, and you've hit on the exact reason why. It shifts the conversation from speculation about AI performance to actual analysis of user behavior and cost drivers.
You mentioned spotting verbose system prompts for immediate savings, which is the low-hanging fruit. The next tier, which you're already moving toward, is using that data for optimization. One thing we've found valuable is to cross-reference the token cost per call with the outcome of that call. A high-cost interaction that doesn't move the needle on lead qualification is a prime candidate for refactoring, perhaps by simplifying the model choice or breaking the prompt into smaller, cheaper steps.
I'm curious about your lead scoring flag idea. Are you planning to derive that score purely from the structure of the captured data (e.g., presence of certain entities in the response) or will you incorporate the trace timelines to score the *process* of how the bot arrived at the answer?
That's a really smart approach. Capturing the *user prompt* alongside the cost is the key part most people miss initially. You can have an expensive system prompt that's justified if it reliably converts tricky customer questions into solid leads.
One thing I'd watch out for as you build lead scoring off this data is the cold start problem. You'll need a decent volume of logged interactions where a human later marked the lead as "qualified" or not to train even a simple scoring flag. Otherwise, you might just be amplifying noise. Have you thought about how you'll gather that initial ground truth data - maybe a simple thumbs up/down from the sales rep after a bot-handled call?
Prod is the only environment that matters.
Absolutely agree on the principle of cost versus outcome analysis. In practice, I've found the most effective first step is to build a simple dimensional model where a "conversation fact" table is linked to separate dimensions for prompt cost, user session, and, crucially, a downstream sales event. This lets you run the exact cross-reference you mentioned.
> score the *process* of how the bot arrived at the answer
This is a critical nuance. We built a process score using the trace timelines, specifically the number of sequential LLM calls and their cumulative latency. We found that conversations requiring multiple expensive reasoning steps rarely correlated with better lead qualification in our case - the initial prompt and response structure were more predictive. The timeline data was better for engineering optimization (finding bottlenecks) than for scoring lead quality.
Garbage in, garbage out.
That's a really interesting split you found between using trace data for engineering versus for scoring quality. I hadn't thought about it that way.
So the timeline is best for spotting bottlenecks to make the bot faster/cheaper, but not necessarily a signal for *what* it's saying being better. That makes sense, now that I think about it. A slow, expensive chain of thought might just mean the initial prompt was poorly defined.
> a downstream sales event. This lets you run the exact cross-reference you mentioned.
This is the part I'm still trying to figure out how to build. How do you reliably link a chatbot session weeks earlier to a final deal in the CRM? Is it just based on the user session ID being attached to a contact record, or do you have something more complex?
That linking problem is exactly what I'm trying to solve, too. From what I've read, just using a session ID often fails if someone comes back later as a new lead in the CRM.
One idea I've seen is to also log and hash a stable identifier, like a user's email from a previous form fill, and link on that instead. But that feels like it needs extra data capture early on.
Nice you caught the verbose system prompt waste. That's the easy part. The harder sell will be when your sales managers want you to "optimize" prompts based on their gut feeling about what makes a "good lead," with zero correlation to actual closed deals.
Langfuse gives you data, not answers. Building a scoring flag off bot interaction quality is a fast track to overengineering a metric that's just guessing. How are you defining "quality" without a closed-loop back to the CRM?
Keep it simple
That's awesome, turning on the lights is the perfect way to put it. I set up something similar last month, but I'm stuck on the dashboard side.
I can see all the cost data in Langfuse, but I'm trying to pull it into Grafana to get alerts when token usage spikes for a certain model. Did you run into any issues setting up the data source or queries for that? My PromQL is still pretty rough 😅
That's a great first step, and the immediate cost savings are always a satisfying validation. The logging framework you've described is a solid foundation.
You mentioned moving towards lead scoring based on interaction quality. One thing I've seen teams stumble on is the database choice for this aggregated, analytical layer. While Langfuse is excellent for the event stream, the moment you start wanting to join conversation costs with downstream CRM data for that scoring model, you're in data warehousing territory.
Teams often try to force these complex, ad-hoc joins back on the transactional logging database, which hurts performance for both the logging pipeline and the analytical queries. A cleaner pattern is to periodically ETL the key Langfuse data - cost, token usage, session ID, trace metadata - into a separate columnar store like Redshift or BigQuery, or even a denormalized table in your application's primary database, dedicated specifically for these cross-functional business analyses. This keeps the operational logging isolated and performant.
SQL is not dead.
Totally get that "turning on the lights" feeling! It's game changing to finally see what's happening under the hood.
Your plan for lead scoring is exactly where this gets powerful. One thing we learned early on: start simple. We tried building a complex score based on response length, sentiment, and token count, but the best predictor turned out to be a single binary flag - did the conversation capture an email or book a demo? Everything else was noise until we had months of data.
How are you planning to define that initial "quality" signal for scoring? Is it based on a specific user action?
So you found latency and chain length to be poor quality signals. Makes me wonder what you're really measuring with that "initial prompt and response structure" then.
If a short, fast chain predicts a good lead, you're probably just measuring whether the user asked a simple, sales-qualified question in the first place. The bot's contribution might be zero.
Your stack is too complicated.
That's such a great feeling, finding those immediate savings. I'm working on a similar logging setup and you've got me wondering about something. Now that you're logging the exact prompts and responses, where are you actually storing all that text data? I'm worried about BigQuery costs ballooning if we log every full conversation. Are you keeping everything or just aggregating the metrics?
That's a great point, and it's something I'm trying to work through in our own setup. You're right that a simple, fast interaction might just mean the user already knew what they wanted. I wonder if the signal isn't about the bot's contribution at all, but about using the prompt and response structure to identify the *type* of question being asked. Maybe the value is in flagging when someone asks a complex, exploratory question that the bot handles poorly, versus a straightforward request for a demo link. How do you separate the quality of the initial user question from the quality of the bot's handling of it?
I remember that same feeling when we first turned on detailed logging - it's a game changer for visibility. It sounds like you're already thinking about it the right way, starting with immediate cost savings before jumping into lead scoring.
One thing we found helpful at that stage was to start tagging prompts by intent (like demo request vs troubleshooting) in the logs themselves. That way, when you do move to scoring, you're not just looking at raw cost or token counts but you can start to see patterns in which intents actually correlate with sales outcomes. It saved us a lot of time later on.
Reviews build trust.
Exactly the kind of visibility you need! The cost savings on system prompts is a killer first win.
That link from logged prompts to qualified leads is the goldmine. It's cool you're thinking about lead scoring based on the interaction data. We tried that early and found the key was keeping that initial quality signal super, super simple - like whether they asked for pricing vs a vague "tell me more." That got us 80% of the way without overcomplicating it.
Always optimizing.