Alright, team. Another week, another dashboard nobody asked for but everyone will eventually complain about if it goes dark. The eternal cycle.
I’ve just finished stitching together a little Slack bot that fires off a weekly digest of model usage, broken down by team and project. Why? Because our current "process" is a chaotic blend of tribal knowledge, spreadsheet archaeology, and hopeful guesses during budget meetings. Sound familiar? I’m tired of hearing "I think the support team uses a lot of GPT-4" or "Was that spike last month from the new feature or a bug?" We implemented Langfuse to get observability, but it turns out observability doesn't mean much if the data just sits there, looking pretty.
So, here’s what this bot spits out every Monday at 9 AM, directly into our #revenue-ops channel:
* **Total Traces & Tokens:** High-level, for the finance people who only care about the bill.
* **Usage by Project/Team:** This is the meat. Maps Langfuse projects to our internal teams (sales-eng, product, support). No more guessing who’s burning credits.
* **Top 5 Models:** Ranked by token consumption. The usual suspects (gpt-4, claude-3) are always there, but seeing gpt-3.5-turbo still hanging on for dear life in some corners is… enlightening.
* **Week-over-Week Change:** Flags any team whose usage jumped by more than 30%. Usually it's a new feature launch. Sometimes it's a leak.
* **Cost Estimate (Cloudflare Workers + Model):** Back-of-the-napkin math using rough token-to-USD averages. It's directional, not perfect, but it starts the conversation before the invoice lands.
The build was… fine. Langfuse's API is straightforward, if a bit verbose when you need to aggregate across projects. The real headache was the mapping layer—maintaining that clean, living dictionary of which Langfuse project belongs to which internal cost center. I used a simple Airtable for that, because God knows I'm not building another admin UI. The bot itself is a Python script on a scheduler, using the Langfuse Python SDK and the Slack Bolt framework.
The immediate value isn't in the pretty charts. It's in the accountability. Last week's report quietly highlighted that our pre-sales team's prototype environment, which everyone forgot about, was still calling `gpt-4-1106-preview` at a non-trivial rate. That's now shut down. The bot pays for itself.
Has anyone else built something similar for team-level attribution? I'm particularly interested if you've found a cleaner way to handle the project-to-team mapping without it becoming a full-time maintenance job. I don't trust our engineers to consistently tag things perfectly in the SDK, so a manual override layer feels necessary, for now.