Exactly. That initial pricing model lures you in, but it completely obscures the variable cost of actual usage. Your $0.028 figure is the kind of data that should be front and center in their own billing dashboard.
Building that internal logging is a must for any team on a "per seat" plan. It forces you to answer the question you didn't know to ask: what are we actually paying for each interaction?
One thing we found helpful was benchmarking that cost per query against other knowledge tools. It puts the Humata spend in perspective and sometimes justifies the expense, or it flags when a simple search tool would suffice.
✌️
Benchmarking against other tools is the crucial second step after you've established your baseline. We did exactly that and found our internal $0.018 per query for Humata was actually favorable compared to the effective per-request cost of our heavily customized Elasticsearch cluster for document search, once you factored in engineering maintenance.
However, the comparison only holds if you're measuring apples to apples. A simple keyword search in another tool versus a complex analytical question in Humata are fundamentally different outputs. Our Grafana dashboard ended up tracking two parallel metrics: cost per query, and a subjective "value score" per query type (1-5) logged by the user post-response. The correlation between those two dimensions revealed where the spend was actually justified.
Latency is a liability
Welcome to the club. You're now maintaining a custom billing system for a service you already pay for. Wait until you add the engineering time you're burning on this to the "true cost" column. That $0.028 just got a lot less charming.
If it ain't broke, don't 'upgrade' it.
That's the whole point of building it. You're not just tracking Humata's cost, you're capturing the true operational expense. If that number is still acceptable after adding engineering hours, then the tool's value is proven. If not, you've got the data to kill it.
We bake our dashboard maintenance time into the quarterly vendor review. It becomes part of the TCO calculation that decides whether we renew.