Skip to content
Notifications
Clear all

ELI5: Why do we need separate metrics for 'relevance' and 'coherence'?

1 Posts
1 Users
0 Reactions
2 Views
(@cloud_cost_fighter)
Estimable Member
Joined: 2 months ago
Posts: 123
Topic starter   [#17886]

Here's the FinOps analogy that finally made it click for me: think of relevance as staying under budget, and coherence as having a clean, auditable bill.

If I ask an LLM for "AWS cost-saving tips for Kubernetes," and it gives me a perfectly grammatical paragraph about the nutritional benefits of kale, that's **high coherence** (sentence flows nicely) but **zero relevance**. It's like getting a beautifully formatted invoice from a vendor you never contracted.

The reverse is also painful. If it spits out a bulleted list of actual AWS services like "use Spot Instances, right-size requests, implement tagging," but the sentences are garbled or contradictory, that's **high relevance** (correct concepts) but **low coherence**. It's like getting a cost report where the numbers don't add up and the explanations are nonsense—you can't act on it.

We track both because they measure different failure modes:
* **Relevance** asks: "Did it answer the *actual question*?" (Cost of misalignment: wasted time/credits.)
* **Coherence** asks: "Is the answer internally consistent and logically structured?" (Cost of misalignment: you can't trust or implement the output.)

In practice, you need both. A relevant but incoherent answer is useless noise. A coherent but irrelevant answer is a perfectly formatted bill for services you didn't use. Neither helps you ship.


Cloud costs are not destiny.


   
Quote