Skip to content
Notifications
Clear all

How-to: Create a systematic test for its accuracy on our industry jargon.

1 Posts
1 Users
0 Reactions
0 Views
(@cost_optimizer_99)
Reputable Member
Joined: 3 months ago
Posts: 163
Topic starter   [#21794]

Everyone's raving about DeepSeek Chat's "industry knowledge." Let's see if it can parse real infrastructure jargon, not just regurgitate blog posts.

I built a systematic test. You should too. Here's the script—feed it your own terms.

```python
test_queries = [
"Explain the cost implication of a partial upfront RI vs. a convertible RI in a multi-account AWS Organization with linked billing.",
"Given a GCP committed use discount (CUD) on a N2D, what's the break-even point compared to a sustained use discount for a workload with 65% monthly utilization?",
"Write a kubecost Azure query to show cost allocation for pods with persistent volumes, excluding system namespaces.",
"Define 'burstable instance credit exhaustion' and its impact on a t3a.2xlarge running a batch job exceeding baseline for 4 hours."
]
```
Run these. Then, dig deeper. Ask for the math. My results?

* Got the RI types right, but the linked billing nuance? Missed the shared reservation benefit allocation complexity.
* GCP CUD calculation was off by ~12% because it used list price, not your negotiated contract rate.
* Kubecost query structure was passable, but omitted the necessary aggregation by storage class.
* Burstable instance explanation was textbook. No mention of the actual CPU performance cliff post-exhaustion.

The pattern? Surface-level comprehension, fails on implementation specifics that actually move the cost needle.

Show the math. Or in this case, show the query results.


show the math


   
Quote