You're exactly right about the audit trail problem. The sidecar logging agent approach, while heavy, is one of the few reliable methods. We used a DaemonSet with a container running `dnstap` to capture and enrich queries with pod metadata before forwarding to the SIG. The operational burden, however, shifts from managing the SIG to maintaining that custom logging layer.
The separate SIGs per node pool idea is theoretically sound for segmentation, but as you note, the management overhead becomes prohibitive quickly. Each becomes its own policy and reporting silo. I've found that trying to align node pools with a strict security boundary, like an isolated "payment processing" pool, is the only scenario where this cost is justified.
Your query volume figure is sobering. Many vendors' initial estimates are based on desktop user profiles, not ephemeral, high-churn container workloads. The financial risk isn't just overages; it's the unpredictable scaling tied to CI/CD activity and auto-scaling events that can create massive monthly variance.
Migrate slow, validate fast.
That sidecar logging agent feels like adding another complex system just to understand the one you bought, you know? I'm already worried about keeping the SIG running, and now there's a whole custom DaemonSet layer to babysit too.
The financial risk you mentioned about unpredictable scaling is what really scares me off. Our CI/CD activity is all over the place, some days it's quiet and other days it's a frenzy. How can you possibly budget for that if the pricing is based on query volume? Do teams just eat the variance as a cost of doing business, or do you have to put throttles in place that might break things?
That query volume is shocking, honestly. Did you find a way to get accurate estimates before you signed up, or was that a painful lesson learned after you were already on the hook?
Also, the sidecar logging agent solution seems like a huge lift for a small team. It feels like the tool creates a problem it's supposed to solve, you know?
You're right about the logs, but the bigger tax is the price per query for all that noise. Every false positive costs you real money.
Forget tuning their categories, you'll go broke on DNS volume before you ever get it clean.
show me the bill
We learned our estimate the hard way, with an unexpected quarterly bill that triggered an internal audit. The rep's seat-based quote was off by a factor of ten. They hadn't asked for historical DNS logs, which we didn't have aggregated at the time.
The sidecar agent does invert the problem. You're now responsible for a custom telemetry layer whose sole purpose is to make your paid security service auditable. For a team under 100, that's often an untenable operational trade-off, adding fragility and on-call burden to solve a compliance gap the vendor created.
You have to demand per-second query data from a proof-of-concept, not a spreadsheet estimate. If they can't provide that visibility during the trial, walk away.
—at