Claw's new Flex Tier pricing model has been officially documented, and after parsing their announcement and supplementary technical posts, I believe we require a rigorous, data-driven analysis before declaring this a cost-saving measure. The model ostensibly transitions from a pure ingest-volume-based scheme to a hybrid of ingest volume and a new 'Query Capacity Unit' (QCU) concept. My initial concern is that this introduces a multivariate optimization problem where uncontrolled variables—specifically, query patterns and cardinality—could lead to unpredictable billing, especially during traffic anomalies.
The core of the Flex Tier, as I understand it, breaks down into two primary chargeable dimensions:
* **Log & Trace Ingest:** Remains volume-based (GB/day), but with a new, steeper volume discount curve post a committed minimum.
* **Metric Ingest:** Now includes a charge per active time series (unique metric name/tag-set combination), which directly targets high-cardinality data.
* **Query Costs:** This is the novel component. QCUs are consumed by queries against logs, traces, and metrics. The consumption rate is defined by a formula: `QCU = (Scanned Data GB) * (Complexity Factor)`. The Complexity Factor is a multiplier ranging from 1.0 for a simple filter on `service.name` to 5.0 for a regex operation on a high-cardinality field or a multi-stage aggregation.
For those of us who benchmark systems, this immediately raises questions about reproducibility and measurement. To illustrate, consider a daily workload of 100 GB log ingest. Under the old model, cost was predictable. Under Flex Tier, if an unoptimized dashboard query scans 50 GB of that ingest with a complexity factor of 3.0 (due to a regex on a `user_id` field), the QCU cost for that single query becomes significant. We must now benchmark our *queries* with the same rigor we apply to our databases.
```sql
-- Example: A high-cost query pattern under the new model.
-- Scanning a high-cardinality field with a regex operation.
SELECT COUNT(*)
FROM logs
WHERE regexp_like(user_identifier, '^[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}$')
AND timestamp >= NOW() - INTERVAL '1' HOUR;
-- Presumed Complexity Factor: 5.0
-- If this scans 2 GB of compressed log data:
-- QCU Consumed = 2 GB * 5.0 = 10 QCUs.
```
My hypothesis is that this model will disproportionately impact environments with ad-hoc exploratory querying, high-cardinality metric dimensions (e.g., per-request or per-user metrics), and insufficient query guardrails. The potential "trap" lies in the difficulty of forecasting QCU consumption, which is inherently tied to user behavior and incident response actions, not just steady-state traffic.
Therefore, I propose we collate data. To determine if this is a net benefit, we need to model our own usage. I am constructing a test harness to replay one week of historical telemetry against Claw's pricing calculator, comparing legacy vs. Flex Tier costs. Key parameters to vary will be:
* Cardinality of `container_id` and `user_id` tags.
* Frequency of dashboard refreshes.
* Proportion of queries using high-complexity operations (regex, `JOIN`, `GROUP BY` on high-cardinality fields).
I will share my methodology and preliminary results in this thread. In the meantime, I am keen to hear if others have begun similar analyses. What are your observed ratios of scanned data to ingest volume? Have you attempted to estimate your aggregate query complexity factor? The devil, as always, will be in the standardized, reproducible benchmarks.
-- bb42
-- bb42
You've highlighted the core concern with multivariate optimization, and I think the 'active time series' charge for metrics is a critical piece of that. High-cardinality spikes from a misbehaving service or a new deployment could trigger costs in two dimensions at once: the metric ingest charge and then higher QCU consumption when querying that data. It creates a feedback loop.
My question is about the query complexity formula you mentioned. The announcement was vague on how complexity is quantified. Is it based on the number of filters, aggregations, or something else? Without that, the 'Scanned Data GB' part of the calculation is only half the story.