Just completed a performance benchmark comparing our legacy CDP's data pipeline throughput against the new vendor's promised specs. The results were within acceptable variance, so we greenlit the migration. Two weeks into a phased schema translation and event backfill, we received a notification: the new CDP is shifting to a consumption-based pricing model effective next quarter.
Our original cost projection was based on their flat-rate enterprise tier. The new model introduces variable costs for compute units and active user profiles, which directly scales with our historical data volume and event ingestion rates.
Our current migration state:
- **Schema Translation:** ~80% complete. We've mapped all core identity and event schemas.
- **Historical Backfill:** Running for our largest segment (Q3 2023 onward). Ingesting ~2 TB of historical events.
- **Downstream Connectors:** Our data warehouse and email service provider integrations are still wired to the old CDP.
The immediate problem is financial predictability, but the technical risk is that the new pricing disincentivizes backfilling our full 3-year history. This could create a permanent data gap in the new system. We are evaluating three paths:
1. **Pause and renegotiate.** Halt all migration work and attempt to lock in the old pricing for a committed term.
2. **Accelerate and absorb.** Rush to complete the migration before the new pricing takes effect, accepting higher short-term engineering cost.
3. **Abort and reassess.** Roll back migrated components and re-evaluate the vendor landscape.
Has anyone executed a mid-migration pivot due to a vendor pricing change? Specifically:
* What were the contractual levers, if any?
* How did you manage the state of partially migrated data pipelines?
* Did you benchmark alternative vendors under a compressed timeline?
Our backfill script logic is now a cost center. A simplified version is below.
```python
# This was a simple throughput script. Now we need to add cost tracking.
for batch in historical_event_batches:
transformed = transform_to_new_schema(batch)
response = new_cdp_client.ingest(transformed)
log_ingestion_success(response)
# New required logic
compute_units_used = estimate_compute_units(batch_size, complexity)
current_cost += compute_units_used * price_per_unit # New variable
```
Benchmarks > marketing.
BenchMark