One of the biggest challenges we faced during our client's CDP migration last quarter wasn't the data mapping itself—it was knowing, with certainty, how complete the migration was at any given moment. Without a clear view, you risk cutting over too early or wasting cycles on edge cases.
We built a simple Grafana dashboard that became our single source of truth. It tracked three core metrics across all user profiles and key events:
* **Profile Translation Fidelity:** Percentage of user profiles in the new CDP where all critical traits (e.g., `lifetime_value`, `account_tier`) matched the source values within an acceptable delta.
* **Historical Event Backfill Progress:** A running count of events successfully replayed into the new system, compared to the total in the source, broken down by event type (e.g., `Page Viewed`, `Order Completed`).
* **Downstream Integration Health:** Status of key connectors (e.g., to Braze, Salesforce) showing successful API calls and error rates from the *new* CDP, ensuring the re-wiring was operational.
This dashboard was powered by a small analytics pipeline that compared daily snapshots from the old and new systems, and logged the new CDP's own outbound connection logs. The visualization made it easy to get stakeholder buy-in for the cutover, as we could point to a 99.8% match for three consecutive days.
What metrics did others track during their migration to build confidence? I'm particularly interested in how you handled validating complex nested object structures or stateful user journeys.
Love this. We did something very similar for a warehouse migration, but we added a fourth metric: **Schema Drift Detection**. We had a panel that tracked counts of columns added/dropped or data type changes in the new system versus the source snapshot. Saved us from a nasty type coercion bug that would've only showed up weeks later.
What did you use for the comparison pipeline? We wrote a few Airflow DAGs that did daily checks, but it got pretty heavy. Curious if you found a lighter-weight way.
Prompt engineering is the new debugging.
Schema drift is a solid addition. Airflow feels like overkill for that, you're right.
We ran the checks as a job in our migration orchestrator (a simple Go service) and pushed the metrics directly to Prometheus. It compared table DDL from source and target snapshots and incremented a counter for each detected drift.
The key was only running it at critical stages, not daily. Did you find the daily cadence actually gave you better signals, or just more noise?
We went with a scheduled Lambda that fires on a cron, dumping findings to CloudWatch Metrics. It was lightweight enough, but honestly, daily was overkill for us too. The signal-to-noise ratio was bad until the final stages.
We got more value from triggering the check after each major batch load finished, like you mentioned. It caught drift when it was introduced, not days later. The daily schedule just added cost and alert fatigue.
Curious about your Go service - did you expose the drift count as a gauge, or did you use a different metric type? We found a simple gauge worked, but we had to be careful about resetting it.
terraform and chill