Everyone's posting their shiny "migration successful!" posts. Did anyone actually *verify* the data? Or just trust the vendor's script and pray?
We didn't. Here's the bare-bones Python validator we ran to compare event streams between the old and new CDP for a week. It's not fancy, but it checks three things the vendor's tools "glossed over":
* Exact payload structure (nested JSON diffs, not just top-level keys).
* Timestamp drift beyond a 2-second tolerance for "real-time" sync.
* Cardinality mismatch on user IDs, which exposed a silent drop of anonymous events.
The core of it just loops, fetches from both APIs, and logs any mismatch. Ran it on a sample of 10 million events. Found a 0.7% discrepancy rate, mostly from truncated strings in one platform's "properties" field. Vendor tried to call it "normalization." We called it a breach of contract.
—aB
—aB
That 2-second timestamp tolerance is an interesting choice. In our stream validation, we found sub-second jitter was common, but consistent drift over that threshold usually pointed to a buffering or batching configuration mismatch in the new pipeline, not just network latency. It's often a sign the new system is grouping events differently before emission.
The cardinality check on user IDs is crucial. We had a similar issue where the new system's SDK was stripping null IDs by default, which dropped all our pre-authentication events. The vendor's dashboard showed 99.9% fidelity because it only counted events with a valid user ID present in both systems.
CPU cycles matter