Everyone talks about migrating *all* their historical data. That’s how you end up paying your new CDP to store a decade’s worth of `button_clicked_test` events.
Why audit before you migrate? Because every event you port is a liability. It becomes part of the new schema, clogs your warehouse syncs, and gives the vendor another vector for usage-based fees. You’re not just moving data; you’re cementing legacy junk into a fresh contract.
Here’s the cynical approach:
First, pull a report of every event key from your current system. Not just the “last 30 days” garbage—get the full timeline. Then, cross-reference it with actual dashboards, segments, and active downstream models. You’ll find at least 30% are orphans. I’ve seen teams migrate deprecated `user_status` events that haven’t been consumed in years.
Next, don’t trust the vendor’s “auto-map” feature. That’s a trap to blindly carry over cruft. Force a manual review of each event that survived the first cut. If no team can articulate its current ROI, kill it. The goal is to arrive at the new platform with a pruned, justified dataset. Anything else is just paying for someone else’s past mistakes.
caveat emptor.
Buyer beware
Totally agree. That cross-reference step is crucial, but it's often the messiest part. You pull the event list and then you're left with a massive spreadsheet trying to map events to dashboards manually.
My trick is to go straight for the API logs or query logs of your BI tool/warehouse. Search for the event key names in the actual SQL queries or API call parameters your reporting tools are making. If an event name doesn't appear in a live query from the last, say, 90 days, it's a dead candidate. It catches those "just in case" events that everyone thinks someone else might be using.
And your point about the auto-map is spot on. I once saw it happily migrate `purchase` and `purchase_event` as two separate streams. That duplicate data sat there for months before we noticed the billing creep. 😅 Manual review is painful but it's the only way to catch those semantic duplicates.
Integration Ian
That query log trick is clever, but it assumes your BI layer is the only consumer. I've seen raw event streams piped directly into some homegrown alerting system or a department's custom Python script that never hits the warehouse logs. You kill the event, and a month later some finance report silently breaks.
Your 90-day cutoff is also a bit optimistic for anything with a quarterly or seasonal cycle. Better to check for absence across a full business cycle, or you'll be resurrecting `black_friday_promo_view` every November.
And yes, the auto-map tools are a trap. They treat migration as a syntax problem, not a semantics one. `purchase` vs `purchase_event` is a classic. The real fun starts when you find `purchase`, `purchase_success`, and `checkout_complete` all representing the same thing, sent by three different legacy code paths.
null
>Force a manual review of each event that survived the first cut. If no team can articulate its current ROI, kill it.
This is where it falls apart. Ask a product team for the "ROI" of an event and you'll get hand-wavy justifications about future plans. They'll never sign off on deletion.
You need a technical kill-switch, not a committee. Flag the orphaned events in staging, shut off the pipeline for them, and wait for the tickets to roll in. If nobody screams in 30 days, you have your answer.
No SOC2, no deal.