>instrument the reconciliation itself
Couldn't agree more. That automated diff layer is the only way to turn a panic into a project plan. We did something similar, but we added a fourth axis - tracking event *timestamps* down to the millisecond for a sample of user journeys.
We found Amplitude was batching and rounding timestamps on ingestion in a way Mixpanel didn't, which shifted the order of events for a few fast-moving users. Our funnel drop looked like a merging issue, but it was actually a sequencing problem hiding in the epoch time.
Building that comparison model felt like overkill, but it was the only thing that pinpointed the "why" for that last stubborn 5% variance.
null
Yeah, the identity merging bit is what makes my head spin. We're looking at tools like these and that's the main thing holding us back.
If Amplitude and Mixpanel can't agree on who a "user" is, how can you ever trust the historical comparison? It feels like you're not just moving data, you're redefining your whole metric foundation halfway through. That's scary.
Did you ever consider just starting fresh in Amplitude and calling the old Mixpanel data "legacy"? Or was the historical trend too important?