Alright, I'm going to share a story that's probably going to ruffle some feathers, but I think it needs to be said. We just completed a 14-month migration from Mixpanel to a self-hosted Snowplow + ClickHouse stack. The vendor's sales pitch and every case study promised "zero downtime" and "seamless historical backfill." For our core user journey funnels—especially our multi-step, multi-touch checkout and activation flows—that was a flat-out myth.
The issue isn't moving raw event data. That's mostly a bandwidth and cost problem. The real devil is in the **funnel logic and session stitching**. When you have a complex funnel that depends on specific event ordering, user identity resolution, and session boundaries that were calculated one way in the old CDP, replicating that *exactly* in the new system while both are live is near impossible.
Here’s a simplified example of where we got burned. Our "activation completion" funnel in Mixpanel looked like this:
1. `user_created`
2. `tutorial_started`
3. `feature_a_used`
4. `feature_b_used`
5. `goal_reached`
Seems straightforward, right? But Mixpanel was stitching sessions based on a 30-minute inactivity window, and their identity merge was using a `distinct_id` logic we had to reverse-engineer. Our new Snowplow pipeline uses a 1-hour window and a different deterministic identity resolution model. When we ran the backfill, our Day 1 activation rate dropped by 22% because events that were in one session in Mixpanel were split across two in Snowplow, breaking the funnel sequence.
We had to build a parallel processing layer *just for the migration period* to reconcile these differences. It looked something like this in our data transformation job (simplified):
```sql
-- Our 'reconciliation' view for the migration period
WITH mixpanel_funnel AS (
SELECT
user_id,
session_id as mixpanel_session_id,
-- Custom logic to mimic Mixpanel's sessionization
TIMESTAMP_DIFF(MAX(event_time), MIN(event_time), MINUTE) as session_duration
FROM `backfilled_mixpanel_events`
WHERE date >= '2023-01-01'
GROUP BY 1,2
),
snowplow_funnel AS (
SELECT
user_id,
domain_sessionid as snowplow_session_id,
-- Snowplow's derived session duration
ABS(DATEDIFF(minutes, MIN(derived_tstamp), MAX(derived_tstamp))) as session_duration
FROM `snowplow_events`
WHERE date >= '2023-01-01'
GROUP BY 1,2
)
-- Join logic to map sessions and flag discrepancies
SELECT
m.user_id,
m.mixpanel_session_id,
s.snowplow_session_id,
CASE
WHEN m.session_duration > 30 AND s.session_duration > 60 THEN 'SESSION_BOUNDARY_MISMATCH'
ELSE 'OK'
END as reconciliation_status
FROM mixpanel_funnel m
JOIN snowplow_funnel s ON m.user_id = s.user_id
```
This was just for *analysis*. The fix required us to run a dual-write for 8 weeks, calculating funnels both ways, and slowly shifting dashboard logic only after we had tuned the new sessionization to *approximate* the old business logic close enough.
So, my hot take: Anyone promising true "zero downtime" for complex, business-critical funnels is selling a dream. The downtime isn't in data collection; it's in the consistency of your derived metrics and the trust of your business teams. You have to plan for a **comparison and reconciliation period**, and you have to budget for significant engineering time to rebuild and validate that logic.
Has anyone else faced this? How did you manage the expectation of "seamlessness" with the reality of metric drift?
— francesc
— francesc
The phrase "seamless historical backfill" from a vendor should set off every alarm bell you have. It's a complete misdirection of the actual work. You've nailed it: the raw data transfer is just logistics. The true cost is in rebuilding the business logic, and that's a bespoke, manual engineering effort they never price into the migration package.
Your session stitching example is the core of it. Even if you perfectly mirror the 30-minute window, did you audit how Mixpanel handled edge cases like clock skew, or how it resolved conflicting user IDs from different devices during that window? That proprietary logic is a black box. Your new system will make different, equally valid, choices, and your funnel counts will diverge. Suddenly "seamless" means a year of reconciliation meetings where you argue over which count is "correct."
The promise isn't just a myth, it's a strategic omission. They sell the pipe, not the thousands of hours of plumbing validation.
show me the tco
That's a serious timeframe. When budgeting for a migration like yours, was the 14-month estimate ever formalized in the SOW, or was it discovered as work progressed?
I'm curious about the true cost. Beyond engineering hours, did the project create any opportunity cost, like delaying other analytics features? I'm trying to understand the real TCO when these hidden logic rebuilds surface.
That 30-minute inactivity window is such a classic trap. Even if you replicate the time boundary, the underlying clock synchronization between your servers and the vendor's ingestion pipeline can create skew you'll never account for. A user might have two events 29 minutes apart in *your* timestamps, but 31 minutes apart in Mixpanel's processed data due to queue delays. That one session becomes two, and your funnel count is permanently off.
We had a similar headache with identity merging during a migration. The old system had a quirky, undocumented rule where it would prioritize an email-based ID over a device ID only after the third event from that device. Rebuilding that "feel" took months of sampling and comparing, not just copying a rule.
So you're spot on - "zero downtime" for data collection might be possible, but zero discontinuity for your existing business metrics? That's the myth right there.
Architect first, buy later
Absolutely, the undocumented rules are the killer. We found that the "feel" of a system is often a collection of hundreds of these tiny, unspoken heuristics that analysts have silently come to rely on. Replicating them isn't engineering, it's archaeology.
Your point about clock skew hits home, too. It makes you realize that "zero downtime" for metrics often assumes a perfectly synchronous world that just doesn't exist. The promise glosses over the fact that you're not just moving data, you're attempting to teleport an entire, living context from one universe into another, and the physics are never the same.
We ended up running both systems in parallel for nearly a year, not because of data transfer, but because we needed that long to rebuild trust in the new definitions. The vendor's timeline never included that psychological runway.
hugo
Your simplified funnel example perfectly illustrates the abstraction leak. You're not just moving five events, you're moving the hidden state machine that decides what constitutes a valid sequence.
The 30-minute window is a parameter, but the real question is: when did that timer reset? On event receipt at their endpoint, or after server-side processing? If it's the latter, as user1168 hinted, your own instrumentation latency becomes part of the funnel logic. A slow beacon dispatch from a mobile app could push an event across the session boundary in the old system, but not in your new, more consistent pipeline. Your funnel step four might now be excluded, and you'll never know why just by looking at timestamps.
This is why parallel runs are mandatory for any metric with business weight. "Zero downtime" for data ingestion is achievable. "Zero discrepancy" for derived business logic is a fantasy. You need that overlap to establish a baseline error rate for your new definitions, not to validate the data transfer.
Trust but verify.