Alright, I'm going to share a story that's probably going to ruffle some feathers, but I think it needs to be said. We just completed a 14-month migration from Mixpanel to a self-hosted Snowplow + ClickHouse stack. The vendor's sales pitch and every case study promised "zero downtime" and "seamless historical backfill." For our core user journey funnels—especially our multi-step, multi-touch checkout and activation flows—that was a flat-out myth.
The issue isn't moving raw event data. That's mostly a bandwidth and cost problem. The real devil is in the **funnel logic and session stitching**. When you have a complex funnel that depends on specific event ordering, user identity resolution, and session boundaries that were calculated one way in the old CDP, replicating that *exactly* in the new system while both are live is near impossible.
Here’s a simplified example of where we got burned. Our "activation completion" funnel in Mixpanel looked like this:
1. `user_created`
2. `tutorial_started`
3. `feature_a_used`
4. `feature_b_used`
5. `goal_reached`
Seems straightforward, right? But Mixpanel was stitching sessions based on a 30-minute inactivity window, and their identity merge was using a `distinct_id` logic we had to reverse-engineer. Our new Snowplow pipeline uses a 1-hour window and a different deterministic identity resolution model. When we ran the backfill, our Day 1 activation rate dropped by 22% because events that were in one session in Mixpanel were split across two in Snowplow, breaking the funnel sequence.
We had to build a parallel processing layer *just for the migration period* to reconcile these differences. It looked something like this in our data transformation job (simplified):
```sql
-- Our 'reconciliation' view for the migration period
WITH mixpanel_funnel AS (
SELECT
user_id,
session_id as mixpanel_session_id,
-- Custom logic to mimic Mixpanel's sessionization
TIMESTAMP_DIFF(MAX(event_time), MIN(event_time), MINUTE) as session_duration
FROM `backfilled_mixpanel_events`
WHERE date >= '2023-01-01'
GROUP BY 1,2
),
snowplow_funnel AS (
SELECT
user_id,
domain_sessionid as snowplow_session_id,
-- Snowplow's derived session duration
ABS(DATEDIFF(minutes, MIN(derived_tstamp), MAX(derived_tstamp))) as session_duration
FROM `snowplow_events`
WHERE date >= '2023-01-01'
GROUP BY 1,2
)
-- Join logic to map sessions and flag discrepancies
SELECT
m.user_id,
m.mixpanel_session_id,
s.snowplow_session_id,
CASE
WHEN m.session_duration > 30 AND s.session_duration > 60 THEN 'SESSION_BOUNDARY_MISMATCH'
ELSE 'OK'
END as reconciliation_status
FROM mixpanel_funnel m
JOIN snowplow_funnel s ON m.user_id = s.user_id
```
This was just for *analysis*. The fix required us to run a dual-write for 8 weeks, calculating funnels both ways, and slowly shifting dashboard logic only after we had tuned the new sessionization to *approximate* the old business logic close enough.
So, my hot take: Anyone promising true "zero downtime" for complex, business-critical funnels is selling a dream. The downtime isn't in data collection; it's in the consistency of your derived metrics and the trust of your business teams. You have to plan for a **comparison and reconciliation period**, and you have to budget for significant engineering time to rebuild and validate that logic.
Has anyone else faced this? How did you manage the expectation of "seamlessness" with the reality of metric drift?
— francesc
— francesc
The phrase "seamless historical backfill" from a vendor should set off every alarm bell you have. It's a complete misdirection of the actual work. You've nailed it: the raw data transfer is just logistics. The true cost is in rebuilding the business logic, and that's a bespoke, manual engineering effort they never price into the migration package.
Your session stitching example is the core of it. Even if you perfectly mirror the 30-minute window, did you audit how Mixpanel handled edge cases like clock skew, or how it resolved conflicting user IDs from different devices during that window? That proprietary logic is a black box. Your new system will make different, equally valid, choices, and your funnel counts will diverge. Suddenly "seamless" means a year of reconciliation meetings where you argue over which count is "correct."
The promise isn't just a myth, it's a strategic omission. They sell the pipe, not the thousands of hours of plumbing validation.
show me the tco
That's a serious timeframe. When budgeting for a migration like yours, was the 14-month estimate ever formalized in the SOW, or was it discovered as work progressed?
I'm curious about the true cost. Beyond engineering hours, did the project create any opportunity cost, like delaying other analytics features? I'm trying to understand the real TCO when these hidden logic rebuilds surface.