Hey everyone, I'm in the early stages of planning a CDP migration for our e-commerce platform. We're moving off a legacy homegrown system and have narrowed it down to mParticle and RudderStack after some initial research.
The core dilemma I'm trying to wrap my head around is the trade-off between raw event throughput speed and the richness of features for transformation. From what I've gathered, mParticle seems incredibly fast for ingesting and forwarding raw events, which is tempting for our high-volume checkout flows. On the other hand, RudderStack's open-source Transformations feature looks powerful for cleaning and shaping data before it hits the warehouse, which would save us a lot of messy SQL modeling later.
I'm particularly stuck on the historical data backfill part. Our legacy data schema is... not great. A lot of nested JSON and inconsistent typing. I'm trying to think through what the migration scripts would look like.
For example, if I'm converting a legacy `user_action` event to a RudderStack `identify` call, how much transformation is best done in the migration script vs. within the CDP itself? With mParticle's focus on speed, does that mean we'd have to do *all* the schema cleanup upfront in our backfill script? That feels like a huge, brittle Python job.
```sql
-- Example of our current legacy 'page_view' event structure in our old DB
SELECT user_id,
properties -> 'page' -> 'name' as page_name, -- sometimes an object, sometimes a string
timestamp,
properties -> 'utm' as utm_params -- nested JSON, needs flattening
FROM legacy_events
WHERE event_type = 'page_view';
```
Has anyone done a similar migration and can share the step-by-step? How did you balance the need for speed with the need for clean, usable data? Did you lean on the CDP's transformation features, or did you handle everything in the migration ETL? Any gotchas with re-wiring downstream connectors to tools like Braze or Segment?