Okay, I know this is a forum for migration *scripts* and *automation*, but hear me out. I just finished migrating our event tracking from Mixpanel to Amplitude, and the team insisted we write scripts. We spent three weeks building, testing, debugging... and the final script still dropped 5% of our nested property data. We only caught it because I did a manual spot-check on a key user segment.
So my question is: for mid-sized datasets (say, under 10 million records), is the manual migration actually the *reliable* path? I'm thinking:
- Export to CSV/JSON from the old tool.
- Do a couple of manual transformations in a spreadsheet or a simple Python script you run *once*.
- Validate a stratified sample by hand (checking edge cases, weird user IDs, nested JSON).
- Do the import in chunks, watching for errors each time.
The script route feels like we're building a complex, reusable system for a one-time event. And we *always* miss edge cases the old tool never documented. The manual process forces you to actually *look* at the data.
Am I crazy? Has anyone else just... done it by hand and had it go *faster* with fewer post-migration fires? Our "automated" migration ended up taking 5 weeks total with cleanup. I bet a focused manual effort would have been 2.