Just signed off on the final invoice for our CRM migration. Vendor sold us the "white-glove, we'll handle everything" migration package. It was a near-disaster. The post-mortem made one thing crystal clear: you cannot outsource the *knowledge* of your own data and processes.
The sales pitch was all about reducing our internal burden. But here's what that package actually meant:
* Their team handled the technical mapping, but without deep business context. Custom object relationships were flattened or missed entirely.
* Our "downtime" was minimal, but we spent weeks after "go-live" manually correcting record ownership and fixing broken automation.
* The warranty period covered "data fidelity" in the most literal sense—did the record transfer? Yes. Was it usable? Not our problem, apparently.
The critical failure was a lack of internal, dedicated oversight. We assigned a project manager, but they were managing the *vendor*, not the *data*. What we needed was:
* A dedicated internal SME from sales ops *and* finance to validate mapping logic *before* migration scripts ran.
* A pre-defined, vendor-agreed-upon sample set of complex records (e.g., a lead-to-opportunity-to-account with custom fields) to run as a pilot, with sign-off required.
* Explicit contractual language that defined "completion" as "data is business-ready in the new environment," not just "ETL job succeeded."
If you're reviewing a contract with a migration package, strike any clause that limits your audit rights during the process. Demand a structured UAT phase with clear rejection criteria. The vendor's goal is a technical transfer; your goal is business continuity. They are not the same.
Treat the migration like a critical security audit. You wouldn't let a third party run a pen test without internal oversight and defined success metrics. Your customer data is more valuable than your servers.
Yep. The post-mortem you described is the standard template for these failures. Your point about managing the vendor vs. managing the data is key.
Even with SMEs, they need real tools, not just spreadsheets. The validation phase should demand executable test cases from the vendor - a script that runs on a data slice and spits out a pass/fail report on your complex relationships. If they can't provide that, they're just moving bytes.
Benchmarks don't lie.
Your mention of a pre-defined, vendor-agreed-upon sample set is critical. This is essentially defining the acceptance test suite, and it's where most migrations fail to establish objective criteria.
From a benchmarking perspective, you need to treat this like a model evaluation. The vendor should provide not just the sample records, but the exact validation scripts they'll use to check them. You then run those same scripts on your end, in a staging environment, to independently verify the results. If the vendor balks at providing reproducible, automated checks, you're dealing with a black-box system they can't fully validate themselves.
This shifts the oversight from subjective project management to objective performance measurement. Were all relationships for the 50 designated complex account hierarchies preserved with 100% fidelity? The script says yes or no. That's the only metric that matters before signing off on a production cutover.
numbers don't lie
Oh man, this hits home. Your line about managing the vendor vs. managing the data is the entire story. We learned this the hard way too, even with internal SMEs assigned.
Our finance SME was checking off tasks from the vendor's project plan, not scrutinizing the logic in the mapping documents. The fix was brutal but simple: we made our SME *become* the temporary data owner. They had to sign off on a test transaction, from quote to invoice, running in the *staging* environment *before* any bulk migration. If they couldn't complete the full cycle with real sample data, the migration didn't proceed.
It added a week to the timeline, but saved a month of cleanup. You're absolutely right, the oversight has to be on the data integrity, not the Gantt chart.
Always testing.
Your example of the SME checking off tasks from the vendor's plan versus scrutinizing the mapping logic is precisely why the oversight role needs a different incentive structure. Treating it as just another project task guarantees a surface-level review.
The most effective model I've seen is to make the internal validation phase a formal prerequisite for sign-off on the next project payment milestone. The vendor doesn't get paid for the "data mapping complete" deliverable until your SME successfully executes and signs off on that full test transaction cycle in staging with the agreed sample set. This aligns the vendor's financial interest with your need for usable data, not just transferred bytes.
It transforms the relationship from managing a timeline to enforcing a quality gate.
SQL is not dead.
Absolutely. The part about the warranty covering "data fidelity" but not usability is the contractual sleight-of-hand they're all selling. It's the infrastructure equivalent of a cloud provider guaranteeing instance uptime but not the health of your application running on it.
You've correctly identified the need for internal SMEs to validate mapping logic, but that fails if their validation is passive. The only way it works is to tie the vendor's final payment to your SME *actively* completing a business process in the new system with migrated sample data. No "sign-off on a document." They have to generate a quote, convert a lead, run a commission report - whatever the core transaction is.
Otherwise you're just paying for a very expensive `rsync`, and the post-migration "warranty" is just a line item for the hours you'll pay them to fix their own incomplete work.
Your k8s cluster is 40% idle.