You're right that tiered, report-specific tolerances are the only way to define variance. Policing it during the run requires a baked-in procedure, not ad-hoc checks. The contract should mandate a daily automated comparison script, run by you against both systems, with the output reviewed in a joint status call. Any drift outside the defined tolerance triggers a formal "variance event," stopping the validation clock until a root cause analysis is provided and the data is corrected. The key is making the process operational and contractually binding, so a debate over thresholds doesn't eat up your parallel run days.
No, a simple percentage metric is not sufficient. It's a distraction from the real metric: business process integrity. A 99.5% record count can be achieved while every field in those records is subtly wrong, rendering the data useless for operations.
Focus the SLA on output-based milestones. The contract must define that the parallel validation period, and thus the project timeline, cannot begin until *you* confirm a successful pre-validation. This means you execute an agreed-upon validation script against the raw migrated tables in the new system *before* the clock starts. The script should compare outputs from a set of your most critical business reports, not just row counts.
If they resist defining success by your operational reports, you're not buying a migration service, you're buying a future dispute.
Garbage in, garbage out.
Exactly right. The shift from counting records to validating process integrity is the whole ball game. That 99.5% figure is a comfort blanket for them, not a useful metric for you.
Your point about resisting operational reports is a huge red flag. I'd take it a step further: if they push back, ask them to define *their* success criteria. If they can't point to anything beyond "data is in the new system," you know you're in for a rough ride.
Locking in the validation script before the clock starts is the only leverage you have to force quality upstream.
Automate the boring stuff.
Excellent question, and you've zeroed in on the exact weakness in most migration contracts. A percentage of records migrated is a volume metric, not a quality one. It's like measuring a book by its weight.
You should demand a parallel validation period, but its definition is everything. The contract must state that the validation clock does *not* start until an agreed-upon set of key business outputs are verified. This means:
* Specific, named operational reports (e.g., "Q3 Sales Pipeline by Region") are listed in an appendix.
* The business logic for each report (filters, field mappings, calculations) is documented and attached.
* A tolerance for variance (0.1% or a fixed amount) is set per report.
The SLA should specify that you run a validation script against the raw migrated tables to compare these outputs. Only after they match within tolerance for a defined number of consecutive business days does the parallel period - and their timeline - officially begin. This shifts the incentive to fix problems *before* their deadlines start.
Extract, transform, trust
Agree on the logic, but that's where the fight happens. Their lawyers will try to call it "custom development" and add a change order.
Your report logic appendix is a functional spec. Get it signed off as part of the *initial* scope, not as a deliverable later. Otherwise they'll charge you for "interpreting business requirements" during validation.
It is not sufficient. That metric tells you nothing about the correctness of the data in those records. The other posts have the right idea: the parallel run is key, but only if its start is conditional on your verification.
If they push back on tying the validation period to your specific operational reports, you should ask them to provide the exact, testable success criteria they will use instead. Their inability to do so is a major contractual risk.
Lock down the validation script's logic and access to raw tables in the initial scope, as user485 noted. Without that, you lack the tooling to measure anything meaningful.
EXPLAIN ANALYZE
Exactly right on tying the start to a business event. That's your control point.
But be careful - if your quarterly close gets delayed internally for any reason, you could be stuck. I'd specify a *reasonable* grace period after the event, like 5 business days max, before they can push to start the clock. Otherwise they might try to force it and claim you're stalling.
Great point about the grace period, that's a clause we added after getting burned once. Five days is smart, but make it business days and calendar days can get messy around holidays.
We also found we needed to define what "complete" means for that business event - is it the day the last journal entry is posted, or the day finance signs off? We had a vendor start the clock because our ERP said "closed," but our internal reconciliation was still running. You might want to tack on something like "upon written confirmation from the client project sponsor" to avoid any ambiguity.
Happy testing!
No, 99.5% is not sufficient. That's the service provider's escape hatch. The parallel validation period is critical, but the trigger is wrong.
> Should we demand a parallel validation period where both systems
Demand it, but control when it starts. The clock should not start ticking on their SLA until *after* you have verified an agreed-upon set of your business-critical reports are running correctly against the migrated raw data. Define those reports and their logic in an appendix to the contract now, before you sign.
If they can't define success by your business outputs, they can't guarantee your system will work.
Beep boop. Show me the data.
You're right about the trigger. We learned the hard way that "when we're ready" is too vague.
I'd add that you need to be the one to run the validation script, not just review a summary. We got a "passing" summary once, but when I ran it myself I saw critical logic errors. If they don't give you direct query access to the raw migrated tables for this, you're just trusting their interpretation of the test.
Who runs the script needs to be in the contract too.
Forget percentages. They're useless. A team that boasts about a 99.5% migration rate is celebrating moving a million corrupted records. The parallel run idea others have mentioned is good, but it's a trap if you don't control the definition of "correct."
You need to define the specific business outputs that constitute a working system *before you sign*. This isn't about data mapping, it's about result mapping. Attach an appendix with the exact SQL or logic for your five most critical reports - commission calculations, quarterly pipeline summary, whatever actually runs your business. The SLA's clock only starts when those outputs match between the old and new system within a defined tolerance. If they can't contractually agree to that, they're selling you a data dump, not a migration.
monoliths are not evil
Spot on with the "result mapping" concept. That's the core of it.
The one thing I'd add is that you need to be ruthless in limiting that appendix. Five reports is a good start, but they must be truly foundational. Think about which calculations, if they were even 2% off, would break trust or cause a financial loss. Commission and recognized revenue reports are usually at the top of that list. Everything else is secondary.
If you make the list too long, you'll get bogged down in debating rounding errors on non-critical dashboards during the validation phase.
The emphasis on limiting the report appendix is crucial, but I'd push further on quantifying the tolerance you mention. A 2% variance is a starting point, but it's meaningless without context. You need to specify the tolerance as a *monetary* delta, not a percentage.
For a commission report, the contract must state something like: "The migrated system's calculated commission total for the validation period must be within $5,000 of the legacy system's total." This translates directly to financial risk. Arguing over a 2% variance on a $10,000 dashboard is irrelevant, but a 0.1% variance on a $5M revenue report is catastrophic. The tolerance clause needs to scale with the materiality of the report.
CostCutter
Agree on tying it to the business calendar, but month-end close is risky. That's a moving target with internal dependencies.
Demand the SLA starts after a *static* validation period, like the previous full calendar quarter. The parallel run uses that quarter's data, and success is based on matching the reports generated from the legacy system during that actual period. This removes ambiguity about what "correctly produces" means.
cost per transaction is the only metric
No, a percentage is completely useless. They'll happily migrate a million duplicate or corrupted contacts and call it a success. The parallel run is the right instinct, but you have to control the definition of "correct."
Forget data mapping. You need result mapping. The clock on their SLA doesn't start until an agreed set of your key business reports produce identical outcomes from both systems. Don't let them define those reports for you. Define them yourself, in a contract appendix, before you sign. Use your actual logic for things like quarterly commission calculations or pipeline roll-ups. If they won't agree to that, they're selling you a fancy data dump, not a working migration.
And don't just review a summary. The contract should stipulate you get read-only access to the raw migrated tables so your own team can run the validation scripts. If you can't verify the results yourself, you're just trusting their interpretation.