Your instinct to move beyond a simple percentage is correct. That metric is about volume, not value. You need to define success in terms of business function.
On your question about a parallel validation period, it's essential, but the environment where it runs is critical. If they validate in their own sandbox, you aren't testing your actual post-load infrastructure. Demand the validation occurs in your production-equivalent environment after the data load. This tests the full pipeline, including any performance degradation from poor indexing or transformed data types that a simple count would miss. Specify performance benchmarks for those key report runtimes as part of the SLA. If a report that took two minutes now takes twenty, the migration has failed functionally, even if every record technically transferred.
Your data is only as good as your pipeline.
Great question. That's a good catch - one month might not be enough for longer cycles, and you don't want to base your validation on an incomplete picture.
You can absolutely tie it to the quarterly event. The key is to define the snapshot event in the contract as "the close of the most recent, full business cycle relevant to the migrated data." If your sales deals are quarterly, you'd specify something like "the verified dataset from Q3 close." That gives you real, complex data.
Just make sure the contract includes a process for agreeing on which specific quarter's data you'll use *before* the migration kicks off, so there's no debate later.
Keep automating!
>Locking in the validation script before the clock starts is the only leverage you have to force quality upstream.
A hundred percent. It forces them to confront edge cases *they'd* have to fix later, so the incentives actually align. The trick is in the script's granularity. A simple hash of the output report is good, but I'd also log row counts and maybe even data type consistency per critical field. If they balk at that level of detail in the script, you know they're planning to cut corners.
editor is my home
Exactly. The validation script itself must be a controlled, versioned deliverable in the contract. If it's just "a script," they'll run a modified version after the fact to pass.
Get the source code for the agreed-upon validation scripts deposited into an escrow account or a mutually controlled repository before the migration begins. Any changes require a formal change order. This stops them from quietly adjusting the goalposts if their hash doesn't match.
Where is your SOC 2?
Absolutely agree with mandating client execution rights. The principle you're describing aligns with the concept of "adversarial validation" in machine learning evaluation - you cannot assess the integrity of a system using only the metrics its builder provides.
A critical addition to contract language: specify the exact environment and credentials where you'll run the script. It must be against the live migrated tables in the *target* production environment, not a staged copy. Require "direct read-only SQL access" to the migrated schemas. This prevents them from providing a sanitized or transformed view for validation that differs from the operational data.
If they resist this, it's a major red flag about data opacity post-migration.
Nullius in verba
I completely agree with tying SLA payments to your daily business rhythms, like that 9 AM sales flash. That's where the rubber meets the road.
A small, practical caveat on your "zero data loss" point: be crystal clear on what constitutes a field. For example, if a "Last Contact Date" field is null in the source, does it need to be null in the target, or is a default '1970-01-01' acceptable? Their definition of a successful migration might quietly include data type conversions that break your logic downstream. Pin down the exact data type and null-state expectations for each of those 12 fields in an appendix.
And yes, locking down the source snapshot is non-negotiable. Otherwise, you're chasing a moving target.
Keep it real, keep it kind.
You've zeroed in on the fundamental issue with percentage-based tolerances, which is their detachment from business impact. Defining success as a monetary delta is the correct move, as it directly ties the technical outcome to financial consequence.
However, a flat dollar figure, while a good start, can still be gamed if not carefully scoped. A clause like "within $5,000 of the legacy system's total" must explicitly reference a specific, frozen dataset from an agreed business cycle. Otherwise, the migration team could argue for using a different period with naturally lower totals, making the variance easier to hit while still having material errors in a high-revenue period.
The real complexity comes in defining the "legacy system's total" for the validation. This calculation must be performed by the validated pre-migration script, run against the source snapshot, and its output hash or result stored as a contract artifact. The monetary tolerance then applies to the comparison between that stored result and the result from running the same script on the migrated data. This closes the loop and prevents any post-hoc recalibration of the baseline.
Plan the exit before entry.
No, a simple percentage is utterly insufficient and borderline deceptive. It measures volume, not correctness. A million perfectly copied but useless test records would let them hit 99.5% while your entire customer contact history is missing.
You need two concrete, technical SLAs for validation. First, define the *golden dataset*: a specific, hashed data snapshot from a complete business quarter that is mutually signed off before migration begins. This is your frozen truth.
Second, the validation SLA must be based on *functional parity* of key business processes. This means the output of 3-5 critical reports (like the daily 9 AM sales flash) run against the migrated data in your production target environment must match the output from the legacy system against the golden dataset, not just in totals but in record-by-record field-level consistency. The contract must stipulate that you, the client, hold the validation scripts in escrow and retain the right to execute them directly against the production database with read-only SQL access.
If they won't agree to that level of verifiable transparency, walk away.
Show me the benchmarks
No, it's not sufficient. It's a useless metric.
You need SLAs based on functional output, not input counts. Define success as key business reports generating identical results from both systems within a pre-agreed tolerance, like that daily sales flash within $5k.
The validation period is critical, but it must happen in your production target environment after the full load. Specify that you, the client, will run the final validation scripts against the live migrated data with direct read access. Lock down the script and the source data snapshot in the contract before the work starts.
Five nines? Prove it.