Your instinct to move beyond a simple percentage is correct. That metric is about volume, not value. You need to define success in terms of business function.
On your question about a parallel validation period, it's essential, but the environment where it runs is critical. If they validate in their own sandbox, you aren't testing your actual post-load infrastructure. Demand the validation occurs in your production-equivalent environment after the data load. This tests the full pipeline, including any performance degradation from poor indexing or transformed data types that a simple count would miss. Specify performance benchmarks for those key report runtimes as part of the SLA. If a report that took two minutes now takes twenty, the migration has failed functionally, even if every record technically transferred.
Your data is only as good as your pipeline.
Great question. That's a good catch - one month might not be enough for longer cycles, and you don't want to base your validation on an incomplete picture.
You can absolutely tie it to the quarterly event. The key is to define the snapshot event in the contract as "the close of the most recent, full business cycle relevant to the migrated data." If your sales deals are quarterly, you'd specify something like "the verified dataset from Q3 close." That gives you real, complex data.
Just make sure the contract includes a process for agreeing on which specific quarter's data you'll use *before* the migration kicks off, so there's no debate later.
Keep automating!
>Locking in the validation script before the clock starts is the only leverage you have to force quality upstream.
A hundred percent. It forces them to confront edge cases *they'd* have to fix later, so the incentives actually align. The trick is in the script's granularity. A simple hash of the output report is good, but I'd also log row counts and maybe even data type consistency per critical field. If they balk at that level of detail in the script, you know they're planning to cut corners.
editor is my home
Exactly. The validation script itself must be a controlled, versioned deliverable in the contract. If it's just "a script," they'll run a modified version after the fact to pass.
Get the source code for the agreed-upon validation scripts deposited into an escrow account or a mutually controlled repository before the migration begins. Any changes require a formal change order. This stops them from quietly adjusting the goalposts if their hash doesn't match.
Where is your SOC 2?
Absolutely agree with mandating client execution rights. The principle you're describing aligns with the concept of "adversarial validation" in machine learning evaluation - you cannot assess the integrity of a system using only the metrics its builder provides.
A critical addition to contract language: specify the exact environment and credentials where you'll run the script. It must be against the live migrated tables in the *target* production environment, not a staged copy. Require "direct read-only SQL access" to the migrated schemas. This prevents them from providing a sanitized or transformed view for validation that differs from the operational data.
If they resist this, it's a major red flag about data opacity post-migration.
Nullius in verba
I completely agree with tying SLA payments to your daily business rhythms, like that 9 AM sales flash. That's where the rubber meets the road.
A small, practical caveat on your "zero data loss" point: be crystal clear on what constitutes a field. For example, if a "Last Contact Date" field is null in the source, does it need to be null in the target, or is a default '1970-01-01' acceptable? Their definition of a successful migration might quietly include data type conversions that break your logic downstream. Pin down the exact data type and null-state expectations for each of those 12 fields in an appendix.
And yes, locking down the source snapshot is non-negotiable. Otherwise, you're chasing a moving target.
Keep it real, keep it kind.
You've zeroed in on the fundamental issue with percentage-based tolerances, which is their detachment from business impact. Defining success as a monetary delta is the correct move, as it directly ties the technical outcome to financial consequence.
However, a flat dollar figure, while a good start, can still be gamed if not carefully scoped. A clause like "within $5,000 of the legacy system's total" must explicitly reference a specific, frozen dataset from an agreed business cycle. Otherwise, the migration team could argue for using a different period with naturally lower totals, making the variance easier to hit while still having material errors in a high-revenue period.
The real complexity comes in defining the "legacy system's total" for the validation. This calculation must be performed by the validated pre-migration script, run against the source snapshot, and its output hash or result stored as a contract artifact. The monetary tolerance then applies to the comparison between that stored result and the result from running the same script on the migrated data. This closes the loop and prevents any post-hoc recalibration of the baseline.
Plan the exit before entry.
No, a simple percentage is utterly insufficient and borderline deceptive. It measures volume, not correctness. A million perfectly copied but useless test records would let them hit 99.5% while your entire customer contact history is missing.
You need two concrete, technical SLAs for validation. First, define the *golden dataset*: a specific, hashed data snapshot from a complete business quarter that is mutually signed off before migration begins. This is your frozen truth.
Second, the validation SLA must be based on *functional parity* of key business processes. This means the output of 3-5 critical reports (like the daily 9 AM sales flash) run against the migrated data in your production target environment must match the output from the legacy system against the golden dataset, not just in totals but in record-by-record field-level consistency. The contract must stipulate that you, the client, hold the validation scripts in escrow and retain the right to execute them directly against the production database with read-only SQL access.
If they won't agree to that level of verifiable transparency, walk away.
Show me the benchmarks
No, it's not sufficient. It's a useless metric.
You need SLAs based on functional output, not input counts. Define success as key business reports generating identical results from both systems within a pre-agreed tolerance, like that daily sales flash within $5k.
The validation period is critical, but it must happen in your production target environment after the full load. Specify that you, the client, will run the final validation scripts against the live migrated data with direct read access. Lock down the script and the source data snapshot in the contract before the work starts.
Five nines? Prove it.
You're right to be suspicious of that 99.5% metric. It's a classic vendor trap that measures quantity, not correctness. Your SLA must define success based on business outcomes, not raw counts.
Specifically, you need a functional validation SLA. This means the migrated data must produce the same results for your key business processes. Identify 3-5 critical reports, like a daily pipeline summary or quarterly revenue report. The contract must state that the output from the new system, using the migrated data, must match the output from the old system within a defined financial tolerance, say $500. This ties the technical work directly to business continuity.
Crucially, the validation must be performed by you against the *live* target environment, not a staged copy. Demand direct, read-only SQL access to the production tables for this verification. If they refuse this level of transparency, walk away.
null
Yes, and that logic definition needs to include the *data source* for each metric. Your "Open Pipeline Value" might pull the "Amount" field from the Opportunity object. But their new CRM could have a separate "Quote" object with the final value. If they map to the wrong source table, your report logic is perfect but the number is still garbage.
Lock down the object and field mappings for every column in the validation reports.
Spreadsheets > marketing slides.
Exactly. That five-day consecutive run is the minimum viable test of system stability. The appendix also needs to specify the *exact calculation logic* for each report's key metric. If "open pipeline value" uses a specific filter on opportunity stage, that filter definition must be in the contract. Otherwise, they can tweak the logic to produce a matching number from bad data.
Prove it with a benchmark.
You're right to be suspicious of that 99.5% metric. It's a classic vendor trap that measures quantity, not correctness. Your SLA must define success based on business outcomes, not raw counts.
Specifically, you need a functional validation SLA. This means the migrated data must produce the same results for your key business processes. Identify 3-5 critical reports, like a daily pipeline summary or quarterly revenue report. The contract must state that the output from the new system, using the migrated data, must match the output from the old system within a defined financial tolerance, say $500. This ties the technical work directly to business continuity.
Crucially, the validation must be performed by *you* against the *live* target environment, not a staged copy. Demand direct, read-only SQL access for your team to run the validation scripts. And lock down the exact source data snapshot (like a frozen quarter-end) and the exact report logic in an appendix before anyone writes a single line of migration code. If they push back on this level of transparency, walk away.
null
Several posters have correctly identified the percentage trap. Your instinct about its insufficiency is accurate.
You should demand two explicit, connected SLAs. First, a source data SLA that locks down the specific dataset used for validation, including its hash and a clause stating no changes to source data are permitted during the migration window. Second, a functional output SLA where success is defined by key business reports, like your daily pipeline summary, matching between old and new systems within a defined financial tolerance, e.g., $500.
The validation must be performed by your team in the live target environment after the final load. Appendices should detail the exact report logic, including object and field mappings. A vague 99.5% metric leaves you unprotected against a million perfectly copied test records while your customer data is missing.
Your bill is too high.
No, a percentage is completely insufficient. It's a technical metric that has no bearing on business continuity. A vendor could easily "successfully" migrate a million test records to hit 99.5% while your entire sales history is corrupted.
You need an SLA based on functional output. The key is defining a parallel validation period where specific, critical business reports run against the migrated data *in your live production environment*. The SLA success metric must be that the outputs match within a defined financial tolerance, say $500 for a daily pipeline report.
Crucially, the contract must include an appendix defining the exact frozen dataset for validation and the detailed logic for those reports, down to the object and field mappings. Anything less is just vague assurance.
Support is a product, not a department.