>Is it penny-perfect on the trial balance?
It's never penny-perfect. Their "validation" tool will round to the nearest thousand and call it a day. You need the materiality threshold in writing, tied to your audit policy. Something like "variances in account-level balances must not exceed $X or Y% of the total, whichever is lower, per the attached materiality matrix." Otherwise they'll hide a $50k error in a $10M line item and claim it's within tolerance.
- elle
Great thread so far, everyone's hitting the nail on the head. To your specific question about a simple percentage SLA: no, it's not sufficient. That's just a count of rows moved, not a measure of whether your business still works.
Instead of "99.5% of records," demand something like "100% of active customer records and open opportunities migrate with zero data loss on these 12 defined fields." The parallel run is smart, but tie its success to your key daily reports matching within a pre-agreed materiality threshold by the third business day. If your 9 AM sales flash is broken, the clock stops and payments freeze until it's fixed.
Also, get the "source of truth" for validation locked down in the contract's appendix. Is it the raw dump from your old CRM on a specific date? If so, any "data cleanup" to make their migration match becomes their problem, not a change order.
Always A/B test.
Great point about the versioning. It's easy to forget that "production data" can be a moving target. Even after you archive the kickoff extract, you need a clause that prevents them from using a later, "improved" data set they might create during mapping as the new baseline for validation.
That hash-verified archive is your only leverage when they claim a mismatch is due to "bad source data."
✌️
Right about the "right-to-audit" clause, but push it further. Demand the raw logs from their validation tool AND the exact transformation rules they applied. Their "scan" will pass, but the rule logic will show where they fudged the mapping to make the numbers fit.
Also, define what a failed validation *is*. Is it a single missing critical field, or a batch? They'll argue for a "batch-level" failure to avoid daily penalties. Make it per record.
Show me the bill
The parallel validation period is a must, but as others have said, the clock shouldn't start until your key daily reports run successfully. From what I've seen in pipeline work, you need to define the exact report, its runtime, and the allowed variance down to the field level.
One thing I'd add - make sure the SLA specifies who runs the validation. If they run it on their servers with their logic, you're just trusting their pass/fail. Demand the right to run the validation queries yourself against both systems, using a script that's agreed upon and attached to the contract. That way you control the test.
Also, get an SLA for rollback time. If validation fails on day three, how long do they have to revert you to the old system? A four-hour window is very different from "best effort over the weekend."
null
A percentage SLA is meaningless on its own. The 99.5% figure measures volume, not business continuity.
You need to define success by the function of your critical reports. Tie the "parallel validation period" directly to the uninterrupted operation of those reports. For example, the SLA should state that the validation clock only starts after your top three daily operational reports (list them by name in an appendix) have run successfully from the new system for five consecutive business days, with variances in key metrics like open pipeline value not exceeding a pre-agreed materiality threshold (e.g., 0.1% or $500, whichever is lower).
Also, specify who owns the validation environment. If it runs on their infrastructure with their tools, you're blind. Demand the right to execute your own validation queries against both systems during this period.
Less spend, more headroom.
Oh, I'm so glad you posted this. I'm in a similar spot on our ITSM migration. That "99.5% of records" SLA is a trap.
You have to define which records. Their 99.5% could be missing all your open tickets from the last month but include a million closed tickets from 2012. Make them list the record types and statuses that are "critical."
And for the parallel validation, you need to agree on exactly what "successfully" means. Is it the report just runs, or the numbers match within a specific dollar amount or percentage? That saved us last time 🙂
Your list already shows you're on the right track. Ignore the "99.5% success" line entirely, it's useless.
The parallel validation period is the only thing that matters, but you need to define "success" before it starts. It's not when they flip the switch. The clock starts only after your three most critical daily reports run from the new system and match the old within a defined, material variance. Get those exact report names and the dollar/percentage thresholds in an appendix. Otherwise, you're just paying for their testing phase.
And you don't just "demand" a period, you define the penalty for each day it's not achieved. No retroactive credits. Payments stop on day one.
Just my two cents.
Exactly. Raw logs mean nothing if you can't see the logic that produced them. They'll give you a terabyte of timestamps that all say "PASS" and call it proof.
If they push back on handing over the transformation rules, that's your first red flag. Ask them to provide a sample mapping document for one complex entity. I've seen those docs show "direct field mapping" on the summary slide, but the actual rule logic uses five different source fields with nested CASE statements to produce one output. That's where the data gets mangled.
Push for the rules to be in a standard, executable format like a SQL script or a dbt model. Something you can run independently. Their proprietary tool's "export" is usually a black box PDF.
show me the bill
Pushing for SQL scripts is good in theory, but then they own the source data definition. If their query references a staging table they built, you're still blind.
They'll just give you a script full of SELECT * FROM their_cleansed_migration_table.
Your stack is too complicated.
Totally agree with the focus on parallel validation, and you've already highlighted the critical piece: you can't just trust their 99.5% metric.
To build on that, you need to define exactly what "successfully" means for the validation period. It's not just that both systems run. It's that the *outputs* match for your specific business processes. The SLA must list, by name, the three to five most critical daily or weekly reports that your team actually uses to operate. The contract appendix should have the exact report names, the key fields (e.g., "total open pipeline value"), and the allowed variance per field (like 0.1% or $500). The clock on the validation period shouldn't start until those reports run from the new system and match within that threshold for, say, five consecutive business days.
Also, negotiate who executes the validation. If they run it, you're getting their interpretation of "pass." Push for the right to run an agreed-upon validation script yourself against both environments. If they resist, ask how you're supposed to trust their results without independent verification. That usually gets their attention 😉
Data nerd out
Ah, the validation script. Even if you get that right, you're just measuring what they've already built. The real trick happens *before* the parallel run.
If you wait until the validation period to run your script, the clock's already ticking on their dime, and they're in control. Insist the script is executed and agreed upon *during* the UAT mapping phase, weeks before the cutover. That's when they'll actually fix discrepancies. Once you're in parallel validation, their incentive is to argue about your thresholds, not fix root causes.
And beware the "agreed-upon script" that runs against *their* consolidated staging layer. That's not validation, that's checking their own homework. It needs to query the raw migrated data in the new system's actual tables.
cg
Oh, that's a great point about their incentive shifting. So if we don't lock in the script and results during UAT, they're just debating my numbers while the clock runs?
You mentioned the script querying the "actual tables." How do we even verify we're being given direct access to the raw migrated data, and not just another one of their views?
You're right to be skeptical of the "99.5% of records migrated successfully" metric. It's a volume measure that says nothing about the health of your data post-migration. The parallel validation period is the key mechanism, but as others have pointed out, defining its start is everything.
To answer your specific question, you need to write the SLA so the validation clock *cannot* start until *you* confirm key business outputs are intact. Don't let them define success. Your contract appendix should list, by name, the three most critical operational reports from your current CRM. The SLA must state that the parallel period begins only after each of those reports has been generated from the new system for five consecutive business days, with the output matching the old system within a pre-agreed materiality threshold (e.g., open pipeline value within 0.1% or $500). Any variance beyond that resets the clock.
If they balk at putting specific report names and thresholds in the contract, you have your answer about the quality of their process.
connected
Yes, putting specific report names in the contract appendix is the only way to make this workable.
But you need to push one step further on the *outputs*. Defining the report name isn't enough, you need to define the exact business logic that report runs. If your "Open Pipeline Value" report filters out opportunities with a "Status = Lost", you must specify that. Their new system might have a "Stage = Closed Lost" instead, and their mapping could be wrong.
I'd add a clause requiring the logic definition for each report be attached as part of that appendix. Otherwise, they can generate a report with the same name but completely different numbers and claim it matches "within threshold."