The part about controlling the definition of "correct" hits home. How do you handle it when a foundational report itself gets tweaked or improved by your own team after the contract is signed? If your commission logic changes slightly in Q2, does that void the validation clause? It seems like you could get stuck validating against an outdated standard.
Performance SLA is a good call, but you're going to get pushback on defining "functional regression." Their team will argue that a 5-minute report is still functional, and the threshold is subjective.
You need to tie it to a hard business process. If the sales team's daily stand-up depends on that pipeline report being ready by 9 AM, then the SLA is that the report must be generated by 8:45 AM in the new system, same as the old. Otherwise it's just another vague metric they can debate.
— skeptical but fair
Glad you're digging into this early. Your instinct is correct: a simple percentage like "99.5% of records migrated" is worse than useless. It provides a false sense of security. The parallel validation period you mentioned is the right path, but its success metric can't be their word.
You need to define what "successfully" means in that clause. Borrow from the good advice here and lock down the validation criteria *before* signing. Specifically, name the three to five business-critical reports that, if they match between old and new systems for a static period (like last quarter), constitute acceptance. That's your SLA trigger.
Without that, the vendor defines a passing grade. I'd also add a clause for direct, read-only database access for your team to run the validation scripts yourself. If they balk at that, it's a red flag.
Read-only access isn't just a red flag, it's non-negotiable. But the scripts your team runs must be frozen as a contract exhibit. If you leave room for "updated logic," you've just given them an infinite loophole.
Lock down the exact queries and the exact dataset (like a quarter-end snapshot) they run against. Any change is a change order.
Simplicity is the ultimate sophistication
That's the right instinct to go beyond a simple percentage. I had to handle a simpler app migration recently, and even with a 99.9% success rate on paper, we lost some key user preferences that caused support headaches for weeks.
The parallel validation is good, but make sure the SLA includes a fixed timebox for them to fix any discrepancies found. Without that, the validation period can drag on forever while they "investigate." Something like a 5-business-day remediation window for any critical report mismatch.
What are they proposing for performance SLAs post-migration? Like, page load times compared to the old system? That's often overlooked.
Totally agree on switching from a percentage to a fixed monetary tolerance. It forces everyone to think about real impact.
Just make sure that $5,000 figure is tied to a specific time window, like a monthly commission run, and not an annual total. Otherwise the risk profile is way off.
Also, consider if that tolerance should be tighter for your top salespeople's individual statements. A small overall variance could still mean a huge, morale-killing error for your top performer.
Infrastructure as code is the only way
The percentage metric is a trap. You need a result-based SLA.
Define acceptance by a fixed validation window (last full quarter, as others said) and 3-5 critical reports matching *exactly*. The SLA is tied to achieving that match. "99.5% migrated" is meaningless if your top salesperson's commission statement is wrong.
Also, put a hard deadline in the contract for them to fix any discrepancies found during that validation. Otherwise it drags on forever.
Exactly is a strong word and can be a point of contention. They might argue that a rounding difference in a financial report is not an exact match. You need to define "match" in the contract appendix, perhaps as "to the penny for financial outputs, and to the record for key data lists."
Also, locking the dataset to "last full quarter" is smart, but ensure that snapshot is taken and mutually verified *before* migration work begins. You don't want the source data changing during the validation.
—HR
Spot on about defining "match". I'd push for tolerance thresholds written right into the exhibit - like "within +/- $0.01 for any individual line item and within +/- $0.01 for all report totals". Cuts off the rounding debate before it starts.
And seconding the verified snapshot. Get a signed hash of the data extract from both sides the day it's pulled. That snapshot becomes your single source of truth.
—b
Signed hash is a great idea. Who actually signs it though, and is it legally binding? Our legal team might get hung up on that detail.
What happens if the validation snapshot is *too* clean? Like, if we only use last quarter's data, we might miss how the migration handles real-time updates during the actual cutover.
The percentage metric is a known trap, and your gut feeling about vague assurances is correct. Moving beyond "99.5% of records migrated successfully" is the first critical shift.
You'll want to define success as the functional output of the system, not the raw data transfer. Tie your acceptance to the exact replication of, say, your three most critical quarterly reports from a mutually verified, static snapshot of data. The SLA should specify the tolerance for any mismatch - for instance, financial totals to the penny, record counts exact - and a strict remediation window for any deviation, like five business days.
Also, that parallel validation period needs a hard stop. Without a contractual deadline for them to resolve discrepancies found during that phase, the project can stall indefinitely while you're billed for "investigation."
—Alex
No, a percentage of records migrated is emphatically not sufficient. It's a comfortable, meaningless metric for the vendor that tells you nothing about whether your business actually works in the new system.
You should demand SLAs tied to business outputs, not data ingestion. The validation isn't that 99.5% of records moved, it's that your top five commission reports run against a locked snapshot produce identical results to the penny. Anything less is a breach, with a defined, short window for them to fix it before penalties accrue.
Your leverage to define these outputs evaporates the moment you sign. Nail down the exact reports and the exact, hashed dataset now. If they balk at that level of specificity, your vague feeling about sales rep assurances is about to be validated in the worst possible way.
Show me the data
I completely agree with this, especially the point about leverage evaporating after the contract is signed. That's the crux of it.
One nuance to add: while locking down the exact reports is vital, you also need to define the exact *parameters* used to run them. If your critical commission report has a dozen filters and sort options, you must specify the precise configuration for validation. A mismatch could be blamed on a "different view" rather than bad data. Getting that documented as part of the hashed snapshot procedure closes another potential loophole.
It's this operational specificity that turns a good intention into an enforceable term.
Stay curious.
That point about "their 99.5%" is so true. Seen it where they count every single archived contact as a successful migration, while active opportunity records are a mess.
To add to your validation point, the definition of "successfully runs" has to include the run-time. If your commission report normally takes 2 minutes but takes 2 hours after migration because of bad data indexing, that's a failure that a simple pass/fail check would miss. Gotta specify performance benchmarks for those key outputs too.
Spreadsheets > marketing slides.
Glad you're pushing on that percentage metric. A few folks here already nailed why "99.5% migrated" is a vanity metric.
On your specific question about a parallel validation period, absolutely demand it, but its structure is key. The period should be fixed length, say two weeks, and it starts only after you've both signed off on a hashed data snapshot. The entire SLA for data integrity should hinge on passing that validation - like the 3-5 critical reports running identically from that frozen dataset.
One thing I haven't seen mentioned yet: you need to specify who owns the validation environment. If they're running the reports in their own test instance, that's another variable. Push for the validation to happen in your production-equivalent environment, post-load. That tests the full pipeline.
✌️