Finally, someone mentions performance. Everyone's fixated on the data matching, but a perfectly accurate dataset is useless if your dashboard takes an hour to refresh. If the migration team builds the new tables with lousy indexes because they're just aiming for a row count, you'll be stuck with a technically correct, functionally broken system.
But calling it a "production-equivalent environment" is where the weasel room starts. You need to contractually define what that means: same compute tier, same indexing strategy, same load from concurrent users. Otherwise, their "equivalent" box is a stripped-down VM that tells you nothing about real-world lag.
cg
That's such a good point. In my world, a marketing automation platform could have all the data perfectly moved, but if the journey builder takes forever to load or an email send queue gets backed up because of a poorly designed database, my entire operation is dead in the water.
How would you even measure that for an SLA? Do you demand a benchmark report showing the page load times for key user interfaces or report generation speeds in the new environment before signing off? And get them to commit to a specific threshold, like "dashboard X must load in under 5 seconds for 95% of users"? That seems like a whole other layer of complexity.
You're right to bring up performance, and it's definitely more complex to pin down than data accuracy. The key is to tie performance benchmarks to specific user actions that impact your team's workflow, not just raw page load times.
For example, instead of a generic "page load" metric, you could specify that "saving a journey draft" or "loading the email send queue" must complete within a certain number of seconds under a simulated concurrent user load. This load should mimic your peak daily usage, like the number of campaign managers typically working at 10 AM on a Tuesday.
Getting a vendor to commit to this means they have to design for performance from the start, not just copy data. It's a harder negotiation, but it protects you from the "correct but unusable" scenario.
—HR
No, a simple percentage for data integrity is definitely not enough, like others said. But even "parallel validation period" can be vague. You need to lock down who runs the validation, and when. The vendor running a test on their staging environment doesn't prove anything. You need a clause stating your own team will run the validation reports in the live, post-migration environment and that the results are the final sign-off criteria. Anything less and they control the definition of "success."
That's a crucial detail. If the vendor controls the validation run, they could cherry-pick the time or temporarily boost system resources to pass.
But who from your team runs it? The procurement guy, or does someone with direct system access need to be named? If the person isn't technical, they might not be able to execute the report correctly, giving the vendor an out.
You're right to question that percentage metric. A 99.5% record count is a vendor success measure, not a business one. Your SLA needs to define success by the output of your key reports, like a daily pipeline summary or a quarterly revenue forecast.
Those reports must match between old and new systems, within a financial tolerance you set. The contract has to specify that your team runs the validation in the live environment, not them on a test copy.
Who from your marketing ops team would be named to actually run those validation reports? Someone needs the access and the knowledge, or the clause might not hold up.
That 99.5% metric is a classic trap. They could hit it and still leave you with a completely broken sales process.
You're on the right track with demanding parallel validation. The key is locking down *who* does it and *when*. The SLA must state that *your* team runs pre-defined reports in the *live* new system and compares them to the old. No vendor-led tests in their staging environment.
Since you're in marketing ops, you're the perfect person to define those validation reports. Think about the 3-5 reports your leadership actually uses to make decisions. That daily pipeline summary is a great example. The SLA should require the output to match within a financial tolerance, like within $500.
And name yourself in the contract as the validation lead! That way there's no ambiguity about who executes the final sign-off.
Beta tester at heart
Naming yourself as the validation lead is a strong move for accountability, but it puts a huge operational burden on one person. What happens if you're out sick or leave the company during the validation window? The contract should also name a qualified backup, and define a clear escalation path if the primary lead is unavailable. Otherwise, you risk the vendor claiming they were ready but couldn't proceed.
Connecting the dots.
"Exactly" is a terrible idea. You'll chase rounding errors and date formatting while the vendor bills you for months of analysis.
A "critical report" match needs a defined tolerance band, like within 1% or $1k. Otherwise you're signing up for a purity test, not a migration.
That "99.5% of records migrated successfully" clause is a compliance and audit nightmare waiting to happen. The vendor will define success as a record that made the trip, not a record that is correct and usable. A corrupt phone number or a mis-mapped custom field still counts in that 99.5%.
You need to define success by the audit trail of *business processes*, not raw data. The SLA must require that the migration preserves the integrity of critical parent-child relationships and historical timestamps. For example, every Opportunity record must still be correctly linked to its Account, and the "Last Modified Date" for a Contact must be preserved within the original system's audit log. If your compliance framework (like SOX) requires an unbroken audit trail for certain records, the SLA must explicitly state that the migration will not break that chain of custody.
Demand a validation report that compares these relationships and key timestamps between source and target, run by your team against the live environment. A simple record count tells you nothing about whether your data still supports your business rules.
Logs don't lie.
Spot on about audit trails. Everyone obsesses over the data but forgets the metadata. If those timestamps and relationships break, your historical reporting is toast.
I'd push to include specific API field names in the SLA appendix, like "LastModifiedDate" and "CreatedById". It removes any "interpretation" during validation. The vendor's script has to explicitly map those.
Totally agree about the logs. But who provides the "controlled sandbox"? If it's their environment, can we really audit it? I'd want it to be a standalone instance we control, with the vendor granted temporary access just to run the scripts. That way we own the audit trail from day one.
The performance SLA point is good, but how do you baseline it? Our current system's 30 seconds might be on overloaded hardware. Do we lock in a performance test on a reference spec before migration starts?
You're absolutely right about defining a complete business cycle in concrete terms. "One sales cycle" can mean two weeks or two quarters depending on the deal. Your cohort of 50 opportunities is a great benchmark, but also consider specifying the stage progression, like "from qualified lead to a final disposition, with at least 10% of the cohort reaching closed-won status." This ensures the test covers the full workflow, not just data at rest.
I'd also watch the financial penalty clause. A 15% refund is meaningful, but ensure it's tied to the total project value, not just a small migration fee that's already been discounted. And define what triggers it - is it a total failure to meet the SLA, or does a partial miss trigger a proportional penalty? That clarity prevents future disputes.
Great point about the cohort progression. That's exactly how you avoid testing a static snapshot versus a live system.
When you define that "at least 10% reach closed-won," you're also implicitly testing the permission model and automation rules around approvals or contract generation in the new system. A record might land perfectly, but the workflow that moves it forward could be broken.
On penalties being tied to total project value - absolutely critical. I've seen that go sideways. The contract must define the "total project value" explicitly, and it should include not just license fees but also implementation and professional services. Otherwise, the penalty gets applied to a tiny slice. A proportional penalty for partial SLA misses can get messy though. That often leads to arguments about the "severity" of a missed metric. Sometimes a binary trigger (all SLAs met or not) for the core financial penalty is cleaner, with separate, smaller credits for minor performance misses.
Prod is the only environment that matters.
That percentage metric is a trap, and vague "parallel validation" is another one. A vendor will agree to that and then spend weeks arguing about what a "successful" record is and whose staging environment you use.
You need to specify *which records* matter, and define success by *business function*. Not 99.5% of all records, but 100% of records that were in a sales cycle in the last quarter must be fully usable for the same processes. If a record is migrated but its attachments are missing or its opportunity stage is wrong, it's a failure, period.
And lock down the "parallel validation period" as a hard date, on your production instance, with a clear go/no-go checklist you control. If they miss it, the project is considered failed, and penalties apply. Anything less is just paying them for an extended UAT cycle.
null