Hot take: Their SOC 2 report is fine, but their pen test scope is too narrow for my use case.

That's a really good catch. A SOC 2 report shows their controls are *in place*, but a narrow pen test doesn't prove those controls are *effective* for the parts you're actually using.

We saw something similar with a performance review tool. Their SOC 2 was fine, but their API, which we needed for data exports, had never been tested. We ended up asking them to include the API in their next testing cycle and share the summary findings with us before we signed. It was a good middle ground.

Have you considered asking them about their roadmap for including API security in future assessments? Sometimes it's just an oversight in their testing cadence.

Good catch on reviewing both documents. A SOC 2 report covers the control framework, but a pen test validates specific implementations. If your primary interaction is through their API and webhooks, that's your actual risk surface, not the core app interface.

We encountered this with a BI platform vendor. Their pen test covered login and dashboard access, but not their Data API or the webhooks used for alerting. We treated it as a material finding and added a clause in the contract requiring any future pen tests to include those components. The vendor agreed, as their next assessment was already scheduled.

Have you asked whether their API gateway or webhook endpoints are covered under a different assessment, like an internal security review or a dedicated DAST scan? Sometimes these are tested separately but not bundled into the main "pen test" report.

You're spot on about treating it as a material finding. The contract clause is the right move, as it shifts the obligation from a verbal promise to a formal requirement.

I'd add that you need to specify what "include those components" means. We once had a vendor agree, but their next pen test only performed a simple credentialed scan of the API endpoints, not the business logic or data flows our integrations used. The scope definition in the appendix matters as much as the clause.

Your point about asking if they're covered elsewhere is key. Often, the API is under a separate DevOps team's purview and gets a lightweight internal scan that doesn't meet third-party assurance standards.

Agreeing to include the API in their next cycle is a pragmatic step. The challenge is that a "next cycle" could be 12-18 months out. Your risk exposure during that interim period isn't covered by the existing reports.

We negotiate for a right to review the updated pen test report, with specific scope, before the contract auto-renews. That creates a tangible deadline for them and an exit ramp for you if the findings are unsatisfactory.

Yep, seen this exact scenario with helpdesk integrations. Their SOC 2 says they have a change management control. A narrow pen test doesn't prove that control actually caught a vulnerability in their webhook logic.

You have to weigh the risk of the unknown surface. For a core feature we'd rely on daily, a narrow test is a dealbreaker, SOC 2 or not. For a minor feature, maybe it's an acceptable risk with a contractual fix for the next cycle.

Have you asked them directly about the exclusion? Sometimes it's a billing issue with their pen test firm, and they can add it quickly if a potential client asks.

Exactly. That "billing issue with their pen test firm" scenario is more common than you'd think. The scope is often defined by a fixed-price SOW, and expanding it means a change order. I've seen vendors who are technically willing to test the API surface, but their current contract with the testing firm only covers the main user-facing app.

The follow-up question isn't just if they can add it, but *how* they would add it. Will they procure a one-off test for the API/webhooks prior to your procurement? Or will they simply promise to include it in the next annual cycle? The former shows a commitment to closing the gap for you; the latter is just kicking the risk can down the road.

This is where asking for their Statement of Work from the last pen test can be revealing. The exclusions section often explicitly lists "API endpoints not consumed by the primary web UI" or similar. If that's the formal, documented scope, then their security controls around the API truly are unvalidated by a third party.

That's a really smart point about asking for the SOW. I wouldn't have thought to ask for that directly. Seeing that formal exclusion in writing would definitely change the conversation for me.

So if they did share it and it says the API is out of scope, is the next step just to ask for that one-off test? Or does that mean we should look at their internal security reviews for that component instead?

Thanks for pointing this out.

This is a classic oversight. Their SOC 2 is about their controls, but your threat model is about your specific data flows through their API and webhooks. They are not the same thing.

Ask them directly for their last pen test SOW to see if the API was formally excluded. If it was, they need to explain their compensating controls for that component. Their internal secure code review is not equivalent to a third-party test.

Treat the untested surface as a risk you have to accept, mitigate, or price into the contract.

Your identification of the gap is precisely on point. A SOC 2 report covers the existence of procedural controls, but it is not a substitute for vulnerability validation of your specific data ingress/egress points.

The critical next step is to analyze this as a FinOps and procurement issue, not just a security finding. You must quantify the potential cost of the risk. Calculate the data transaction volume you plan to push through their untested API and webhooks, then model the financial impact of a hypothetical breach or service degradation originating there. This gives you concrete leverage.

In negotiations, move past asking about their roadmap and instead request their last penetration testing Statement of Work and a detailed asset inventory. This will show if the API is formally excluded. If it is, you have two clear options: demand a one-off test as a condition of signing, or bake a significant discount into the contract to offset your assumed risk. A promise for "the next cycle" is financially irresponsible.

That's a smart contractual lever to pull. I've found that right to review needs a very clear trigger date, though. Something like "report to be provided 60 days prior to auto-renewal" versus a vague "before renewal." Without it, they might deliver the report 5 days before, leaving no real time for assessment or negotiation.

Also, consider what happens if the report is late. Does the auto-renewal pause? Is there a financial penalty? The teeth in that clause matter as much as the clause itself.

Your gap is real. SOC 2 tests controls, not your specific data flow.

I've required vendors to share their last pen test SOW. If the API/webhook is formally excluded, you need to see their internal test results for it. Internal scans often miss logic flaws that an external test would catch.

The risk depends on your data volume. Model the impact of a breach via that untested channel. That number is your leverage.

Internal scans rarely carry the adversarial perspective needed for a proper risk assessment. They verify known patterns, not novel attack vectors.

If their only compensating control for the excluded API is an internal review, you should ask for the methodology. Was it a SAST scan, a manual peer review, or a threat model? The difference matters. A manual review might catch logic flaws, a SAST scan likely won't.

Requiring them to disclose that methodology forces them to justify the adequacy of their internal control, which is often the weakest link in the argument.

Quantifying the financial risk is the correct escalation path. However, translating that model into a contractual discount is often met with heavy resistance. They'll counter that their liability is already capped.

A more effective approach is to link the risk directly to a specific contractual obligation. Instead of a discount, negotiate a *contingent* right to terminate without penalty if the one-off test, to be performed within 90 days of signing, reveals critical findings they fail to remediate within an agreed SLA. This turns an abstract financial risk into a concrete, time-bound security deliverable. The procurement team understands termination clauses far better than nebulous risk offsets.

You're right that internal review isn't equivalent, but calling it "not equivalent" is letting them off too easy. It's an apples to orangutans comparison. A third party test looks for ways to break it. An internal review looks for ways it meets their own checklist.

The real question is why a critical external interface like an API is only getting internal review in the first place. That's a design flaw in their security program, not just a scope gap.

If they offer internal reviews as a compensating control, ask for the CVEs discovered in the last year from that process. The answer will be telling.