After evaluating four vendors for a managed data analytics platform, I realized the security sections of their RFP responses were functionally useless. They were dense, filled with certifications listed by acronym (SOC 2 Type II, ISO 27001, etc.), and relied on the phrase "industry best practices" as a blanket answer. To cut through this, I added a simple, non-negotiable step to our process: each shortlisted vendor had to schedule a one-hour call where they were to explain their security model in plain English, without slides.
The results were revealing. Two vendors sent their sales engineers, who faltered and reverted to jargon. One sent a pre-sales architect who performed admirably. The fourth sent their actual CISO, which was the most instructive meeting of all. The exercise was less about the specific answers and more about observing *how* they answered. Here is my rubric for what constituted a passing explanation, which I recommend incorporating into any technical evaluation.
A satisfactory plain-English explanation had to cover these points without using undefined acronyms:
* **Data Segregation:** "How do you ensure my customer's data is kept separate from your other clients' data at rest and in transit? Is it a logical separation via software controls, or a physical separation per tenant?"
* **Access Path:** "Walk me through the steps, from a public internet request to accessing a piece of my encrypted data at rest, that one of your support engineers would take. Where are credentials stored, and how is that access logged and audited?"
* **Incident Response:** "Describe, step-by-step, what happens on your end if your monitoring detects an anomalous data export from my instance. Who is notified on your side and on my side, and within what timeframe?"
* **Key Management:** "Who controls the encryption keys for my data at rest? If it's you, what is the process for key rotation and who has access to the master keys? If it's me (customer-managed keys), explain the integration mechanism and how it affects your support capabilities."
The vendor that sent their CISO excelled by using concrete analogies. Instead of "zero-trust network access," they said, "We treat every access request as if it's coming from an untrusted coffee shop Wi-Fi, regardless of where it originates inside our network, and we verify each step." They also proactively disclosed their "blast radius" limits and the mean time for internal versus customer-facing notifications during a security event.
The takeaway is that the ability to explain a complex model simply often reflects a deeper, more operational understanding of that model. Vendors whose teams could only parrot marketing terms or certification names were deprioritized, as this indicated a potential disconnect between their security marketing and engineering reality. I now include this mandatory "plain English" call in all our technical RFPs.
-- Liam
Always check the data transfer costs.