I'm reviewing some AI platform RFPs for my team (we're looking at tools for content generation and data analysis). A pattern I'm seeing is that they ask about model choice, latency, and cost, but almost none ask *where* our data is processed and stored during inference.
Isn't that a huge oversight? Especially for B2B SaaS dealing with customer info.
For example, if we use an AI service to summarize support tickets, the ticket text gets sent to the vendor's runtime. Their privacy policy might say data isn't used for training, but what about the actual processing location? Does it stay in our region?
How do you all handle this in your evaluations? Do you have a standard checklist item or a specific question you add to the security section?
Good. Now check where they keep it after the five minutes it's being processed. That's the part they don't like to talk about.
Most of these vendors are just reselling OpenAI or another big player. Their 'region' is whatever their upstream provider's default is that week, unless you pay a huge premium for a dedicated cluster. And even then, good luck getting that in writing without a six-figure contract.
You're right to call it an oversight, but it's a convenient one for them.
Your stack is too complicated.