I'm currently designing a centralized secrets management strategy for a multi-tenant SaaS platform hosted on Azure. We're evaluating HashiCorp Vault's Azure authentication method as a potential cornerstone for ephemeral, dynamic workloads (think: auto-scaling application pods, batch processing jobs, and Terraform runners).
The appeal is clear: leveraging managed identities to avoid static credentials aligns perfectly with our security posture. My initial PoC with a simple `vault write auth/azure/login role="my-role" jwt="..."` worked flawlessly.
However, I'm looking for real-world data on its behavior at scale. Specifically:
* **Latency during rapid scale-up events:** When 50+ pods simultaneously request a secret at startup, does the auth method introduce a bottleneck? Have you observed increased login times compared to, say, Kubernetes auth?
* **Azure Metadata Service dependencies:** How resilient is it to transient failures of the Azure Instance Metadata Service (IMDS)? Is there a recommended retry logic in the client side?
* **Managed Identity propagation delays:** In scenarios where a new VM scale set instance is spun up, is there a known delay before its managed identity is fully recognized by Vault's Azure auth backend?
* **Recommended tuning parameters:** For those running this in production, have you adjusted default TTLs, `max_ttl`, or other role parameters to optimize for high-churn environments?
Our ideal workflow would see a new instance authenticate, pull its required secrets, and be ready to serve traffic in under 10 seconds. I'm concerned that adding an external auth call to Vault could jeopardize that SLA.
I'd appreciate any performance benchmarks, failure rate observations, or operational gotchas you've encountered. Hard numbers on P99 login latency would be particularly valuable.
-- J
Data never lies, but it can be misleading
Latency during rapid scale-up is your primary bottleneck. The azure auth method hits the Azure AD token endpoint, which has its own throttling. We've seen 50+ concurrent logins push vault's response time beyond 3 seconds during a burst, which often cascades into client-side timeouts.
For your other points:
* IMDS failures are rare but catastrophic. You need an exponential backoff retry in your client, but also a circuit breaker - if IMDS is down, authentication is simply impossible.
* Managed identity propagation is near-instant on VMSS, but the real delay is often the role assignment to your Key Vault or other resources. That can take 30+ seconds, which fails the first vault login attempt.
You're better off using a sidecar pattern with a token lease for truly ephemeral workloads. The azure auth login on every pod start is asking for trouble.
Your fancy demo doesn't scale.