I've been deep in the evaluation phase for a new event-driven microservice project, and I keep circling back to AWS Lambda as the obvious candidate on paper. The pricing model aligns perfectly with our spiky traffic patterns, and the operational overhead reduction is a huge selling point for our small team.
However, every single proof-of-concept I run brings me back to the same wall: cold start latency. I've read all the older threads and AWS's own documentation on mitigations (provisioned concurrency, warming tricks, etc.), but I'm struggling to see how this isn't a fundamental architectural constraint for any user-facing service with even modest responsiveness requirements.
My testing shows cold starts for a Python function with a few dependencies still routinely hitting 1.5-2 seconds for the first invocation in a period of inactivity. This is for a simple API handler. I have to assume Java or .NET runtimes are even more pronounced.
Before I commit to a design or start negotiating with AWS, I need a reality check from those running this in production. My specific questions are:
* For those of you using Lambda for customer-facing HTTP APIs (via API Gateway), what is your actual acceptable p95 or p99 latency threshold, and how do you consistently meet it? Are you just eating the occasional 2-second response?
* Is the answer simply to throw money at provisioned concurrency for all your critical functions? When I model the TCO with that included, the cost savings versus a constantly running container service (like ECS or App Runner) start to evaporate, especially for a baseline load.
* How do you handle this in your vendor evaluation? Do you benchmark against alternatives like Google Cloud Run or Azure Container Instances specifically on this cold start criterion? I'd be very interested in any comparative latency numbers from real implementations.
* What does your implementation timeline look like when factoring in the need to architect around cold starts? Does it add significant development complexity with custom tooling for warming or concurrency management?
The promise of serverless is so compelling, but the cold start issue feels like a hidden tax—on both performance and architectural simplicity. I'm concerned about building a system where the exit strategy away from Lambda (if the latency becomes untenable) would be a significant re-architecture. Am I over-indexing on this problem, or is it still the core trade-off everyone is quietly managing?