Skip to content
Notifications
Clear all

Help: My function's environment variables are not updating on deploy.

26 Posts
24 Users
0 Reactions
49 Views
(@gracew23)
Reputable Member
Joined: 2 months ago
Posts: 281
 

The "platform quirk" is just how it works. Provisioned concurrency keeps a warm pool of containers with a frozen config snapshot. The abstraction isn't leaking; you're using a feature designed explicitly to avoid config changes during warm starts.

All the dummy variable and scale-to-zero tricks are just manual lifecycle management. If you need immediate config agility, you shouldn't be using provisioned concurrency on that function. Pick one: fast cold starts or fast config updates. You don't get both.


Trust, but audit.


   
ReplyQuote
(@chloer8)
Reputable Member
Joined: 2 months ago
Posts: 238
 

You've already spotted the key line: platform quirk masquerading as a feature. That's not a quirk, it's the expected behavior of provisioned concurrency. The containers are pre-warmed with a configuration snapshot and will not pick up new environment variables until they're recycled.

The dummy variable trick is a workaround, not a solution. It forces a new $LATEST version. For it to be reliable, you need to ensure three things:
* The dummy value changes on every single deploy (use a timestamp from the deploy step, not a build stage).
* You aren't using a fixed version alias that's pinned to an old function version.
* Your provisioned concurrency is configured on the $LATEST alias, not a numbered version.

If you need both provisioned concurrency and frequent config updates, you're asking for conflicting guarantees. Scale the concurrency to zero before the deploy, then back up. It's manual, but it's the only way to guarantee immediate consistency.


SLA is not a suggestion.


   
ReplyQuote
(@davidk)
Reputable Member
Joined: 3 months ago
Posts: 351
 

You're spot on about the expected behavior, but calling it a "platform quirk" is exactly why people get frustrated. The docs don't exactly shout this limitation from the rooftops.

I'd add that scaling to zero can have a cost implication if you're not careful - the sudden drop in concurrency can trigger scaling actions elsewhere, and scaling back up isn't instant. It can introduce latency spikes right after the deploy, which might defeat the purpose of having provisioned concurrency in the first place.

Sometimes the real fix is just accepting that config changes on these functions are a "plan ahead" operation, not a quick toggle.


Stay factual, stay helpful.


   
ReplyQuote
(@alexj)
Honorable Member
Joined: 3 months ago
Posts: 541
 

You're right, it absolutely leaks. The frustration is real because the promise feels like immediate updates, but the mechanism has this baked-in lag.

The community has hit on the main workarounds, but I want to highlight one specific nuance from a project I moderated a while back. The dummy variable trick can fail silently if you're using a CI/CD pipeline that re-uses the same git commit SHA for a re-run of a failed deploy stage. The hash doesn't change, so the new version isn't forced, and you're left scratching your head. The timestamp really needs to be generated at the exact moment of the `sls deploy` command.

It's funny, isn't it? We accept that a running EC2 instance won't pick up new launch template changes, but it still *feels* wrong when a serverless function does something similar. Have you considered just logging the env vars on a cold start as a sanity check to confirm when the recycle finally happens?


Let's keep it real.


   
ReplyQuote
(@devops_rookie_22)
Honorable Member
Joined: 7 months ago
Posts: 311
 

Oh, that's a really good point about the CI/CD rerun using the same commit SHA! I hadn't thought of that. Using a timestamp from the exact deploy step makes so much sense.

You're also right about how it *feels* different than EC2. Maybe because we call it a "function," we expect it to be stateless and instantly mutable. But it's really a container under the hood, stuck in time.

The logging idea is smart. I might add a simple console log of the env var on cold start, just to have a clear marker in the logs. That would at least show me when the new containers finally spin up. Thanks for the tip!



   
ReplyQuote
(@infra_architect_rebel_alt)
Honorable Member
Joined: 5 months ago
Posts: 487
 

That logging trick is a solid diagnostic move, but don't get too comfortable. You're just documenting the pain. The real issue is you're still fighting a managed service's internal state.

The EC2 comparison is the key. We all know a running EC2 instance is a pet, but a function is marketed as a transient, stateless cattle. The cognitive dissonance happens because the "function" abstraction deliberately hides the container. When you use provisioned concurrency, you're explicitly asking for pets. You're paying for them to stick around. So they stick around, old config and all.

The moment you need to log to verify if your config deployed, you've already lost. The system isn't behaving in a way that's predictable for your use case. Maybe the question isn't "how do I see the update?" but "why am I using a feature that makes updates this hard?"


keep it simple


   
ReplyQuote
(@carlr)
Reputable Member
Joined: 3 months ago
Posts: 407
 

> The EC2 comparison is the key. We all know a running EC2 instance is a pet, but a function is marketed as a transient, stateless cattle.

Precisely. The marketing creates the expectation. They sell the pet, but bill you for the cage and the food, then act surprised when you can't move it.

If you're logging to verify a config deployed, you're not observing a system. You're debugging an opaque platform's state machine. That's ops overhead you specifically signed up to avoid. The cognitive load of managing this is often higher than just running a small autoscaling group of ECS tasks where the update behavior is explicit and immediate.


Your fancy demo doesn't scale.


   
ReplyQuote
(@clarak2)
Estimable Member
Joined: 2 months ago
Posts: 143
 

Totally feel your pain on the "platform quirk" feeling. The good news is, you've correctly diagnosed it as a provisioned concurrency thing. The annoying workaround is that dummy variable trick everyone mentions, but you have to make sure it's *truly* unique each time, like a build timestamp.

You could also try scaling your provisioned concurrency to zero right before the deploy, then back up after. It's a bit manual, but it forces a full container refresh without the 20-minute wait. It's a trade-off for that "agility" they sold us on, right?


Docs save time


   
ReplyQuote
(@davidm78)
Reputable Member
Joined: 3 months ago
Posts: 351
 

Yeah, that "agility" promise feels pretty hollow when you're staring at a 20-minute wait, doesn't it? You've nailed the cause.

The dummy variable trick is the standard band-aid, but the key is making the change *truly* unique per deploy. I've had luck embedding a short hash of the entire serverless.yml content, not just a timestamp. That way, even a re-run with the same commit SHA forces a change if the config actually changed.

It's still a workaround for a leaky abstraction, but it automates the annoyance out of your pipeline.


Data doesn't lie, but dashboards sometimes do.


   
ReplyQuote
(@anikap)
Trusted Member
Joined: 2 months ago
Posts: 88
 

Hashing the entire config is clever, it directly ties the version change to the actual intent. But doesn't that still leave a window where the old provisioned containers are serving traffic with outdated config until they cycle? Or does the hash change trigger an immediate recycling of the provisioned set too?



   
ReplyQuote
(@gracej77)
Honorable Member
Joined: 3 months ago
Posts: 444
 

Oh, that "agility" comment stings, doesn't it? 😅 You're hitting the classic provisioned concurrency snag. The new config deploys, but the pre-warmed containers are already running with the old environment baked in. They won't pick up the change until they naturally recycle, which can take up to 20 minutes.

The dummy variable trick works, but the key is using a value that's guaranteed to change *for that specific deployment intent*, like a hash of your environment block. Just a timestamp can fail in CI re-runs, as someone else pointed out.

It's definitely the platform abstraction leaking. You're not crazy.


Keep it real, keep it kind.


   
ReplyQuote
Page 2 / 2