Everyone's obsessed with "best practices" from the vendor docs. They'll tell you to allocate 2GB per agent and call it a day. That's a great way to waste money and still have sessions die.
The real answer is it depends entirely on your session length and what you're storing. A 4-hour support session with full chat history, co-browsing artifacts, and uploaded screenshots is a different beast than a 10-minute quick query.
For a truly long-running session (think hours), you need to stop thinking about the agent and start thinking about the backing store. The agent should be stateless. Your memory config is just a cache. The real configuration is your database's connection pool and your object storage TTL.
If you're self-hosting, you're probably using Postgres for history and Redis for live state. Don't let Redis become a memory sink. Set aggressive maxmemory policies and key expirations. The agent itself should have a small heap, just enough to handle the concurrent load of a few sessions. If it's holding more than that, you've architected it wrong.
Your vendor is not your friend.
I'm an analytics lead at a mid-market SaaS company, we self-host our support platform and handle sessions averaging 2-3 hours.
* **Primary Data Store**: Don't size memory for the chat server, size your database. For long sessions, we provision our Postgres instances with 30-40% more RAM than our calculated working set, which for us was around 16GB per replica. The agent's own memory limit is secondary and set to 1GB.
* **Live State Management**: We use Redis for session state and enforce a `maxmemory-policy allkeys-lru` with a 1GB cap. Every session key has a TTL of 8 hours (our max session length plus buffer). This prevents leaks.
* **Artifact Handling**: Co-browsing data and screenshots go straight to S3-compatible storage, never through application memory. The agent just holds signed URLs. This was the biggest cost saver.
* **Failure Mode**: The real breakpoint isn't memory, it's database connections. We had to tune our PgBouncer pool from the default 20 to 100 connections per agent pod to handle the concurrent long sessions without blocking.
My pick is the backing-store-first approach you described. If you're self-hosting, I'd recommend tuning Postgres connection pools and object storage before touching agent memory. For a clean call, tell us your average concurrent sessions and whether you're using a managed DB or running your own.
data over opinions
You had to bump your PgBouncer pool to 100 connections per pod. That's not a tuning success story, that's a symptom of a chatty application layer putting too much load on the connection pool. Sounds expensive.
If your agent pods need that many concurrent database connections just to hold state for a few long sessions, maybe the problem is in your architecture, not your pool size. Have you profiled what those connections are actually doing, or is this just throwing hardware at a leak?
Your stack is too complicated.
Exactly. 100 connections per pod screams "chatty app". I've seen this when the agent polls for state changes instead of using a proper event stream.
Each poll hits the DB. You get a connection spike every time a new message lands in a long session. Connection pooling doesn't fix the pattern.
Profile it. If you see a high query rate for `SELECT * FROM session_state WHERE id = ?`, your config problem is in the app layer, not PgBouncer. Move that state to Redis and keep the DB for persistence.
YAML all the things.
Completely agree about treating the agent as stateless. The "cache" analogy is perfect. I've run into teams that treat the agent's memory as the primary ledger and then get shocked when a pod restart loses "live" context.
One nuance I'd add: even with aggressive Redis TTLs, you can get memory pressure from one nasty session if you're not careful. Like, a single session with a massive, uncapped chat log or a huge base64-encoded file uploaded mid-conversation. Your maxmemory policy might not save you if that one key balloons before LRU kicks in. We had to implement a size check before storing any session variable.
So yeah, the agent's heap is just the spillway. The real dams are further back.
Data nerd out
Yeah, the "stateless agent" point makes so much sense. So when you say the memory config is just a cache, does that mean we should basically set the agent heap as small as it'll go without crashing from normal traffic? Like, ignore the session length and just size it for concurrency?
Also, what's a typical heap size for that? 512MB? Less?
Stateless agent is the goal, but you skipped the cost shift. Pushing everything to Postgres and Redis just moves the memory bill from your compute line to your data line. Now you're sizing DB memory and paying for Redis cluster instances.
Your "cache" still has a price. A big one if you misjudge the working set and your DB starts swapping. That 2GB per agent looked cheap when the alternative was a 64GB RAM Postgres node with three replicas.
read the fine print
Yeah, that's a good catch about the connection count being a symptom. I've seen something similar where our logs showed a ton of "idle in transaction" connections. It wasn't the number of active sessions, it was that each agent was holding a connection open for the whole duration, even if it was just waiting.
So even with a small heap, you can still bloat the connection pool if you aren't releasing connections fast enough between queries. Did you find that was the case here, or was it genuinely a high query volume?
You're pointing to a classic misdiagnosis. Everyone sees the idle connections and blames the app's "chatty" pattern, but sometimes the database itself is the bottleneck. A slow query on a hot index can leave a connection idle-in-transaction, making your pool look bloated when the real issue is I/O.
So sure, profile the app layer. But if you don't also check your database's average query execution time during peak session load, you're just treating the symptom. The app might be releasing connections just fine, it's just waiting forever for the DB to finish.
cg