Hey everyone, new to managing Elastic Agent fleet servers here. I set one up on a t3.medium EC2 instance (4GB RAM) to handle maybe a hundred agents, and it's constantly maxing out the memory.
I'm not seeing a huge number of agents or events in the logs. The config is pretty much the default from the quick start. Has anyone else run into this? Wondering if there's a common misconfiguration or if 4GB is just not enough for a basic fleet server setup. Trying to keep costs low while learning 😅
Yep, that t3.medium with 4GB is tricky for Fleet Server. It's right on the edge.
The defaults can be too aggressive, especially for the internal monitoring policy. I'd try dialing back the monitoring metrics collection frequency first - that often gives you a quick 500MB-1GB back. Also, check if any of your hundred agents are shipping bulky logs or metrics you might not need right away for your testing.
You can probably make it work on that instance with some tuning, but if you're just learning, maybe bump to a t3.large for a bit more headroom. It's not much more per hour and saves you the headache while you get things stable.
Data doesn't lie, but dashboards sometimes do.
The default monitoring policy is the main culprit. It's logging far too much internal data by default.
Check your agent's heap usage - you can cut it by half with the right JVM flags. Also, reduce the number of heap dumps on OOM from 3 to 1. That saved me over a gigabyte on a similar setup.
T3.medium *can* work for a hundred agents if you're just doing heartbeat and basic logs, but you have to strip out all the telemetry.
Prove it with a benchmark.
The t3.large suggestion is solid for avoiding immediate firefighting, but that's a 25% bump in cost for a "while learning" setup. That adds up.
Better to tune on the t3.medium first. If you *do* size up later, do it as a Spot Instance or with a Savings Plan commitment - otherwise you're just burning cash on idle headroom.
> dialing back the monitoring metrics collection frequency
This is the first knob to turn. Also, set `agent.monitoring.http.enabled: false` if you don't need that HTTP endpoint. The default internal collection is hilariously chatty for a small test fleet.
If you still need the headroom after tuning, a t3.large Spot Instance is often cheaper than the on-demand t3.medium you're struggling with.
- elle