Hi everyone, I've been running Twingate's Linux connector in Docker for a few months now, and overall it's been great for our zero-trust setup. Lately though, I've noticed the container process is consistently using high CPU—often sitting at 30-50% even with minimal network traffic. This is on a t3.medium EC2 instance that's otherwise idle.
Has anyone else run into this and found any configuration tweaks or Docker settings that help? I'm wondering if it's related to the health check intervals, logging verbosity, or perhaps a known issue with a specific version.
My current setup is pretty standard:
```yaml
version: '3.8'
services:
twingate-connector:
image: twingate/connector:latest
container_name: twingate-connector
restart: unless-stopped
environment:
- TWINGATE_ACCESS_TOKEN=${ACCESS_TOKEN}
- TWINGATE_REFRESH_TOKEN=${REFRESH_TOKEN}
- TWINGATE_NETWORK=${NETWORK_ID}
- TWINGATE_LOG_LEVEL=info
ports:
- "443:443"
```
I've already tried pinning to a specific tag instead of `latest` and setting `TWINGATE_LOG_LEVEL=warn`, but the CPU usage didn't change much. I'm curious if adjusting the health check frequency or the resource limits in Docker might be the next step. Any insights or experiences would be super helpful!
-- Amy
Cloud cost nerd. No, I don't use Reserved Instances.
That log level change you tried is a red herring; it mostly just throttles what gets written to disk, not the internal processing overhead. The culprit is almost always the default health check probing, which can get aggressive.
You're on the right track pinning a version, but the better move is to explicitly set the health check intervals with something like `--health-interval=30s` in your docker run or the compose healthcheck definition. The default is often much faster than needed for a stable deployment.
Also, check if you're on a kernel that's had issues with TLS handshake overhead in userspace networking modes. I've seen similar spikes vanish by forcing the connector to use the host network (`network_mode: host`), though that's a bigger config change.
Data over dogma.
The network mode switch can indeed reduce overhead, but I've seen it introduce routing complications in multi-NIC environments. If you go that route, verify your outbound traffic paths match expectations, especially if you're using custom iptables rules.
The health check interval adjustment is valid, though I'd recommend correlating it with your actual recovery time objectives. If you're running in a high-availability pair, shorter intervals might be necessary despite the CPU cost.
One additional factor I've observed is memory pressure triggering frequent garbage collection cycles in the connector's runtime. It's worth checking if your container has a memory limit set that's too restrictive, forcing aggressive cleanup.
Plan the exit before entry.
Good call on the memory pressure angle - it's easy to overlook. I've seen the same garbage collection churn when the connector is crammed into a tiny memory limit.
If you're running in Docker, check if you have a `mem_limit` set from an old compose file. The default in newer Docker versions is usually fine, but if you've capped it below 512MB, you're asking for trouble. The connector's JVM needs a bit of breathing room for its heap.
The log level and tag pinning attempts confirm the issue isn't superficial, but the provided compose snippet is missing the critical `healthcheck` and `cpuset` parameters, which are the primary levers here. The default health check likely runs every few seconds, causing constant probe evaluation overhead.
You should define a custom health check with a significantly longer interval, like 30s, and a longer timeout. Combine that with pinning the container to specific CPU cores using `cpuset` to prevent it from bouncing across vCPUs and incurring context switch penalties, which is particularly impactful on AWS's t-series instances.
Also, add `--cpus 0.5` or similar in your compose under `deploy.resources.limits` to directly throttle the container's CPU share. The high idle usage you're seeing suggests the process isn't being constrained by the scheduler, allowing it to consume more cycles than it strictly needs.