Let’s be frank: deploying OpenClaw in a production Kubernetes environment using their default Helm charts is a recipe for a post-mortem. The vendor’s documentation touts a simple “helm install” for a “scalable, enterprise-ready” deployment, but their defaults are dangerously optimistic for any meaningful load. Our team learned this the hard way after a series of cascading failures during a forecast-critical quarter-end. This guide is the result of that painful, data-driven analysis and subsequent hardening exercise.
Our primary issues stemmed from three core architectural oversights in the default configuration:
* **Resource requests and limits were either absent or comically low**, leading to node pressure and unpredictable pod evictions.
* **The default PostgreSQL subchart used an unoptimized configuration** with no connection pooling, causing database connection storms under concurrent user load.
* **Liveness and readiness probes were overly aggressive and poorly configured**, creating restart loops that amplified the very outages they were meant to mitigate.
The following step-by-step configurations represent the necessary hardening we applied to achieve stability. These are not theoretical best practices but concrete changes that resolved our specific incidents.
**1. Resource Allocation & Quality of Service**
We moved from no defined limits to guaranteed QoS. The vendor’s default values.yaml had `resources: {}`. Our production values now enforce the following for the main `openclaw-api` deployment:
```yaml
resources:
requests:
memory: "512Mi"
cpu: "250m"
limits:
memory: "1Gi"
cpu: "500m"
```
We derived these numbers from analyzing a 95th percentile of usage over a 30-day period in a staging environment under load-testing. This alone eliminated the “OOMKilled” events plaguing our nodes.
**2. Database Tuning & PgBouncer Sidecar**
The included PostgreSQL chart was a major bottleneck. We disabled the subchart (`postgresql.enabled: false`) and connected to a managed database service. For teams that must run a database in-cluster, we strongly recommend deploying PgBouncer as a sidecar. The critical addition was this pool configuration for the main application container:
```yaml
env:
- name: DATABASE_URL
value: "postgresql://$(DB_USER):$(DB_PASSWORD)@localhost:6432/openclaw_prod?pool_timeout=10&pool_size=20"
```
We also adjusted the `shared_buffers` and `max_connections` on the PostgreSQL instance itself, aligning with the compute resources of the node.
**3. Probe Reconfiguration**
The default probes used an HTTP GET to `/health`, which performed a full dependency check on each call. This caused the database to be queried every 10 seconds. During database latency spikes, this would fail and restart healthy pods. We reconfigured for a layered health check:
```yaml
livenessProbe:
httpGet:
path: /health/liveness
port: http
initialDelaySeconds: 90
periodSeconds: 30
failureThreshold: 3
readinessProbe:
httpGet:
path: /health/readiness
port: http
initialDelaySeconds: 30
periodSeconds: 10
failureThreshold: 3
```
The `/health/liveness` endpoint checks only the process state, while `/health/readiness` includes critical dependencies. The increased `initialDelaySeconds` was vital for the JVM to fully initialize.
**4. NetworkPolicy and Ingress Timeouts**
The vendor's guide neglected network isolation and timeout settings. We implemented a default-deny NetworkPolicy for the namespace and explicitly allowed only necessary traffic. More critically, we had to increase Ingress timeouts as some of OpenClaw's reporting endpoints are long-running:
```yaml
annotations:
nginx.ingress.kubernetes.io/proxy-connect-timeout: "30"
nginx.ingress.kubernetes.io/proxy-send-timeout: "180"
nginx.ingress.kubernetes.io/proxy-read-timeout: "180"
```
**Would we renew?** Yes, but only after implementing these changes. The core OpenClaw application is powerful for revenue analytics, but its packaging as a “Kubernetes-native” solution requires significant operational maturity. The vendor’s support was responsive but their initial posture was that we were under-provisioning our cluster; our metrics clearly showed inefficient configuration, not a lack of resources. The lesson here is to treat the provided Helm charts as a starting point for development, not a production blueprint. The total engineering effort to harden this deployment was approximately 18 person-days.
-- revops_nerd
trust but verify