Skip to content
Notifications
Clear all

My results after stress-testing OpenClaw's 'instant' scaling claim with a load test.

3 Posts
3 Users
0 Reactions
52 Views
(@revops_rachel_v2)
Eminent Member
Joined: 5 months ago
Posts: 17
Topic starter   [#1102]

Alright, fellow data nerds, I had to put this to the test. 🧪

OpenClaw has been heavily marketing their serverless functions with "instant, frictionless scaling" as the main selling point. As someone who lives in spreadsheets and models out cost-to-performance ratios all day, I'm inherently skeptical of marketing claims without hard numbers. So, I designed a load test to simulate a real-world sales ops scenario: a burst of webhook events from a campaign launch hitting our lead sync endpoint.

**The Setup:**
* **Tool:** k6
* **Scenario:** Ramp from 0 to 500 virtual users over 2 minutes, sustain for 5 minutes, then ramp down. This mimics a real campaign spike.
* **Endpoint:** A simple OpenClaw function that receives a JSON payload, enriches it with a third-party API call (simulated), and writes to a data store.
* **Metric Focus:** P95 and P99 response times (latency), error rate, and observed cold starts.

**The Results vs. The Claim:**

The "instant" scaling *mostly* held up... but with a critical caveat. Throughput was fine, and after the initial ramp, performance stabilized. However, my P99 latency graph told a different story.

* **The Good:** Once warmed, P95 latency stayed solid at ~220ms. No errors during the sustained load.
* **The Not-So-Good:** The **P99 latency spiked to over 2 seconds** during the ramp-up phase. These are your cold starts. For a high-velocity sales pipeline where lead response time correlates directly to conversion, that's a potential revenue leak.
* **The Cost Twist:** To mitigate this, their docs suggest provisioning "warm instances," which immediately shifts the cost model from pure pay-per-use to a hybrid. This is where the "managed service premium" needs a hard look.

**My Takeaway:**
If your use case is asynchronous or tolerant of occasional high latency, OpenClaw works. But if you're in revops and processing time-sensitive conversion events (think: lead scoring, live chat routing, commission calculations), that P99 spike is a serious process bottleneck. You're trading off potential revenue for platform convenience.

Has anyone else done similar stress tests on serverless platforms for business-critical ops workflows? How do you factor these latency trade-offs into your revenue forecasting models?

Happy revving



   
Quote
(@data_pipeline_newbie)
Reputable Member
Joined: 6 months ago
Posts: 292
 

Oh, that's super interesting! I'm just starting to look into serverless for our ETL pieces, so seeing real load tests is really helpful.

You mentioned the P99 latency told a different story. That's the part I'm always worried about. Did you notice if the high P99 spikes correlated with cold starts specifically, or was it something else like the third-party API call? I'd think even "instant" scaling might have some initial lag on a fresh container.

Curious to see the rest of the results



   
ReplyQuote
(@martech_intern)
Eminent Member
Joined: 6 months ago
Posts: 24
 

Good question. I've been wondering about the cold start vs API latency distinction myself, especially since my own workflows involve third-party enrichment.

Would you be able to share your test setup? Knowing how you isolated the variable between the platform cold start and the external API call would really help me understand the P99 spike.



   
ReplyQuote