I've been running a 30-day deep-dive test on Claw, the new AI coding assistant that's been making waves for its claimed "sub-100ms" latency on code completions. My setup: I integrated it into VS Code, using their official extension, and ran it through my typical daily workload—a mix of TypeScript/React frontend work and some Python data processing scripts. I also built a simple load simulator to push it beyond my normal keystroke pace.
The headline claim about latency is, in my experience, **absolutely true**. In normal, paced development, the completions appear almost instantaneously. It's a genuinely different feel from other assistants I've used (Cursor, Tabnine, Cody). To quantify it, I logged response times for 1,000 completions during routine work:
```json
{
"test_conditions": "normal paced typing, local project",
"p50_latency": "87ms",
"p95_latency": "112ms",
"p99_latency": "145ms"
}
```
The perceived speed is incredible and does reduce cognitive friction when you're in a flow state. It feels less like waiting for a suggestion and more like your editor is just magically finishing your thoughts.
However—and this is a big however—the performance profile changes dramatically under sustained, rapid-fire load. When I used my simulator to mimic a high-intensity pairing session or just rapid code deletion and rewriting, the error rate spiked. I'm not just talking about less relevant suggestions; I mean outright failures: empty completions, or the dreaded "Connection lost, retrying..." toast in the IDE.
My load test simulated a developer typing at >100 WPM with minimal pause, triggering a completion request on every plausible keystroke. Here's what I observed:
* **Latency remained surprisingly good** (p95 under 150ms), showing their backend can handle the queue.
* **The failure rate (empty or useless completions) jumped from <1% in normal use to over 15%** during sustained 2-minute bursts of high load.
* The extension **does not degrade gracefully**. It doesn't fall back to a local model or show a "rate limited" message; it just serves empty lines or old suggestions until the storm passes.
This tells me their architecture is optimized for the single-user, bursty pattern of normal coding, but the global rate-limiting or context-switching logic hits a wall when a single user floods the queue. For a solo dev, it's mostly a non-issue. But if you're the type who goes into rapid refactoring sprints, or if your team all gets on a frenzied deploy at the same time (and shares a provisioned throughput pool), you might hit these walls.
So the "so what"? Claw is a fantastic technical achievement for day-to-day coding fluidity. If latency is your primary bottleneck with other assistants, it's a game-changer. But if you need consistent, rock-solid reliability during peak creative (or destructive) coding sessions, or you're evaluating for a team that might create concurrent load patterns, you need to be aware of this trade-off. I'm curious if others have stress-tested it in similar ways or if my experience aligns with the community's.