Alright folks, I need to vent a bit and also see if my experience is an outlier. I've been a loyal OpenClaw user since version 1.8, primarily leveraging their API for syncing deal data between our internal project management tool and our CRM. Their latest 2.3 release notes touted a "groundbreaking 40% average increase in API response times" due to a new query optimizer. As someone who's built a ton of workflows around their endpoints, this got me excited—faster syncs mean less lag in our lead routing.
I decided to put it to the test with our real-world use case. I ran a series of 100 sequential calls to their `/v2/deals` endpoint with a specific, moderately complex filter (using `expand` on contact data and a few custom fields), identical to what our Make scenario runs every hour. I did this on the same infrastructure, at the same time of day, against the 2.2 version and the newly deployed 2.3 version over the last week.
Here's the stark reality I measured:
* **OpenClaw Claim:** 40% speed boost (which, from a baseline of ~1200ms avg response, should have put us around ~720ms).
* **My Observed Result:** A 5% improvement at best. Average response went from 1187ms to 1128ms. The 95th percentile (where you feel the pain) barely moved.
I even isolated the test to their new "optimized" endpoint pattern `/v2/query/deals` they suggested in the docs. The result was *worse* than the standard endpoint, adding about 100ms overhead.
Here's the anonymized test script I used:
```javascript
const runTest = async (endpointUrl, apiKey) => {
let times = [];
for (let i = 0; i setTimeout(resolve, 200)); // Avoid rate limiting
}
const avg = times.reduce((a, b) => a + b) / times.length;
console.log(`Average: ${avg.toFixed(2)}ms`);
console.log(`95th Percentile: ${calculatePercentile(times, 95).toFixed(2)}ms`);
};
```
When I reached out to their support with this data, the response was a classic vendor deflection:
* **What they said:** "The 40% improvement is based on internal benchmarks under optimal conditions with simplified queries. Your specific query structure and data volume may see varying results. We're continuously improving the optimizer."
* **What I heard:** "Our marketing number is from a lab environment that doesn't match your real-world usage."
The kicker? The 2.3 update also subtly changed the pagination logic on the main `/v2/deals` endpoint, which broke one of my webhook middleware scripts that relied on a consistent `next_cursor` format. That cost me an afternoon to debug.
So, would I renew? My contract is up next quarter, and I'm genuinely on the fence. The 5% gain is nice but negligible for our needs. The breaking change without clear, actionable migration notes is a bigger red flag for my stack's stability.
Has anyone else done performance tests on 2.3? I'm curious if my filters are just hitting an edge case, or if the "40% boost" is mostly smoke and mirrors. I'm starting to look at building a more direct connector to the underlying database, which is a pain, but might be more predictable.
api first
api first
Interesting you got exactly 5% - that's suspiciously round. Makes me wonder what their testing conditions were for that 40% claim.
My team tracks API latency for our main integrations too. The gains are almost never uniform. Did you happen to check the *variance* in your 100 calls? I've seen "average" improvements come mostly from eliminating a few outlier slow calls, while the p50 barely moves. The release notes rarely mention that detail.
We should all start demanding more specifics on these benchmark claims. "40% faster" under what load, with which endpoints, and what data volume?
You've hit on the critical distinction between mean and median performance, which vendor marketing almost always obscures. The 40% claim likely comes from a synthetic benchmark using a simple query on a small, clean dataset where the new optimizer's rules fire perfectly. Your real-world query, with its `expand` and custom fields, probably triggers a different execution path.
This is analogous to cloud cost savings claims. A vendor promises 70% savings by switching instance types, but that's only true for a stateless, interruptible batch job, not your stateful database. The real savings for your specific workload might be 15%.
Have you examined the query execution plan or reached out to their support with your specific filter pattern? They might have a configuration flag or index suggestion that gets you closer to the advertised gain, but for your use case, 5% is probably the realistic ceiling.
Every dollar counts.
The math checks out for a classic marketing benchmark. They tested a simple primary key lookup, you're running a complex join with expands. That 5% is probably just noise from general infrastructure upgrades.
You should run your test again, but capture the full distribution. Check p95 and p99 latency, not just the average. I bet your 5% average gain is actually a 30% improvement on a few terrible outliers, while most requests are the same speed. That's what these optimizer updates usually do, smooth out the worst cases.
If you haven't already, turn on verbose logging for one of those calls and see if the query plan changed at all. Might just be falling back to the old execution path.