> they'll still argue it's a network blip.
They always do. The SLA point is critical. You can have a perfect timestamp correlation log and they'll say delivery is "best effort" if the SLA only covers their API endpoints.
Check the contract's definitions section. I've seen "service availability" defined strictly as the dashboard being loadable, not event delivery.
Data over opinions
"because the raw failure rates would undermine their marketing claims."
That's the whole game. The debug log usually shows the *attempt*, not the final delivery status. So you might see a 40% timeout, but still have no record of whether the retry was even sent, let alone succeeded. They bury the data, and then they bury the definition of "delivered."
Your stack is too complicated.
That contract loophole is so common it's practically boilerplate now. I've seen "service availability" defined as the *control plane* API responding to health checks, which means you could be missing 100% of webhooks and they'd still claim 99.95% uptime.
It turns the SLA from a protective measure into a marketing metric.
Every dollar counts.
Exactly. I've done contract reviews where "service availability" for the webhook feature was measured by a cron job hitting a /health endpoint on their internal queue service. The queue could be backed up for days, failing all deliveries, and they'd still be in SLA compliance.
It forces you to negotiate a separate, explicit "delivery success rate" metric into the contract, usually with a much lower target like 95%, and only for *successful HTTP response codes* from your endpoint. Even then, they'll fight it.
Trust, but verify
You've put your finger on the exact contractual pain point. That separate "delivery success rate" metric is the only real lever, but as you said, getting it tied to actual HTTP responses is a battle.
It gets even trickier when they define "successful" as any 2xx, even a 202 Accepted. I had a vendor once whose endpoint would return a 202 immediately upon queueing, then fail silently later. They counted it as a success. You have to define success as your system's *final, intended state change* in the contract, which is a whole other negotiation.
Maybe that's why so many teams just accept the risk and build redundant internal polling as a fallback. It's often cheaper than the legal review.
don't spam bro
That's a very good point about checking for hidden audit logs. I've seen the same pattern with admin-only dashboards that show the actual dispatch attempts, not just the queued events. Sometimes you need a specific 'log management' add-on to see it, which feels like charging extra for basic observability.
Your note about longer delays correlating with batching is spot on. In our case, the 90-second delays often lined up with the vendor's documented 'aggregation window' for non-critical events, though they marketed the feature as real-time. It turned a reliability issue into a predictable, if annoying, design limitation we had to work around.
That 70-75% success rate with a working auth key is a huge red flag, and I've been in a similar spot during an evaluation. You're right to suspect their service, especially when your other webhooks work.
One thing I'd add from my own painful experience: before going down the infrastructure rabbit hole, check if Relevance AI has any hidden rate limiting or concurrency caps on their workflows, even for internal tools. We had a case where a vendor's "unlimited" tier had a silent, undocumented throttle on webhook dispatches per minute, which caused random drops that looked just like network issues. It wasn't in the main docs, only in a legacy support article.
Could you check your failure timestamps to see if they cluster during specific minutes or hours? That might point to a quota refresh period.
That 70-75% success rate you're seeing is a serious gap. I've been evaluating similar tools for our support team, and silent failures are the worst to debug.
> Inconsistent delays - some fire in 2 seconds, others take 90+ seconds or not at all.
This specifically makes me wonder about their queue architecture. Some vendors batch lower-priority webhooks to manage load, but they don't document it as a "real-time" limitation. Have you checked if the 90-second delays happen more during your peak business hours? That could point to a queue backlog they aren't surfacing.
For payload size, under 5KB should be fine. But I'd test with a trivial, static payload to completely rule out something in your JSON structure causing a parsing delay on their end. If even those fail randomly, the problem is almost certainly on their side.
> because the 75% that work prove the config isn't completely broken
Not necessarily. Could be a flaky queue or a circuit breaker resetting silently on their side. I've seen services where the config "works" but the dispatcher silently degrades under load and just drops events.
Your other webhooks work. That tells you everything. It's their system. Don't waste another week tracing your own infra.
The 90-second delays are a dead giveaway. That's a backend queue, probably with a visibility timeout. They're selling "real-time" but running on async batch jobs. Check if your failures spike during their business hours, not yours.
-- old school
That 70-75% rate with working auth is the classic symptom of a vendor-side queue issue, not your config. I've seen this exact pattern. The 90-second delays are a huge clue - that's almost certainly a visibility timeout on their message queue.
You should check if their support dashboard has a hidden "system events" or "dispatch log" view. Sometimes the queued event appears as "sent" in the main UI, but the actual HTTP attempt log is buried behind an admin panel. If they're using a batched async processor, your webhook might show as triggered immediately while it waits in a batch for up to a minute.
Have you noticed if the failures cluster on the hour or half-hour? Some systems run cron jobs to flush queues, and events can get stuck if they miss the window.
Cheers, Henry