I was reviewing their status page after yesterday's downtime, and their webhook delivery reliability over the last quarter is hovering around 97%. That's concerning for any process where you need a guaranteed event receipt.
If you're building anything critical—like syncing new customer data to your CRM or triggering fulfillment—you can't afford to lose 3 out of every 100 events. I've had to implement a few patterns to handle this for clients. The core idea is to treat their webhook as a best-effort notification, not the source of truth.
Here’s a basic resilience pattern I often set up in Zapier or Make:
1. **Immediate Acknowledgement:** Configure your endpoint to return a `200` immediately upon receipt.
2. **Queue for Processing:** Push the webhook payload into a reliable queue. I use tools like this:
```javascript
// Example: Pushing to a queue service (like Google Pub/Sub via a Zapier webhook)
const payload = JSON.parse(request.body);
await queueService.publish('incoming-webhooks', payload);
```
3. **Separate, Retry-able Process:** Have a separate zap/automation that pulls from the queue and performs the main action (e.g., updating Salesforce). This flow can have its own retry logic.
For more complex scenarios, I sometimes use Workato to add a persistent data store that logs every webhook call, and then a job periodically checks for missing IDs against the source system's API.
What strategies are others using to add reliability to shaky webhook sources? Have you found a particular middleware tool handles retries and logging better for these cases?
That's a solid architectural pattern to add reliability on your side. I've had to build similar queuing systems after getting burned by a vendor's "best effort" delivery.
But I'm always left wondering about the financial side of that trade-off. You're now responsible for building, monitoring, and maintaining a queuing layer. You also take on the cost of that infrastructure and the dev hours to implement it.
When I see a core service reliability of 97%, my next step is to pull the contract. Does their SLA for the webhook service actually guarantee anything higher than that? And if it doesn't, are we paying a premium for a "reliable" service that's shifting the resilience burden onto us? I'd rather a cheaper, explicit "best effort" tier and plan for my own queue from the start, than pay for an SLA that doesn't match the reality.
Trust but verify - especially the pricing page.
That pattern makes sense, especially the part about treating their webhook as just a notification. But I'm stuck on a more basic step.
When you say "configure your endpoint to return a 200 immediately," what does that endpoint look like in practice for someone using Zapier? Is it a separate Zap that just catches the webhook and passes it to the queue, or something else?
I'm trying to picture the actual setup before you even get to the reliable queue part.
Help me decide