That last point about owning the reliability is so key. I went the DIY route too and the biggest surprise was the hidden work. You're not just coding the enrichment logic, you're building all the monitoring and alerting for the pipeline itself. When it fails at 2 AM, you're on call for your own glue code.
For deduplication, we started with timestamps but got burned. We ended up using a hash of the core alert payload and storing it with a TTL in Redis. It's not perfect, but it handles out-of-order delivery better. The real pitfall, as others said, is the batch API. You're forced to choose between freshness (frequent polling) and load (slower cycles). Neither feels great.
The flexibility is intoxicating, but you're right to question if you're just rebuilding a queue manager. Have you looked at using something like PagerDuty's Event Orchestration as a middle layer? It can handle the state and dedup, and you push enriched events to it. Offloads some of the ops burden.
The hidden work part hits home. You're suddenly on call for your own integration, and that's a big mental shift.
I hadn't thought of using PagerDuty's orchestration as a middle layer, that's an interesting idea. Does it handle the polling/API part too, or do you still have to build that piece and just send the events to them? If you're still polling, I guess you'd still own that potential breakage.
Your point about integration flexibility is spot on. We hit a similar wall trying to enrich alerts with data from an internal risk scoring service; Prisma's workflow couldn't do a simple HTTP call without a convoluted proxy step.
That 1.2 second p95 for your enriched ticket flow is impressive, but what's your p99 during an internal service degradation? That's where our DIY setup showed cracks - one downstream API lagging would stall the entire enrichment chain because we'd implemented simple sequential calls. We moved to a fan-out pattern with timeouts, but that added state complexity.
The six-month feature request wait tracks with our experience. It's not just about adding a step, it's about the velocity of adapting the workflow itself.
Commit early, deploy often, but always rollback-ready.
You've touched on the key architectural trade-off. Moving to a fan-out pattern was the correct move, but that state complexity you mentioned is a non-trivial overhead. It's not just about adding timeouts; you now need a coordinator to handle partial failures and manage which enrichments are essential versus optional for ticket creation.
We also learned the hard way about sequential calls stalling. Our p95 was similar, but our p99 ballooned to over 8 seconds during a downstream database lag. The fix required moving from a simple Lambda to a Step Functions state machine to manage parallel tasks and define failure thresholds. This is exactly the "hidden work" others are noting - you're building a distributed system coordinator.
The six-month wait for a native feature is indeed about adaptation velocity. With a DIY approach, you can implement that risk scoring call in days. But you then spend the next five months hardening the orchestration layer you just built. The question becomes whether you're investing in your core product or in a more resilient integration framework.
Data > opinions