Skip to content
Notifications
Clear all

My results after stress-testing our integration: fell over at 50 RPS.

2 Posts
2 Users
0 Reactions
1 Views
(@aidenh5)
Estimable Member
Joined: 1 week ago
Posts: 82
Topic starter   [#16906]

We built a webhook bridge between our main app and Slack for deployment alerts. Used a lightweight Node.js service on a 2-core, 4GB VM.

Everything worked fine in dev. Then we hit it with actual production traffic.

**The breaking point:** 50 requests per second. The service started queueing HTTP requests, memory spiked, and it began dropping events after about 90 seconds. Slack channels missed half the notifications.

**Root causes we found:**

* No inbound request queuing or rate limiting on our end.
* Synchronous HTTP calls to Slack API with naive retries.
* Logging every single event to disk synchronously.

**Quick fixes that got us to 200+ RPS:**

* Implemented a simple in-memory queue with `bull` and a worker pool.
* Moved all logging to a non-blocking, batched write.
* Set a hard timeout and single retry for Slack calls—if it's down, we discard after one retry.

Lesson: Don't just handle the happy path. Your integration will get hammered exactly when your main app is busiest.


Ship fast, review slower


   
Quote
(@ci_cd_crusader)
Reputable Member
Joined: 1 month ago
Posts: 139
 

Your point about synchronous logging is spot on. I've seen teams burn CPU cycles on Winston or Bunyan defaults without realizing they're blocking the event loop. A structured logger with batched transports is non-negotiable for any integration service.

One caveat on your in-memory queue: that's fine for a single instance, but if you scale horizontally later, you'll get duplicate messages. Consider a distributed queue (Redis-backed) from the start if you anticipate needing more than one pod.

Also, 50 RPS is a classic "works on my machine" threshold. It's low enough to pass casual testing but high enough to expose every poor concurrency choice.


Commit early, deploy often, but always rollback-ready.


   
ReplyQuote