Hyperproof's API webhook failures under load aren't just annoying—they're a cost center. Failed calls mean manual reconciliation, wasted engineer hours, and delayed compliance cycles.
From our logs, failures spike above ~50 concurrent uploads. The system returns HTTP 429 or 502, but the retry logic in their webhook spec is insufficient.
Key issues:
* Default webhook setup lacks exponential backoff.
* Payloads often exceed 5MB, triggering timeouts.
* No built-in dead-letter queue for evidence.
Our workaround: We built a proxy queue. All evidence uploads first go to an internal SQS queue, then a lambda processes them at a controlled rate to Hyperproof.
```python
# Pseudo-code for controlled dispatch
def send_to_hyperproof(evidence_item):
retry_count = 0
max_retries = 5
while retry_count < max_retries:
try:
response = post_with_timeout(evidence_item, timeout=30)
if response.status_code == 200:
return True
except (Timeout, TooManyRequests):
sleep(2 ** retry_count + random.uniform(0, 1))
retry_count += 1
send_to_dead_letter_queue(evidence_item)
return False
```
Has anyone else quantified the operational overhead of these failures? What's your acceptable failure rate for compliance evidence?
cost per transaction is the only metric