Our team has been evaluating Braintrust for the last quarter as a potential replacement for our existing, fragmented suite of code quality and security scanners. The primary value proposition for us was the unified findings feed and the promise of a single, actionable API for all scan results. However, the true operational test is always in the integration: can these findings be woven into existing developer workflows without adding cognitive overhead? For us, that meant piping critical findings directly into our primary Slack channels, but with enough context to be immediately useful, not just noisy alerts.
This post details the technical implementation we landed on, the trade-offs we made, and the performance characteristics we observed. We rejected the native Slack integration provided by Braintrust's UI as it was too coarse-grained, offering only broad category alerts. Instead, we built a lightweight orchestration service that polls the Braintrust API, filters and enriches the data, and posts formatted alerts. The goal was to trigger an alert only for new, high-severity findings on the default branch of our repositories.
Our orchestration service runs as a Kubernetes CronJob every 10 minutes. It performs the following sequence:
1. **Poll the Braintrust Projects API** to get the list of projects we monitor.
2. **Fetch findings for each project** using the `/findings` endpoint with specific filters: `branch=main`, `state=open`, `severity=high,critical`.
3. **Deduplicate against a persistent cache** (we use a simple Redis store keyed by finding ID) to ensure we only act on findings not previously alerted on.
4. **Enrich the finding data** by pulling the associated commit hash and author from the Git provider API (using the `git_commit` field in the finding).
5. **Format and post to Slack** using a Block Kit payload designed for clarity.
The core filtering logic in our service looks something like this:
```python
# Pseudo-code for the critical filtering step
def get_new_actionable_findings(project_id):
params = {
'branch': 'main',
'state': 'open',
'severity': 'critical,high',
'sort_by': 'created_at',
'limit': 50
}
findings = braintrust_api.get(f'/projects/{project_id}/findings', params)
actionable = []
for finding in findings:
# Skip if we've already alerted on this finding ID
if redis.get(f"alerted:{finding['id']}"):
continue
# Enrich with git info if available
if finding.get('git_commit'):
finding['git_info'] = git_service.get_commit_info(finding['git_commit'])
actionable.append(finding)
redis.setex(f"alerted:{finding['id']}", 604800, 'true') # Cache for 7 days
return actionable
```
The Slack message template was carefully constructed to include all necessary context without requiring a click-through to Braintrust for initial triage. Each alert shows:
- The project and repository name.
- The finding title and severity (with color-coded borders).
- The exact file path and line number(s).
- The commit hash and author who introduced it.
- A direct link to the finding in the Braintrust UI for deeper investigation.
- A one-click button to create a Jira ticket (using Slack's message actions).
Performance and cost observations:
- The API response time for the `/findings` endpoint averages ~450ms per project under our current load. We monitor 12 projects, so the full poll cycle takes ~6 seconds of compute time.
- We have not hit any rate-limiting issues with the 10-minute interval.
- The primary cost is not Braintrust's API, but the additional compute for our enrichment service and Redis operations. This is negligible (< $5/month) compared to the engineering efficiency gained.
- The most significant pitfall was initially not caching `git_info` separately, leading to duplicate calls to our Git provider API. We added a second Redis layer to cache commit data for 24 hours.
The integration has been running in production for eight weeks. The alert volume is manageable (2-5 critical alerts per week), and the development team's response time to new security vulnerabilities has decreased significantly because the alert appears in their natural communication flow with full context. The key was moving beyond the out-of-box notification system to create a tailored, data-rich, and low-noise integration.
-ek
Show me the numbers, not the roadmap.