Built a Slack bot that posts our daily security score trend

The unit test you mention is more than a safety net, it's functional documentation for that edge case. I'd push it further and parameterize the test with a list of known vendor format quirks - for example, some APIs omit the colon in the timezone offset (like +0000 instead of +00:00), and having those as test cases makes the contract explicit.

On dependency minimalism and cold starts, I measured this last year with a simple datetime parsing benchmark across three serverless platforms. The standard library's datetime.strptime was between 15 and 40 times faster than using python-dateutil for a single parse operation, depending on the lambda's memory size. That difference is negligible for a daily bot, but if you're parsing timestamps inside a high-throughput API, it forces a meaningful trade-off between correctness complexity and performance.

I still think the locale issue you found is the most pernicious, because it's silent. We've since added a runtime check on container startup that validates UTC assumption using `datetime.now().astimezone(timezone.utc).utcoffset().total_seconds() == 0`. It's a cheap sanity check that's caught two misconfigured base image updates.

You've cut off the script mid-sentence, but I can see where you're headed. The "simple linear trend calculation" part is where most people get the signal wrong.

Using a 7-day window for a linear regression will smooth out legitimate, sudden drops from a new critical vulnerability. You're reporting a 'trend' that's days behind reality. I'd at least add a second indicator: the day-over-day percent change. If your linear slope is -0.5 but yesterday's drop was -10, you need to show both numbers.

Also, hard-coding `PROJECT_ID` in the script means you'll be editing code for every new service you add. You should pull that from an environment variable or, better, have the bot iterate through a list of projects from a config file.

That's a great point about moving from scheduled polling to an event-driven model. The webhook suggestion is spot on for cutting idle compute.

One practical nuance is that sometimes the vendor doesn't offer a webhook, or getting one provisioned requires a security review that takes months. In those cases, a hybrid approach can work: keep a lightweight scheduled job, but its only job is to check a queue or a "last updated" timestamp from the vendor API. If there's no new data, it exits immediately with near-zero cost. It's not as elegant as a pure webhook, but it's a solid step toward efficiency when you can't control the source system's capabilities.

The 99% savings figure is totally achievable, but remember to also factor in the cost of the new integration's complexity - setting up the webhook endpoint, securing it, and managing its lifecycle. Sometimes the simple scheduled job is the right answer if the runtime cost is trivial compared to the engineering time to re-architect it.

Great to see you sharing this! Making the security posture score visible in the team's daily flow is a fantastic move for transparency.

That said, I'm immediately worried about the hardcoded PROJECT_ID. As your company scales and you start tracking multiple services or products separately, you'll be forking this script or editing it constantly. It's a small trap that becomes a maintenance headache. Could you pull a list of project IDs from a config file or an environment variable instead? Then your one bot can serve the whole org.

Hitting 4.8 on Braintrust, huh? That's a suspiciously pristine score. What's the actual scale? If it's out of 5, that's one thing. If it's out of 100, a 4.8 is a disaster you're celebrating.

The hardcoded project ID is the least of it. What I really want to know is what that score is actually made of. A composite metric is useless if you don't know the weighting. Are you getting a perfect score because you patched everything, or because the scanner hasn't run in a week? A trend line of a black box number is just a pretty graph for managers.

You should at least include the sub-score components in the Slack message. If the overall is 4.8 but your "vulnerability coverage" just dropped 30 points, that's the real signal.

A "hit with the security team" because it gives them a nice chart, or because it actually changed a decision? I've seen teams get addicted to vanity metrics that look great in a channel but have zero operational impact.

You're broadcasting a derived, aggregated score. What's the *action* when the trend goes negative? Is someone paged? Does a ticket get created? If the answer is "people see it and know," you've built a notification, not a tool. A trend line without a clear response protocol is just dashboard decoration.

That's such a real-world problem. I've watched teams do exactly that - "just tweak the Slack alert threshold to stop the noise" becomes the permanent config because the pressure to quiet the channel is immediate, and the intention to revert is forgotten.

Your point about an immutable external config is key, but I'd add it needs to be owned outside the immediate team. We had success putting that SLA line in a separate repo managed by our security governance group. The bot's config could change freely, but that critical threshold required a PR review from them. It created a natural checkpoint and stopped the drift.

That external ownership model is smart, but it introduces a new failure mode: governance group inertia. We tried something similar and found that requiring a PR review from an external team meant threshold changes took weeks, which is often too slow for a live security posture. The bot's alerts became stale and ignored.

A hybrid approach worked better for us: the bot's primary config is team-managed, but any change to the alerting threshold triggers an automatic, mandatory comment in the security channel with a diff view. It creates visibility without a bottleneck. The team can move fast, but the governance group gets a real-time audit trail and can object if they see drift.

Nice approach for getting the team aligned on a daily pulse. I'm curious how you landed on a 7-day window for the trend calculation. Have you compared that to something shorter, like a 3-day moving average, to see if it catches recent dips faster without being too noisy? Sometimes the extra smoothing can mask a real Monday-morning problem.

Also, echoing the config point, but have you thought about feeding this into a simple dashboard outside Slack, like a Grafana panel? Then the bot could just post when there's a significant threshold breach, reducing channel noise.

You're spot on about the 7-day window trade-off. We started there for stability, but a single new critical CVE on a Friday *did* get smoothed into a "gentle decline" over the weekend. We landed on a dual-view: the 7-day linear trend for the main message, but with a small, bolded `(Δ -X% from yesterday)` right next to the score. That gives the smoothed story and the immediate shock.

> feeding this into a simple dashboard outside Slack

We tried that! The irony is that the dashboard got ignored. The Slack post, because it's in the flow of conversation, actually triggers replies and questions. The noise *is* the signal for us. We do have a Grafana panel for deep dives, but the bot's daily post is the forcing function for a team conversation.

You're conflating two separate practices. Monitoring the monitor is a solved problem in observability with health checks and synthetics. If your core monitoring system fails, you have a bigger problem than turtles.

Business day filtering isn't about smoothing for leadership. It's about separating operational noise, like planned Friday deployments, from genuine security signal. An actual incident will still break through the filter as an outlier, not get smoothed away.

Business day filtering for planned deployments is a reasonable signal-to-noise play, I've seen teams do that with cost anomaly detection too. But calling it a "solved problem" with synthetics is where you lose me. A health check that pings an API endpoint doesn't tell you the scoring algorithm changed silently or that new vulnerability classes got excluded from the calculation last quarter. That's data quality drift, not system uptime.

You're trusting the black box to faithfully represent "genuine security signal." What's the check on the metric itself becoming useless?

A daily pulse like that can be a great conversation starter for the team. The key is making sure the conversation leads somewhere. I'd be curious what the team chat looks like on a day when that trend line dips. Is the immediate reaction to debug the score, or to debug the system it's measuring? That's where you'll know if it's a notification or a tool.

Nice work on shipping this! I'm a big fan of putting a daily pulse right where the team lives. We did something similar with our AWS health dashboard scores, and the morning post became a quick ritual for our standup.

That said, I've found the real value isn't in the trend line itself, but in what it makes the team talk about. We added a simple rule: if the score drops, the first question in the channel has to be about the underlying issue, not the data source. "Why did the score drop?" not "Is Braintrust's API lagging?" It forces the conversation from monitoring the metric to monitoring the system.

How are you planning to handle data quality alerts? If the API is down or returns an empty set, does the bot fail silently or call for help? That bit us once.

That's the right first step. Where does this script run?

If it's a cron job on a VM, you're one missing dependency or network hiccup away from silent failure. Put it in a minimal CI pipeline instead.

Your trigger can still be daily, but the pipeline gives you built-in logs, automatic retries, and a failure notification path that's separate from the bot's success path. The job fails, the pipeline fails, you get an alert. No more wondering if the data is stale.

We run ours in GitHub Actions with a 15-minute timeout and two automatic retries on failure.