Love the idea of making security posture visible with a daily trend! That kind of ambient awareness is really powerful.
Your code snippet reminded me of a subtle issue with date parsing in scheduled jobs. If Braintrust's API returns UTC timestamps and your job runs in a local timezone, that `datetime.fromisoformat()` call could give you unexpected dates around midnight UTC, effectively making your "last 7 days" shift. I've been bitten by that before. It's often safer to treat all dates from the API as UTC and then convert to local for display only, or ask the API for a date range directly instead of a fixed `limit`.
Also, for the trend calculation, you might consider using a rolling window average alongside the linear trend. Sometimes a single noisy day can make the linear slope look alarming when the underlying week-over-week movement is flat. A 3-day moving average smooths out the noise for a clearer signal.
Prod is the only environment that matters.
The timezone issue is a critical one that can create silent data errors. Beyond consistent UTC handling, I'd recommend storing all timestamps with explicit timezone metadata in your data pipeline. If you're using a database, store them as `TIMESTAMP WITH TIME ZONE` or equivalent. This prevents assumptions from creeping in during later analysis stages.
On the smoothing, a rolling average is a good start. For security scores, which often move in step functions, I've found a median filter over a 3-day window can be even more effective than a mean at ignoring single-day anomalies without overly smoothing a genuine step change. It's a simple change that better reflects the discrete nature of these metric changes.
—BJ
Totally agree on the median filter for security scores. We've been using a 7-day rolling median in some of our dashboards specifically because it preserves those step changes while ignoring the noise from partial scan completions or vendor hiccups.
The `TIMESTAMP WITH TIME ZONE` advice can't be overstated. I've seen a team waste a week debugging a daily trend that looked fine in Grafana but was completely wrong in their offline analysis because of implicit timezone conversions in their reporting layer. Once you store it correctly at the source, the problem just disappears downstream.
Sleep is for the weak
The week-long debugging session over timestamps is painfully familiar. I'd add that even when you store them correctly, you still need to check the query layer. Some BI tools have a "timezone for display" setting that's separate from the underlying data, and analysts can unknowingly apply it, reintroducing the same error. It's a two-part fix: correct storage, then rigid query standards.
On the median filter, the 7-day window is solid, but watch out for weekly patterns. If your security scans have a predictable weekly cadence (e.g., heavy scans on Sunday), a 7-day window might smooth out a legitimate recurring dip. Sometimes a 5-day window, excluding weekends, gives a clearer signal.
Migrate once, test twice.
Oh man, the vendor-calculated-trend-but-not-exposed API tax is a classic. It's like they're charging you rent for the data twice - once to generate it, and again for you to pull it and re-calculate the same thing on your dime. Always worth poking around the docs for a hidden `/trend` or `/summary` endpoint.
For weekends and missing scores, we skip posting entirely. Interpolation feels like lying to ourselves - if the scanner didn't run, there's no new signal. The bot just logs a "no new data" and quits. The real trick is setting the channel expectation that a missing daily post doesn't mean the sky is falling, it just means nothing changed.
Though, this does expose a secondary cost: you're still paying for the compute to run the Lambda or container that checks and finds nothing. Over a year, those "no-op" executions for weekends and holidays add up to a stupid amount of wasted spend.
Nice approach! I love the simplicity. That `datetime.fromisoformat()` line makes me nervous, though. If Braintrust returns timestamps with a 'Z' (like `"2023-12-01T00:00:00Z"`), slicing off the last character with `[:-1]` removes the 'Z', making it a naive datetime. Your trend could silently shift if your job's server timezone isn't UTC.
Better to parse it with timezone awareness:
```python
from datetime import datetime, timezone
# Keep the Z and parse correctly
dates = [datetime.fromisoformat(s['date'].replace('Z', '+00:00')) for s in scores_data]
```
Or use `dateutil.parser.isoparse` if you're open to another dependency. It handles the 'Z' natively.
Saves you from that midnight UTC bug user705 mentioned!
Clean code is not an option, it's a sanity measure.
That `[:-1]` slice is dangerous and unnecessary. If you're getting RFC 3339/ISO 8601 strings with a 'Z' suffix, you should parse them correctly. Using `fromisoformat` requires you to replace the 'Z' with '+00:00' as user288 shows, but that's clunky. The standard library has a better option since Python 3.7:
```python
from datetime import datetime
dates = [datetime.fromisoformat(s['date'].replace('Z', '+00:00')) for s in scores_data]
```
But honestly, if you're already pulling in `requests` and `slack_sdk`, just add `python-dateutil` to your dependencies and use `dateutil.parser.isoparse`. It handles all the edge cases, including the 'Z', without any string munging. The dependency cost is minimal compared to the risk of a silent timezone bug corrupting your trend data.
Benchmarks or bust
Good catch on using `dateutil`. I'd actually recommend against adding it just for this specific parsing, though, if the format is stable. The `.replace('Z', '+00:00')` method is a bit clunky but it's explicit and keeps your dependency list minimal, which has its own value in deployment and security reviews.
A middle ground is to wrap the parsing in a small utility function. That way, if the vendor's API format ever changes or you need to handle more complex cases, you have a single place to swap the implementation, and you can still avoid the extra dependency until it's truly justified.
Also, don't forget to consider the runtime environment's locale settings. Even with the '+00:00' replacement, `fromisoformat` is generally safe, but I've seen edge cases with alternative calendar systems on some container images. A unit test with a few known-good timestamps from the API is a cheap sanity check.