We built a detection rule for anomalous location logins using Zscaler NSS logs pushed to Grafana Loki. The goal was to flag user logins from geographic locations they've never been seen in before, without waiting for a SIEM correlation.
Key components:
* Zscaler NSS logs (specifically `ZSCALER_NSS_FIREWALL_LOGS`) streamed to a Loki instance via Fluent Bit.
* A Grafana `sql` alert rule that runs a query to find new, unique country codes per user.
The alert query logic looks for any user+country combo that hasn't been seen in the last 30 days. We tuned it to ignore our common corporate VPN exit nodes.
```sql
WITH current_logs AS (
SELECT DISTINCT
parsed.user_login as user,
parsed.country_code as country
FROM logs
WHERE parsed.event = 'auth'
AND parsed.country_code != ''
AND $__timeFilter(timestamp)
),
historical_logs AS (
SELECT DISTINCT
parsed.user_login as user,
parsed.country_code as country
FROM logs
WHERE parsed.event = 'auth'
AND parsed.country_code != ''
AND timestamp >= NOW() - INTERVAL '30' DAY
AND timestamp < $__timeFrom
)
SELECT
cl.user,
cl.country
FROM current_logs cl
LEFT JOIN historical_logs hl ON cl.user = hl.user AND cl.country = hl.country
WHERE hl.user IS NULL
AND cl.user NOT LIKE 'svc-%'
```
Alert fires, sends a notification to our security channel with the user and country. Team can then verify or dismiss. Works faster than our old SIEM workflow.
Biggest pitfall was log volume—had to adjust the Loki retention and chunk size. Also, Zscaler's internal service IPs sometimes get mapped to unexpected countries; we maintain a small exclusion list for those.
—cp
—cp
Interesting approach using Loki and Grafana for this. The 30-day historical baseline is a practical choice, but have you considered the risk of seasonal employees or contractors who might legitimately log in from a new country after a longer period of inactivity? Your rule would flag them.
Also, the static exclusion for corporate VPN nodes is a good start, but it requires manual upkeep. A drift in those IP ranges could cause false negatives. You might want to layer in a secondary check, like looking for concurrent logins or impossible travel if you ever get timestamps with enough precision.
—at