Hey folks! 👋 I've been living in the ServiceNow GRC module for a while now, and while the dashboards are great, I really wanted to get those critical findings in front of my team *where they already are*. For us, that's Slack.
So I built a simple bot that monitors for new high-severity findings (think Sev 1/Critical) and posts a formatted alert to a dedicated channel. It's been a game-changer for response time.
The core of it is a Python script that runs on a schedule (we use a lightweight VM, but a serverless function would work too). It basically polls the ServiceNow API for `sn_grc_finding` records filtered by severity and a "created" timestamp. Here's a simplified version of the key part:
```python
import requests
import json
from slack_sdk import WebClient
SN_INSTANCE = "your-instance.service-now.com"
SN_TABLE = "sn_grc_finding"
SN_USER = "your_integration_user"
SN_PASS = "your_pass"
SLACK_TOKEN = "xoxb-your-token"
SLACK_CHANNEL = "#grc-alerts"
# Query for High/Critical findings created in the last 10 minutes
query = "severity=1^createdONLast10 minutes@javascript:gs.minutesAgoStart(10)@javascript:gs.minutesAgoEnd(0)"
url = f"https://{SN_INSTANCE}/api/now/table/{SN_TABLE}?sysparm_query={query}"
response = requests.get(url, auth=(SN_USER, SN_PASS), headers={"Accept": "application/json"})
findings = response.json().get('result', [])
slack_client = WebClient(token=SLACK_TOKEN)
for finding in findings:
number = finding.get('number')
short_description = finding.get('short_description')
sys_created_on = finding.get('sys_created_on')
grc_link = f"https://{SN_INSTANCE}/nav_to.do?uri=%2F{SN_TABLE}.do%3Fsys_id%3D{finding.get('sys_id')}"
message = f"🚨 *New Critical Finding*: n*Description*: {short_description}n*Created*: {sys_created_on}"
slack_client.chat_postMessage(channel=SLACK_CHANNEL, text=message)
```
We've added some extra logic to avoid duplicates and include the responsible business unit, but this is the essence. It's been super reliable.
The beauty is it's decoupled and simple. No heavy middleware, just a direct API call. It saves our risk team from constantly refreshing the GRC workspace and has really improved our mean-time-to-acknowledge. Anyone else doing something similar? I'd love to compare notes or hear about other clever integrations.
ship it
ship it
The polling approach with a timestamp window is a classic and functional start. Have you considered the performance implications at scale? Using `createdONLast10 minutes` with `gs.minutesAgoStart/End` can become problematic if your script execution interval drifts or if you have a high volume of findings, leading to potential gaps or overlaps in detection.
I'd suggest implementing a stateful checkpoint system. Instead of relying on a relative time window, your script should persist the last successfully processed `sys_created_on` timestamp (or the record's `sys_id`) to a small local SQLite database or a cloud key-value store. On each run, query for records where `sys_created_on > last_processed_timestamp`, ordered ascending. This gives you idempotency and guarantees you won't miss records during script downtime or failures.
Also, for production resilience, you should wrap the API call and the Slack posting in separate try-catch blocks with retry logic and dead-letter queue semantics. What's your current strategy for handling a Slack API outage when a finding is detected?
Data first, decisions later.
Good call on the stateful checkpoint, that's a solid next step. I actually started with the window method because it was quick to prototype, but you're right about the drift potential.
For the outage handling, right now it's just logging to a file if Slack's API fails, which is... not great. A dead-letter queue would be a big upgrade. Maybe even a simple SQS setup to retry those failed alerts.
I'm curious, have you set up something similar? How are you handling the checkpoint storage - SQLite feels fine for a single runner, but if I move to serverless, I'd need something like DynamoDB or just a simple S3 file, I guess.
Beta tester at heart