BeyondTrust's REST API is surprisingly robust for extracting operational health data, but turning that into actionable observability took a bit of work. I wanted a single pane of glass for our Privileged Access Management posture—think session counts, vault status, credential check-in/out rates, and failed authentication attempts.
I built a Python collector that runs on a schedule, pulls the key metrics, and pushes them to Prometheus. From there, it's a standard Grafana dashboard. The real value isn't just in seeing the numbers, but in setting alerts on trends, like a slow but steady rise in concurrent sessions that might indicate credential sprawl. The script handles the authentication flow and maps the JSON responses to clear metric names.
Here's the core of the collector for anyone looking to do something similar. You'll need to adjust the endpoints and add your own credentials, obviously.
```python
import requests
import prometheus_client as prom
from prometheus_client import start_http_server, Gauge
import time
# Define Prometheus Gauges
active_sessions = Gauge('beyondtrust_active_sessions', 'Current active PAM sessions')
vault_health = Gauge('beyondtrust_vault_health_status', 'Vault health status (1=healthy, 0=unhealthy)')
credential_checkouts = Gauge('beyondtrust_credential_checkouts_total', 'Total credential checkouts')
def fetch_beyondtrust_metrics(api_base, api_key):
headers = {'Authorization': f'Bearer {api_key}'}
# Fetch active sessions
sessions_resp = requests.get(f'{api_base}/api/sessions/active', headers=headers)
active_sessions.set(len(sessions_resp.json() if sessions_resp.ok else 0))
# Fetch vault health (simplified example)
health_resp = requests.get(f'{api_base}/api/system/health', headers=headers)
is_healthy = 1 if health_resp.ok and health_resp.json().get('status') == 'OK' else 0
vault_health.set(is_healthy)
# Additional metric collection would go here...
```
The dashboard now gives our security team the same observability mindset we apply to our infrastructure. It's helped us spot issues like a vault sync delay before it impacted users. I'm curious—has anyone else instrumented their PAM solution? What metrics did you find most valuable for day-to-day health versus audit compliance?
- GG
- GG
Nice approach with the custom collector. One thing you'll want to instrument is the API client latency and error rate from the collector itself. I've seen these external integrations become a silent point of failure. Wrap your main request calls in a try/except and expose a `beyondtrust_api_client_errors_total` counter.
Also, consider adding a metric for the timestamp of the last successful scrape. It's a simple sanity check for the pipeline's health, separate from the PAM system's health.
The authentication flow you didn't show is the part that usually breaks. If you're using OAuth, you need to instrument token refresh failures separately from the data scrape. That's a silent killer.
Also, you're pushing to Prometheus. Are you registering those gauges once, or redefining them on every loop? If you're redefining, you'll get errors after the first registry push. Do it like this outside your loop:
```python
METRICS = {
'active_sessions': Gauge('beyondtrust_active_sessions', '...'),
# ...
}
```
Finally, have you run this at scale? The API might have pagination limits you haven't hit yet. You'll need to handle `next` links for endpoints like `/api/sessions`.
Show me the query.
Instrumenting the API client latency is a solid suggestion. I usually wrap the core request function to time the duration and expose it as a histogram. That way you can track if things are slowing down over time, which is often the first sign of trouble.
Your point about the `last_successful_scrape` timestamp is good. It becomes a dead-man's switch in your monitoring. Just remember to label it with the collector's instance name if you're running multiple, or you won't know which one is down.
null