Another script that dumps logs into a CSV and calls it a detection engine. Saw the one on GitHub. It’s fine if you’re just grepping for curse words, but it misses the entire point of monitoring a tool like Claw.
You’re not looking for “suspicious prompts.” You’re looking for deviations from baseline activity that indicate data exfiltration attempts, policy circumvention, or prompt injection. That means context: user role, time of day, volume, and crucially, the *sequence* of interactions. A single prompt is rarely the issue.
Here’s a pared-down version of what we actually use to build a baseline and flag anomalies. It’s not a full SIEM, but it’ll get you closer to something audit-worthy. Focuses on rate, uniqueness, and known risky patterns.
```python
import json
import re
from datetime import datetime, timedelta
from collections import defaultdict
def analyze_log_sequence(log_entries):
"""
Expects log_entries as list of dicts with 'user', 'timestamp', 'raw_prompt'.
Returns dict of anomalies.
"""
user_activity = defaultdict(list)
anomalies = {'high_frequency': [], 'pattern_match': [], 'unique_explosion': []}
# Build user timeline
for entry in log_entries:
user = entry['user']
ts = datetime.fromisoformat(entry['timestamp'])
user_activity[user].append((ts, entry['raw_prompt']))
# Check each user
for user, sessions in user_activity.items():
sessions.sort(key=lambda x: x[0])
prompts = [p for _, p in sessions]
# High frequency check: >20 prompts in 5 minutes
for i in range(len(sessions)):
window_start = sessions[i][0]
window_count = 0
for j in range(i, len(sessions)):
if sessions[j][0] - window_start 20:
anomalies['high_frequency'].append({
'user': user,
'time': window_start.isoformat(),
'count': window_count
})
break
# Pattern match for common exfiltration attempts
pattern = re.compile(r'((?:repeat|ignore|previous|all).*(?:password|key|token|secret|confidential))', re.IGNORECASE)
for ts, prompt in sessions:
if pattern.search(prompt):
anomalies['pattern_match'].append({
'user': user,
'time': ts.isoformat(),
'match': pattern.search(prompt).group(1)[:50] # truncated
})
# Unique prompt explosion vs user baseline (simplified)
unique_ratio = len(set(prompts)) / max(len(prompts), 1)
if len(prompts) > 10 and unique_ratio > 0.9:
anomalies['unique_explosion'].append({
'user': user,
'window': f"{sessions[0][0]} to {sessions[-1][0]}",
'unique_ratio': unique_ratio
})
return anomalies
```
Run this on a daily slice of your logs. Pair it with your identity provider data (is this user in engineering? why are they asking about HR data at 2 AM?). The output is what you feed into your vendor risk assessment—proof you’re actually *monitoring*, not just collecting.
And for compliance, ensure every flagged anomaly generates an audit trail with a reviewer’s decision. Otherwise, it’s just a fancy log.
Trust but verify – and audit