Hey folks — been putting Panther through its paces for a few months now, mostly for SOC 2 compliance monitoring and cloud log analysis. I kept wondering: “How well is it actually covering my environment? Am I missing obvious detections?”
So I built a small Python tool that cross-references Panther’s built-in detections against our AWS resources and common attack patterns. It’s basically a coverage grader.
Here’s what it checks:
- Which AWS services we use vs. which have Panther default detections
- High-risk API calls (like `DeleteBucketPolicy` or `ConsoleLogin`) that might lack alerting
- IAM roles and S3 buckets without active monitoring rules
- Gaps in common MITRE ATT&CK techniques relevant to our stack
A few things I found in our setup:
- Panther had great coverage for CloudTrail and GuardDuty, but we were light on some CloudWatch Logs detections.
- Several S3 buckets with sensitive data weren’t flagged for public write attempts — we added custom rules for those.
- IAM role assumption patterns weren’t being baselined; we added anomaly detection.
The tool isn’t fancy — just uses the Panther API and boto3 — but it helped us visualize gaps and prioritize which custom rules to write first.
Anyone else run a similar coverage assessment? Curious how you’re validating detection breadth, especially if you’re multi-cloud or hybrid.
✌️
✌️
This is a great idea. I've been meaning to do something similar for our Panther setup but wasn't sure where to start. Did you use a specific library to map your resources to the MITRE ATT&CK techniques, or was that a manual process?
Nice approach! I've found that mapping to MITRE ATT&CK manually is doable for a focused cloud environment, but it gets pretty time-consuming. For a recent project, I ended up using the `mitreattack-python` library to programmatically pull the technique list and map our custom detections. It saved a ton of time, though you still have to make some judgment calls on what constitutes coverage for each technique.
The Panther API is solid for pulling your active detection list. Combining that with a resource inventory from boto3 and a library for the ATT&CK framework seems like a perfect combo. How are you handling the scoring? A simple percentage match, or something more weighted for high-risk resources?
ship it