Everyone talks about "automating evidence collection" for audits and compliance. Most of it is still a manual slog. We just ran our first real test of Hyperproof's automated connectors for GitHub and AWS CloudTrail. Here's what actually happened.
The setup was straightforward. You connect your GitHub org and an AWS trail, then map specific events to controls in your framework (we're using SOC 2). For GitHub, we configured it to pull evidence for:
* Branch protection rules on critical repos
* Required code reviews before merge
* MFA enforcement on organization accounts
* Regular vulnerability scanning alerts
For CloudTrail, we mapped controls around:
* IAM user creation/deletion
* Security group changes
* S3 bucket policy modifications
* GuardDuty alert generation
The good: It works. On the scheduled pull, the evidence landed in the correct control cards. No more downloading CSV logs, taking screenshots, and manually uploading files. The time saved on the CloudTrail items alone is significant.
The not-so-good: It's not magic. You still have to interpret the evidence. A list of "CreateUser" events doesn't prove you're reviewing them—you need a separate process log for that. Also, the initial mapping logic is critical; get it wrong and you pull irrelevant noise.
Bottom line: This moves the needle from "completely manual" to "managed automation." It eliminates the grunt work of gathering raw logs. But your team still needs the expertise to configure it correctly and validate the output. For the price, it's a justifiable tool if you have multiple frameworks or frequent audit cycles. If you're doing a one-off annual audit, the ROI might be thin.
Has anyone else pushed these connectors further? I'm particularly interested in how you're handling false positives or filtering out dev/test environment noise from the evidence stream.
—JW
—JW