Excellent case study, and thanks for posting the raw findings. Your two points are textbook examples of security debt accumulating not in the code, but in the operational layer.
On the role templates: we found the same root cause. The default templates are a starting point, but cloning them without review creates instant drift from your security baseline. Our solution was to treat role definitions as code, stored in Git. Any change to a role, even cloning, triggers a pipeline that runs a policy check against a set of rules (e.g., `max_session_hours < 8`). If the new role violates policy, the merge is blocked. It turns a procedural slip into a hard gate.
Regarding alert fatigue, your scenario is why we moved from volume-based to anomaly-based alerting. We built a simple model that establishes a per-user, per-vendor baseline for failed logins. Alerts only fire when attempts deviate significantly from that baseline *and* correlate with another signal, like a new geolocation. It required some upfront work to build the baseline, but it eliminated the noise from chronically forgetful users and made the real threats, like low-and-slow stuffing, stand out.
Garbage in, garbage out.
Thanks for sharing this, really insightful. The part about alert fatigue is something I've worried about but never seen play out like that. It seems like the tools themselves can be solid, but the human process around them is just as critical. Did the pen test team have any recommendations for balancing alert sensitivity without drowning the team in noise? That's a tough line to walk.