I was really excited to roll out Amazon CodeGuru Reviewer across our engineering team's primary repositories. The promise of automated security and performance findings from a major cloud provider seemed like a solid step forward for our code quality. We've now had it running on about 30 repos (mix of Java and Python) for a few months, and while it *has* caught some legitimate issues, the experience hasn't been as smooth as the marketing led us to believe.
The biggest surprise has been the sheer volume of what I'd call "context-blind" recommendations. It will flag a piece of code as a potential resource leak or performance hotspot based on a very narrow, textbook analysis, completely missing the larger pattern in the codebase that makes it a non-issue. This has created a fair bit of noise in our review queues, and more importantly, it's starting to train some of our junior devs to just click "dismiss" without critical thought. For example, it consistently recommends replacing a `for` loop with a streams-based operation in a performance-critical path where we've explicitly chosen the loop for clarity and measured performance reasons.
My other gripe is with the integration friction. The onboarding was straightforward, but the way findings are presented in the PR—as comments from a bot account—lacks the actionable depth I was hoping for. It often just states a problem without offering a concrete, ready-to-apply fix, which leaves the developer to figure out the remediation from scratch. This can sometimes take longer than if a human reviewer had spotted it.
Has anyone else run into similar issues with CodeGuru, particularly around the relevance of its suggestions in a mature codebase? I'm wondering if we need to tweak our configuration significantly, or if this is just the nature of the tool. I'm also curious about how you've managed the "signal vs. noise" balance—did you create a lot of baseline exclusions?
~Harry
~Harry
Not surprised. We tried it on a few high-velocity Java repos and saw the same.
The "context-blind" problem is real. It'll flag a potential NPE in a method that's only ever called after a specific validation step it can't see. You end up dismissing a lot.
Bigger issue for us was the lag. By the time it ran on a PR, the dev had often moved on. The feedback loop was too slow to be useful in active development.
It's decent for a post-commit audit, but you need a strong lead to filter the noise. Don't let juniors treat it as an oracle.
Show me the bill
That lag issue you mentioned resonates. We tried integrating its PR feedback via a webhook to Slack, thinking faster notifications would help. The delay was baked into the analysis itself, not the delivery. It felt like getting a detailed architectural review for a shed you finished building last week.
The "noise to signal" problem gets worse when you're dealing with integration code. It once flagged a connection pool not being closed in a Python Lambda handler. The handler's runtime container gets frozen and reused by AWS - closing the pool actually hurts performance. CodeGuru was applying server-side patterns to a serverless context.
Your point about juniors treating it as gospel is spot on. We had to add a rule to our PR template: "If you're suppressing a CodeGuru finding, you must cite the architectural reason in a comment." That at least forced some thought.
APIs are not magic.