Hi everyone. I’ve been following this forum for a while but this is my first post. I’m not an engineer, but I manage a lot of cross-functional projects with our engineering teams. We’ve been using GitHub Copilot across our engineering org for about a year now, and I’ve been gathering feedback from the teams I work with.
The consensus seems to be that it’s great for boilerplate and speeding up repetitive code, but I’ve heard some consistent, specific stories about where it really trips up. Since my role involves a lot of vendor research and evaluating tools, I wanted to share a few concrete failure cases the engineers have reported to me, to see if they match what others are seeing.
One big theme is around refactoring. An engineer told me they asked Copilot to rename a method across a large codebase, and it confidently changed the method name in most places, but also “updated” several similar-looking method names in unrelated files, which broke some integrations. They said the suggestion looked plausible at a glance, but they had to spend hours reverting and checking cross-references.
Another case was with API suggestions. A team was integrating with a third-party service, and Copilot suggested using an API endpoint and parameters that simply didn’t exist in the vendor’s actual documentation. It looked real—it had what seemed like valid syntax—but it was completely fabricated. This caused a blockage because the junior developer trusted it and spent time building around a non-functional call.
I’m curious if others have run into similar patterns. As someone who isn’t deep in the code myself, I’m trying to understand the real pitfalls so I can help our teams use these tools more effectively and know when to double-check.