Your test plan covers the basics. The biggest difference you'll see immediately is that Copilot is far more assertive and opinionated, often overriding your internal patterns with generic GitHub trends. For your React/Node stack, expect it to default to popular libraries (like `fetch`) unless you explicitly steer it in a comment.
> context awareness
It's weaker than CodeWhisperer here. Chat only sees the current file. For inline suggestions, it uses open tabs, but its tendency is to pull from its public training data first.
For the chat, treat it like a junior dev that needs precise instructions. "Explain this legacy auth middleware" works. "Refactor this" without strict boundaries will break things. It's not a gimmick if you're directive.
Skip the PR description test for anything non-trivial. It hallucinates details.
You've touched on a critical limitation with the fresh context problem. I've found that inconsistency to be the most jarring part of the transition. A file save and reopen is indeed the current workaround, but it breaks the flow state.
The monorepo boundary you mention is even stricter than it seems. It doesn't just fail to see unopened directories; it will often make confidently wrong suggestions based on public data. I once asked it where to place a shared types file in a `packages/` structure, and it suggested a `src/utils` folder that hadn't existed in our codebase for two years. It was clearly pulling from a common open-source pattern.
This forces a specific workflow where you must have the exact relevant module open to get a useful suggestion, turning what should be a discovery aid into a manual navigation task first.
null
You've zeroed in on the exact friction point for regulated teams. That "black box" feeling around telemetry is real, and it's a hard sell after having everything tied to your AWS CloudTrail.
>It hallucinates dependencies
This is the killer. The PR description feature can be useful for internal reference, but you're right that it fails as a source of truth. We had a similar incident where it inserted a non-existent API endpoint. It makes you question every other output, which defeats the purpose of an assistive tool.
The license risk is a quiet, long-term liability. Even with an SCA tool, the overhead of tracking down the origin of a small, clever snippet that slipped through feels like it undermines the productivity gain.