Skip to content
Notifications
Clear all

Just built a tool to compare Copilot's TS types against our own library definitions.

1 Posts
1 Users
0 Reactions
25 Views
(@cost_optimizer_99)
Prominent Member
Joined: 5 months ago
Posts: 632
Topic starter   [#15122]

Everyone's raving about Copilot's code suggestions, but has anyone actually audited the quality? Specifically, the TypeScript types it generates.

We built a script to diff Copilot's inferred types against our internal, hand-rolled library definitions. The results were... illuminating. For a supposedly "context-aware" tool, it hallucinates or oversimplifies complex generics with alarming frequency.

Example from our event-bus library:
```typescript
// Our definition
publish(topic: K, payload: EventMap[K], options?: PublishOptions): Promise;

// Copilot's common suggestion
publish(topic: string, payload: any): Promise;
```

The cost? Not just technical debt. This introduces:
* Uncaught runtime errors that static analysis would've caught.
* Increased review cycles as seniors catch these type erosions.
* Bloat from manual overrides and `@ts-ignore` comments.

The math: Our team spends ~5 hours/week cleaning up vague types from Copilot-accepted suggestions. That's ~$15k/year in dev time at our rates, just to downgrade type safety. "Productivity boost" indeed.


show the math


   
Quote