Just built an internal dashboard showing Copilot acceptance rates per developer.

Alright, let's get something straight. We're paying for this Copilot seat license for the whole engineering team, and I wanted to know if we're just burning money or if it's actually being used. Management asked for "ROI," which is a fancy way of saying "prove it's not a toy." So I slapped together an internal dashboard to track the real metric: acceptance rates. Not just suggestions made, but suggestions *kept*.

I'm not using any fancy external SaaS for this. It's a simple pipeline that taps into the GitHub API, since Copilot telemetry is available there if you have the right permissions. The core of it is a scheduled job that pulls the data, aggregates it, and dumps it into a time-series database (Prometheus, because it's already there for monitoring). Then it's just a Grafana dashboard on top.

Here's the gist of the collector script's logic. It's run via a Jenkins job (yes, Jenkins, it works and I don't have to explain YAML pipelines to it) every 6 hours.

```python
#!/usr/bin/env python3
"""
Queries GitHub API for Copilot metrics per user in our org.
Requires a PAT with `read:org` and `copilot:read` permissions.
"""
import requests
import os
from datetime import datetime, timedelta

API_URL = "https://api.github.com/orgs/YOUR_ORG/copilot/metrics"
HEADERS = {
"Authorization": f"token {os.getenv('GITHUB_COPILOT_TOKEN')}",
"Accept": "application/vnd.github.v3+json"
}

# Get last 7 days
end_date = datetime.now()
start_date = end_date - timedelta(days=7)
params = {
'since': start_date.isoformat(),
'until': end_date.isoformat(),
'per_page': 100
}

response = requests.get(API_URL, headers=HEADERS, params=params)
data = response.json()

# Data structure is a list of daily entries per user
for day in data.get('days', []):
for user_metric in day.get('users', []):
user = user_metric['login']
suggestions = user_metric['suggestions_count']
acceptances = user_metric['acceptances_count']
acceptance_rate = (acceptances / suggestions * 100) if suggestions > 0 else 0
# Emit to Prometheus pushgateway or print for scraping
print(f'copilot_acceptance_rate{{user="{user}"}} {acceptance_rate}')
print(f'copilot_suggestions_total{{user="{user}"}} {suggestions}')
```

The dashboard panels show:
* Acceptance rate (%) per developer, over time.
* Total suggestions accepted vs. rejected, stacked bar per team.
* A simple leaderboard (controversial, I know) of who's getting the most *useful* suggestions.
* A breakdown by language (from the API's language field) to see if it's more useful for our Python services vs. the frontend TypeScript.

What we've learned so far isn't surprising, but it's concrete:
* The average acceptance rate across the team is about 35%. Huge variance, from 15% to 60%.
* Senior devs have a lower acceptance rate but higher total acceptances. They're getting more suggestions and filtering harder.
* Junior devs have a higher acceptance rate, but the suggestions they accept are for boilerplate and simple patterns. They report it helps them learn API conventions.
* For our legacy Perl scripts (don't ask), the acceptance rate is near zero. It's useless there.

This data finally lets us have a real conversation. It's not about "is Copilot good?" It's about "for which parts of our work is it a net positive?" We're considering adjusting licenses or focusing training based on this, instead of just blindly renewing for everyone. The next step is correlating this with PR cycle time to see if higher acceptance rates link to faster delivery, or just more code.