I've been using Aider for a few months now, mainly for refactoring and feature additions in a Python/JS codebase. While I love the concept, I found myself wondering: how effective is it, really? I was asking for changes, but was I just generating a lot of suggestions I'd ultimately reject?
To answer that, I built a simple internal dashboard that tracks Aider's suggestion acceptance rate. The goal was to get a concrete metric on its efficiency for my workflow.
Here’s the basic flow I set up using a combination of tools:
* I use a wrapper script to launch Aider, which logs each session start to a small backend service.
* The service parses the conversation from `aider-chat.md` after each session (using a post-commit hook).
* It uses some simple regex to identify Aider's proposed code blocks and then checks the git diff to see which changes were actually applied.
* This data gets sent to a Tinybird workspace, which powers a simple dashboard showing:
* Acceptance rate over time (weekly/daily)
* Raw counts: suggestions offered vs. accepted
* Breakdown by project/file type (Python vs. JS had a noticeable difference for me)
The initial findings were pretty interesting. My acceptance rate hovers around 65-70%, but it dips significantly when I'm vague with my prompts. It's become a great feedback loop for me to write better instructions for Aider.
Has anyone else tried to measure Aider's performance in a quantitative way? I'm curious if others have built similar tracking or have thoughts on what other metrics would be valuable to log.
Oh that's a cool idea. I've been wanting to try Aider but worried about the same thing - spending more time correcting it than just writing code myself.
>Breakdown by project/file type (Python vs. JS had a noticeable difference)
That's really interesting. Which one had the better acceptance rate in your case? I'd guess Python, but maybe the JS ecosystem's flexibility helps?
Did you find the dashboard changed how you prompted it? Like, did you start phrasing requests differently to try and boost the rate?