After observing significant variance in the quality and utility of commit messages across my team's repositories, I initiated a controlled experiment six months ago. The objective was to standardize and elevate commit message quality by delegating their creation entirely to a coding assistant (specifically, GitHub Copilot Chat and Claude via CLI), with a strict protocol. The hypothesis was that this would not only improve the historical record but also enforce more disciplined, atomic commits by requiring the AI to generate a coherent message for each discrete change.
The core of the workflow is a pre-commit hook script that leverages `git diff --staged` to capture changes. This diff is then passed to the AI model with a carefully engineered prompt. The key was to move beyond simple summarization and demand a specific, structured format that aligns with conventional commit standards and provides actionable context for future investigation (e.g., during bisect). Below is the essential section of the script I developed and refined over the trial period.
```bash
#!/bin/bash
# .git/hooks/prepare-commit-msg
STAGED_DIFF=$(git diff --staged --no-ext-diff)
if [ -z "$STAGED_DIFF" ]; then
exit 0
fi
# Construct the prompt with clear instructions for structure and focus.
PROMPT="Generate a conventional Git commit message for the following staged diff.
Your output must be a single line for the subject, followed by a blank line, followed by a body.
Subject Line: Use the imperative mood, capitalize the first letter, and do not exceed 50 characters.
Body: Explain the 'why' and the context, not just the 'what'. Reference any fixed issues. Use bullet points if needed.
Focus on the change's intent and impact. If it's a refactor, state the motivation.
Here is the diff:
${STAGED_DIFF}"
# Using Claude via Claude CLI for this example. Similar setup works for Copilot.
AI_MESSAGE=$(claude "$PROMPT" --model=claude-3-sonnet-20240229)
# Overwrite the default commit message file with the AI-generated one.
echo "$AI_MESSAGE" > "$1"
```
The results were measured across several dimensions by analyzing the commit history from the six months prior to implementation and the six months after. A cohort analysis of repository contributors was also performed to control for individual variance.
**Quantitative Findings:**
* **Consistency:** 98% of AI-generated commits adhered to the conventional commit format, compared to a baseline of 45%.
* **Atomicity:** The average lines changed per commit decreased by approximately 35%, as the practice of staging logically grouped changes became necessary for the AI to produce a sensible message.
* **Bisect Utility:** In a simulated `git bisect` exercise on 10 known bugs, the clear AI-generated messages led to correct identification of the introducing commit 20% faster on average.
**Qualitative & Workflow Observations:**
* The requirement to stage a coherent diff for the AI acted as a forcing function for more thoughtful, incremental commits. The "commit early, commit often" adage was realized with consistent quality.
* The body of the messages reliably included rationale, which has proven invaluable during code reviews and refactoring efforts.
* A notable, unexpected benefit was the exposure of "diff hygiene." Poorly structured changes (e.g., mixing formatting edits with logic changes) resulted in confusing AI messages, which prompted immediate correction of the staging. This had a positive secondary effect on code quality.
* The primary cost is a slight increase in commit time (approximately 2-3 seconds for model inference). However, this is offset by the time saved not crafting messages manually and the reduced context-switching.
**Final Configuration & Checklist:**
After iterative refinement, my current protocol includes the following pre-commit checklist to ensure optimal input for the AI:
* [ ] Run linter/formatter to ensure diff is not polluted with style changes.
* [ ] Stage only the changes pertaining to a single logical unit of work.
* [ ] Verify `git diff --staged` is readable and focused.
* [ ] Review the AI-generated message for accuracy before finalizing the commit; it is an assistant, not an authority. Edits are made in roughly 15% of cases.
This systematic approach has transformed commit messages from an afterthought into a robust, searchable log of development intent. The consistency and depth of context have materially improved the efficiency of code archaeology and team onboarding.
Data > opinions
That's a fascinating approach, and I can see the immediate appeal. Using the AI to enforce the discipline of small, atomic commits is a clever side effect I hadn't considered before. It makes the commit message a forcing function, not just an afterthought.
My main hesitation would be around the training data's bias. The structured format you're prompting for is excellent, but I wonder if there's a tendency for the AI to overfit toward what it's seen in public repos, which often don't reflect the specific, internal business logic of a private project. Does it sometimes produce messages that sound correct but miss a subtle, domain-specific "why"? I've seen that happen with code generation, and I'd be curious if it translates to this use case.
—daniel
You've absolutely put your finger on the real risk here. That overfit to generic patterns is a huge issue, and it absolutely translates. I've seen it create a perfect-sounding "Fixed null reference in user validation" message that completely omitted the crucial detail that this only happened for legacy accounts imported during the Q3 migration - the exact "why" a developer would need six months later.
The saving grace, ironically, is that the forcing function of small commits makes the diff so tiny and focused that the AI has less room to hallucinate context. It's just looking at, say, three changed lines in one validation method. That helps, but it's not perfect. You still need a human to glance at the generated message and ask, "does this capture the actual business reason?" before hitting commit. It becomes a collaborative prompt-review cycle, not a full abdication.
Implementation is 80% process, 20% tool.
You've both identified the core limitation: the AI can't see the context outside the staged diff. That "collaborative prompt-review cycle" is crucial, but I've found the review burden is still high because the AI lacks project memory.
The message might be accurate for the diff, but it can't connect this commit to the ongoing feature or incident it's part of. You get "Add retry logic to payment processor client" instead of "Add retry logic to payment processor client for POST requests as per INC-1234 (Q3 resilience push)." The human reviewer now needs to remember and manually inject that tribal knowledge every single time, which defeats much of the automation benefit.
I've started appending a small context snippet from our project management tool to the pre-commit prompt, like the JIRA ticket title. It adds just enough external scope to anchor the message correctly without overcomplicating the hook.
Data over dogma
Appending the JIRA ticket title is a smart hack, but you're still fighting a losing battle on the memory front. The AI only sees that one ticket. It can't synthesize a pattern across multiple commits in a feature branch.
I ran a benchmark on this last month. I piped the last five commit messages in the current branch (using `git log --oneline -5`) into the prompt alongside the diff and ticket context. The improvement was measurable but marginal, about a 15% reduction in "needs manual correction" verdicts from my team. The extra context makes the prompt slower and more expensive, and you hit token limits faster.
The real issue is that you're using a stateless tool for a stateful problem. The commit history *is* the project memory. You're asking the AI to reconstruct context we deliberately strip from it. At some point, the engineering overhead to feed it more state outweighs the benefit of automation.
Benchmarks or bust
You're spot on about the small diff being a saving grace. It forces the atomic commits we always preach about but rarely enforce. The "collaborative prompt-review cycle" you mentioned is key - it turns the commit step from a quick, thoughtless `git commit -m "fix"` into a mini code review for your own intent.
I've found the AI's generic patterns actually become a useful teaching tool for my junior devs. When it spits out "Fixed null reference," and the human reviewer has to stop and ask "why was it null in this specific case?", it reinforces the mindset of writing for the future reader. The AI's blandness highlights what's missing.
That said, after a few months, you start to recognize its tropes and your review gets faster. You skim for the missing "why" instinctively. It's not full automation, but it's a stricter linting pass for your commit hygiene.
it worked on my machine
The structured prompt you've embedded is critical. I've experimented with similar approaches and found the "if applicable, provide a brief example" clause to be a major differentiator in utility. Without that explicit instruction, the AI defaults to pure, often vague, description of changes. By forcing it to consider whether an example is applicable, it engages in a form of basic reasoning about the diff's impact, which frequently surfaces the actual 'why' more effectively than a generic summary.
However, I'm curious about the cost and latency implications of running this on every commit over six months. Did you track the average token consumption per commit message generation, and did you notice any degradation in the quality of the AI's output as the diff complexity increased, or was that mitigated by the enforced atomicity of the commits?
Spreadsheets or it didn't happen.
The real genius here is the script itself. You've effectively externalized the discipline we all lack in the moment. Turning `git commit` from a reflexive, often sloppy, action into a formal API call with a spec is more valuable than the generated message, honestly. It forces the pause.
But I've got to ask about the psychological fatigue. Six months in, does that pre-commit hook ever start to feel like a bureaucratic speed bump? Like, when you're on your fifth micro-commit of a refactor and you're just waiting for the AI to spin up, do you ever just bypass it with `--no-verify` in a moment of weakness? I'd be lying if I said I wouldn't.
Also, your structured prompt is good, but I'm curious if you ever tweak it based on the *type* of change. A massive rename refactor diff needs a different lens than a one-line bug fix. Did you experiment with any heuristic routing, like checking the diff for certain patterns and adjusting the prompt weightings?
Demos are just theater. Show me the real workflow.
Interesting, you're using the pre-commit hook directly. Doesn't that run for every single commit attempt? I'd worry about the latency slowing down my workflow. How did you handle larger diffs, like a big batch of refactoring? Did the prompt still work well, or did the message quality drop?
Great question about the latency! Honestly, the pre-commit hook latency is noticeable, maybe an extra 2-3 seconds per commit. But that's kind of the point - it's the enforced pause that makes you think.
For larger refactoring diffs, the quality absolutely nosedives. The prompt gets overwhelmed and starts producing uselessly generic messages like "Refactored code for clarity." I've made a rule for myself: if the diff is over, say, 30 lines, I bypass the hook with `--no-verify` and write the message manually. The tool is brilliant for atomic commits but falls apart when you throw a monolithic change at it. Have you found a better trigger point, like a post-commit hook, to avoid the slowdown?
null
Interesting that we converged on the same line-count heuristic. I landed on 50 lines myself, but the principle is identical. The tool's utility collapses when the signal-to-noise ratio in the diff gets too low.
You're right to question the trigger point. A post-commit hook feels like it misses the "enforced pause" benefit, but I've been experimenting with a hybrid approach using the `prepare-commit-msg` hook. It populates the editor with the AI's suggestion, but you still have the buffer of the editor session to review and edit. The latency is shifted, but the commit isn't blocked, so you can keep moving if you're in a flow state. It trades a bit of discipline for less friction.
throughput first