Hey folks, I've been experimenting with Aider over the last few weeks, mostly for code generation and refactoring tasks in our AWS Lambda projects. It's been a game-changer for that. But I've hit a bit of a wall trying to use it for a different purpose: generating and updating our technical documentation.
We have a mix of READMEs, architecture decision records (ADRs), and internal runbooks. The dream was to point Aider at our existing docs and a new feature's code, then have it draft updates or even create new documents. The reality has been... mixed.
Here's a concrete example. I tried to get it to expand a runbook for a new S3 event processing flow. I fed it the existing runbook template and the new Python handler code. The initial draft it produced had the right structure but kept hallucinating steps that didn't exist in the actual code, like referencing environment variables we never defined.
```bash
# My typical command pattern
aider --file runbook.md --file lambda_handler.py
> "Update runbook.md to include the new S3 failure handling logic from lambda_handler.py"
```
It excels at the mechanical parts—like inserting code blocks with correct syntax highlighting or reformatting tables. But for maintaining factual accuracy about *what the system actually does*, it required so much manual correction that I'm not sure it saved time.
Has anyone else pushed Aider into this technical writing space? I'm particularly curious about:
- Strategies for keeping generated documentation truthful to the codebase.
- If using smaller, more frequent prompts (like "just update the error codes section") works better than asking for a full document.
- Whether mixing in architecture diagrams (like a Mermaid diagram description) has worked for you.
I love the idea of automating doc upkeep, as our team's docs are the first thing to rot when we're under pressure. But I'm starting to think LLMs might need a tighter feedback loop with the code than Aider currently provides for this use case. Maybe a multi-step process of code summarization first, *then* doc generation? Keen to hear your experiences.
cost first, then scale
Yep, same experience. The problem is it's just a really fancy auto-complete trained on code. Docs require a different kind of reasoning - connecting intent to actual implementation. It can't do that.
You can sometimes force it to work by feeding it smaller, more atomic prompts. Instead of "update the runbook," try "list all environment variables used in lambda_handler.py" and then "add a troubleshooting section for error X." It's more manual, but you avoid the worst hallucinations.
Honestly, for runbooks, you're better off with a simple template and a checklist. No AI needed.
Keep it simple
I've pushed Aider into similar territory for generating ADRs and high-level architecture overviews, and found a critical success factor is providing it with extremely structured inputs. The hallucination issue you're seeing, like the phantom environment variables, is predictable because Aider lacks the runtime context of your actual deployment.
What worked for me was creating a strict, data-first prompt chain. Before asking it to touch the runbook, I'd extract the concrete facts into a separate temporary file. For example, I'd run a command to parse the Lambda code and output a JSON manifest of actual handlers, IAM permissions, and environment variable keys, then feed *that* to Aider alongside the template. You're essentially using it as a formatting and templating engine at that point, which it's good at, rather than asking it to infer semantics.
It's an extra step, but it bypasses the guesswork. The real value isn't in full automation, it's in reducing the friction of taking those manually curated facts and assembling them into the correct prose and structure. Have you tried pre-processing your source files into a structured context file before the main Aider call?