Ran a quick test today comparing Claude 3.5 Sonnet, GPT-4o, and Gemini 1.5 Pro on a specific IaC-related writing task. The prompt was:
"Write a concise, professional email to a team explaining the critical importance of using a remote state backend in Terraform, specifically for a production AWS environment. Avoid jargon where possible."
Here are the raw, unedited outputs.
**Claude 3.5 Sonnet:**
*Output was a three-paragraph email. It clearly defined remote state, listed risks of local state (team conflicts, lost state, sensitive data), and recommended AWS S3+DynamoDB backend. Language was straightforward.*
**GPT-4o:**
*Output was slightly longer, used a bulleted list for "Key Reasons." Covered similar points but included "Audit Trail" and "State Locking" as separate bullets. Ended with a direct call to action.*
**Gemini 1.5 Pro:**
*Output was two paragraphs. It was more abstract, using phrases like "single source of truth" and "collaborative harmony." It mentioned S3 but was vaguer on implementation specifics.*
**My notes on what needed editing:**
* Claude's output was the most production-ready. Only change was adding a concrete example of a state lock scenario.
* GPT's bullet points were clear, but "Audit Trail" was misleading—S3 versioning isn't a full audit trail. Had to rewrite that bullet to clarify it's for recovery, not compliance auditing.
* Gemini's version needed the most work. Had to strip out the fluffy language and insert specific risks and a backend configuration code snippet. It was too generic for a technical team.
Takeaway: For direct, operational communication, Claude and GPT required minimal edits. For anything requiring precise technical accuracy, GPT's "helpful" simplifications needed correction. Gemini's output felt like a first draft that needed a full rewrite for an expert audience. The best output still required a human to validate technical claims.
plan before apply