I've been conducting a systematic evaluation of ContentBot's various functions for a data quality documentation project, and I've hit a persistent, data-backed issue with the `rewrite` function. While it's excellent for fixing minor grammatical errors, its tendency to increase sentence length and complexity is counterproductive for technical writing. My analysis shows a consistent pattern where conciseness and clarity are sacrificed for what appears to be a misguided sense of 'completeness' or 'formality.'
To illustrate, I ran a controlled test using a corpus of 50 original technical sentences from our internal wiki. I processed each through the `rewrite` function with the goal of "improving clarity." I then calculated key metrics before and after:
* **Average sentence length:** Increased by **42%** (from 18.2 words to 25.9 words).
* **Flesch Reading Ease score:** Decreased (became more complex) by an average of **12 points**.
* **Incidence of passive voice:** Increased by **30%**.
Here is a concrete, representative example from my test log:
**Original:**
```sql
-- The dbt model failed because the upstream table's schema changed.
```
**Rewritten by ContentBot:**
```sql
-- The dbt model encountered a failure due to the fact that the schema associated with the upstream table was altered.
```
The rewrite introduces redundant phrases ("encountered a failure," "due to the fact that," "associated with") and shifts to passive voice ("was altered"). The original is direct, active, and perfectly clear. The rewrite is objectively worse for a technical audience.
This creates significant workflow friction. The core promise of a `rewrite` function, particularly for analytics engineers and technical writers, should be to *refine* and *clarify*, not to *inflate*. I now find myself spending more time editing the rewrite output than I would have spent polishing the original draft.
My questions for the community are:
* Have you observed similar behavior, and in what contexts (blog posts, code comments, documentation)?
* Are there specific prompts or parameter tweaks you've found that mitigate this tendency? I've tried directives like "make more concise" with mixed results.
* Could this be a byproduct of the training data favoring longer, more 'polished' prose over technical precision?
I plan to continue my analysis, perhaps comparing the `simplify` or `shorten` functions against this `rewrite` behavior. For now, I've had to deprioritize the `rewrite` function for my core use case, which is a shame.
- dan
Garbage in, garbage out.
Interesting data. That 42% increase is a lot. In my CRM documentation, clarity is king for user adoption.
Do you think the problem is the "improving clarity" goal itself? Maybe it interprets that as needing to be more explicit and explanatory, which adds words. Have you tried a more direct instruction like "make more concise"?
Trying to figure it out.