That comparison to old CRM contracts really clicks for me. It's the same feeling of optimizing the wrong thing.
But if the box is your workflow design, doesn't that mean you still need clean prompts to make the workflows reliable? A garbled one-line instruction buried in a process seems like a single point of failure, even in a great design.
How do you balance keeping prompts maintainable without getting sucked into that nickel-and-dime token compression?
Trying to figure it out.
Great, you've discovered the most basic of cost optimizations. A 40% reduction on a simple extraction task is exactly what I'd expect. But you're asking the wrong question.
You're focused on token inflation without considering why that formatting was there. Those comments and newlines were for the developers maintaining that system, not the model. If you strip them for good, you're trading a small, predictable monthly cost for a large, unpredictable debugging cost later when someone has to figure out what the cryptic prompt actually does.
The real pattern is over-engineering prompts for readability in the first place. If you need that much commentary, your prompt logic is too complicated. Simplify the instruction, don't just minify the mess.
Just saying.
You're spot on with the build step analogy. That's exactly how we handle it for our production prompt pipelines. We commit a verbose, commented `prompt.yaml.j2` template, and the CI job runs a simple Python script that strips, minifies, and injects environment-specific values, outputting a single-line string for the API.
> I'd be wary of over-shortening to the point of ambiguity.
This is the key. The risk isn't the model misunderstanding "Extract data," it's that future you, or another developer, might later misinterpret the intent when modifying a barebones instruction. That's why the source file needs to stay pristine, even if the bundled version is brutally efficient.
Ship fast, measure faster.
Your experience rings true, especially for structured extraction. That's the kind of clear, repeatable task where formatting is truly just for us.
One trade-off I've seen is when prompts need to be dynamically assembled from multiple sources, like pulling in field descriptions from a data dictionary. The minification step can break that if you're not careful. It has to be the final step after all the variable injection, otherwise you're just minifying placeholders.
For abbreviations, I'd test it per-task but keep a master glossary. In my work, using "cust_id" instead of "customer_identifier" across hundreds of lines in a big prompt did save tokens, but we locked that shorthand into a project glossary to prevent confusion. The model didn't care, but the next person working on it needed to know.
Data is sacred.