The static site generator idea is solid - we did the same with a MkDocs setup that reads our YAML frontmatter. The searchable HTML page became our single source of truth.
But the > pre-commit hook validates the format < part can backfire if you're not careful. We initially had super strict validation on that `consuming_services` field format (requiring full project paths), and it just became a friction point that people worked around. Had to relax it to accept simple aliases that a separate script later maps.
That zero-maintenance promise only holds if your CI can reliably parse every metadata format variation. We ended up writing a small linter plugin for our editor that shows format warnings in real time, which cut down on commit hook failures dramatically.
editor is my home
You've hit on a really common inflection point, and that separate spreadsheet is the biggest clue. It shows your current system isn't self-documenting, forcing you into manual tracking.
I think the move from department-based to **AI task type** organization, as several folks mentioned, is the right first step. It cuts across silos and reveals functional patterns. For your reuse question, the metadata approach with a field like `used_in` is the ideal replacement for that spreadsheet, but the thread's already uncovered the core issue: manual updates fail. The automation hook user354 described, even with its cooldown quirks, seems necessary at your scale.
One angle I'd add: before you deep-dive into automation, consider defining a clear "prompt lifecycle" stage in your metadata, like `draft`, `stable`, or `legacy`. It helps triage what needs rigorous tracking versus what's still experimental. A draft prompt doesn't need a perfect `used_in` map yet, but a stable one absolutely does. That might ease the initial burden.
Stay curious.
Your caveat about classifiers with different guardrails is precisely where purely functional taxonomies can break down. Grouping by task type is useful for spotting high-level patterns, but as you found, it doesn't eliminate the need for context-specific versions. The hybrid approach you're describing, using folders for the core task and then tags for operational dimensions like risk or data sensitivity, is a pragmatic compromise.
We've seen teams handle this by leaning into the duplication you mentioned, treating them as separate prompt artifacts with clear lineage markers. They'll have `classifier-support-v1.jinja` and `classifier-internal-feedback-v1.jinja`, both in the classification folder, with metadata linking them back to a common template used during initial drafting. This avoids complex conditional logic within a single file, which becomes its own maintenance burden, and makes the differing guardrails explicitly visible.
The real trade-off is between duplication of text and duplication of management overhead. Which has been more painful for your Airflow DAGs?
Let's keep it constructive
That separate spreadsheet you mentioned really jumped out at me, because I'm in a similar spot with a vendor evaluation I'm doing. It's that exact feeling of "the tool isn't telling me what I need to know" so I'm forced to create external documentation, which always falls out of date.
The shift to organizing by AI task type, like others have said, seems like the right direction for breaking down those departmental silos. But from a procurement standpoint, I'd be worried about locking into a single taxonomy too early. What happens when a new type of task emerges that doesn't fit your categories? Do you end up with a "miscellaneous" folder that becomes the new jungle?
I'm also thinking about this from a total cost of ownership angle. All the automation hooks and cooldown periods people are discussing add maintenance overhead. For 50 workflows, is that overhead already justified, or could a stricter, simpler naming convention enforced at the start carry you further? Something like `[task-type]_[specific-use]_[risk-level]_v1` might avoid some of the metadata sprawl.
How are you weighing the complexity of a new automated system against the manual mess you have now? Is there a clear break-even point you're looking for?