The prevailing narrative suggests that large language models will revolutionize the generation and maintenance of compliance documentation—think SOC 2, ISO 27001, GDPR, and internal policy frameworks. However, after extensive, methodical testing across multiple platforms, I have arrived at a contrary conclusion: current models are fundamentally unreliable for this purpose, rendering them nearly useless for any serious compliance effort. The failures are not mere edge cases; they are systemic and rooted in the models' core operational mechanics.
The primary failure mode is the generation of **plausible-sounding fabrications.** When tasked with drafting a data retention policy section, for example, a model will confidently produce a detailed clause citing a specific, non-existent subsection of a regulation. It will invent retention periods that sound reasonable ("180 days for audit logs") but may directly contradict the actual legal requirement or your industry's best practices. This hallucination extends to:
* **Fictive Controls:** Inventing security controls or audit procedures that do not exist in the standard being implemented.
* **Misattributed Requirements:** Attributing a requirement to the wrong framework (e.g., stating a PCI DSS requirement is part of GDPR Article 32).
* **Inconsistent Logic:** Generating policy statements that conflict with one another across different sections of the same document.
To illustrate, I prompted a leading model with a task many believe to be within its capabilities. Below is a condensed version of the interaction.
**Prompt:**
"Draft a section for an employee security awareness policy regarding the handling of PII. Reference specific GDPR articles where applicable. Include a procedural step for reporting a suspected data breach."
**Model Output (Excerpt):**
"...Employees must ensure all Personally Identifiable Information (PII) is encrypted at rest and in transit, in accordance with GDPR Article 17(2). Any suspected breach of PII must be reported to the Data Protection Officer (DPO) and the relevant supervisory authority within 48 hours of discovery, as mandated by GDPR Article 33(1). The DPO will then initiate the internal incident response plan, detailed in Annex B of this policy..."
**The Critical Failures:**
1. **Hallucinated Citation:** GDPR Article 17 is the "Right to erasure ('right to be forgotten')." It contains no subsection (2) mandating encryption. The model fabricated a citation to lend false authority to a (otherwise good) security practice.
2. **Procedural Oversimplification:** The 48-hour reporting timeline is correct for Article 33, but the model omits the crucial nuance that this is "without undue delay," with 72 hours as the maximum, and only if the breach is *likely to result in a risk to the rights and freedoms of natural persons*. It fails to mention the requirement for a reasoned justification if the notification is delayed beyond 48 hours.
3. **Referential Hallucination:** It references a non-existent "Annex B" of the policy, creating a false dependency and an incomplete control loop.
The danger here is not a simple error, but a **compounded, authoritative-sounding inaccuracy.** A compliance officer without deep, immediate recall of the GDPR text could easily be misled. The document *appears* rigorous and specific, yet it is built on a foundation of false references. In an audit, such a document would be worse than useless—it would be evidence of a misunderstanding of the regulatory landscape.
Therefore, these models cannot be trusted as authoritative sources or even as reliable drafting assistants for compliance. Their utility is limited to very early-stage brainstorming or templating for well-understood, generic policy language, where every single output must be meticulously validated line-by-line against the primary source material. The resource expenditure required for this validation often negates any efficiency gains. For now, human expertise, coupled with vetted templates and robust governance workflows, remains the only viable path for compliant documentation.
Method over hype