Hi everyone, hoping you can help me unpack something from our recent audit prep. I'm the main point person for our Vanta setup, and our external auditor just flagged an issue I wasn't expecting.
They're saying that the read-only user access reports Vanta generates aren't sufficient as audit evidence on their own. The report shows user lists and permissions, but the auditor wants to be able to trace *how* we generated that report and verify the data's source directly. They called it a "system-generated snapshot" without a clear audit trail back to the raw system data (like our IdP or GitHub).
Has anyone else run into this? I thought the whole point of these automated reports was to satisfy this exact requirement.
Our auditor suggested we might need to supplement with direct database queries or logs from the source systems, which seems to defeat the purpose of having Vanta pull it all together. I'm trying to figure out if this is a limitation in how we have Vanta configured, or a common critique of the tool.
For example, our "Users with Admin Access" report from Vanta is essentially a PDF. The auditor is asking: "How do I know this list is complete and current as of the time stated? Can I see the query or the API call that fetched this data?"
I'm comfortable writing SQL, but I'm not sure where to even start to bridge this gap. Do we need to build custom integrations that pull the same data but keep a verifiable log? Or is there a way within Vanta to provide this level of transparency?
Yeah, we hit this exact roadblock last year. The auditor isn't really questioning Vanta's data, they need the provenance - the trail showing the report is built directly from source systems without manipulation in between.
Our fix was to include the "source evidence" links Vanta captures for each finding right in the audit package. For a user report, we'd also snapshot the corresponding IdP admin console page (showing the same user list/roles) from the same date. It's a bit manual, but it bridges the gap.
It feels redundant, but it satisfied our auditor. They just want to see you can trace the output back to a raw system, not just trust the tool's curated PDF.
Data doesn't lie, but dashboards sometimes do.