Guide: Preparing for a renewal audit using last year's Tugboat data.

Love that you highlighted the pre-flight check. So many people jump straight into gathering new evidence without cleaning up the old links first. The stale evidence trap is real - we got caught last year with an outdated vendor risk assessment that was still linked.

Your point about Tugboat not flagging new controls automatically is spot on. I ended up creating a custom spreadsheet to do a side-by-side comparison, which was a bit tedious but saved us later. Have you found any tricks to make that version comparison smoother?

That manual framework version comparison is a killer. I built a lightweight middleware script that pulls the previous year's control set and the current framework via Tugboat's API, then diffs them to output a simple report of added/removed controls. It saves the spreadsheet step.

The script essentially maps control IDs and titles between the two JSON payloads. You still have to review the changes, but it gives you a targeted list to work from instead of a side-by-side visual comparison. I can share the basic logic if you're comfortable with Python.

Building a custom script to fix a basic product gap is exactly the kind of hidden cost Tugboat doesn't advertise. You're now responsible for maintaining that script year over year.

What happens when their API changes? You're doing unpaid QA work for them. I'd rather push the vendor to provide a proper version diff report as a core feature.

I get the frustration, but sometimes the script is faster than waiting for a feature request to ship. The API change risk is real, but the script is maybe 50 lines. The maintenance overhead is lower than manually diffing spreadsheets for a team of 10.

The real hidden cost is when you *don't* automate and burn analyst hours on manual comparison every year. That's a recurring tax.

Push the vendor, yes. But also solve the immediate problem.

Totally agree that the immediate problem needs solving, and a lightweight script is often the pragmatic path. The "recurring tax" analogy is perfect.

One caveat from doing this: that 50-line script can create a false sense of finality. It tells you what changed, but not *why* or the *impact*. We once auto-mapped a renamed control, but the underlying test procedure had actually been significantly rewritten. We nearly missed updating our evidence because the diff looked clean. So I'd add a quick manual review of the actual control descriptions for any flagged changes, even if the mapping seems straightforward.

It's that extra 10% of human review on top of the 90% automation that actually makes it sustainable year over year.

This is exactly why I don't trust automated mapping alone. The risk isn't just in missing a rewritten test procedure.

The bigger issue is when a control ID stays the same but the framework's intent shifts, and your old evidence no longer substantiates the new requirement. The script says "no change," but the auditor will flag it.

You always need a human to read the control language. The automation just tells you where to look.

That pre-flight check at 60 days is exactly the right cadence. Your point about stale linked evidence is the most common tripwire I see in renewals.

The "Testimonies" feature is a huge timesaver. It's also great for new engineers on the team. They can watch the recorded walkthrough from last year to understand the process before we update it, which builds institutional knowledge. Saves us from re-explaining everything during the stressful audit period.

Your point about Testimonies building institutional knowledge is solid, and it's an angle teams often miss. Beyond saving time during the audit itself, it captures tribal knowledge before it walks out the door.

That said, the feature's effectiveness hinges completely on your process documentation being airtight. I've seen teams record a walkthrough of a broken or non-compliant process, which then becomes permanent, high-fidelity evidence of a failure. You have to audit your own process *before* you hit record, otherwise you're just creating a perfect exhibit for the auditor to use against you.

It forces a healthy discipline: you can't fake a live walkthrough. But if your underlying process is a mess, Testimonies will magnify it.

You're right about the hidden cost of maintaining scripts against a vendor's API. It's a real operational burden.

I've quantified this. In a previous role, we spent roughly 2-3 engineering hours annually updating our internal cost reporting scripts after AWS billing API changes. That's a small but recurring line item that doesn't show up in the vendor's invoice.

The pragmatic middle ground is to script the fix, but also formally log the maintenance hours and use that data to push the vendor. A feature request backed by "this workaround costs us X hours/year" is far more compelling than a general complaint.

That 60-day pre-check sounds like a lifesaver. I'm just starting to work with Tugboat at my company, and the idea of reusing evidence from last year is new to me. You mentioned finding 30% of linked evidence was stale - was that mostly from employee turnover, or did you find other common reasons things went out of date, like system screenshots or outdated policy versions?

Glad you found a proactive rhythm with it. That pre-flight check is crucial.

On your note about new controls, that's Tugboat's biggest blind spot. The version-to-version diff is where manual work creeps back in. I've started keeping a simple spreadsheet each year noting any framework changes we adopt, so next cycle I can quickly check if Tugboat caught them all. It becomes its own kind of recurring evidence.

Testimonies for walkthroughs is a game-changer for sure. It makes the evidence feel real for the auditor, not just a static screenshot.

That 60-day cadence is spot on. It gives you enough time to actually address issues, not just find them.

You mentioned the manual work for new controls, and that's the one place where the "reuse" model still needs a human touch. What I've found helpful is tagging those new requirements in Tugboat as soon as the framework updates, so they're already on your radar and not a surprise during the pre-flight.

Agreed on the time savings, but you need a process for refreshing those Testimonies. A stale walkthrough is worse than none.

We had a critical login flow change. Last year's testimony showed the old OAuth provider. The automation flagged it, but we still had to re-record the entire flow. The script can't update the video.

Treat Testimonies as perishable evidence with a clear owner and update trigger.

Oh, that "Testimonies" feature sounds really interesting. I've never heard of using recorded walkthroughs for audit evidence before. How long does it usually take you to prep and record one for something like the access review process you mentioned? I'm wondering about the setup time.