Good to see you sharing the actual script, makes the discussion concrete. The community's really drilling into the edge cases and versioning, which is crucial.
You mentioned it flattens fields like `industries_targeted` and `associated_actors`. That's the kind of output I'd want for my procurement checklists. But from a vendor management view, I'd add a step to flag any normalized entry where *all* your critical output fields (maybe you pick 3-5) are null or empty. That's a signal the mapping might be broken for a new object type, even before a version change.
Have you thought about how you'd log that for a weekly review? A simple count of "incomplete" outputs would be a great health metric.
Ask me about my RFP template
You're totally right about making the threshold field-dependent. A 10% default rate for `ttp_list` means your playbook is missing a huge chunk of context.
For storing variance, I'd push those raw counts and timestamps into a git-tracked CSV as part of the pipeline. Makes it trivial to pull up a quick graph during an incident review or a vendor call. That's the hard proof you need to stop a slow creep from becoming a permanent problem.
What do you think about tagging alerts based on the field's criticality? `ttp_list` jumping 5% could page someone, while `reporting_org` at 10% might just be a weekly dashboard metric.
git push and pray
Love the focus on flattening for playbook triggers. That predictability is key.
But I'm curious about your "few simple mappings." How are you handling empty fields? If `associated_actors` is null in the raw data, does your output list it as an empty array or just drop the key? My playbooks broke once because I expected the key to always be present.
Also, have you seen any weirdness with the `industries_targeted` list formatting across different object types yet?
Demo or it didn't happen