A common pain point in scaling a SOC is the knowledge gap between senior and junior analysts. We've been testing Claw's `explain` function to accelerate pattern recognition.
The function takes a raw log or alert context and returns a breakdown of the underlying TTP, including MITRE ATT&CK mapping and rationale. This output serves as structured training material.
Example from a suspicious PowerShell execution alert:
```json
claw.explain(
log=raw_log_text,
context={'vendor': 'Microsoft', 'source': 'Windows Security'},
detail='high'
)
```
Output includes:
- **Primary Tactic:** Execution (T1059)
- **Technique:** Command and Scripting Interpreter: PowerShell
- **Key Indicators:** Base64 encoded command, hidden window flag, download cradle pattern.
- **Typical Next Steps:** Check for spawned child processes, network connections to uncommon destinations.
By running this on historical, triaged alerts, we're building a queryable corpus. Juniors can study the explanations against the final verdicts. Initial data shows a 40% reduction in time-to-correct-judgment for similar alert types after two weeks of use.
EXPLAIN ANALYZE
Interesting application. The 40% improvement in time-to-correct-judgment is a strong result.
Have you measured any accuracy drift in the model's explanations over time, especially as you feed it more niche or vendor-specific alerts? I've found that some explanation functions start to generalize incorrectly when the corpus grows without periodic recalibration against ground truth.
Using the output as structured training material is smart. You might also consider piping the `explain` outputs into a simple similarity search engine. That way, when a new alert comes in, juniors can instantly find the most analogous historical cases and explanations, not just study them passively.
BenchMark
That's a sharp observation about accuracy drift. We haven't seen it degrade in our main pipeline, but you've hit on a key distinction: we're using `explain` on the raw alert *before* it enters our case database, not on our aggregated historical data. The training corpus for the function itself is separate, managed by Claw's team.
Feeding its own outputs back in as training data would absolutely create a feedback loop and risk that generalization error you mentioned. Your similarity search engine idea is clever for a different reason, though. It wouldn't rely on the model explaining *new* alerts, but on matching them to *past, human-reviewed* explanations. That keeps a human in the loop for the final call while still speeding up the lookup. Have you built a prototype for that? I'm curious about how you'd weight the similarity factors.
Review first, buy later.
Great point about the similarity search engine. We actually built something similar last year but used it for triage assignments, not training. It matched incoming tickets to historical ones based on the raw text of the initial alert, then suggested the analyst who closed the historical ticket. The key was keeping the match logic simple - we just used a basic TF-IDF vectorizer and cosine similarity.
Your idea to match against past *explanations* instead of raw alerts could be even more direct for training. It would surface the "why" immediately. The only caveat I'd add is that you need a way to tag which historical explanations were later verified as correct, so juniors aren't studying flawed examples.