Hot take: The vendor over-promises on 'AI' and under-delivers on basic UX.

I've been conducting a long-term performance evaluation of our security stack's orchestration and automation layer, with ThreatConnect as a central component under scrutiny for the past 18 months. While my primary focus is on quantitative metrics, the qualitative user experience directly impacts analyst velocity and, by extension, mean time to respond (MTTR). My assessment leads me to a concerning conclusion: the platform's heavy emphasis on its "AI-driven" features is not only misleading but appears to have come at the direct cost of foundational usability and stability.

Let's deconstruct the "AI" promise. The platform markets features like "predictive prioritization" and "natural language processing" for indicator enrichment. In controlled benchmarking against our own deterministic playbooks, the so-called AI scoring consistently failed to outperform our rule-based priority assignments. More critically, the black-box nature of these scores introduces unquantifiable variance. When I instrumented the API to log scoring inputs and outputs over a 30-day period, the results showed high inconsistency for similar threat clusters.

```json
// Example of API log variance for similar indicator clusters
{
"date": "2023-10-05",
"indicators": ["malicious-domain-a.com", "185.163.45.22"],
"ai_score": 87
}
{
"date": "2023-10-12",
"indicators": ["malicious-domain-a.com", "185.163.45.22"],
"ai_score": 64 // Same cluster, different day, significant delta.
}
```

This inconsistency becomes a performance liability. Meanwhile, basic UX workflows remain fraught with latency and unnecessary complexity. For instance:

* **Dashboard Load Time:** The main analyst dashboard, a critical path for daily operations, takes an average of 8.2 seconds to become interactive in our environment. This was measured via synthetic browser monitoring (using k6 browser modules). A comparable Grafana dashboard for our threat feeds loads in under 1.5 seconds.
* **Playbook Editor Lag:** The graphical playbook editor exhibits severe input latency (over 300ms per node interaction) when a workflow exceeds 15 nodes. This forces analysts to work in smaller, less efficient fragments or revert to the API, negating the visual editor's value.
* **API Rate Limiting & Consistency:** The REST API's rate limiting is unusually aggressive for internal automation. More problematic is the paging inconsistency; the `page[size]` parameter is not reliably honored across different endpoint groups, requiring custom error handling and retry logic in all our integrations, which adds to processing latency.

The resource allocation seems misplaced. The computational overhead for the "AI" features is non-trivial (observed as a 15-20% higher baseline CPU usage on application nodes compared to when these features are disabled), yet the ROI is unproven and unmeasurable in our context. This overhead likely contributes to the sluggish UI and API performance. I would posit that engineering effort would be better spent on:
1. Stabilizing and optimizing the core data ingestion and indexing pipelines.
2. Providing a true low-code, high-performance playbook engine with deterministic execution paths.
3. Delivering a consistent, well-documented, and performant API.

Has anyone else performed systematic load testing or benchmarking on ThreatConnect's core operational interfaces? I am particularly interested in comparative latency metrics for indicator bulk creation, playbook execution times under load, and the stability of the WebSocket connections for real-time dashboards. My data suggests a significant gap between marketed capabilities and the operational reality faced by performance-conscious teams.