I've been benchmarking Anomali against other SIEM tools for a structured logging use case, and their advertised "300+ integrations" was a key factor in the selection. After a month of implementation, I have to conclude that this list is largely a marketing feature. A significant portion of these connectors are either deprecated, poorly maintained, or simply do not work as documented.
In my testing, I attempted to set up ten integrations from their catalog for common cloud services. Of those ten:
* Three failed to authenticate with the current API versions, throwing `InvalidGrant` or `404` errors that pointed to outdated endpoint URLs.
* Two pulled data but used a default schema that ignored critical event-specific fields, flattening everything into a generic `event_data` JSON blob—useless for proper alerting.
* One required a custom Python script to act as a bridge, which defeats the purpose of a pre-built connector.
For example, their documented configuration for a specific SaaS tool looked like this:
```yaml
connector: saas_audit_logs
version: 2.1
auth:
api_key: !env SAAS_KEY
endpoint: https://api.saastool.com/v1/logs
```
The actual API endpoint had moved to `/v3/logs/audit`, and the authentication required OAuth 2.0, not a simple API key. The provided documentation was two years out of date.
This pattern suggests they are counting every minor variant or legacy version as a separate "integration." The operational burden falls on the analytics or security engineering team to troubleshoot and maintain what is sold as a turnkey solution. From a data warehousing perspective, this creates unreliable data pipelines. You can't build robust dashboards or alerts on streams that break with every vendor API update.
Has anyone else done a systematic evaluation? I'm particularly interested in the state of their cloud infrastructure connectors (AWS CloudTrail, Azure Audit Logs, GCP Admin). My benchmark shows a 40-50% failure rate on initial setup for non-"tier one" services. If this is the community's experience, the effective integration count is far lower than advertised.
> the advertised "300+ integrations"
Yeah, and I bet the other half require you to run their proprietary middleware on-prem to "facilitate" the connection. Classic.
Real integration means a stable API spec, not a pile of abandoned config files. Their sales deck probably still lists support for that SaaS tool's v1 API.
Ever tried their Kubernetes log collector? It's just Fluentd with a broken custom plugin and a 300% markup. You're better off writing your own connector - at least you'll know when it breaks.
Ouch, that endpoint example hits close to home. I ran into the same thing last week with a different connector - the docs still pointed to a `/v2` endpoint that's been sunset for a year.
How do you even vet these integrations before committing to a tool? Is it just trial and error, or is there a trick to spotting the abandoned ones?
Vetting these integrations systematically is indeed possible, but it requires moving past the marketing materials. I treat any vendor-provided integration list as an unvalidated hypothesis, not a feature set.
My approach is to design a small, reproducible benchmark for the specific integration I need. Before committing to a tool, I'll request a temporary instance or a trial and attempt to:
* Stand up the connector using only the public documentation.
* Generate a known data set (e.g., 10,000 structured log events) from the source system.
* Measure latency, schema fidelity, and failure rate over a 48-hour period.
The broken endpoints and deprecated schemas surface immediately under this load. If the vendor cannot facilitate this basic validation for their "key" integrations, it's a strong indicator about the health of the other 290+ on the list. The abandoned ones consistently fail the schema fidelity check, mutating the event structure.
Ultimately, you're not vetting the integration, you're vetting the vendor's engineering and documentation update cycle. A stale doc is a major red flag, but it's just the surface symptom.
-- bb42
That endpoint example is spot on, and I've seen the same pattern with the generic `event_data` blob. It usually means the integration is using a legacy "catch-all" parser that hasn't been updated for the source's actual schema changes.
One extra step I take now is to check the connector's last-modified date in their public GitHub repo or config store, if available. It's a dead giveaway - I've found connectors touted as "supported" where the YAML file hasn't been touched in 2+ years. The ones that just wrap a generic Fluentd or Logstash plugin are always the culprits for flattening data.
Your point about the custom Python script is the kicker. When a "pre-built" connector requires you to maintain a script anyway, you've just inherited a liability instead of gaining a feature.
— francesc
You're right about the last-modified date being a useful signal, but I've found it can be misleading in curated repositories. A stale YAML file might indicate abandonment, but I've also seen vendors push silent updates to a core ingestion library that all connectors depend on, leaving the individual connector configs untouched for years. The broken schema mapping persists, but the "last updated" timestamp looks fresh.
The deeper issue with the generic `event_data` blob isn't just legacy code, it's a strategic choice to avoid schema maintenance. By dumping everything into a single JSON field, the vendor offloads the parsing responsibility onto your analytics team downstream. You're forced to write and maintain the normalization logic they advertised as pre-built. It transforms their integration from a product feature into a bare transport layer.
This pattern often correlates with connectors built by professional services during a one-off engagement, then later added to the general catalog without a plan for ongoing support. Checking the commit history for that specific connector's directory, rather than just the file date, usually tells that story more clearly. Look for a burst of activity two years ago and then nothing but README updates.
—BJ
Your example with the endpoint shift is the core of the problem. It's not just broken, it's a documentation and maintenance failure. A vendor listing an integration implies they're tracking the source API. If they aren't even updating URLs, the "300+" number is meaningless. It becomes technical debt on arrival.
—AF
> flattening everything into a generic `event_data` JSON blob
That's the pattern. I see this with vendors that acquire other products and just rebrand the old connectors. The schema flattening means they never engineered a proper parser for that source. It's a lazy integration.
Your 3/10 failure rate on basic auth aligns with what I've seen. It's not a technical debt problem, it's a lifecycle process failure. A real integration list should have a publicly visible status for each connector: maintained, community, deprecated. Most won't do that because the numbers would drop by half.
If you're stuck with them, you can force it: try to downgrade the source system's API version to match the connector. If that's not possible, you've just proven the integration is dead. Use that to push back on support or cost.
Metrics don't lie.