Best open-source SIEM for a Python-heavy AWS stack in 2026

Given the accelerating convergence of Python-based data processing (Lambda, Glue, Airflow) and infrastructure-as-code (CDK, Terraform) within AWS environments, the traditional agent-centric SIEM model is increasingly misaligned. By 2026, the primary challenge won't be log collection, but rather the contextual correlation of ephemeral resource metrics, application trace data, and CloudTrail events with custom security logic defined in Python. This necessitates a SIEM that is inherently programmable and can natively integrate with the AWS observability suite and modern Python instrumentation libraries.

Therefore, the core evaluation criteria for an open-source SIEM in this context must be:
* **First-class Python SDK/API:** The ability to ingest, query, and manage security content via Python, not just REST calls. This allows security logic to be embedded within CI/CD pipelines and data transformation jobs.
* **AWS Native Telemetry Ingestion:** Direct, efficient integration with Amazon Security Lake, OpenTelemetry Collector, and VPC Flow Logs (Parquet format) without requiring heavy forwarders.
* **Composability with Existing Python Security Tooling:** The SIEM should act as a correlation engine for findings from tools like `Bandit`, `TruffleHog`, `Prowler`, and custom `boto3` audit scripts, not replace them.
* **Stateful Processing for Serverless:** Ability to maintain and query ephemeral state related to Lambda execution chains and Step Function workflows across multiple log events.

Based on these parameters, the two most viable contenders appear to be **Apache Metron (incubating)** and **Wazuh**, though both require significant adaptation.

**Apache Metron** offers a compelling data pipeline architecture. Its parsing and enrichment topology can be extended with Python UDFs, and it can leverage Amazon Security Lake as a primary data source. However, its development velocity is a concern. A proof-of-concept enrichment script for Lambda metadata might look like:

```python
# Example using PySpark within a Metron enrichment topology (simplified)
def enrich_lambda_event(event):
from botocore.client import Config
import boto3
client = boto3.client('lambda', config=Config(read_timeout=300))
# Enrich log event with concurrent execution metrics
function_metrics = client.get_function_concurrency(
FunctionName=event['function_name']
)
event['reserved_concurrent_executions'] = function_metrics['ReservedConcurrentExecutions']
# Cross-reference with prior events from a state store
if is_rapid_invocation(event):
event['threat_score'] = 0.85
return event
```

**Wazuh** provides robust AWS integration out-of-the-box and a more active community. Its major advantage is the lightweight agent, which can be deployed as a Lambda layer for runtime introspection. The limitation is its primarily JSON-based rule system, which becomes cumbersome for complex, multi-service correlation logic. Integrating Python analysis requires deploying custom `active-response` scripts or piping alerts to an external Python service, adding latency.

The critical open question is whether a purely log-based SIEM can suffice, or if we should be evaluating a composite architecture: using **OpenTelemetry for trace/span collection**, a **time-series database (VictoriaMetrics) for metric anomaly detection**, and a **rules engine (like Apache Flink with a custom Python wrapper)** for stateful correlation, with a lightweight dashboard like **Grafana** for visualization. This approach sacrifices integrated incident management for greater programmability and cost control.

I am currently modeling the event throughput and cost implications of both the integrated SIEM and composite approaches for a stack comprising 300 Lambdas, 50 ECS Fargate tasks, and a significant MSK-based data pipeline. I would be particularly interested in any production experiences integrating OpenTelemetry Python instrumentation (auto-instrumentation or manual) directly into a security correlation workflow, or benchmarks on the per-event processing latency of Wazuh's Python active-response module at scale.