My team is currently evaluating ingestion pipelines for a multi-vendor, hybrid security environment, and the Common Event Format (CEF) connector for on-premises Fortinet devices is a critical component. Initial deployment was driven by the promise of a standardized, agentless collection method, but our operational experience over the last eight months has revealed significant deviations from the documented stability.
The core architectural premise—syslog forwarding to a Linux machine running the AMA (Azure Monitor Agent) and the CEF connector—introduces several potential points of failure. Our primary issues have been:
* **Buffer Management and Log Spikes:** During targeted penetration tests, our FortiGate firewalls generated event bursts exceeding 15,000 EPS. The default `rsyslog`/`syslog-ng` buffer configurations on the collector VM were insufficient, leading to dropped messages. This required manual tuning of ring buffer sizes and `imuxsock` parameters.
* **Connector Health Monitoring Gaps:** The connector's status in the Azure portal (`Sentinel > Data connectors`) often shows as "Connected" even when the local `cef` service on the collector is in a failed state. We had to implement a secondary monitoring layer via custom queries on the `Heartbeat` table and agent-local health scripts.
* **CEF Field Mapping Inconsistencies:** While standard fields like `deviceVendor` and `deviceProduct` map reliably, extended custom fields from Fortinet's application control and web filtering modules often land in the `DeviceCustomString` or `DeviceCustomNumber` generic fields without consistent indexing, complicating alert rule creation.
Here is a sample of the `rsyslog` configuration fragment we found necessary for stability under load, which goes beyond the Microsoft documentation:
```
# Increase default message processing rate
$SystemLogRateLimitInterval 0
$SystemLogRateLimitBurst 0
# Configure memory buffer for imudp
$ModLoad imudp
$UDPServerRun 514
$UDPServerAddress 0.0.0.0
$imudpRcvBufSize 16777216 # 16MB buffer
$imudpSndBufSize 16777216
# Template to ensure correct CEF formatting for the connector
$template CEFTemplate,"%TIMESTAMP% %HOSTNAME% %syslogtag% - - - - - %msg%n"
if $programname == 'fortigate' then {
action(type="omfile" file="/var/opt/microsoft/azuremonitoragent/log/cef_input.log" template=CEFTemplate)
stop
}
```
**Key Question for the Community:** Have you conducted long-term stability or load tests on this pipeline? I am particularly interested in:
1. Observed maximum sustainable EPS on a single collector node (e.g., D4s v3) before logs are queued or dropped.
2. Experience with the redundancy model—whether deploying multiple collector VMs behind a load balancer with the same source IP affects Sentinel's ingestion logic.
3. Any operational patterns for schema management when FortiOS updates introduce new log field types.
Our cost model is also impacted by these stability concerns, as log gaps necessitate longer data retention for forensic coverage and increase management overhead. Anecdotal "it works" assurances are insufficient; I'm seeking data on mean time between failures (MTBF), recovery time objectives (RTO) during collector failover, and detailed resource consumption metrics (CPU/IOPS of the collector VM).
-ek
Show me the numbers, not the roadmap.