Hey folks, been wrestling with AWS WAF logs in CloudWatch for a few months now, trying to build some dashboards and alerts. Am I going crazy, or is the default log format just... painfully unusable for any real operational insight? 😅
The JSON structure is so deeply nested, and the most critical pieces of info (like the actual **matched rule** or the **action taken**) are buried. Trying to parse this in CloudWatch Insights feels like a workout. Here’s a snippet of what you’re dealing with:
```json
"httpRequest": {
"clientIp": "192.0.2.1",
"country": "US"
},
"ruleGroupList": [
{
"ruleGroupId": "arn:aws:wafv2:us-east-1:123456789012:...",
"terminatingRule": {
"ruleId": "AWS-AWSManagedRulesCommonRuleSet",
"action": "BLOCK",
"ruleMatchDetails": null
},
"nonTerminatingMatchingRules": []
}
]
```
And that's *after* I've trimmed 80% of the fields! To find out *which* Common Rule Set rule triggered, you need to cross-reference with `managedRuleSetName` and `ruleName` elsewhere. It's a lot of joins in your head.
What I’ve had to do:
* Set up a Lambda function just to parse and flatten the logs before they hit CloudWatch.
* Build a custom dashboard with widgets that query the transformed data.
* Create alarms on specific rule counts, which required custom metrics.
Without this, trying to answer "what's blocking our traffic right now?" takes way too long. Has anyone else built a better pipeline for this? I'm considering exporting to OpenSearch for better querying.
Would love to hear your setups or if you've found a secret sauce. Maybe I’m missing a native AWS tool that makes this easier?
-pipelinepilot
Pipeline Pilot