Guide: Creating a 'learning mode' rule using CloudWatch and Lambda.

Hey folks, been deep in the weeds with AWS WAF on a recent project and wanted to share a pattern I've found super useful. We all know the pain: you deploy a new WAF rule, it's a bit aggressive, and suddenly you're blocking legitimate traffic. I wanted a way to let a rule "learn" what normal traffic looks like before it starts blocking—a sort of 'learning mode' or observation phase.

Here's the core idea: instead of setting a rule's action to `BLOCK` immediately, you set it to `COUNT`. Then, you use CloudWatch Logs to capture those counted requests, analyze them with a Lambda function, and only after reviewing the patterns do you decide to flip the rule to active blocking. This gives you a safety net.

My setup uses a Python Lambda (could be Go too, of course) that triggers on the WAF's CloudWatch log group. The function filters for requests that matched your 'learning' rule, extracts key details, and pushes them to a DynamoDB table for review. You could also send a digest to an SNS topic.

Here's a simplified snippet of the Lambda's core logic:

```python
import boto3
import json
import gzip
import base64
from datetime import datetime

dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('WAFLearningObservations')

def lambda_handler(event, context):
# Decode and decompress CloudWatch log data
cw_data = event['awslogs']['data']
json_string = gzip.decompress(base64.b64decode(cw_data)).decode('utf-8')
log_entries = json.loads(json_string)['logEvents']

for entry in log_entries:
message = json.loads(entry['message'])
# Filter for your specific rule (by rule ID from the log)
if message.get('terminatingRuleId') == 'your-learning-rule-id':
item = {
'requestId': message['requestId'],
'timestamp': message['timestamp'],
'uri': message.get('httpRequest', {}).get('uri', 'N/A'),
'country': message.get('httpRequest', {}).get('country', 'N/A'),
'action': message.get('action', 'COUNT'),
'reviewed': False
}
table.put_item(Item=item)
```

**Workflow:**
1. Create your WAF rule with action set to `COUNT`.
2. Enable WAF logs to a CloudWatch Log Group.
3. Deploy this Lambda, triggered by new log events in that group.
4. Let it run for a defined period (e.g., a week of peak traffic).
4. Review the captured patterns in DynamoDB.
5. Adjust your rule criteria if needed, then change the action to `BLOCK`.

The beauty is you avoid false positives from day one. You can even automate the review step later, but I like a manual check first. Has anyone else tried a similar approach? Curious if there are pitfalls I've missed, especially around log volume costs.

--builder