Having observed numerous discussions on WAF efficacy that lack operational visibility, I've found that simply enabling a WAF rule group is insufficient. True security posture management requires correlating events with application traffic patterns in real-time. To that end, I've integrated Cloudflare's Logpush data with Grafana Cloud to create a dashboard that moves beyond the built-in analytics, focusing on actionable signals rather than raw log volume.
The architecture is straightforward but requires specific configuration to ensure low-latency alerting. I use a Logpush job streaming to an S3 bucket, which is then ingested by Grafana Loki via the S3 Log Fetch source. The key is the logfmt parsing of Cloudflare's HTTP request schema, which allows for rich labeling and filtering. Below is the core `logql` query used to track WAF rule triggers over time, segmented by rule description and client country code, which is invaluable for identifying geographic attack patterns.
```logql
sum by (RuleDescription, CountryCode) (
rate(
{job="cloudflare-waf"}
| logfmt
| SecurityAction="challenge" or SecurityAction="drop" or SecurityAction="jschallenge"
| RuleID != ""
| __error__=""
[$__interval]
)
)
```
The dashboard itself is organized into several key panels:
* **WAF Events per Second by Action**: A stacked bar chart differentiating `drop`, `challenge`, and `jschallenge` actions. A sudden spike in `drop` actions, for instance, correlates directly with an active mitigation event.
* **Top 10 Triggered Rules**: A table showing the most frequently triggered WAF rules, including the rule ID and description. This is critical for tuning false positives; we identified a rule blocking a legitimate API client due to a missing content-type header.
* **Requests by Country vs. WAF Actions**: A dual-axis chart comparing total request volume from a country against WAF actions from that same country. This quickly surfaces if anomalous traffic from a new region is malicious.
* **Top Targeted Hosts & Paths**: Understand which endpoints are under the most scrutiny. We found our login endpoint (`/api/v1/auth`) was subject to 70% of all challenge requests.
From a cost perspective, this setup leverages Grafana Cloud's generous free tier for Loki, with the primary cost being S3 storage for the Logpush archives (negligible for logs under 10GB/month). The Terraform configuration for the Cloudflare Logpush job is essential for reproducibility and version control.
```hcl
resource "cloudflare_logpush_job" "waf_json" {
zone_id = var.cloudflare_zone_id
name = "WAF JSON Logpush to S3"
dataset = "http_requests"
enabled = true
output_options {
field_names = ["ClientIP", "ClientRequestHost", "ClientRequestMethod", "ClientRequestURI", "ZoneID", "SecurityAction", "RuleID", "RuleDescription", "CountryCode"]
output_type = "json"
}
destination_conf = "s3://${aws_s3_bucket.logs.bucket}/cloudflare/?region=us-east-1"
}
```
The operational benefit has been a reduction in mean time to acknowledge (MTTA) for WAF-related incidents from hours to minutes. By setting alerts on the rate of change for `drop` actions, our on-call engineers are now reacting to active attacks before our internal monitoring thresholds (based on origin load) are even breached. The next iteration will involve correlating these WAF logs with origin latency metrics from our Kubernetes ingress controllers to quantify the performance impact of mitigated attacks.
That query's going to choke on cardinality within a week if you keep all those RuleDescription and CountryCode labels. You're not just summing rates, you're turning high-cardinality string fields into labels. Loki will have a bad time.
And calling this "real-time" with an S3 to Loki pipeline is generous. What's your actual latency from event to panel?
Trust but verify.
You're right to flag the cardinality risk. I've burned myself on that with Loki before. A more sustainable approach is to keep those high-cardinality fields as parsed log attributes and use metric queries that aggregate before labeling. For example, you can create a recording rule that generates a metric series pre-aggregated by just RuleID or Action, then visualize the details from the logs on drill-down.
On the latency, you're correct. "Real-time" is a stretch with S3 as an intermediary. I switched to a Pub/Sub-to-Loki setup for a similar pipeline, and the cost was comparable to S3 storage plus batch processing, but the event-to-panel time dropped to under 10 seconds. The trade-off is operational complexity versus needing true real-time alerts.
CloudCostHawk