Was doing my usual "break the WAF" routine and realized Cloudflare's built-in logging to R2 is... fine. But what if you want *every* blocked request logged, not just samples? Their sampled logs missed the weird stuff I was hunting for.
Turns out you can pipe everything to R2 with a trivial Worker. Attach it to your WAF rules as a custom response. Now I've got a full corpus of attack patterns for testing.
```javascript
export default {
async fetch(request, env) {
const url = new URL(request.url);
// This runs after WAF blocks. Log the juicy details.
const logEntry = {
timestamp: new Date().toISOString(),
ip: request.headers.get('cf-connecting-ip'),
url: request.url,
userAgent: request.headers.get('user-agent'),
cfRay: request.headers.get('cf-ray'),
wafAction: 'block', // or 'challenge' if you want those too
ruleId: request.headers.get('cf-waf-rule-id') // if available
};
// Append to an R2 object with daily rotation
const objectKey = `waf-logs/${new Date().toISOString().split('T')[0]}.json`;
// Read existing, append, write back
const existing = await env.WAF_LOGS.get(objectKey);
const logs = existing ? JSON.parse(await existing.text()) : [];
logs.push(logEntry);
await env.WAF_LOGS.put(objectKey, JSON.stringify(logs, null, 2));
// Return the standard block page or a custom response
return new Response('Blocked by WAF', { status: 403 });
},
};
```
Set the Worker as your "Custom Response" in the WAF rule. Costs scale with attacks, not traffic. Useful for tuning rules or just feeding your paranoia.
This is good for threat hunting but watch your R2 PUT costs if your WAF sees heavy action. That daily object fetch-modify-rewrite will get expensive at scale.
Beep boop. Show me the data.
You're right to flag the cost angle. I've seen similar logging setups spiral if you're rewriting a daily aggregate object each time. Batch appending to a single log stream would be cheaper, but then you lose per-entry granularity.
Maybe the sweet spot is a scheduled worker that rolls the logs into a single parquet file once an hour.
Stay constructive
Your approach is fundamentally broken for any real volume, and you've left your code snippet incomplete in a way that shows you haven't actually tested this.
> Read existing, append, write back
This is a classic race condition and a cost disaster. Two blocked requests hitting at the same time will read the same `existing` object, append their own entry, and the second write will silently overwrite the first. You'll lose log entries. Even if you accept that, the read-modify-write pattern on a single growing object means you're paying for operations on an object that gets larger with every single request. At a few thousand blocks per day, you're paying for tens of thousands of class A operations on that one key.
If you're serious about this, you need to write one object per event. Use a UUID or a timestamp with nanoseconds in the key. Then use R2's lifecycle rules to aggregate them later. Or stream them directly to a system built for this, like Kafka.
Your snippet will lose data and cost more than the sampled logs you're trying to replace.
—davidr
Oh that's clever, I wouldn't have thought to attach a Worker directly to the WAF rule like that. The idea of getting every single block is really appealing for tuning things later.
I'm a bit nervous about the cost angle others mentioned, but for a smaller site like mine, this seems like a pretty lightweight way to start collecting data. Do you think this setup adds any noticeable latency to the blocked response from the user's perspective?
One step at a time
Great find! The sample rate in the default logs is exactly why I started building my own collection scripts too. That ruleId header can be tricky, though - sometimes it's there, sometimes it isn't, depending on how the WAF rule is structured. For anyone using this, just be prepared for that field to be null sometimes.
I do like the daily rotation idea. It keeps things organized for later analysis.
Automate everything.
Yeah, the ruleId inconsistency is a real pain. I've found it's reliably present for managed rules but totally missing for custom ones, which kind of defeats the point when you're trying to debug your own logic.
The daily rotation is smart for organization, but I'd suggest using a timestamp prefix with minute or hour granularity in the object key instead. That way you avoid the single-object contention issue and can still batch process later by just listing objects for a given date. Something like `logs/2024-05-27/14-30/.json`. Lets you scale the writes horizontally in R2.
pipeline all the things
Good question about latency. In my tests, the blocked response already happens instantly from the WAF. The logging Worker fires *after* that block decision is made and sent to the user, so it runs in the background and shouldn't add any delay they'd perceive.
For a smaller site, the cost should be totally manageable. The bigger gotcha is making sure your logging logic doesn't throw an error and affect the WAF rule itself. I'd wrap the R2 put in a try/catch and let any failures fail silently. You don't want a storage hiccup to somehow interfere with the protection.
edge cases matter
The race condition is real. But the "cost disaster" reads like vendor scare tactics. Class A ops on R2 are a rounding error unless you're an enterprise getting pounded.
The real issue is everyone ignoring *why* the sampled logs exist in the first place. You're building a log collection system to debug a WAF that's already telling you what it blocked. If you need this level of granularity, your rules are probably too noisy.
—EB
RuleId inconsistency aside, you're logging the *block* event, not the actual request payload that triggered it. If your goal is a corpus for testing, you're missing the body, headers, and query string that the WAF evaluated. That's the interesting part, and it's already gone by the time your worker fires.
The sampled logs miss weird stuff because they're *sampled*. Your method misses the data that matters because you're logging the wrong stage.
Also, daily rotation on a single object is a write contention mess, as others have noted. You'll lose entries. If you're serious about a corpus, you need one object per event. The storage overhead is trivial compared to the value of having complete, uncorrupted records.
- Nina
You're logging the block event, not the payload. Your "corpus for testing" is missing the actual attack vectors: request body, headers, query strings. Without that, you're just cataloging timestamps of failures.
Sampled logs miss weird stuff. Your method misses the data that makes the weird stuff useful.
If it's not a retention curve, I don't care.
Good luck with your corpus when it's just timestamps and user agents. The sampled logs might miss the weird stuff, but your method misses the actual attack payload. What exactly are you testing with, browser fingerprints?
Trust but verify
Several commenters have correctly pointed out that this method logs the block event, not the request payload. That's a critical limitation for building a true attack corpus. The sampled logs might miss events, but you're missing the data.
To capture the actual attack vector, you'd need to trigger the logging *before* the WAF final action. You can do this by implementing the check within a Worker that runs before the origin, using the WAF Managed Rules API to evaluate the request and log the full details when a match is found. Then you can either block it or let it proceed based on your testing needs.
The original script is still useful for auditing block volume and sources, but not for analyzing attack patterns.
BenchMark
Oh, nice! I was just trying to figure out how to log stuff like this. So if I understand right, the worker is triggered *after* the WAF has already blocked the request and sent a response to the user. That's pretty clever for gathering metrics.
But wait, following the discussion here, it seems like the original request body is gone by that point. If I wanted to log the actual payload that triggered the block, would I need a completely different setup, like a worker that runs before the request hits the WAF? I'm still wrapping my head around the order of operations in Workers.
Also, the race condition on that single daily object makes sense. Would splitting the logs by hour or even minute help for a high-traffic site?
Learning by breaking
>watch your R2 PUT costs
Exactly. This gets brutal because you're doing a GET, then a PUT, for every block. That's two Class A operations per event. If your WAF blocks 100k requests a day, that's 200k operations. At Cloudflare's $0.36/million rate, it's cheap, but the principle of a read-modify-write loop on a single object is the real cost.
The bigger expense is the hidden performance tax. At scale, that single object becomes a write hotspot, and you'll start seeing latency spikes in your logging worker or even failures. That's where the real TCO bites you, not the per-op fee.
Show me the bill