Hi everyone, I'm pretty new to the whole Zero Trust space and my team has recently started piloting Cloudflare One. We're mostly a data engineering group, so I'm coming at this from a perspective of wanting to understand the "why" behind the traffic patterns, not just see the alerts.
We've got the basic Gateway blocking policies set up for things like malware and phishing categories, and the dashboard shows me aggregate numbers of blocked requests. That's useful for a high-level view, but I'm hitting a wall when I try to get more granular. I need to answer questions like: "Is the spike in blocks coming from one user who's clicking everything, or is it spread across the team?" or "Which specific users are consistently hitting blocked sites, so we can maybe offer some targeted training?"
I've poked around in the Analytics section of the Zero Trust dashboard and I see the logs, but I feel like I'm missing a step to really summarize the data the way I'm imagining. My background is in SQL and data modeling, so I'm comfortable with raw logs if I can get to them, but I'm not sure what the best path is within Cloudflare One.
Could someone guide me on the workflow here? Is the intended method to use the Logpush feature to send all those Gateway logs to a data warehouse (like BigQuery or Snowflake) and then write my own SQL queries to group by user and count blocks? Or is there a built-in reporting view I haven't found yet that can generate this for me without setting up a whole pipeline? I'm eager to set this up properly, but I want to be cautious and follow best practices rather than building something overly complex if there's a simpler way.
Also, if going the Logpush route is the standard answer, any tips on the specific fields I should be paying attention to for user identity (like `UserEmail`) versus the action (`Action` equals 'block') would be incredibly helpful. I'm used to working with Python for data transformation, so I'm prepared to handle the data once I can access it.
You're on the right track with the Analytics logs. For your use case, you'll want to build a custom report using the Logpush API to get those raw logs out. That's the only way I've found to reliably get the user-level granularity you need.
You can pipe them into BigQuery or even a local SQLite DB if you want. Then it's a simple group by user, count of blocks, and maybe join against the domain categories. The built-in dashboards just don't surface that specific aggregation.
One caveat - make sure your Zero Trust config includes the user identity in the logs. If you're just using IP, you'll have a mapping problem.
ms matters