The disconnect between `session_start_time` and `last_activity` is a critical operational detail that often gets overlooked. You're right to highlight it as a standalone metric. We've used that exact delta to create a separate "idleness index" in our dashboards, which helps flag policies that might need session timeouts adjusted for specific applications. It turns a data oddity into a potential action item.
Regarding pagination, we did encounter a quirk where using only a `created_at` filter on the audit log endpoint could miss events if the volume during a polling interval exceeded the page limit. The solution wasn't in the docs; we had to implement a hybrid approach using the filter alongside a check on the total returned record count. If the count matched the page limit, we'd recursively fetch with a narrower time window until it fell below the limit, ensuring completeness. Did your approach need similar safeguards?
Let's keep it constructive
That initial `Active Sessions Over Time` graph is such a revelation. It's the moment you stop seeing your ZTNA as a binary on/off switch and start seeing the actual workload patterns. I had a similar experience where the default console just showed "connected," but the custom graph revealed a nightly backup job was spawning dozens of parallel sessions, which was technically allowed but completely unnecessary and wasteful of concurrent connection limits.
Storing it in your own TimescaleDB is the critical step. The vendor's retention window is always too short when you need to look back at a trend from three months ago to justify a license upgrade or a policy change. One caveat I'd add from doing this with other services: make sure your Python script is idempotent and can handle being re-run for a past date range. You'll inevitably discover a gap in your data collection, and being able to backfill from the API without creating duplicates is a lifesaver.
What's your retention policy on the TimescaleDB side? Are you using continuous aggregates or native compression to manage the growth? The audit log volume can get surprisingly large once you start polling every minute.
Retention for our ZTNA logs is aggressive. We keep raw audit logs for 90 days, then roll them into 15-minute aggregates kept for 3 years. Timescale's native compression on the hypertable gets us about an 80% reduction, which is essential when you're storing every session event and API call.
That idempotent script advice is critical. We learned the hard way when a network blip caused duplicates that skewed our session concurrency metrics for a week. Now we use `ON CONFLICT` on a composite key of (hashed_user_id, session_id, event_timestamp) and log any conflicts to a separate table for inspection.
The real challenge isn't retention or compression, though. It's defining what a "session" actually costs when your ZTNA egresses to three different cloud regions. Have you mapped your session activity to the actual data transfer line items yet? That's where the vanity metric becomes a budget forecast.
Right-size or die
The retention and idempotency points are spot on. Our finance team asked the exact same question about mapping costs, and the answer was surprisingly messy.
Our ZTNA provider bills based on aggregate bandwidth, but the cloud egress bills from AWS, Azure, and GCP are itemized by region and service. We built a lookup table to map each internal application gateway identifier to its primary cloud provider and region. It's not perfect because some traffic gets routed dynamically, but the approximation was good enough to show that 70% of our "session cost" was actually egress from a single region housing a data-heavy internal tool. It changed the conversation from "sessions are expensive" to "this one app's workflow is expensive."
How are you handling the approximation gap between your provider's session logs and the granularity of the actual cloud bills?
ship early, test often
Oh, that initial Active Sessions Over Time graph is a total game-changer, isn't it? It's like flicking on a light in a room you've only ever felt your way around in. Everyone here is spot on about the storage and idempotency stuff, but for me, that first real time-series graph was the hook that made the whole project feel worthwhile.
The immediate "aha" for us was spotting daily usage patterns we'd completely assumed wrong about. We thought our engineering team in Europe was the heaviest user of a particular gateway, but the graph clearly showed the bulk of sustained sessions were during APAC hours, which traced back to a support team workflow we'd totally overlooked. It shifted our whole approach to scheduling gateway maintenance.
What are you planning to track in your next panel? Once you have that baseline, adding something like authentication success rate by geo or average session duration per gateway can start to tell some really compelling stories about how your network is actually being used.
hugo
That pattern discovery is the real value, but it's fragile. You're right that seeing the APAC usage changes maintenance plans. The problem comes when your ZTNA provider quietly changes the "gateway" identifier in their API response or merges two logical gateways in an infrastructure update. Your beautiful time-series graph suddenly shows a 50% drop in one gateway and a mysterious spike in another.
The next panel I'd build isn't more metrics, it's a simple sanity check logging the unique gateway IDs pulled each day. You need to catch those definitional shifts before you start making decisions based on them.
Your CRM is lying to you.
Ah, the classic "let's build a bespoke observability platform for our proprietary SaaS" move. You've just traded squinting at spreadsheets for squinting at your own code when the vendor changes an API field without notice.
I'm all for pulling your own data, but you're replicating the core problem - you're still dependent on Perimeter 81's data model. That "simple graph" showing true peaks? It's only true until they redefine what an "active session" is in their next backend sprint.
The real free alternative is to push the vendor for a proper, exportable data stream or open spec. You're doing their product development for them.
FOSS advocate