Hey folks, hoping to tap into the collective wisdom here. I've been digging into Appgate SDP's logging capabilities, and while the audit logs in the admin portal give a good high-level overview, I'm hitting a wall trying to get a fine-grained, chronological trace of *everything* a specific user account does in a session. Think less "user connected to resource X" and more "user accessed file Y on server Z at this exact timestamp, with this client IP, and here was the exact request path."
My use case is debugging a weird access pattern reported by one of our devs—they swear they didn't touch a particular system, but logs suggest otherwise. I want to reconstruct their session move-by-move to see if it was a misclick, a script running, or something else. I'm used to instrumenting my IDE with verbose logging plugins to trace every linting rule or language server request, so I'm hoping for a similar level of detail here.
From what I've pored over in the documentation, I know the system generates a ton of log data. I've been experimenting with the Collector and the `system` logs. Has anyone built a reliable method to filter and correlate these? I'm thinking along the lines of:
* **Key fields to grep/filter on:** Is the user's `subjectId` or `subjectName` consistently captured across all log types? What about the `claimId`?
* **Log sources:** Which log files or API endpoints are most fruitful for access events? The `access-log`? `gateway-log`? Or is the data spread and needs joining?
* **Timezone normalization:** I've seen timestamps in UTC and local—what's the best practice to ensure the sequence is correct?
I tried a quick CLI query on a Collector node, something like:
```bash
grep -r "jsmith@company.com" /opt/appgate/logs/gateway/ --include="*.log" | head -20
```
But the output is a bit messy and doesn't always show the resource accessed in a clear way. I'm wondering if the Admin API's `audit` or `events` endpoints are more structured for this. Maybe something like:
```json
// Hypothetical API query filter
{
"filter": {
"subjectName": "jsmith@company.com",
"from": "2024-05-20T09:00:00Z",
"to": "2024-05-20T17:00:00Z",
"types": ["ACCESS_ALLOWED", "ENTITY_ACCESSED"]
}
}
```
Has anyone scripted something like this? Do you pipe it to a log aggregator (Splunk, ELK) for better visualization? I'm particularly curious about capturing details for HTTP/S and SSH connections through the SDP—are those protocol-specific details (like URLs or commands) logged somewhere, or just the connection event?
Any pointers on the exact log locations, field names, or API parameters would be incredibly helpful. I'm also interested in any gotchas—like certain log levels that need to be enabled, or if user-specific detailed logging incurs a performance hit.
editor is my home
Hey, you're definitely on the right track with the Collector and system logs. For your specific need to trace a user's every move, you'll want to filter the `gateway` logs, not just the `system` ones. The gateway logs hold the detailed connection and traffic data per session.
If you pipe those logs to a SIEM or even a structured file, you can filter by the user's SDP identity ID and the session ID. That gives you the chronological chain you're after - every resource accessed, the client IP for each connection, and timestamps down to the millisecond. It's a bit of a query to set up, but it's how we've reconstructed similar "I swear I didn't do that" scenarios.
Just a heads up, make sure your logging level is set to capture that detail, it might be set to 'info' by default. Good luck!
Stay curious, stay skeptical.
That's a solid analogy, comparing it to your IDE's verbose logging. The key detail to add is that you'll need to look at the gateway logs from the specific Gateway the user was connected through. The system-level logs are more about the SDP platform's health, while the gateway logs hold the session-specific breadcrumbs.
A good next step is to check your Gateway logging profile settings. There's a specific "Transaction" log type that captures the level of detail you're after - each individual request and its metadata. It's often not enabled by default because of the volume, but it's exactly what you'd need to reconstruct the session. Have you checked which log types are active on your gateway logging profile?
—daniel
That verbose IDE logging comparison is where you're going wrong. You're hunting for mouse clicks in a protocol designed for tunnels and policies.
The "Transaction" logs user1539 mentioned will drown you in TCP handshakes and DNS queries, not "accessed file Y". You need the actual resource server logs, not the gateway's noise. SDP tells you *which door* they used. You have to look inside the room.
If your dev's script ran a background job, you'll see the connection in SDP logs, but the "access" detail is in Apache/nginx or the OS audit trail on the target server. Correlate the session ID from SDP with timestamps on the server.
-- old school
You're right that the gateway's "Transaction" logs can be noisy for that level of detail, and correlating with the resource server logs is often the definitive step. I think the SDP session ID is the crucial link here - it's the common identifier that lets you stitch the gateway's "door" event to the specific actions "inside the room."
One caveat is that for some internal web apps or APIs, the gateway logs might actually show the exact HTTP path if full URL logging is enabled in the policy. But for most file-level forensics, you're absolutely pointing them to the right next step.
Keep it constructive.
Correlating logs with a session ID is the textbook answer. It's also usually useless.
By the time you've got server logs pulled and timestamps aligned, the issue is already in a post-mortem doc. The real problem is Appgate not giving you the damn detail in one place. If it can tunnel the traffic, it should be trivial to log the path. "Full URL logging in the policy" is a band-aid on a design flaw.
Stop stitching systems. Demand better from your tools.
> "Correlating logs with a session ID is the textbook answer. It's also usually useless."
You're not wrong about the frustration. The latency in stitching systems kills momentum in an investigation. But calling it useless is where I push back; it's the only forensic method you have when you're trying to prove a user's script did, in fact, touch the production database at 2 AM.
The real band-aid isn't the policy logging, it's the refusal to build a proper log aggregation pipeline from day one. If your SIEM or even a centralized logging stack is an afterthought, you'll always be in this mess, regardless of the SDP vendor. The session ID is your foreign key. If you don't have a system to join the tables efficiently, that's an ops failure, not just a tool flaw.