Skip to content
Notifications
Clear all

Walkthrough: Using the API to pull all malop data for a compliance audit

7 Posts
7 Users
0 Reactions
81 Views
(@benjamink)
Estimable Member
Joined: 4 months ago
Posts: 202
Topic starter   [#27639]

We just completed a third-party compliance audit that required a full historical record of all malops (malicious operations) from our Cybereason instance over the last 24 months. While the UI is great for day-to-day, pulling this volume of data reliably meant using the API. Here’s how we structured the workflow, along with a few integration tips we picked up.

**Our main steps were:**
* Setting up a dedicated service account in Cybereason with the necessary API permissions (View & Investigate).
* Writing a script to handle pagination and rate limiting—the default result limit is 100, so this is crucial for a full dataset.
* Mapping the JSON response to our internal fields for the audit report, focusing on malop creation time, severity, status, and last update time.
* Staging the data into a cloud storage bucket for the auditors to access securely.

A couple of things to watch for:
* The timestamp format in the API response is epoch milliseconds, which needed conversion for our compliance team’s spreadsheet.
* We initially missed that the API call only pulls malops from the “default” environment; if you have multiple, you’ll need to adjust the query.

For anyone in a similar boat, the key is to start the data pull well ahead of the audit deadline. The process is straightforward, but the volume means it takes several hours to run. We also found it helpful to run a smaller test query first to validate the field mappings before executing the full 24-month pull.

Happy to share more specifics on the field mappings or the pagination logic if it’s helpful—just ask in the replies.

— benk


automate everything


   
Quote
(@brandonj)
Reputable Member
Joined: 4 months ago
Posts: 253
 

Great point about the environment scope. We ran into that too, but with roles. The service account needs "View & Investigate" *and* "Monitor" to catch everything if you're using custom dashboards. Also, staging to cloud storage is smart for security. Did you find the auditors were picky about the exact field mapping, or did they just want the raw JSON?


—b


   
ReplyQuote
(@annab8)
Estimable Member
Joined: 4 months ago
Posts: 184
 

The auditors were surprisingly hands-off about field mapping, they just wanted the raw JSON plus a simple glossary from us. The bigger hiccup was making sure our script preserved the full nested structure, not just the top-level fields. Some of the evidence details they needed were buried a few levels down.

On the roles, you're right, Monitor is key for dashboard contexts. We actually had to go back and add it after our first test run missed a category of malops linked to a real-time monitoring view. It's easy to overlook if you're only thinking about historical queries.



   
ReplyQuote
(@hannahr2)
Reputable Member
Joined: 4 months ago
Posts: 233
 

Oh, fantastic walkthrough. Your note about the environment scope is a lifesaver, it's so easy to forget when you're scripting in a single-tenancy setup. We learned a similar lesson about pagination the hard way - our first script just looped based on a total count, but if new malops were generated during the pull, we'd get duplicates and miss others. The key for us was using the `lastUpdatedTime` from the most recent result as a cursor for the next request.

Also, for the timestamp conversion, we built a small lookup table in our staging database to handle all the epoch fields at once (creationTime, lastUpdateTime, etc.) and output them in ISO 8601 format. It kept the auditors happy and made the data way easier to re-use for our own quarterly reviews later.


Measure twice, automate once.


   
ReplyQuote
(@contrarian_kevin)
Honorable Member
Joined: 4 months ago
Posts: 418
 

Calling this a success story is premature. You just completed the audit pull, but you haven't had to maintain that bespoke script for three years of quarterly requests yet.

That "secure" cloud bucket? It's another vendor dependency and a potential cost trap once data retention policies start clashing with storage tiers. And the minute Cybereason changes their API response schema, your field mapping breaks and your internal glossary is worthless.

All this custom scripting for basic data extraction is exactly the lock-in they want. You're building their export tool for them.


Just saying.


   
ReplyQuote
(@dianar)
Honorable Member
Joined: 4 months ago
Posts: 487
 

You're missing the key failure mode: your script doesn't handle an API outage during the multi-hour pull. If the connection drops at page 300, you have to restart from the beginning or build a state checkpoint, which you didn't mention. That's a major reliability gap for a compliance dump.

Also, storing raw data for auditors in a cloud bucket is fine, but you must hash the dataset before and after transfer. Otherwise, you can't prove the integrity of the evidence chain if they question it.


Five nines? Prove it.


   
ReplyQuote
(@crm_hopper_2025)
Honorable Member
Joined: 6 months ago
Posts: 339
 

That's a great starting point for anyone facing their first big audit pull. The environment scope trap is real, I've seen teams pull only a fraction of their data because they didn't specify the right sensor group or environment in the query parameters.

On your point about pagination, using a simple offset can get messy with very large datasets. We had better luck using the malop GUID from the last record of each batch as the exclusive start point for the next query. It felt more stable than relying on page numbers, especially for a 24-month window.

Have you thought about setting this script to run on a schedule, even quarterly? It'd help catch any schema changes in the API response before the next audit panic.



   
ReplyQuote