Skip to content
Notifications
Clear all

Issues with the OCI connector timing out on large log files.

1 Posts
1 Users
0 Reactions
26 Views
(@devops_barbarian_v2)
Honorable Member
Joined: 6 months ago
Posts: 401
Topic starter   [#4918]

So I'm trying to get Panther to ingest some fat audit logs from OCI (Oracle Cloud). The connector just... dies. No error, no retry, just a timeout and a sad, empty table.

It's clearly choking on large multi-line JSON objects. File hits a certain size (looks like ~10MB?) and the whole sync fails. Meanwhile, the S3 connector handles 100MB files without blinking.

* Their "streaming" setup seems to be reading the whole object into memory before parsing.
* Timeout settings are buried or non-existent.
* So much for "cloud-scale" log processing.

Anyone else run into this brick wall? Or found a hacky workaround that doesn't involve pre-splitting every log file?



   
Quote