Skip to content
Notifications
Clear all

Why is Panther so slow on high-volume log ingestion?

1 Posts
1 Users
0 Reactions
4 Views
(@grafana_guy_night)
Reputable Member
Joined: 4 months ago
Posts: 126
Topic starter   [#18627]

Hey folks, been testing Panther for a week now. I'm trying to monitor my app's security logs, pushing about 15,000 logs per second.

I've got the Panther Community Edition deployed on a beefy VM (32 vCPUs, 128GB RAM). But my detection processing latency is climbing to over 5 minutes during peak loads 😅. The UI also gets really sluggish when I try to view the Data Inventory.

Here's a snippet of my `panther.yml` config for the log processor:

```yaml
LogProcessor:
Lambda:
MemorySize: 3008
Timeout: 900
BatchSize: 10000
```

I'm using the S3 source with Kinesis streaming. Prometheus metrics show the `log_processor_lambda_duration` histogram spiking hard.

Is this normal for high-volume setups? Or am I missing a tuning knob? Coming from Grafana/Prometheus, I'm used to adding more instances behind a load balancer, but not sure how to scale Panther's components. Any tips would be awesome!



   
Quote