A Log Source Extension (LSE) is a custom Python script you drop into QRadar to make it understand a log format it doesn't natively recognize.
You write one when:
* Your device/vendor isn't in the supported App Framework list.
* The default parsing fails or is inaccurate for your custom app logs.
* You need to extract specific, non-standard fields for your use case.
Think of it as a translator. QRadar gives the LSE a raw log line, your code extracts key fields (source IP, event name, severity) and hands back a dictionary QRadar can store and correlate.
Example structure:
```python
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Official QRadar LSE skeleton
def extract_log(line):
# Your parsing logic here
parsed_data = {}
# ... parse 'line' ...
parsed_data['sourceAddress'] = found_src_ip
parsed_data['username'] = found_user
parsed_data['eventName'] = "CUSTOM_EVENT"
return parsed_data
```
You deploy it, map the parsed fields to QRadar's schema, and your logs are now first-class citizens. Without it, they're just unsearchable text blobs.
Great analogy calling it a translator. That's spot on.
The example structure is the official skeleton, but in practice, most of the battle is in the regex to actually split that raw `line`. Also, one huge caveat: you've got to be super careful with performance. If your `extract_log` function is too complex for high-volume logs, you can really bog down the pipeline.
My rule of thumb: if you're writing an LSE for a commercial product, always check the IBM community site first. Someone might have already solved 90% of it.
✌️
Yeah, checking the community first is solid advice. I've saved myself days of work that way.
Totally agree on the performance point. Regex is the obvious culprit, but I'd add that you need to watch for string operations in loops too. A sloppy split() or find() on a massive line can hurt just as much.
Found that out the hard way with some verbose custom audit logs, haha.
That "hard way" experience hits home. I had a similar one where a simple split() on pipe characters inside a massive JSON payload was causing timeouts. The logs looked clean in testing but exploded in volume.
The community tip is gold, but sometimes you find a script that's close but not quite. Tweaking someone else's regex for your weird timestamp format can be just as tricky as starting fresh, honestly. Still, better than nothing!
Oh, that example structure helps a lot, thank you. So just to make sure I'm following - once you get that dictionary back, is the field mapping to QRadar's schema something you do in the script itself, or is that a separate configuration step later? Asking because I'm picturing my team trying to maintain this later.
One step at a time