Help: LogRhythm isn't parsing our custom application logs correctly

Your 5-sample validation rule is the bare minimum. I've seen parsers pass 20 test logs and still fail silently in production because they hit a log line with a null value in a nested field that wasn't in any sample. The parser doesn't just skip that field, it discards the entire event.

You can mitigate it by setting every XPath mapping to optional in Dev Studio, but that's a manual checkbox for each field and it's easy to miss one. Even then, a null in a critical field like timestamp will kill the parse. The silent failure is the worst part; you only find out when your dashboards go empty.

Spot on about the silent failure. That's the real danger.

You can have "optional" set on every field and still lose events. It's not just nulls. A missing closing brace, a single quote where it expects a double quote, any malformed JSON that's still technically delivered by the syslog daemon will just vanish.

The TCO isn't just the build time, it's the ongoing maintenance and hunting for missing data.

Exporting samples is a good start, but that "live parse rate" metric you mentioned is a lagging indicator. By the time you see it drop, you've already lost data. You need a proactive validation step before deployment.

Instead of just mapping fields, build a small Python script that uses LogRhythm's API to test parse your 20-30 samples against your parser *before* you push it to Dev Studio. Use their test endpoint to see the raw parsed output. You'll catch those optional field misses immediately, not hours later. It adds maybe 30 minutes to your process and saves you from the silent failure hell everyone's describing.

Welcome to the community, and great question. You've already put your finger on the core issue - the built-in UI config and XML rules are really meant for flat structures, so they'll butcher nested JSON.

Given your product analytics mindset, I'd say dive straight into the FlexParser. That schema-mapping step everyone's mentioned will feel very natural to you, and it's the only way to properly define your hierarchy. But before you even open Dev Studio, go double-check your log source is set to 'Verbose' in the Console. If it's not, the parser gets stripped data and you'll be building a solution for a problem that doesn't exist anymore.

The silent failure on nulls or malformed JSON is real, though. Once you build that mapping document, consider using the API test endpoint to validate your parser against a wide sample set, including edge cases, before you deploy. It catches things the UI won't.

You've already hit the nail on the head with the problem, coming from a product analytics mindset. The default JSON parser is treating your nested object as a string because it's designed for simple KV pairs.

Given your background, the FlexParser in Dev Studio is absolutely your path. It works directly with the raw, structured log, letting you define XPaths like `/event/user/department`. But there's a big "before you do that" step everyone's missing.

Your `event.user.department` example suggests you control the application code. Before you spend a week in Dev Studio, consider if you can add a lightweight log processor. A small Go service that sits between your app and LogRhythm could flatten or rename those nested fields into something the default parser can handle. It's a bit more infra, but it's often less total work than maintaining a complex custom parser for every schema change.

You're right about the processing profile, that's another hidden knife. But on your second point, I have to push back a little.

That mapping time isn't hidden. It's just shifted. You either spend it upfront building the parser, or you spend it later every day building dashboards with half parsed string blobs and running manual SQL queries to unpack them. At least the parser work is a one time cost if your log format is stable.

You're exactly right that the mapping time is a real cost, but it's not just a one time tax if your schema evolves. We've had to maintain FlexParsers across major application version updates where the nesting structure changed, and each time it's a full regression test against the API endpoint. That overhead can become a hidden recurring cost.

If the application team owns the log format, a better long term solution might be embedding a transformation step directly in the CI/CD pipeline that emits the logs. A simple Logstash filter or a Fluentd parser definition checked into the service repo shifts the schema ownership back to the developers, who understand the data model best. Then LogRhythm ingests a normalized, flat structure.

This also sidesteps the silent failure problem, because malformed JSON fails the pipeline stage visibly, before it ever reaches the SIEM.

The built-in XML rules are for flat, delimited logs like IIS. For hierarchical JSON, you're looking at the FlexParser in Dev Studio. It's essentially mapping an XPath to each target metadata field.

But given you own the app code, there's a third option no one's mentioned: schema-on-write versus schema-on-read. You could add a log shipper with a simple transform. If you're already using Vector or Fluent Bit, a 10-line Lua script could flatten that `event.user.department` to `user_department` before it ever hits LogRhythm. This moves the parsing complexity to a component you fully control and version with your app.

You're correct to question the UI versus FlexParser approach, but I'd argue the architecture decision is more fundamental. The FlexParser is a schema-on-read solution applied to a stream, which creates a tight coupling between your SIEM's parsing logic and your application's log format. Every schema change now requires a SIEM admin to update an XML mapping and run validation tests, which is an operational bottleneck.

Given you control the application code, implement schema-on-write at the edge. Deploy a log sidecar (like Fluent Bit) with a transform that flattens the JSON to a SIEM-friendly structure before egress. This decouples your team's deployment velocity from the SIEM's configuration management. You can version the transform alongside your microservice, and your SIEM ingests simple, flat logs using its default parser. It's a more cloud-native pattern than bending a legacy SIEM module to fit a modern app.

I just went through this exact same pain setting up logs for our new workflow engine. That `event.user.department` showing up as a raw string? Classic. The UI config just can't handle nesting.

I agree with everyone pointing to FlexParser, but if you're already in a trial, there's a huge speed bump: access to Dev Studio. In our trial, it took a support ticket just to get it enabled, which burned a week. If you haven't already, open a ticket asking for "FlexParser access in Dev Studio" now, while you explore other options.

The log shipper transform idea (like with Fluent Bit) is a solid workaround to get things moving immediately. You could prototype a flat structure today and see if it unblocks your evaluation, then build the proper FlexParser later when you have access. It keeps your trial from stalling over a permissions issue.