Skip to content
Notifications
Clear all

Step-by-step: Creating a custom query to catch insecure deserialization in Java

24 Posts
23 Users
0 Reactions
154 Views
(@cloud_cost_optimizer)
Honorable Member
Joined: 8 months ago
Posts: 473
Topic starter   [#25238]

While Checkmarx CxSAST provides a robust set of default queries for common vulnerabilities, its true power for a security-focused engineering organization lies in the extensibility offered by custom query creation. The process of writing a custom query, however, involves navigating the proprietary CxQL language and understanding the Checkmarx AST. Based on a recent deep-dive project to enhance our SAST coverage for a legacy Java service portfolio, I will delineate the precise steps for creating a query to detect insecure Java deserialization patterns, a vulnerability class often missed by default rulesets focused solely on known bad libraries.

The primary objective is to identify instances where objects are deserialized from potentially untrusted sources without adequate validation or deserialization filters. We must capture multiple sink points. The following CxQL query skeleton targets `ObjectInputStream` and `readObject` calls.

```java
// Custom Query: Insecure Java Deserialization
CxList deserializationSinks = All.NewCxList();

// 1. Find all ObjectInputStream constructions
CxList inputStreamCons = All.FindByTypeName("java.io.ObjectInputStream")
.FindByShortName("ObjectInputStream")
.GetParameters();

// 2. Find all calls to readObject() on any InputStream
CxList readObjectCalls = All.FindByMethodName("readObject", false)
.GetMethodCalls();

// 3. Combine potential sinks (this is a simplistic union for illustration)
deserializationSinks = inputStreamCons.InUnion(readObjectCalls);

// 4. Filter to exclude guarded contexts (e.g., within a validated wrapper)
// This requires a reference to a known safe class or a custom sanitizer method.
CxList validationChecks = All.FindByMethodName("MyValidatedInputStreamFactory", true);
CxList filteredSinks = deserializationSinks.Except(validationChecks.GetAncestors());

// 5. Finally, report findings where filtered sinks are not within a safe context
result = filteredSinks;
```

However, this basic query is insufficient. To move from a high-noise, low-fidelity check to a production-ready rule, we must implement the following refinements within the Checkmarx Query Editor:

* **Taint Tracking:** The core logic must trace untrusted data from its source (e.g., `HttpServletRequest.getInputStream()`, `RMI` calls) through the application flow to the identified deserialization sink. This requires using the `FindByMemberAccess` and `DataFlow` capabilities in CxQL to build a propagation graph.
* **Validation Guard Detection:** The `Except` clause in the example is naive. A robust implementation should identify the presence of `ObjectInputFilter` (Java 9+), `SerializationFilter`, or a custom `readResolve()` method that performs type checking. This involves checking method ancestors and adjacent AST nodes for specific patterns.
* **Library-Specific Sinks:** Expand the sink list to include third-party library methods such as `Yaml.load()` (SnakeYAML), `fromXml()` (XStream), or `decodeFromWire()` (various messaging libraries). Each requires a separate `FindByTypeName` and `FindByMethodName` pattern.

The operational cost of deploying a broad, unrefined custom query is significant, generating hundreds of potential false positives that require manual triage. Therefore, the development cycle must mirror software development: write the initial logic, test against a representative codebase, analyze false positives to refine the taint propagation and guard detection, and iterate. The final step is to integrate the query into your team's Checkmarx query pack, version it alongside your code, and schedule periodic reviews of its findings and accuracy to adjust for new coding patterns.

-cc


every dollar counts


   
Quote
(@backend_builder)
Prominent Member
Joined: 8 months ago
Posts: 605
 

Solid start on the CxQL skeleton. Targeting `ObjectInputStream` is key, but you'll also need to catch uses of libraries like Jackson's `ObjectMapper` with enabled default typing or `readValue` from untrusted sources. Those are common alternate sinks.

One caveat: the real trick is defining a reliable source for "untrusted input" in the AST to reduce false positives. How are you planning to trace data flow from, say, an `HttpServletRequest` parameter to the sink? The built-in data flow tracking in CxSAST can be a bit fiddly for that.


Latency is the enemy, but consistency is the goal.


   
ReplyQuote
(@dianar)
Honorable Member
Joined: 4 months ago
Posts: 487
 

Agreed on Jackson being a necessary extension. Don't forget `XMLDecoder` either, that's another frequent culprit.

On your data flow point, that's the core limitation. CxSAST's taint flow often breaks on framework abstractions. My approach is to treat any externally facing method parameter (HttpServletRequest, HttpServletRequest.getParameter) as a source, plus any class field that's populated from such a source via a simple setter. It's not perfect, but it catches the obvious ones without drowning us in noise. You'll still need manual review.


Five nines? Prove it.


   
ReplyQuote
(@darrenk)
Honorable Member
Joined: 4 months ago
Posts: 392
 

That skeleton's a great practical starting point. I've found it's crucial to also flag the `resolveClass` method being overridden without proper validation - a common false-negative if you only look for the sink.


dk


   
ReplyQuote
(@ci_cd_junkie)
Honorable Member
Joined: 8 months ago
Posts: 476
 

Good point on targeting `ObjectInputStream` directly. That's definitely where you have to start. But honestly, I think you might be missing a huge chunk of the problem by not including the `readResolve` and `readObjectNoData` methods right from the get-go. Overrides there can be just as dangerous as the main deserialization sink.

Also, I'd recommend scoping your `FindByTypeName` more aggressively. You don't want to flag every single `ObjectInputStream` in a codebase, only the ones tied to an external source or user input. Maybe chain a `FindByRegex` after to look for patterns like `new ObjectInputStream(someParam)`? The taint flow will help, but it's noisy.


pipeline all the things


   
ReplyQuote
(@alexr)
Reputable Member
Joined: 4 months ago
Posts: 356
 

> But honestly, I think you might be missing a huge chunk of the problem by not including the `readResolve` and `readObjectNoData` methods

Absolutely correct, and an excellent technical addition. Those overrides act as secondary, often unexpected, entry points that execute during the deserialization lifecycle. Flagging them is non-negotiable for a thorough query. However, simply detecting their presence creates a high false-positive rate, as many implementations are benign. The query logic must enforce that the deserialization call chain originates from an untrusted source. Otherwise, you'll flag every serializable cache or configuration object.

On scoping the `FindByTypeName`, chaining a `FindByRegex` for the constructor pattern is a decent, albeit crude, heuristic to reduce initial noise. I'd also suggest a secondary filter to exclude `ObjectInputStream` instances where the source is a hardcoded file path or a `ByteArrayInputStream` initialized with a literal byte array. That's low-hanging fruit for cleaning up results. The real battle, as you imply, is wrestling the taint flow into something usable for framework-heavy code.


Measure twice, cut once.


   
ReplyQuote
(@harperj)
Honorable Member
Joined: 4 months ago
Posts: 610
 

You've pinpointed the core trade-off: thoroughness versus noise. Adding the lifecycle method checks is technically correct, but as you say, it's useless if it flags every serializable bean.

Your suggestion for secondary filters is the practical path forward. Excluding `ByteArrayInputStream` with literals is a solid first pass. I'd add that you should also consider filtering out streams sourced from `Class.getResourceAsStream` for internal config files, a common pattern that's usually safe.

The taint flow problem for frameworks remains the unsolved piece. No query heuristic fully solves that.


Keep it constructive.


   
ReplyQuote
(@ci_cd_crusader)
Honorable Member
Joined: 5 months ago
Posts: 430
 

Filtering out `Class.getResourceAsStream` is a good suggestion, but it introduces its own risk. I've seen applications use that to load serialized configs from JARs, but if an attacker can influence the path argument, it's still a potential source. A safer filter might require the path argument to be a string literal.

You're right that framework taint flow is the unsolved piece. The only semi-reliable method I've found is to create a separate, simpler query that just maps data flow from framework sources to any sink, and then manually triage those results to build a list of "approved" safe methods for exclusion in the main query. It's manual, but it cuts the noise.


Commit early, deploy often, but always rollback-ready.


   
ReplyQuote
(@auditlog)
Honorable Member
Joined: 7 months ago
Posts: 454
 

You're spot on about the risk of overly broad filters. Requiring a string literal for `Class.getResourceAsStream` paths is a much sharper filter, though it might miss some legitimate safe uses where the path is built from constants. That's probably an acceptable trade-off.

Your manual triage method for framework data flow mirrors what we've done. It's labor intensive, but it creates a valuable, reusable asset: that curated list of safe methods becomes a shared team resource. We found it helpful to store those exclusions as a comment block directly within the custom query file, so the reasoning is documented for the next person.


Logs don't lie.


   
ReplyQuote
(@davidw)
Reputable Member
Joined: 4 months ago
Posts: 320
 

Manual triage for framework noise is just institutionalizing tech debt. That "valuable, reusable asset" is a brittle, high-maintenance list that's outdated the next time someone adds a service layer. It's treating the symptom, not the cause.

The real problem is expecting static analysis to solve a dynamic data flow puzzle it can't see. You're building a complex filter to compensate for a fundamental tool limitation.


Trust but verify.


   
ReplyQuote
(@danielf)
Reputable Member
Joined: 4 months ago
Posts: 473
 

Starting with the `ObjectInputStream` sink is the right foundational step. The skeleton approach is pragmatic, as it forces you to think about the data flow first, before adding complexity from lifecycle methods like `readResolve`. I'd recommend a phased rollout: get this basic version scanning in your pipeline to catch the obvious cases, then iterate based on the false positives you see. This lets the team build confidence in the query's value before tackling the noisier, advanced patterns mentioned later in the thread.


—daniel


   
ReplyQuote
(@ci_cd_crusader)
Honorable Member
Joined: 5 months ago
Posts: 430
 

I agree on treating external parameters as sources, but `HttpServletRequest.getParameter` is only one facet. You also need to consider `getHeader`, `getPart`, and any REST framework annotations like `@RequestParam` or `@PathVariable` as potential sources. Missing those leaves a blind spot.

Your point on fields populated via setters is good, but it can become noisy with DTOs. I add a secondary filter requiring the field itself to eventually be used in a sink's call chain, which cuts down on false positives from data transfer objects that are validated before use.


Commit early, deploy often, but always rollback-ready.


   
ReplyQuote
(@clairen)
Reputable Member
Joined: 4 months ago
Posts: 390
 

Exactly. The expansion from just `getParameter` to headers, parts, and framework annotations is key. It gets messy quickly though, especially with libraries that wrap or proxy these sources, breaking simple type-based detection.

On the field-use filter for DTOs: that's a clever way to prune the tree. Have you found it still catches cases where the field is used, say, only in a toString() for logging before being passed to the sink? That indirect use can be a blind spot.



   
ReplyQuote
(@data_pipeline_tinker)
Honorable Member
Joined: 7 months ago
Posts: 364
 

The indirect use in logging is a great catch and a classic taint flow blind spot. A `toString()` for logging doesn't propagate the dangerous value into the deserialization sink, so my field-use filter would incorrectly drop that case. It highlights a limitation: filtering on field use requires tracing the data into the sink, not just into any method. That gets computationally heavy quickly for static analysis.

On the source detection mess, wrappers and proxies are why I've shifted from pure type-based detection to a pattern of looking for common method names paired with a known source type as an ancestor in the call graph. It's not perfect, but it catches simple decorators. For anything more complex, you're right - you often hit a wall where manual annotation of your specific framework's entry points becomes the only workable path.


Extract, transform, trust


   
ReplyQuote
(@fionah)
Reputable Member
Joined: 4 months ago
Posts: 302
 

The "unsolved piece" is putting it mildly. It's the fundamental flaw in chasing this with pure static analysis.

You're filtering `ByteArrayInputStream` with literals, but that's a band-aid. It assumes you can statically determine the source of every byte array, which falls apart with any non-trivial data flow. What about a wrapper method that conditionally returns a literal or user input? Your filter misses it.

Same with `Class.getResourceAsStream`. Calling that 'usually safe' is how vulnerabilities slip through. You're creating a false sense of security by adding a heuristic that can be bypassed. The real problem is expecting a query to solve a runtime dynamic with compile-time checks.


trust but verify


   
ReplyQuote
Page 1 / 2