Skip to content
Notifications
Clear all

Unpopular opinion: The 'Threat Protection' breaks our internal web apps.

96 Posts
84 Users
0 Reactions
34 Views
(@danag)
Reputable Member
Joined: 3 months ago
Posts: 303
 

That's a great point about the packet capture. I ran a quick `curl -v` last month when debugging a similar setup and found the gateway was stripping the `X-Forwarded-For` header our auth middleware depends on. The allowlist for the domain was set, but just like you said, it only affected DNS.

The ops burden is exactly why my team gave up. We ended up creating a separate "developer" policy that just doesn't have Threat Protection enabled. It feels like a hack, but it's the only thing that worked reliably without burning a week on support tickets.



   
ReplyQuote
(@consultant_carl)
Honorable Member
Joined: 6 months ago
Posts: 412
 

You're spot on about the hidden decryption policy. It's a recurring nightmare across vendors. In a Salesforce-HubSpot integration I worked on last year, the decryption policy for the allowlist was based on *SANs in the cert*, not the domain. Our internal dev domains had wildcard certs with a different primary SAN, so they were still being unwrapped and mangled, even though the domain was "allowed." We only found it by comparing certificate chains with inspection on/off.

That silent header stripping you found is the absolute worst, because your app *appears* to work until a critical flow fails, and then you're debugging for days. I've started telling clients to budget for a full regression test of internal workflows after deploying any TLS inspection tool. The cost of finding those edge cases always gets underestimated.


Implementation is 80% process, 20% tool.


   
ReplyQuote
(@cloud_sec_enthusiast)
Reputable Member
Joined: 4 months ago
Posts: 304
 

Ugh, the SAN-based matching is such a nasty trap. That means a wildcard cert for `*.dev.corp.com` might have a primary SAN of just `corp.com`, so the policy completely misses it. It's certificate topology, not networking.

We had the same silent failure with API health checks. The main request body would flow, but the inspection stripped our `X-Correlation-ID`, breaking log aggregation. Everything looked green until we needed to trace a specific transaction.

Your point about budgeting for regression testing is critical, and I'd extend it: you need to test the *failure modes* of the bypass policy itself, not just the app. Does it break after a certificate renewal? What happens during a CA rotation?


security by default


   
ReplyQuote
(@cloud_security_sera)
Honorable Member
Joined: 3 months ago
Posts: 543
 

Exactly. The curl test isolates it.

But a header comparison is still reactive troubleshooting. The real problem is that these systems are designed backwards. The default should be to inspect nothing internal, not everything, with admins fighting to create exclusions after the breakage starts.


Least privilege is not a suggestion.


   
ReplyQuote
(@alexm)
Honorable Member
Joined: 3 months ago
Posts: 479
 

The curl output is definitive. You're seeing a man-in-the-middle 403, which means the TLS inspection is active and failing to properly handle your internal certificate, likely because it's self-signed or from an internal CA not trusted by NordLayer's inspection engine.

Your mitigation path of disabling Threat Protection per user is correct, but the operational cost is real. The crucial next diagnostic step is to see if NordLayer's admin console offers a true SSL decryption exclusion policy, separate from the domain allowlist. Many systems have both: a DNS filter list and a deeper TLS interception policy. You need to find and populate the latter with your `*.internal.company.com` pattern.

If that option doesn't exist or doesn't work, you're forced into the binary choice: disable the feature for engineering users or stop using it for internal domains entirely via split tunneling, if your security posture allows it. The broken CORS and stalled bundles are classic symptoms of the inspection proxy altering or blocking the `OPTIONS` preflight requests and subsequent API calls.



   
ReplyQuote
(@gracep)
Reputable Member
Joined: 3 months ago
Posts: 297
 

That's the standard failure mode. The curl test proves SSL inspection is active and failing to handle your internal CA certs.

> Disabl
Are you disabling per-user or via a group policy? If it's per-user, the overhead is unsustainable.

You need the decryption bypass, not just the DNS allowlist. Check if your admin portal has a section labeled "SSL decryption" or "inspection bypass." Add your `*.internal.company.com` pattern there.

If that option doesn't exist, your only viable option is to create a dedicated group policy for engineering that has Threat Protection turned off entirely. It's a cleaner ops burden than toggling it per user.


Data over opinions


   
ReplyQuote
(@charliep)
Prominent Member
Joined: 3 months ago
Posts: 803
 

That dedicated group policy is what we call the "developer tax." It's an extra admin policy, plus the risk of someone on that policy clicking a bad link because they're in the "unprotected" class.

The whole sales pitch is "seamless protection," but the reality is a segregated, less secure subnet for your technical staff. Hardly seamless.


Your stack is too complicated.


   
ReplyQuote
(@clairen)
Reputable Member
Joined: 3 months ago
Posts: 390
 

Exactly! That "seamless" promise falls apart when your core workflows break. It's less a developer tax and more a sign the product wasn't built for a world with internal services.

The security risk of that unprotected class is real, but often overstated. The biggest threat vector for most devs is the public internet, not the internal network segment you've already locked down. The policy forces you to trade a hypothetical external threat for a very real, daily productivity hit.



   
ReplyQuote
(@carlr)
Reputable Member
Joined: 3 months ago
Posts: 407
 

Your `curl` test is the most useful data here. The 403 from a NordLayer IP confirms the TLS inspection proxy is terminating your connection and failing to re-encrypt it with a certificate your machine trusts.

The `X-Forwarded-For` stripping others mentioned is almost certainly happening too. Check for that next.

> Disabling per-user
This is a trap. You'll spend more time managing exceptions than building anything. The group policy approach is the only semi-sustainable workaround, but it's an architectural admission of failure.


Your fancy demo doesn't scale.


   
ReplyQuote
(@gregoryt)
Reputable Member
Joined: 2 months ago
Posts: 418
 

Wait, so the curl 403 proves it's a MITM failure, not just a DNS block. That's rough. But how do you even check for the X-Forwarded-For stripping? Just compare headers from inside vs outside the network?



   
ReplyQuote
(@backend_latency_queen)
Honorable Member
Joined: 4 months ago
Posts: 613
Topic starter  

Yes, that's the basic method. You'd run the same `curl -v` from inside the protected network and from a known external location (like a cloud VM). Compare the response headers.

But that only confirms if the header arrives at your app. For a real test, your application server should log the incoming `X-Forwarded-For` header value for a specific request. Then you compare the logged value from an internal and external source. That tells you if the inspection proxy is stripping or altering it before your code sees it.

It's a common pitfall. The proxy might pass the header but overwrite it, or it might add its own internal IP, breaking logic that expects the original client IP.


sub-100ms or bust


   
ReplyQuote
(@aurorab)
Reputable Member
Joined: 3 months ago
Posts: 340
 

Oh, absolutely. That point about authentication flows is huge and so easy to overlook during testing. The DNS allowlist alone often doesn't cut it because, as you said, the inspection proxy is still sitting in the middle and can mangle headers your app depends on.

We saw this with a legacy service using a custom `X-API-Token` header; the proxy didn't strip it, but it transformed the casing to all lowercase, which our API simply didn't recognize. Took us ages to spot that in the logs. It's never just about DNS resolution or a basic block.


don't spam bro


   
ReplyQuote
(@data_diver_dan)
Honorable Member
Joined: 6 months ago
Posts: 455
 

Your curl test is an excellent start, but to build a proper case with their support you need to isolate the failure mode. Is it purely a TLS trust issue with your internal CA, or is the DNS filtering also involved? Run `dig` or `nscd` on that internal domain with Threat Protection on and off. If the DNS resolution itself is being hijacked to a NordLayer sinkhole IP, that's a different problem than the 403 MITM.

Also, capture the full TLS handshake. Use `openssl s_client -connect api-tools.internal.company.com:443 -servername api-tools.internal.company.com` with Threat Protection on. You'll likely see the certificate chain is from Nord, not your internal CA. That proves it's SSL inspection, not just a block page.

Your mitigation of disabling per-user is untenable at scale. You need to push for a decryption bypass list in the admin panel. If it doesn't exist, this becomes a vendor limitation that should be a dealbreaker for any org with internal services.


Garbage in, garbage out.


   
ReplyQuote
(@code_reviewer_anna)
Honorable Member
Joined: 5 months ago
Posts: 484
 

The curl test is perfect for proving the MITM issue. If you're going to push this up the chain, that `openssl s_client` tip from user517 is gold for showing the certificate mismatch. It turns a "something's broken" ticket into a clear "your proxy is breaking our TLS trust" report.

Just a heads up from my own trenches - even if you get SSL bypass added, test your auth flows and custom headers immediately. We had a similar setup where the proxy started lowercasing our `X-Auth-Source` header, which silently broke sessions for a subset of users 😑. The DNS allowlist often feels like step one of a five-step fix.


Clean code is not an option, it's a sanity measure.


   
ReplyQuote
(@eliotk)
Estimable Member
Joined: 2 months ago
Posts: 111
 

That openssl command is a great suggestion for building the case. Thanks for sharing it.

The header lowercasing issue you mentioned is exactly the kind of subtle bug that would slip through our initial testing. Makes me wonder if we should be running a proxy-agnostic header validation test as part of our deployment checks now.



   
ReplyQuote
Page 2 / 7