Skip to content
Notifications
Clear all

Troubleshooting: PSM connectors keep disconnecting from our Oracle databases.

2 Posts
2 Users
0 Reactions
4 Views
(@alexm)
Reputable Member
Joined: 1 week ago
Posts: 147
Topic starter   [#12873]

Our team has been experiencing persistent and seemingly random disconnections of PSM connector sessions to our Oracle 19c backend databases. The issue manifests as abrupt session termination from the PSM server side, logged in the CyberArk PVWA as "PSMConnectFailure," while the corresponding Oracle listener logs show "TNS-12535: TNS:operation timed out" or "TNS-12537: TNS:connection closed." The disconnections occur inconsistently, sometimes after minutes of idle time, other times during active query execution, and are not correlated with any observable load spikes on the target Oracle hosts.

We have ruled out basic network instability and have confirmed that direct connections from the PSM servers to the Oracle databases using SQL*Plus or other clients remain stable under identical patterns. This points to a configuration or behavioral mismatch specifically within the PSM connector framework. Our current environment setup is as follows:

* **CyberArk Components:** PSM 12.6, using the standard Oracle PSM connector (PSM-Connect-Oracle).
* **Oracle Environment:** Oracle Database 19c (19.0.0.0.0), listener on default port 1521.
* **Network:** Low latency (<2ms), no intermediate firewalls resetting idle connections beyond the corporate standard 30-minute TCP idle timeout.
* **Current PSM Connector Configuration** (relevant excerpts from the `Oracle.ini` file):
```ini
[General]
SessionDisconnectedGraceTime=10
ConnectionPort=1521
; No explicit HeartbeatInterval or QueryKeepAlive settings configured.
```

My primary hypothesis centers around the lack of a keep-alive mechanism at the application protocol level. The corporate 30-minute TCP idle timeout is likely being triggered because the PSM connector, in its default configuration, does not send periodic heartbeats during idle periods. The `SessionDisconnectedGraceTime` parameter appears to govern post-disconnection behavior, not prevention.

I am seeking a definitive, data-backed comparison between the available keep-alive strategies within the CyberArk PSM connector for Oracle. Specifically:

1. **HeartbeatInterval vs. QueryKeepAlive:** What is the operational difference between setting `HeartbeatInterval` (seemingly connector-level) versus using a periodic `SELECT 1 FROM DUAL` via the `QueryKeepAlive` parameter? Which is more robust against intermediary devices that may scrutinize and reset non-standard TNS packets?

2. **Parameter Synergy:** Should these parameters be used mutually exclusively, or in tandem? For example:
* Setting `HeartbeatInterval=300` (seconds)
* Setting `QueryKeepAlive=SELECT 1 FROM DUAL@300`
* A combination of both

3. **Impact on Session Monitoring:** Does a `QueryKeepAlive` generate observable "query" events in the session monitoring log, adding noise, whereas a `HeartbeatInterval` might not?

4. **Oracle Net Timeouts:** Have we sufficiently addressed the server-side `SQLNET.INBOUND_CONNECT_TIMEOUT`, `SQLNET.EXPIRE_TIME`, and `SQLNET.RECV_TIMEOUT` parameters, or is the solution purely client-side (PSM) in this scenario?

I intend to run a controlled benchmark, modifying one parameter at a time across three test PSM servers, logging disconnection events, and capturing TCP dump samples during idle periods. Before embarking on this, I would appreciate validation of the test matrix and any experiential data from the community on stable configurations for long-lived Oracle PSM sessions.



   
Quote
(@chrisp)
Estimable Member
Joined: 1 week ago
Posts: 115
 

Interesting - the fact that direct SQL*Plus connections stay stable while PSM drops out strongly points to something in the connector's session handling. I've run into similar gotchas with other CyberArk connectors where the PSM's own idle timeout or an aggressive SQLNET.EXPIRE_TIME on the Oracle side aren't aligned.

Have you checked the SQLNET.EXPIRE_TIME parameter on the Oracle end? Sometimes even a small mismatch (like 10 minutes on the database vs. 5 minutes on the PSM connector) can cause those TNS-12535 timeouts, especially if the connector doesn't send keepalives. Also worth looking at the PSM connector's own timeout settings in the configuration file. I'd A/B test two different idle timeout values just to see if the disconnection pattern shifts.

Another angle: are you using the default PSM-Connect-Oracle or a customized version? I've seen cases where the connector's SQLNET.OUTBOUND_CONNECT_TIMEOUT interacts badly with the listener. Might be worth rolling back to a known-good connector version or trying the generic telnet/SSH approach as a control. What does the PSM trace log show right before the disconnect?


✌️


   
ReplyQuote