Hi everyone. Still fairly new to the Splunk and SOAR world, so I'm hoping this is just a simple config issue I'm missing.
We have Splunk ES generating notables, and they're supposed to trigger playbooks in our SOAR platform. The notables are created fine, but the adaptive response actions just... don't fire. No errors in the usual logs I check (like `sa_notable_events.log`).
Has anyone run into this before? I've double-checked the adaptive response framework setup and the SOAR server connection details, which seem okay. Are there specific permissions or a service account that needs to be configured for this handoff to work? Any common pitfalls I should look at next? 😅
Really appreciate any pointers.
Hey, I'm stuck on a similar integration with a different SOAR platform. Have you checked the ES notable event review status? I remember reading somewhere that if the status isn't set to something like "new" or "pending," the adaptive response might not trigger automatically.
Also, maybe the action is set to "ask" instead of "execute"? That got me once.
Good luck, hope you find it!
Yeah, that "no errors in the usual logs" phase is the most frustrating part - everything looks fine on the surface. Since you mentioned checking the framework and SOAR connection, the next layer I'd peel back is the user context for the action itself.
The adaptive response action runs as the owner of the correlation search that created the notable. If that owner's account doesn't have the correct SOAR permissions or API key configured in its user-specific storage (like the `sa_*` KV store collections), the handoff will fail silently. It's a common oversight because we test the main connection with an admin account, but the system uses the search owner's context for execution.
Also, have you looked at the `audit` logs, specifically the events for "adaptive response action execution"? Sometimes the failure gets logged there instead of the notable events log, with a bit more detail about the authentication or network timeout.
Let's keep it real.
Welcome to the fun world of ES and SOAR integrations, I remember that initial setup phase well. The silent failure when logs look clean is a classic symptom.
You're right to suspect permissions and service accounts - that's often the heart of it. The connection might test successfully in the framework settings, but the adaptive response action executes in the context of the correlation search owner. If that user's account lacks the specific SOAR API credentials stored in their `sa_user_credentials` KV store lookup, the action will fail without a clear error in the notable events log.
A good next step is to impersonate that search owner account and manually run the `sendnotable` command with the same parameters your notable uses. That usually surfaces the real auth or permission error that's being swallowed during automated execution. Also, check the audit log for entries related to that specific search owner and adaptive response actions.
Architect first, buy later
That "simple config issue" feeling is so familiar. You're on the right track checking the SOAR connection at the framework level, but the execution context is probably the culprit.
The system uses the correlation search owner's credentials, not the global connection. Even if the framework test passes, the action will fail silently if that specific user lacks the proper API key in their `sa_user_credentials` lookup. Try impersonating that user and run the `sendnotable` command from Search to see the real error.
Also, don't just check `sa_notable_events.log`. Take a look at the audit logs - filter for "adaptive response" events. That's where I usually find the missing piece, like a permission mismatch on the SOAR side for the specific app or playbook.
Integrate or die
Oh, that "no errors in the usual logs" stage is such a headache, isn't it? You've done the right first checks on the framework and connection.
I'd really focus on what user688 and user163 just mentioned about the execution context. Even if your main SOAR server connection tests okay in the adaptive response setup, the specific correlation search that creates the notable has an owner. That user's personal credential storage is what's actually used when the action tries to fire.
One thing I'd add to their advice - sometimes the issue isn't just missing credentials for that user, but that the credential format is wrong for the SOAR app. The KV store lookup expects a very specific JSON structure. Maybe try running `| rest /servicesNS/-/-/storage/passwords` while impersonating the search owner to see what's actually stored there? That saved me a ton of time once when an API key was stored but formatted incorrectly.
test everything twice
That credential format point is crucial. I've seen setups where the API key gets stored as a clear string instead of the required JSON with `api_key` and `server_url` fields, and the framework just chokes on it silently.
To build on your `| rest` command, I usually pipe it to `| table username, realm, clear_password` to see the actual stored data quickly. If the format is wrong, you'll need to delete the bad entry and recreate it using the SOAR app's setup modal - manual edits to that KV store can get messy.
cost optimization, not cost cutting
Oh man, I was *just* wrestling with something almost identical last week. I feel that pain of "the logs look fine but nothing happens."
Everyone's advice here about the search owner context is spot on. That was my exact issue. I set everything up with my admin user, and the connection test passed, but the actual playbook trigger was a ghost.
One extra thing that tripped me up after I sorted the credentials: the SOAR app's "allowed roles" in its setup. Even with the right API key in the KV store, if the correlation search owner's role isn't listed there, it'll fail silently too. Maybe check that while you're impersonating the user?
Good luck! Let us know what you find.
null
Oh, the "allowed roles" setting! I almost forgot about that one. It's easy to miss because the connection test seems to work fine from the admin side.
When I finally got my credentials sorted, I still had nothing happening, and it turned out my search owner only had "user" role access. The SOAR app was set to only allow "admin" and "power" roles. No error, just... silence.
Did you find a good way to quickly check which roles are allowed across all your SOAR app integrations?
Still learning.
That's a really good catch about the allowed roles. I hadn't thought to check that either.
> a good way to quickly check which roles are allowed
Not a quick way, I'm afraid. I ended up having to go into each SOAR app's setup page one by one. It's in the configuration where you first set up the connection, under the "advanced" options usually. A bit tedious, but it was the only place I could find it.
Makes you wonder why the connection test doesn't validate that too, right?