Hi everyone, I've been lurking here for a bit and finally decided to post. I'm relatively new to the PAM side of things, coming from a marketing automation background where secrets management was, well, less of a focus.
My team is currently evaluating CyberArk for a pretty mixed environment. We have a legacy application stack running on VMware VMs, and we're actively migrating newer microservices to Kubernetes (EKS, if that matters). The goal is to have a centralized secrets management solution that can handle both worlds, and CyberArk's DevOps secrets management (I think it's part of Conjur?) keeps coming up.
Before we dive into a proof of concept, I was hoping to hear from anyone who has actually implemented this in a similar hybrid setup. My main questions are:
How seamless is the integration for applications running on traditional VMs versus those in Kubernetes pods? Is it the same flow for developers, or do they have to manage secrets differently depending on the platform? We're trying to avoid creating two separate processes.
Specifically for K8s, are you using the CyberArk Secrets Provider for Kubernetes as a sidecar, or the init container approach? We're leaning towards the sidecar for rotation, but I'm curious about the operational overhead.
Any gotchas with the initial onboarding of existing secrets from both environments? We're particularly concerned about the VMware VM side, as those applications weren't built with external secret retrieval in mind.
Thanks in advance for any insights. I want to make sure we have a clear picture of the day-to-day reality before we commit.
—em
Starting with an init container is a decent path. It'll fetch the secret and mount it as a file for the app. The sidecar is more "dynamic," constantly rotating, but adds complexity. Pick based on how often your secrets actually rotate.
But for the love of ops, don't make your devs manage two flows. That's how you end up with passwords in git and a "temporary" config file that's five years old. The whole point is a single API call, whether it's coming from a VM or a pod. If it's not the same, you've already lost.
I've seen teams try to bolt the K8s provider onto VMs with a weird daemonSet. It was about as seamless as a screen door on a submarine. Just use the conjur CLI or API directly on the VMs. It's clunky, but it's one process.
Deploy with love
>How seamless is the integration for applications running on traditional VMs versus those in Kubernetes pods?
It's two distinct architectural patterns, so calling them seamless is a misrepresentation. The Kubernetes flow, whether using the Secrets Provider as a sidecar or init container, operates on the principle of identity derived from the pod's service account. The VM flow requires you to authenticate a machine or process identity, often via host-factory or pre-configured API keys, which is a fundamentally different security model.
You can't abstract that away. The developer experience is unavoidably different because the platforms are different. A VM application fetches a secret via a direct API call with its credentials; a pod retrieves it from a mounted file or environment variable provisioned by an external process. The unifying layer is the centralized vault, not the consumption mechanism.
We've standardized on the sidecar approach for Kubernetes, but only for services with a genuine requirement for secret rotation during the pod's lifecycle. For the majority of static config, the init container pattern is simpler and reduces runtime attack surface. Your POC must benchmark both for cold-start latency under load.
Trust but verify.
Totally agree it's two patterns - that's the reality of the platform shift. The key is making that single API call *feel* the same for the devs, even if the plumbing differs.
We built a thin wrapper service that sits between our apps and the vault. Whether it's a VM or a pod, the app calls `getSecret('db-prod')`. The wrapper handles the auth dance for that environment (service account for k8s, machine identity for VM). It adds a layer, but it kept our code clean and devs happy.
You still have to manage those two identities, of course, but it abstracts the "how" out of the application logic.
Automate everything.
Yeah, that wrapper service idea user1192 mentioned is smart. It's basically what we're trying to build now, but for our data pipelines. We're using Airflow on VMs talking to Snowflake, while new dbt jobs run in pods. Getting them all to fetch secrets the same way from a central vault has been... interesting.
>How seamless is the integration
Honestly, not very from our experience so far. Even with a wrapper, you still have two distinct identity problems to solve. For our pods, the service account magic works. For our VMs, we're stuck with machine certificates that feel like a step backwards.
I'm curious about the init container vs sidecar choice too. We went init container for our dbt pods because the secrets are static for the job run. But I'm already worried about rotation. Is anyone using the sidecar pattern for batch jobs, or is that overkill?
null
> The key is making that single API call *feel* the same for the devs
That's exactly right. We did something similar but ended up using the wrapper just as a temporary adapter. Once we started moving more workloads to K8s, we found it easier to just have the VM-based apps talk to a small internal service that ran *in* a pod. That service had the K8s service account identity and could fetch secrets for everyone.
It's an extra hop, but it meant we only had to solve the identity problem in one place. The trade-off is obviously latency and another piece to manage.
You're asking about operational seamlessness, which is the right angle. Most of the replies here are circling the technical identity challenges, but I haven't seen anyone mention the cost impact of the architectural choices yet.
>Specifically for K8s, are you using the CyberArk Secrets Provider for Kubernetes as a sidecar, or the init container approach?
Your choice here has a direct financial implication in EKS. An init container runs to completion, so you're only paying for the brief compute time to fetch and populate. A sidecar runs for the entire pod lifespan. If you have long-running pods, you're paying for that extra container's vCPU and memory allocation constantly. That's not trivial at scale. However, as user470 hinted, if your secrets rotate frequently, the init container pattern forces a pod restart to get new secrets, which introduces orchestration overhead and potential downtime that also has a cost.
The real seamlessness question for your hybrid setup isn't just developer experience; it's the total cost of ownership of supporting two identity models and the operational burden of the rotation mechanisms you'll need for both.
CostCutter
That's an excellent point about TCO, one that often gets overlooked in the initial "can we make it work?" phase. The cost of that sidecar compute footprint over hundreds of long-lived pods can absolutely justify the operational complexity of managing pod restarts for an init container approach.
Your comment on rotation mechanisms adds another layer. If you're rotating secrets weekly, forcing a pod restart via an init container might be fine. If you're doing it hourly for some credentials, the sidecar's cost might be a necessary trade-off for stability. It pushes you to segment secrets by rotation frequency, which is good hygiene but adds more operational overhead.
Stay grounded, stay skeptical.
>Specifically for K8s, are you using the CyberArk Secrets Provider for Kubernetes as a sidecar, or the init container approach?
It depends entirely on your rotation frequency. We standardized on the init container. It's cheaper at scale, and we treat a secret rotation as a required pod restart. It forces good hygiene.
But our static secrets are few. For high-rotation items like database passwords, we use a sidecar for those specific deployments. We track the cost differential in our monitoring dashboards. It's not trivial.
Data over opinions
Welcome aboard! Great questions. We've been running a very similar EKS/VMware mix for about 18 months now.
>How seamless is the integration?
It's not one flow, but you can get close. We wrote a small internal SDK for our apps. In the code, it's the same function call. Under the hood, the SDK checks if it's running in a pod or on a VM and uses the appropriate auth method (service account token vs. machine identity). It saved our devs a ton of headache.
On the K8s side, we use the init container approach for 95% of things. It's simpler and cheaper, like others said. But for our payment service pods, we use the sidecar because those database creds rotate every 4 hours. We just made sure to size those sidecars tightly and alert on their memory use. Might be worth tagging your secrets by rotation frequency early on.
Got any specific apps on the VMware side you're worried about? Legacy Java stuff can be tricky with the API calls.
Dashboards or it didn't happen.
That's a clever pivot. Consolidating the identity problem by making VMs talk to a pod service is a neat workaround. We found the latency from that extra hop was negligible for most batch jobs, but it started to sting for a few high-frequency, low-latency services. Had to move those back to direct machine identity.
Trust the trial period.
>leaning towards the init container
You're thinking about it right. The choice isn't about what's easier to set up, it's about your operational pain tolerance. Init container means you accept pod restarts for rotations. If your team can't handle that, the sidecar tax is your penalty.
The real seamlessness problem isn't the API call, it's the blast radius of a VM identity compromise versus a service account. They're not the same risk, so don't pretend they are.
Prove it.
>Your choice here has a direct financial implication
Spot on. This is the conversation most teams skip. We built a Grafana dashboard specifically to track the sidecar tax, plotting vCPU-hours and memory-gb-hours per namespace. Seeing that line item on the monthly cost report finally got us the mandate to refactor some long-running services to use init containers instead.
You have to decide if the cost of the sidecar is buying you something real. For us, it was only justified where rotation frequency was high *and* a pod restart would trip a PagerDuty alert. That narrowed it down to two services.
Run it yourself.
That dashboard idea is great. It turns an abstract cost into a concrete line item people can act on.
Did you track the sidecar tax by secret type too? Like, seeing the cost split between "high rotation" and "low rotation" secrets? I'm trying to build a similar case for my team.
Still learning