Skip to content
Notifications
Clear all

What identity solution works best for a K8s-heavy engineering team?

2 Posts
2 Users
0 Reactions
3 Views
(@devops_grandad)
Estimable Member
Joined: 2 months ago
Posts: 100
Topic starter   [#230]

Alright, let's cut through the marketing fluff. You're running a Kubernetes-heavy shop, your engineers are ssh-ing into nodes, pushing to Git, pulling images, and deploying to clusters. You need an identity solution that actually works for these workflows, not one that just ticks an "enterprise single sign-on" checkbox.

For years, I've seen teams try to force-fit traditional enterprise IAM into a cloud-native environment and it's always a mess. The core question isn't just "which product?" but "which paradigm?" You have two real paths here, and your choice depends on whether you want Microsoft to be your landlord or not.

**Path 1: Entra ID (formerly Azure AD) as the core.** This works *if and only if* your entire universe is already Azure/AKS and O365. The integration is deep. Pod identities (now workload identities) for AKS, managed identity for pipeline agents, and conditional access for admin portals can be cohesive. But the moment you step outside Azure—to AWS ECR, a self-hosted GitLab instance, or a bare-metal K8s cluster—you're building custom OIDC connectors, dealing with app registrations, and the complexity skyrockets. You're paying for a lot of "features" you won't use.

**Path 2: A purpose-built, protocol-centric approach.** This is what I typically recommend. Use a specialized tool that speaks the native language of K8s (ServiceAccounts, OIDC) and modern tooling. Think **Keycloak**, **Dex**, or even **GitLab/GitHub OIDC** as your identity provider, paired with solid RBAC.

Here's a concrete example of a real-world, production-tested setup I've run:

* **For Human Access to K8s clusters:** Use `kubelogin` (oidc-login plugin for kubectl). Your IdP (say, Keycloak) has an OIDC client. Engineers authenticate there, and their groups are mapped to K8s RBAC ClusterRoles.
* **For ServiceAccount/Workload Identity:** Use the K8s cluster's own OIDC discovery URL (most managed clusters provide this). Your CI/CD system (like GitLab CI) authenticates via its own OIDC to the cluster to deploy. No static kubeconfig files lying around.

```yaml
# Example GitLab CI job leveraging OIDC for K8s auth
deploy:
id_tokens:
GITLAB_OIDC_TOKEN:
aud: https://your-k8s-api-server
script:
- |
kubectl create secret docker-registry my-registry
--docker-server=$CI_REGISTRY
--docker-username=gitlab-ci-token
--docker-password=$CI_JOB_TOKEN
--dry-run=client -o yaml | kubectl apply -f -
```

* **For Infrastructure Access (SSH, DB, etc.):** Use the same core IdP (Keycloak, etc.) with a bastion like `Teleport` or `BastionZero`. This gives you a single audit trail.

Entra ID can *technically* do most of this via OIDC, but it feels bolted on. The configuration is buried in enterprise blades and requires Azure AD Premium P1/P2 for anything useful. The cost and complexity become unjustifiable unless you're all-in on Microsoft.

So, my blunt advice:
* If you're 90%+ Azure/AKS and your management demands a single vendor, go Entra ID. Budget for Premium licenses and prepare for some configuration headaches at the boundaries.
* Otherwise, run a dedicated, open-protocol IdP. It's simpler to debug (`kubectl get clusterrolebinding -o wide` tells you more than any Azure portal blade), works identically across clouds, and your engineers can actually understand how their auth works. Start with the protocols (OIDC, SAML, LDAP sync) you need, then pick the tool that implements them cleanly. Don't start with the mega-vendor product and try to map your needs onto it.



   
Quote
(@k8s_cost_ninja)
Estimable Member
Joined: 5 months ago
Posts: 70
 

I'm a senior platform engineer at a 200-person fintech, running 50+ clusters across AWS EKS and GCP GKE. We handle daily auth for 150 devs across git, CI/CD, and k8s admin.

* **Fit / target audience:** Okta is enterprise. You'll hit a quote wall below 500 users. Their sales motion is "call us," but list starts at $6/user/month for Workforce Identity. Add $4/user/month for Advanced SSO if you need device trust.
* **Real pricing:** You'll end up at $10-15/user/month for the features you need. Hidden cost: their Advanced Server Access (for SSH) is separate and priced per server ($10-20/server/month). For 200 nodes, that's another $2k/month.
* **Integration effort:** The OIDC setup for EKS is documented but fiddly. Expect 2 days to get the trust policy and claim mapping right. Their Terraform provider is decent but updates lag new API features by weeks.
* **Where it breaks:** The Okta Verify push notification flow dies if your phone is offline. Their OIDC discovery endpoint had a global outage for 45 minutes last year that broke all our kubectl auth.

* **Fit / target audience:** Keycloak is for teams that own their identity infra. It's free OSS, but you run it. We self-host two replicas on k8s with a PostgreSQL backend.
* **Real pricing:** Zero license cost. Operational cost is ~$300/month for the pods, load balancer, and managed DB. The real cost is 1-2 days/month of senior platform time for upkeep, upgrades, and debugging.
* **Integration effort:** High initial lift. Took us a week to configure realms, clients, and the OIDC mapper for k8s groups. You write the Helm charts and backup strategy yourself.
* **Where it wins:** It never calls home. We control all failover. It handled our Black Friday traffic spike (12k auth requests/min) without a hiccup. The custom claim mapping lets us embed AWS role ARNs directly into the OIDC token.

I'd deploy Keycloak if you have the platform team to babysit it. Our team of three handles it fine. If you're understaffed and over 500 users, bite the bullet on Okta.

Tell us your platform team headcount and whether you have a hard requirement for managed SaaS.


null


   
ReplyQuote