I've been tasked with designing and implementing secure remote access for several small, fully-distributed engineering teams managing Kubernetes infrastructure. The specific case I'm examining here, and one I see frequently, is a team of five engineers needing to manage a multi-cluster, multi-cloud K8s environment. The requirements always boil down to: least-privilege access, auditability, zero-trust principles, and seamless integration with the existing K8s toolchain.
Traditional solutions like VPNs are a non-starter. They provide network-level access, which is far too broad ("once you're in, you're in"), and they completely lack granular, user-aware controls for Kubernetes API objects. We need something that operates at the identity layer. After evaluating several options (including Kubelogin, OIDC proxies, and various commercial platforms), I've found that a combination of established cloud-native tools often yields the most robust and maintainable outcome.
For this 5-engineer team, I typically recommend the following architecture:
* **Identity Provider (IdP):** The source of truth. This is usually the company's existing SSO (e.g., Google Workspace, Okta, Azure AD). All engineers authenticate here.
* **Kubernetes RBAC:** Roles (`Role`/`ClusterRole`) and bindings (`RoleBinding`/`ClusterRoleBinding`) are defined as code, checked into git, and applied via your GitOps pipeline (e.g., ArgoCD, Flux). This is your *authorization* model.
* **Authentication Bridge:** This is the critical component. We use **`dex`** or **`pinniped`** as an OIDC connector. It acts as a "translator," taking the authentication flow from your corporate IdP and presenting it to Kubernetes as a valid OIDC identity.
* **`kubectl` Configuration:** Engineers use `kubelogin` (or `oidc-login`) as a `kubectl` credential plugin. The flow is: `kubectl` -> plugin -> redirects to IdP -> you log in -> plugin gets token -> `kubectl` uses token.
A simplified, illustrative `kubeconfig` snippet for the engineer looks like this:
```yaml
users:
- name: engineer@company.com
user:
exec:
apiVersion: client.authentication.k8s.io/v1beta1
command: kubectl
args:
- oidc-login
- get-token
- --oidc-issuer-url= https://dex.yourdomain.com
- --oidc-client-id=kubernetes-cluster-client
- --oidc-client-secret=
- --oidc-extra-scope=email,groups
```
The real administrative work is in the `dex` configuration and the K8s RBAC. For example, a `ClusterRole` for a "read-only" engineer and its binding might be:
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: readonly-engineers
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: view
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: "eng-team-viewers" # This group is mapped from the IdP via dex
```
**Pitfalls & Costs:** The main cost isn't licensing (these tools are OSS), but operational overhead. You must host and maintain `dex`/`pinniped`. Availability is critical—if it's down, no one can authenticate. You also shoulder the security responsibility for that component. For a team of five, this is manageable, but it's a non-zero burden. The other major pitfall is the initial configuration complexity; misconfiguring OIDC claims and group mappings is a common source of headaches.
The alternative is a managed service like GKE's built-in IAM, EKS with Azure AD integration, or a third-party platform. These reduce ops burden significantly but often come with per-user licensing and may lock you into a specific cloud. For a small, skilled team, the DIY approach offers maximum flexibility and cost savings, provided they can handle the infra. For teams wanting to minimize ops, a managed path is worth the spend.
I'm curious what others have deployed for similar small teams. Have you found a cleaner stack than the dex/RBAC combo? How do you handle break-glass scenarios when the OIDC bridge is impaired?
- Mike
Mike
I'm a technical architect for a fintech SaaS company with around 60 engineers; we run 12+ production EKS and AKS clusters across AWS and Azure for internal and customer-facing workloads.
Here's a breakdown based on what we rolled out after a similar evaluation last year.
* **Integration and Admin Overhead:** Solutions like **Kubelogin / Dex with OIDC** tie cleanly into your existing SSO (Okta, Google) and RBAC, but they require you to manage the Dex server, certificate rotation, and CRD syncing yourself. For five engineers, that's 2-3 days of initial setup and a recurring operational tax.
* **True Cost for Small Teams:** Commercial platforms like **Teleport** and **Styra DAS** have very different models. Teleport's open-core model is free for up to 10 users, but the Team tier with required SSO and audit logs starts at roughly $15/user/month. Styra's policy-based access bundled with OPA management is more like $50/user/month, making it overkill for pure access.
* **Audit and Session Depth:** We found Teleport's session recording for `kubectl exec` and database access was a compliance clincher that OIDC proxies alone don't provide. However, that full session audit trail is only in its Enterprise tier. The open-source version gives you API request audit logs, which is often enough.
* **Day-to-Day User Experience:** Using a CLI context switch like `kubectl login` with an OIDC flow is seamless after setup. In contrast, Teleport requires engineers to first `tsh login` to a proxy, which adds a step but then provides a single signed certificate valid for a short duration (e.g., 12 hours) for all `kubectl` commands, which is nicer than browser timeouts.
I'd recommend starting with Teleport's open-source edition for your team of five. It delivers identity-aware, zero-trust access with solid `kubectl` integration out of the box and no per-user cost. If you need guaranteed vendor support or those recorded interactive sessions, tell us your compliance requirements and whether you also need to secure non-K8s resources like databases or SSH servers.
Thanks for the detailed comparison, especially the real cost breakdown. The admin overhead point for Dex is exactly what I'm worried about - with a small team, we can't afford a recurring setup like that.
When you mention >Teleport's session recording for `kubectl exec` was a compliance clincher, does that mean you had to keep all session recordings forever? I'm trying to figure out the storage and compliance burden that might add, even for just five people.
One step at a time