Skip to content
Notifications
Clear all

Flux for multi-tenant setups - any success stories?

3 Posts
3 Users
0 Reactions
23 Views
(@devops_barbarian_v3)
Honorable Member
Joined: 6 months ago
Posts: 403
Topic starter   [#1795]

Tried to wrangle Flux v2 for a multi-tenant cluster where each team gets their own namespace and git repo. Needed isolation, not a free-for-all.

Made it work with `Kustomization` targeting specific namespaces and a `GitRepository` per tenant. RBAC via `ServiceAccount` and `ClusterRoleBinding` per tenant is the real key. Kept the Flux controllers cluster-scoped but each tenant's reconciler only sees their own stuff.

```yaml
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: tenant-a-apps
namespace: flux-system
spec:
serviceAccountName: tenant-a-reconciler
interval: 5m
path: "./manifests"
prune: true
sourceRef:
kind: GitRepository
name: tenant-a-source
targetNamespace: tenant-a
```

Anyone else pulled this off without it becoming a total RBAC nightmare? How do you handle tenant onboarding/offboarding?



   
Quote
(@tom_w_analytics)
Eminent Member
Joined: 6 months ago
Posts: 19
 

Your approach is the standard "tenant per namespace" pattern, but you're right about the RBAC boilerplate becoming a nightmare. The onboarding/offboarding script is the real product here.

We automated it with a templated GitRepository and Kustomization, plus a small operator that creates the needed RBAC from a custom Tenant resource. The trick is making sure your cleanup actually works - orphaned RoleBindings are the silent killers of multi-tenant clusters.

Have you hit the "cross-tenant dependency" problem yet? When tenant A needs a ConfigMap from tenant B's namespace, everyone's neat isolation model goes out the window and you're back to shared secrets.


Tom W.


   
ReplyQuote
(@cloud_ops_learner_3)
Honorable Member
Joined: 5 months ago
Posts: 479
 

Yeah, onboarding scripts feel like they become half the project. Did you write your operator for the Tenant resource yourselves, or is it something like Crossplane?

Cross-tenant dependencies are my next headache. We haven't hit it yet, but I can already see the "just one shared secret" request coming. How do you handle it without blowing up the isolation? Do you create a separate shared namespace?



   
ReplyQuote