So the prevailing wisdom in the GitOps cathedral seems to be that you can just slap an ArgoCD ApplicationSet on top of your Helm charts and call your multi-tenancy problem solved. I've been watching this pattern gain cult-like adherence, and I have to ask: has anyone actually *tried* this beyond a hello-world example in a sterile lab environment, or are we all just parroting the same optimistic blog post from 2021?
The promise is seductive, I'll grant you. Define a single ApplicationSet generator (probably a cluster generator or a matrix combining git and clusters), point it at your Helm chart repo, let ArgoCD proliferate instances across namespaces or clusters, and bask in the glory of centralized management. But the devil, as always, is in the concrete details, and that's where the whole facade starts to crack. The moment you need per-tenant overrides that are more complex than a couple of values, you're plunged into a configuration nightmare. Are you using `values.yaml` in separate git paths? A monstrous `values-matrix` in the ApplicationSet spec? Or have you surrendered and started templating your Helm *templates* with yet another layer of text substitution before Argo even sees them?
Let's talk about the real failure scenario I've witnessed. A team attempted to use a ClusterDecisionResource generator to dynamically onboard new tenants (whole namespaces) in a SaaS-like internal platform. Their Helm chart had a dozen conditional dependencies based on feature flags, and the required overrides were substantial. The ApplicationSet's `template` field became an unmaintainable beast of inline YAML, mixing Argo application spec with nested Helm values. The final blow came when they needed to execute a Helm post-install hook for one tenant but not another, based on a parameter. The solution involved... well, it involved creating a separate ApplicationSet, which defeated the entire purpose. They were left with fragmented automation and more complexity than they started with.
I'm genuinely curious about real-world implementations. Not the "look at my perfect git repository structure" demos, but the gritty ones. How do you handle:
- Secret management per tenant when Helm values need to be injected from external sources like Vault? Does your ApplicationSet generator pull and inject, or does that break the GitOps purity model?
- Helm chart dependencies that vary per tenant (some get Redis, some get RabbitMQ, some get neither). Are you maintaining multiple subcharts and toggling them with `tags`, or do you have a chart per service profile?
- The inevitable drift when someone manually tweaks a resource in a tenant namespace because of a firefight, and your GitOps pipeline now views that namespace as a perpetual conflict zone.
I suspect the happy path is incredibly narrow. For truly simple, cookie-cutter deployments, it works. But the moment your tenants have divergent needs, the abstraction leaks like a sieve, and you're left with a toolchain that's fighting itself. Is there a pragmatic middle ground, or are we all just waiting for the next tool to repackage these same problems with a shinier logo? 🧐
—Bella
Price ≠ value.
Yes, we've run this exact pattern in production for a data platform serving over two dozen internal tenants. The configuration nightmare you're anticipating is absolutely real.
The monolithic `values-matrix` becomes unmaintainable beyond about ten tenants. We abandoned it. Our solution was to combine generators: a Git generator pulls the per-tenant `values.yaml` from a dedicated directory structure (`tenants//values.yaml`), and a Cluster generator supplies the target cluster context. The ApplicationSet merges them. This at least keeps the tenant-specific overrides isolated in separate files.
However, this just pushes the complexity down the road. The real friction starts when you need to make a coordinated change to a subset of tenants. You're now forced to manage a directory of YAML files programmatically, which is essentially writing a bespoke, less-featured GitOps operator on top of your GitOps operator. It works, but it's meta.
Query first, ask questions later.