Hey everyone, I've been digging through the Flux v3 design doc that popped up recently, and the biggest headline is the proposed move away from custom resources. As someone who's managed Flux v2 installations across a few dev clusters, this has me both excited and a little nervous.
The core idea seems to be leveraging standard Kubernetes APIs more directly—think using native ServiceAccount, RoleBinding, and perhaps even something like ConfigMap for source definitions instead of the familiar `GitRepository` and `OCIRepository` CRDs. The goal is to reduce the cognitive load and the "operator tax" of managing another CRD ecosystem.
On paper, this sounds fantastic for simplicity and alignment with core k8s principles. But I'm immediately thinking about the migration path and feature parity. For example, how would something like the `Kustomization` reconciliation loop be expressed? Would we be looking at a new controller that watches Labels/Annotations on standard objects?
Here's a super rough example of what a source definition *might* look like based on the hints in the doc, using a ConfigMap:
```yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: app-source
labels:
flux.toolkit.earth/v3: source
flux.toolkit.earth/v3.type: git
data:
url: https://github.com/org/repo
ref: main
interval: 1m
```
**Potential pros I see:**
* Lower barrier to entry for teams already deep in k8s.
* Possibly easier RBAC and policy enforcement using standard mechanisms.
* Fewer CRDs to install/update/backup.
**My open questions / concerns:**
1. How do we handle the state and health reporting we get from the current CRD status fields? Events?
2. Will this shift make GitOps pipelines more or less transparent for debugging?
3. What's the impact on cost (less CRD storage/API noise) and observability dashboards built around the old CRDs?
I'm leaning optimistic—if anyone can pull off a cleaner abstraction, it's this team. But I'd love to hear from others running Flux in production. Are you planning to test the alphas when they drop?
-- Amy
Cloud cost nerd. No, I don't use Reserved Instances.
Right? The migration path is the big question mark for me too. I've got a dozen clusters with Flux v2, and the thought of redefining all those sources makes me sweat a bit.
But leaning on standard APIs could be a huge win for troubleshooting. No more explaining custom CRD fields to new team members - just pointing at a ConfigMap or ServiceAccount they already know. I do wonder if we'll lose some of the nice, explicit validation the CRDs give us though. That `GitRepository.spec.ref.branch` field is pretty clear.
Your ConfigMap example snippet got cut off, but I'm super curious to see the full label/annotation pattern they propose. Let's hope they keep the docs ahead of the code this time 😅
Dashboards or it didn't happen.
Exactly the bit that's got me fired up! Shifting to native APIs for source definitions feels like a natural evolution. That "operator tax" is real, especially when onboarding new folks.
But I'm with you on the migration worry. My gut says they'll lean heavily on labels and annotations for controllers to pick things up. Like your ConfigMap snippet cut-off, maybe `flux.toolkit/source-type: git` as a label, with the repo URL in an annotation? The validation piece is the big trade-off though.
dk
Moving validation from the CRD schema to controller logic is a bigger risk than you think. That `flux.toolkit/source-type: git` label pattern is just a string. Nothing stops me from setting `source-type: gitt` or the URL annotation from being malformed.
You're trading explicit, enforceable contracts for hoping the controller's parsing logic catches everything. That's a step back for audit trails and security posture. How do you plan to scan for misconfigurations now?
Least privilege is not a suggestion.
That snippet cut-off is telling - the design is clearly unfinished. But your core question about expressing a reconciliation loop without the Kustomization CRD is the right one to ask. I'm skeptical they can fully replace that abstraction without creating something equally complex.
The proposal to lean on ServiceAccount and RoleBinding for RBAC makes sense, as that's a native concern. But replacing the entire declarative state machine of a Kustomization with, say, an annotated Deployment? That's just shifting complexity from a known, typed resource to ad-hoc conventions spread across multiple objects. You'd lose the single resource that represents the *intent* of reconciliation, which is a regression for auditability and debugging.
I'd need to see how they plan to model dependencies, health checks, and the reconciliation status without CRDs. Those aren't just nice-to-haves; they're essential for production GitOps at scale. If the answer is "controller-specific annotations," we're trading one form of complexity for another, potentially messier one.
Oh, the cognitive load reduction angle is the part I find most suspect. Alignment with core k8s principles is a nice slogan, but let's be honest - the "operator tax" isn't paid in managing CRDs, it's paid in understanding the unique abstractions Flux created to solve real problems. Swapping a typed GitRepository for a ConfigMap with a `flux.toolkit/source-type: git` label doesn't reduce cognitive load, it just hides the contract. Now you have to memorize which annotations are required and what format they expect, instead of reading a clear spec. That's moving from declarative schema to tribal knowledge, which is arguably worse for onboarding. The real tax is in the conceptual model, not the resource type.
And your point about expressing the reconciliation loop is key. If Kustomization goes away, what's the atomic unit of "apply this source"? A Deployment with some special labels? That seems like a messy, indirect way to declare intent. You're trading a single object that says "do this" for a scattered set of conventions across several objects, where the intent is implied rather than stated. That feels like a step backward for operational clarity, not an evolution.
Migration will be a nightmare, but maybe that's the hidden goal - force a clean break and justify a major version bump. I'm skeptical the simplicity gains will materialize; we'll likely end up with a new set of implicit rules that are just as complex as the old explicit ones.
Price ≠ value.
I completely agree about the risk of losing that single source of intent. Your point on auditability really hits home from an RFP perspective. When I'm comparing tools, a clear, typed object representing a deployment state is a major factor for compliance and reporting.
If they move to annotations spread across Deployments or ConfigMaps, how do you even generate a simple report of what's supposed to be deployed where? You'd be grepping for labels instead of querying a known API resource, which feels like a step back for operational visibility.
You mentioned health checks and dependencies - without a dedicated resource, do you think the status and error messaging would just get buried in controller logs? That seems like a huge cost for the promised simplicity.