We've been running OpenShift 4.x for about two years on our own hardware. Last quarter, we completed a full migration to a vanilla upstream Kubernetes distribution (using kubeadm). The main drivers were operational overhead and cost.
OpenShift does a lot for you, but you pay for it:
* **Resource footprint:** The control plane components plus all the OpenShift-specific operators were consuming nearly 4 vCPUs and 16GB RAM before we even deployed our first app. Our plain K8s control plane uses less than half of that.
* **Upgrade friction:** Every minor version upgrade felt like a major project. The operator-based model meant waiting on numerous individual components, and we hit several snags where OpenShift-specific customizations blocked the process. With kubeadm, the upgrade path is predictable and faster.
* **Hidden constraints:** We weren't using the built-in registry, monitoring (we use Prometheus/Grafana), or service mesh. But we were still carrying the maintenance burden and complexity. The networking defaults (OpenShift SDN) also felt heavier than our current Calico setup.
The switch wasn't trivial. We had to build out our own CI/CD pipeline integrations and take ownership of security policies (which we were mostly doing anyway). But the transparency is worth it. We now have a clear line of sight from a performance issue or cost to a specific component we control.
For our scale (~50 nodes, ~400 pods), the "free" managed experience of OpenShift wasn't free in terms of resource consumption and team cycles. If you need the integrated platform and support, it's a valid choice. If you're already cloud-native tooling savvy, the upstream distribution is significantly leaner.
cb