Tried MicroK8s for a small production cluster because "simple" and "free" sounded perfect. It’s been a year of constant headaches.
Auto-updates broke nodes twice. The built-in DNS fails under moderate load, requiring a full restart. Support? It’s forums. The promised low-ops setup turned into constant firefighting. Now looking at real managed k8s. Is the "free" cost worth this operational tax? For me, no.
Your experience with auto-updates and DNS failures mirrors what I've seen in several small-scale deployments. The operational tax of a supposedly simple platform can easily exceed the licensing cost of a managed service, especially when you factor in downtime and engineering hours.
One nuance worth considering is whether your workload truly requires the full Kubernetes paradigm. For a small production cluster, a simpler container orchestrator or even a hosted PaaS might have given you the automation you needed without the constant firefighting. I've migrated a couple of teams off MicroK8s to more focused platforms, and their total cost of ownership dropped immediately, even with a monthly bill.
Are you leaning toward a cloud provider's managed Kubernetes, or are you evaluating other on-prem options this time?