So, I usually work with workflow automation tools, but I've been pulled into a project that needs a proper container orchestration setup. We've got three decently sized VMs provisioned, and I want to use them to learn Kubernetes hands-on before we potentially move to a managed service.
My primary goal is a simple, stable cluster for running a handful of services. I'm wary of overcomplicating things. I've seen k3s, kubeadm, and even minikube (though that seems more for single-node). My concerns are:
* **Operational overhead:** I don't want a full-time job managing the cluster.
* **Networking defaults:** Something that works out of the box without deep CNI research.
* **Upgrade path:** Being able to move to a newer version without a full rebuild.
Given my background, I appreciate tools that feel like "low-code" for infra—where the complexity is managed for you. What's the closest fit for a 3-node setup?
From my initial poking around, k3s looks promising. I might try a quick provisioning script to see how it works. Something like this on all nodes:
```bash
curl -sfL https://get.k3s.io | INSTALL_K3S_CHANNEL=stable sh -
# On the first VM, get the token for joining nodes:
sudo cat /var/lib/rancher/k3s/server/node-token
```
Then join the others. But is this too simplistic for a "real" starting point? What are the trade-offs compared to a kubeadm-built cluster?
if it's manual, it's wrong
I'm in a similar boat, coming from the martech side where I just want the infra to work! I tried both k3s and kubeadm on a small set of VMs last quarter.
Your instinct on k3s is solid for the "low-code infra" feel - it really does just run. The quick start script you found is exactly how I got my test cluster up. One caveat: the default sqlite database for the server is great for simplicity, but for a bit more resilience on a 3-node setup, you might consider using the embedded etcd option from the start. It adds a tiny bit more complexity to the install command, but makes upgrades and node failures less stressful later.
The networking defaults are indeed painless. Flannel just works. My only gotcha was with some ingress configurations on the stable channel, but switching to the latest channel fixed it.
What's your plan for storage? That was the next head-scratcher for me after the cluster was running.
You're absolutely right about the embedded etcd being the sweet spot for a three-node setup. I made the mistake of starting with sqlite on my first k3s cluster, and when my primary server VM had a storage hiccup during a kernel update, the whole control plane got... confused. Recovering was messier than it needed to be.
Your question on storage is the real next hurdle. The default local-path provisioner is fine for truly ephemeral stuff, but for any persistent data, you'll need a plan. Since you've got three VMs with presumably some local disk, I'd suggest looking at the Longhorn project (also from Rancher). It installs as a Helm chart onto your k3s cluster and gives you block storage replicated across your nodes. The overhead is minimal, and it feels like a natural fit for that "just works" vibe you've got with k3s and Flannel.
Did you run into any particular service that forced your hand on storage, or are you just planning ahead? I got bitten by a simple Postgres container I thought I could just leave running on a single node.
MigrateMentor
Oh, that's really helpful, thanks. I hadn't even thought about the storage piece coming back to bite me, but you're right, the local-path provisioner probably won't cut it if I'm experimenting with anything stateful.
> I got bitten by a simple Postgres container I thought I could just leave running on a single node.
This is exactly the kind of mistake I'd make. I was planning to try a small app with a database, so I guess I should look at Longhorn from the start. Do you find its replication adds noticeable latency, or is it pretty seamless for a learning setup?