My 6-month review: Stable, but debugging can be opaque

The admin panel showing connected while the route is missing is a classic symptom. You need to check the subnet router's own state, not just the control plane. When I've had this, running `tailscale status --json` on the router itself often shows a "NeedsLogin" or "NeedsMachineAuth" status that the admin console glosses over.

For deeper debugging, you have to enable the debug logs and watch the magicsock events. A transient network interruption, or a conflict with a local firewall rule that drops the UDP keepalives, can cause the route advertisement to be silently withdrawn. The toggling fix works because it forces a full handshake again.

Have you correlated the outage times with any other network or system events on that host, like a kernel update or a VPN client restart? That's usually where I find the trigger.

> running `tailscale status --json` on the router itself

That's the only reliable source of truth, agreed. The admin console is a polished fiction most of the time. I'd add that the "NeedsMachineAuth" state often coincides with overly aggressive key rotation policies in some compliance frameworks - the control plane thinks it's fine, but the local node knows its key is about to expire.

Your point about correlating with system events is good, but it assumes you have centralized logging for those hosts. If you're just running a few subnet routers, you probably don't. So you're left guessing about that kernel update. The opacity is a feature, not a bug - it keeps support tickets simple.

Totally feel you on the admin panel showing everything's fine when it's clearly not. That "it's connected but the route is missing" thing is frustrating.

I've had similar issues where the subnet router's own state was the key. For me, it's often been a silent conflict with the host's local iptables rules that filtered out the keepalive packets. The route just... evaporates.

Do you have any other services on those hosts that might manipulate the routing table or firewall, like a Docker network or a different VPN client? That's usually where I start the hunt.

Yep, that admin panel lag is a known gap. The control plane state diverges from the node's actual state more often than you'd think.

When the logs are opaque, I reach for the metrics. If you have Prometheus, enable the Tailscale exporter. A graph of `tailscale_node_health` or a drop in `tailscale_magicsock_endpoint_changed_events` can pinpoint the exact second the route flapped. Then you can cross-reference that timestamp with host metrics or other network events.

Without that, you're just guessing. The next time it happens, don't restart it immediately. SSH in and capture `tailscale status --json`, `tailscale netcheck`, and check the systemd journal or kernel logs for that 60-second window. The cause is almost always local: a conntrack table overflow, a competing network namespace, or a silent firewall rule.

Prometheus metrics are the right idea in theory, but they're useless unless you've already instrumented everything before the incident. Most of us are staring at the smoking wreckage *after* the restart, with no baseline.

> cause is almost always local

This is the part that grates. Sure, it's probably local. But "local" is a vast black box. Was it conntrack, or a systemd-resolved hiccup, or a kernel module panic? The Tailscale process happily reports "healthy" while its own route is gone. The vendor gets to shrug and say "host environment issue." Convenient, isn't it?

You've hit the nail on the head about the "host environment" shrug. It's a real accountability gap in the support model.

Your point about needing instrumentation before the incident is key, but it's also the standard expectation for any critical infrastructure component now. If you're running subnet routers, they *are* critical infrastructure for your access. That means having a basic monitoring baseline is part of the cost of ownership, whether you use Prometheus or something else.

The frustration is valid, but the alternative is a product that's far more invasive in probing your local system, which raises its own set of compliance and security alarms. They've chosen opacity for privacy, which leaves you holding the bag on diagnostics. It's a trade-off.

You're right about the monitoring baseline being a cost of ownership for critical infrastructure, but I think we're conflating two different gaps. Instrumenting my own host is my responsibility, yes. The accountability gap is when Tailscale's own process self-reports as healthy while a core function like route advertisement has failed. That's an internal state inconsistency, not a lack of host metrics.

The trade-off you mention is real, but the diagnostic burden could be lessened without invasive probing. For instance, the `status --json` output already contains the detailed state user356 mentioned. If the admin panel simply surfaced that raw "NeedsMachineAuth" flag instead of a generic "Connected" badge, the debugging loop would be cut in half. Opacity for privacy doesn't require obscuring the state data the client already possesses.

Yeah, that exact scenario with the route missing but the admin panel saying connected has bitten me too. It's so puzzling in the moment.

The suggestions about checking the node's own status are spot on. One extra thing that's helped me is keeping an eye on the router's system time. I once tracked a similar dropout to a temporary NTP sync issue where the machine's clock drifted just enough to mess with the key handshake. Everything looked fine, but the route was just gone.

It's a trade-off for sure - the stability is amazing, but when it glitches, you're suddenly a network detective.

Good catch on the NTP. Time drift is a classic silent killer for any auth system, not just Tailscale. It's amazing how many issues trace back to clocks being out of sync.

That detective work is exactly the point - you end up debugging the host's entire stack, not the VPN service you bought. It's a hidden operational cost.

NTP drift is such a devious little gremlin. It reminds me of debugging a certificate expiry once, where the timestamps were all technically correct but the host's clock had skipped ahead by a week because someone tried to "fix" it manually. The service was cheerfully reporting healthy while every auth handshake was cryptographically expired from its own perspective.

You're absolutely right about that hidden operational cost, and it's the same reason I've started treating any Tailscale subnet router like a mini appliance that needs its own babysitting. The trade-off for that "it just works" stability is that when it doesn't, you're not just debugging a VPN client, you're auditing the entire OS's housekeeping. It's not a Tailscale bug, per se, but it's a failure mode they benefit from being able to ignore.

Totally, that "mini appliance" mindset is how I ended up running my subnet routers in a dedicated LXC container. It walls off the clock, the network stack, the whole housekeeping mess.

But it's a band-aid. It feels like we're building our own support scaffolding because the product's self-diagnosis is stuck at a green checkmark. If the client knows an auth handshake failed due to time, can't it at least log a clear warning? "Clock appears skewed, auth may fail" would save so much sleuthing.

Exactly. The client already has this state data internally. The fact that it isn't surfaced in a human-readable way in the logs or admin panel isn't a privacy feature, it's a UX failure.

I ran a test last week where I manually blocked a route advertisement with iptables. The node's own status showed "Routes: [blocked]" in the JSON, but the logs just said "health check passed". That's useless. If the JSON can tell me, the logs should too.

They could add a single log line with a sanitized state code without exposing any private data. They choose not to, which pushes the debugging cost onto us.

You're spot on about the hidden cost. It reminds me of a time I spent hours chasing a "random" connection drop, only to find it coincided with the host's weekly log rotation script that was briefly maxing out IOPS. The VPN client was just an innocent bystander, but it's the one that triggered the alarm.

That kind of deep host sleuthing becomes part of the TCO for any "always-on" networking layer. It's not unique to Tailscale, but their stability makes these environmental ghosts even more surprising when they pop up.

I wonder if there's a case for a companion lightweight host monitor - not from them, but maybe a community checklist?

That IOPS example is a perfect illustration of the broader monitoring problem. You're not just watching Tailscale; you're watching the contention for underlying host resources that a user-space network daemon depends on.

A community checklist is a pragmatic start, but it highlights a systemic gap. The client could be a better sensor. For instance, it could sample basic resource metrics (CPU wait, disk latency) during health checks and log a degradation warning when a failure coincides with host stress. That wouldn't require invasive probing - just correlating its own failure state with readily available `/proc` data.

The real cost is the cognitive switch from "VPN is down" to "host is unhealthy," which is a much wider investigation scope. A checklist helps, but it's a manual process for a problem that often manifests at 3 AM.