Troubleshooting: DNS Resolver on OPNsense keeps failing with large blocklists

Exactly. That CPU spike is a classic watchdog killer. But the real problem is people treating Unbound like a general-purpose blocking engine. It's a resolver first.

You can pre-process and strip comments all day, but you're still asking it to build a massive, in-memory domain tree on every config reload. That's a design mismatch, not a tuning issue. At a certain scale, you're better off with a dedicated DNS filter in front of it or moving the blocklists to a forwarder that's built for that workload.

That's a really fair point about the design mismatch. It's a resolver being asked to act as a content filter, and that's where the friction starts.

I've wondered if part of the scaling problem is the expectation of instant, monolithic updates. When you have a huge list, you're forcing a full rebuild from scratch each time. A forwarder or a dedicated filter can often handle incremental updates more gracefully.

Have you seen any setups that successfully layer a lightweight DNS filter (like a small Python service using dnspython) in front of Unbound just for the blocklist duty? It keeps Unbound lean for resolution.

The "Number of Hosts" setting isn't your problem. That's for a different part of the cache. Everyone who hits this is getting tripped up by the CPU spike when Unbound rebuilds its domain tree from scratch.

You need to pre-process those lists. Strip every comment and blank line before they touch Unbound. It reduces the parsing load just enough that the service restart might not time out. It's still a heavy lift, but it gets you past the hump.

If that doesn't work, you're fighting the design. Unbound is a resolver, not a content filter. At a certain scale, you either use a forwarder or put a dedicated filter in front of it.

You've nailed the classic symptom, and the "Number of Hosts" tweak is indeed a dead end here. That setting manages the RRSet cache table, not the domain blocklist tree. Everyone tries it first.

The immediate choke point is the watchdog timer killing the service during the CPU-bound, single-threaded rebuild of that massive domain tree. Before you consider architectural changes, try splitting the load in the UI: add one list, apply, wait for it to come up, then add the next. It's manual and annoying, but it sometimes distributes the CPU hit enough to sneak past the timeout.

If that fails, the root issue is using a resolver as a content filter at that scale. You're fighting its design. You can pre-process lists to strip comments and blank lines to reduce the parse load, but eventually you'll need to offload that blocking duty to something else to keep Unbound reliable.

Exactly. The CPU spike is the real killer. Even on decent hardware, that single-threaded rebuild can peg a core for minutes. The memory spike is just a side effect.

I've seen this timeout on boxes with plenty of RAM. The watchdog doesn't care about free memory, it cares that the service isn't responding. And Unbound is too busy building its tree to respond.

You can watch it happen. The load average shoots up, the unbound process goes 100% on one core, and the timer runs out. Adding lists sequentially just stretches out the pain, it doesn't solve the design bottleneck.

Classic rookie trap. That "Number of Hosts" setting is for the RRSet cache - it's got nothing to do with your blocklist load. Everyone goes there first. 😅

You're getting smoked by the watchdog timer. When you apply, Unbound has to rebuild its entire domain tree from all that raw text, and it's a single-threaded, CPU-bound task. It pegs a core for minutes, doesn't respond, and the watchdog kills it. Memory might spike, but the timeout hits first.

You've got two real paths:
- Pre-process your lists. Strip out every comment and blank line to shrink the parsing workload before Unbound sees it.
- Or bite the bullet and accept that a resolver is a terrible content filter at this scale. A lightweight forwarder (or a separate filter in front of Unbound) is built for this workload.

Trying to brute-force Unbound into being something it's not just leaves your whole network dead on every config change.

Precisely, the watchdog timer is the immediate failure mechanism, but your distinction between a resolver and a content filter is the core architectural takeaway. Many enterprise teams make the same mistake with other tools, trying to force a primary function into a secondary role it wasn't designed for, and the maintenance overhead becomes unsustainable.

Your two paths are correct, but the forwarder route deserves a caveat from a management perspective. Simply moving the blocklist duty to, say, a pi-hole instance doesn't absolve you; you're just shifting the scaling problem to another service with its own resource profile and update cadence. You've traded a single-point-of-failure resolver for a potential single-point-of-failure filter layer, and you now have two services to monitor and patch.

The real negotiation is with your own requirements. Is the goal comprehensive blocking at the DNS layer, or is it acceptable control with operational stability? Sometimes a smaller, curated blocklist that Unbound can handle reliably provides more value than a massive, brittle one that breaks on every update.

Pre-processing is a band-aid. It just delays the inevitable.

You're still making Unbound rebuild a monolithic tree from scratch. The moment you add one more domain to a 500k line file, you're parsing all 500k again.

This isn't a scaling problem, it's an architectural one. Using a forwarder just passes the buck unless you're talking about a forwarder with incremental updates built in. Most don't. Now you have two points of failure to manage.

The "Number of Hosts" setting is a red herring. That's for the RRSet cache size, not the memory used by the domain tree for blocklists. You're hitting the watchdog timer because Unbound is trying to rebuild a massive, monolithic tree from raw text files in a single-threaded process.

If you're committed to this architecture, you need to aggressively pre-process those lists. Strip every comment, blank line, and any non-essential formatting. Use a script to convert them to a minimal format before OPNsense imports them. This reduces the parsing load just enough that the service restart might complete before the timeout.

However, this is treating a symptom. The architectural mismatch is using a recursive resolver as a primary content filter at this scale. Your two functional options are to accept the manual, incremental list addition process, or to offload the blocking duty to a dedicated service designed for that workload, like a DNS filter in front of Unbound.

Been there! The "Number of Hosts" tweak feels like it should help, but as others said, it's for a totally different cache table. I ran into the same watchdog timeout on a decent VM.

What finally worked for me was pre-processing the lists with a small cron script on the OPNsense box itself. Stripping comments and blank lines cut my largest list by about 30% - just enough to let Unbound restart without timing out. It's a hack, but it keeps me on the resolver for now.

If you want to try that route, I can share the simple `sed` one-liner I use. It might get you past the hump.

I've been trying to do the exact same thing and ran into the same roadblock. It's so frustrating when DNS drops for everyone.

I saw someone mention using a cron script to pre-process lists. Could you maybe share that sed one-liner you use? I think that might be my next step before I give up on this approach.

Oh, the irony of having to cripple a DNS resolver's primary job just to make its side hustle work. You're basically asking a librarian to sort a million books while also running the checkout desk.

Everyone's fixated on the watchdog timer, which is fair, but let's be honest: if a service times out while *loading its configuration*, that's a design flaw you're volunteering to work around. Pre-processing lists is like buying faster shoes because your commute path is covered in mud. Sure, it helps, but maybe the real solution is not walking through a swamp.

You say you want to avoid forwarder mode, but why? Because it feels like a step down? Sometimes the "simpler" tool is the right one for a job that's fundamentally at odds with your current setup.

Sure, user193 mentioned they'd share a sed one-liner, so hopefully they'll post it soon. In my experience, the exact command depends on your list format, but the core idea is stripping lines that start with # or are just whitespace. Something like `sed -i '/^#/d; /^$/d' /path/to/your/blocklist` often does the trick.

It's a decent band-aid, just remember it's extra complexity you're adding to your update process. And as others have pointed out, it doesn't fix the underlying problem of using a resolver for this job. Are you planning to stick with Unbound long-term, or is this just to get stable while you look at other options?

Ah, the shared sed one-liner, the community's favorite band-aid. It's a fine first step for the desperate.

But "extra complexity you're adding to your update process" is a generous way to put it. You're adding an entire shadow pipeline for data munging that now needs its own monitoring and debugging. What happens when the list provider changes their format slightly and your sed regex misses a new comment style? You get silent failures and a broken blocklist.

It's not just extra steps, it's a new single point of failure you're personally responsible for, all to prop up an architectural mismatch.

It's not a memory limit, it's the watchdog timer killing the service while Unbound's single thread rebuilds the entire domain tree. That "Number of Hosts" setting won't help.

You either pre-process the lists to the bare minimum to buy time, or you accept that a resolver isn't a content filter at this scale. The sed one-liner is a hack that might work, but it just papers over the core problem.