RT-AC86U intermittent DNS_PROBE_FINISHED_NXDOMAIN

[BlN4RY]

Hi guys, long-time lurker, first-time poster.

For the last two months, I've been getting intermittent issues with websites not loading and getting stuck on DNS_PROBE_FINISHED_NXDOMAIN.
A manual reload of the website or waiting for chrome to auto-refresh causes the page to load.
This issue is observed on all devices under the router.

Setup: RT-AC86U with Merlin's 386.2_2 + Raspbi Pi4 (Pihole+Unbound+Media Server)
Raspberry Pi's IP is entered on WAN DNS, LAN DNS under DHCP, and DHCP filter is set to the Raspbi's IP for pihole and unbound to work. (Raspbi Pi was given a no filtering option)
Unbound is set up as a caching forwarder for DNS over TLS to Cloudflare.
All 4 ports and 2 USBs are populated on the router. One USB for scripts and the other for file sharing.
The setup was working without any errors for the past year.
The unbound config from Martineau's script was used with some device-specific modifications.

There are no specific logs when the issue happens on the router.
I've tried L&LD's nuclear reset and tried with 3 different versions of merlin.
Even a reset to the stock firmware with minimal config caused the issue to persist. (Using automatic WAN DNS and Ai Protection off)

The temperature on the CPU is hovering around 80C with a router-mounted fan in a room with a 35C ambient temp. (No thermal throttle on the logs)

Any idea on what could be causing these intermittent failures?

Please let me know if any additional information is needed.

 

[Cornel]

Also started noticing intermitent DNS_PROBE_FINISHED_NXDOMAIN on AC86U after moving to 386 release. Since then I factory reset twice following upgrades, tried playing with DNSSEC, using various DNS servers. I hoped more would report the same issue, but apparently not many enough.

 

[NotA+]

Are you using something like Nextdns or their DNS servers added to the router?

 

[BlN4RY]

Also started noticing intermitent DNS_PROBE_FINISHED_NXDOMAIN on AC86U after moving to 386 release. Since then I factory reset twice following upgrades, tried playing with DNSSEC, using various DNS servers. I hoped more would report the same issue, but apparently not many enough.

Glad to know I'm not the only one with this issue.
From when did you start noticing the issue? And any scripts on the router? Or a pihole?

 

[Cornel]

I believe it started following upgrade from 384.19 to 386.1. No pihole, or Nextdns used. After upgrading to 386.1 I performed a full router reset and remade all settings manually (also reset after 386.2). I aimed to keep thing simple so I also stopped using any AMTM AddOns such as Skynet, Entware or even an external USB drive, all used before without issues. I have jffs active though for a ddns-start script that I have been using for years to update Dnsomatic. Other than the above I use VPN clients and server, Adaptive Qos, AiProtection. I tried disabling DNSSEC validation to no avail, I have now disabled DNS Privacy with no errors so far, but will confirm later if this works (which would be really frustrating as I used it successfully in the past). The occurences are quite random, affecting various browsers, both connected via VPN or not.

 

[BlN4RY]

I believe it started following upgrade from 384.19 to 386.1. No pihole, or Nextdns used. After upgrading to 386.1 I performed a full router reset and remade all settings manually (also reset after 386.2). I aimed to keep thing simple so I also stopped using any AMTM AddOns such as Skynet, Entware or even an external USB drive, all used before without issues. I have jffs active though for a ddns-start script that I have been using for years to update Dnsomatic. Other than the above I use VPN clients and server, Adaptive Qos, AiProtection. I tried disabling DNSSEC validation to no avail, I have now disabled DNS Privacy with no errors so far, but will confirm later if this works (which would be really frustrating as I used it successfully in the past). The occurences are quite random, affecting various browsers, both connected via VPN or not.

The similarities that we have then are, AiProtection, VPN Server (IPSec). I have tried disabling Ai Protection, but it didn't have any change. Could VPN Server cause such an issue?

Also, any changes made to the DNS entries seem to delay the issue for a while. However, I have noticed the NXDOMAIN occurring frequently, when I ran more scripts on the router.

 

[sbsnb]

I occasionally experience this issue where a website will not load because of a failed DNS lookup and after one or more reload attempts it succeeds. While investigating the issue with nslookup I have seen that dnsmasq returns the error instantaneously without any time elapsed between pressing enter and receiving the error, with no possibility that it had actually submitted a query to upstream DNS providers and received a response. This has been going on for years and I have not been able to successfully diagnose the issue. I had always chalked it up to something with DNSCrypt proxy, but seeing your post makes me think perhaps it isn't.

 

[NotA+]

I was using Nextdns and DoT . In the end I removed nextdns as DNS. I switched to Cloudflare Gateway (free for upto 50 users) & that was fine no issues, which makes me think it was a Nextdns issue.

You could try Nextdns CLI but I did not.

 

[BlN4RY]

I was using Nextdns and DoT . In the end I removed nextdns as DNS. I switched to Cloudflare Gateway (free for upto 50 users) & that was fine no issues, which makes me think it was a Nextdns issue.

You could try Nextdns CLI but I did not.

I originally started facing the issue with cloudflare as my DNS. I've also tried ISP's DNS, Google DNS, cloudflare with DoH and DoT. Faced the issue with all of them.

Trying cloudflare gateway now.

 

[NotA+]

I had issues with Cloudflare free but no issues with the gateway version.
Nextdns in theory is a good idea but poorly executed.
I will roll out pfsense instead to block ads etc.

 

[BlN4RY]

I had issues with Cloudflare free but no issues with the gateway version.
Nextdns in theory is a good idea but poorly executed.
I will roll out pfsense instead to block ads etc.

Routed all DNS requests via Cloudflare gateway from unbound with DoT for about 12 hours, but the NXDomain issue was still there.

 

[NotA+]

Have you considered disabling unbound to see if that makes a difference?

 

[BlN4RY]

Have you considered disabling unbound to see if that makes a difference?

I have previously, but not with Cloudflare gateway. I'll try and report back.

 

[BlN4RY]

I have previously, but not with Cloudflare gateway. I'll try and report back.

Bypassed pihole and unbound altogether. DNS set on router WAN, the issue still persists.

 

[ColinTaylor]

Maybe this isn't a local problem or a DNS issue, but a problem with your ISP's network generally. Can you setup a continuous ping to somewhere like www.google.com and see if you loose all internet connectivity?

 

[BlN4RY]

Maybe this isn't a local problem or a DNS issue, but a problem with your ISP's network generally. Can you setup a continuous ping to somewhere like www.google.com and see if you loose all internet connectivity?

I just get a random NXDomain, its frequency seems to increase with the router's uptime and with the number of scripts running. The weird thing is, even if I refresh immediately, the webpage starts to load. So, might not be related to losing internet connectivity?

Did a continuous ping to google. There were some packet drops.

Ping statistics for 142.250.183.174:
Packets: Sent = 2301, Received = 2279, Lost = 22 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 14ms, Maximum = 64ms, Average = 14ms


Attached a connmon screenshot with ping to 1.1.1.1 for the last 7 days. Is this normal?

 

[ColinTaylor]

I don't use connmon so I don't know how accurate that graph is. But if it is accurate I would be concerned about that level of packet loss.

Your ping results also show an unacceptable level of packet loss IMHO.

I suggest you run a continuous ping on your main PC and the next time you get the DNS error look at it to see if it coincides with ping failures. That would indicate that it's likely an ISP issue.

 

[BlN4RY]

I don't use connmon so I don't know how accurate that graph is. But if it is accurate I would be concerned about that level of packet loss.

Your ping results also show an unacceptable level of packet loss IMHO.

I suggest you run a continuous ping on your main PC and the next time you get the DNS error look at it to see if it coincides with ping failures. That would indicate that it's likely an ISP issue.

Alright. Will try and see if the packet drop coincides with the DNS resolution failure.

Out of curiosity tho, since I have unbound and pihole's caching DNS, shouldn't repetitive queries be answered from the cache? I remember adding a "serve expired" on unbound config.

 

[Cornel]

I had more than 24h without one single NXDomain error with DoT deactivated. I'm not necessarily suggesting DNS Privacy is the issue here as long as a decrease in DNS performance is advised anyway, but this problem hasn't existed before 386.1 (at least not often enough to be noticed).

 

[ColinTaylor]

Out of curiosity tho, since I have unbound and pihole's caching DNS, shouldn't repetitive queries be answered from the cache? I remember adding a "serve expired" on unbound config.

I don't use unbound but my understanding is that would only apply to the first query (from the client), the query from unbound and any subsequent queries would still be sent upstream. And of course this has no effect on new uncached entries. Besides, you still had the same problem even without using unbound.