DNSSEC DNS on RT-AX86U Pro causing some websites not to load properly

[Intrepid]

When using Quad9 DNS on my router, if I turn ON DNSSEC support on my ASUS RT-AX86U Pro router business.comcast.com will not load properly. When I turn DNSSEC Off the problem is resolved. I'm working with Quad9 technical support currently. Has anyone else heard of DNSSEC on Asus Routers causing issues like this? Thanks!

 

[bbunge]

When using Quad9 DNS on my router, if I turn ON DNSSEC support on my ASUS RT-AX86U Pro router business.comcast.com will not load properly. When I turn DNSSEC Off the problem is resolved. I'm working with Quad9 technical support currently. Have anyone else heard of DNSSEC on Asus Routers causing issues like this? Thank You.

Yes. That is why I use Cloudflare secure. 1.1.1.2 and 1.0.0.2

 

[dave14305]

I tallied 94 different domains that would have to pass DNSSEC validation when that page loads. One of them failing might be enough to break the page loading.

You can learn a lot watching the browser console Network tab during the page load.

 

[Intrepid]

Yes. That is why I use Cloudflare secure. 1.1.1.2 and 1.0.0.2

Thanks. Do you use DNSSEC and/or DOT with Cloudflare secure. 1.1.1.2 and 1.0.0.2?

Quad9 replied:

"DNSSEC should not be enabled on a DNS forwarder or DNS client using a recursive service like Quad9 which already performs DNSSEC validation

https://docs.quad9.net/Quad9_For_Or...der_Best_Practices/#disable-dnssec-validation
["Since Quad9 already performs DNSSEC validation, DNSSEC being enabled in the forwarder will cause a duplication of the DNSSEC process, significantly reducing performance and potentially causing false BOGUS responses."]

Although we don't have a Setup Guide for ASUS routers with DNS over TLS support, all of our other open-source router guides on https://docs.quad9.net explicitly state that DNSSEC should be disabled at the forwarder level.
["DNSSEC is already enforced by Quad9, and enabling DNSSEC at the forwarder level can cause false DNSSEC failures."]

I'm guessing some other FQDN used for loading resources on that page is failing to resolve properly when DNSSEC enabled on the router."

I think ASUS router firmware should be changed to disable the use of DNSSEC with Quad9.

 

[Intrepid]

Quad9 replied:

"DNSSEC should not be enabled on a DNS forwarder or DNS client using a recursive service like Quad9 which already performs DNSSEC validation
https://docs.quad9.net/Quad9_For_Or...der_Best_Practices/#disable-dnssec-validation

Although we don't have a Setup Guide for ASUS routers with DNS over TLS support, all of our other open-source router guides on https://docs.quad9.net explicitly state that DNSSEC should be disabled at the forwarder level.

I'm guessing some other FQDN used for loading resources on that page is failing to resolve properly when DNSSEC enabled on the router."

I think ASUS router firmware should be changed to disable the use of DNSSEC with Quad9.

 

[bbunge]

Release - ASUS RT-AX86U Pro Firmware version 3.0.0.6_102_34313 - 2024/05/28

Firmware version 3.0.0.6_102_34313-g90fb2f8_385-g2830d - Release Note - Bug Fixes and Enhancements： - Fixed IPV6 connection issue. Available from router webUI... upgraded my AiMesh without reconfiguration. Available from non-US website...

www.snbforums.com

 

[Intrepid]

This is pretty much BS. The purpose of DNSSEC is to validate the packets coming from the upstream resolver to make sure thy have not been tampered with in transmission. Many other upstream resolvers do not have an issue with DNSSEC or the combo of DNSSEC with DoT. Unfortunately, Quad9 resolvers do not play well with DNSSEC or DoT. And this is nothing new. Back in the day when we were working to get Stubby to work on Asus routers I continually had to switch to some other upstream resolver because the connection to Quad9 failed. Back then Stubby did DoT as well as DNSSEC validation.

Oh, it is not just Asus routers that have problems with Quad9 and DNSSEC. I have had issues with Pi-Hole on Quad9 with DNSSEC enabled.

So, don't blame Asus or Merlin. Switch to Cloudflare Secure or Cleanbrowsing.

Thanks. This is why I prefer Quad9 over Cloudflare. I primarily care about malicious site blocking and security (video queued):

 

[bbunge]

Thanks. This is why I prefer Quad9 over Cloudflare. I primarily care about malicious site blocking and security (video queued):

Actually, Quad9 was rated lower than other services.

So, if you must just do not use DoT and DNSSEC. Better yet, set up a pihole and use several of the available blocking lists.

 

[Intrepid]

Actually, Quad9 was rated lower than other services.

So, if you must just do not use DoT and DNSSEC. Better yet, set up a pihole and use several of the available blocking lists.

Huh? No, it was rated the best. It blocked the most malicious sites (missed only 0.79%) while Cloudflare missed 33%.

 

[bbunge]

Huh? No, it was rated the best. It blocked the most malicious sites (missed only 0.79%) while Cloudflare missed 33%.

Not talking about Cloudflare 1.1.1.1 and 1.0.0.1

Use Cloudflare Secure 1.1.1.2 and 1.0.0.2

Ah, I see that idiot did review Cloudflare Secure.

And you are going to believe someone on Youtube? If so I have some prime land to sell you in Southern Florida....

 

[RMerlin]

"DNSSEC should not be enabled on a DNS forwarder or DNS client using a recursive service like Quad9 which already performs DNSSEC validation
https://docs.quad9.net/Quad9_For_Or...der_Best_Practices/#disable-dnssec-validation

I disagree with that. DNSSEC in that setup would validate that there is no DNS interception between Quad9 and the authoritative DNS. That does not mean there couldn't be DNS interception between Quad9 and you if you keep DNSSEC disabled on your end. Unless I'm missing something there, which is entirely possible.

 

[Intrepid]

Not talking about Cloudflare 1.1.1.1 and 1.0.0.1

Use Cloudflare Secure 1.1.1.2 and 1.0.0.2

Ah, I see that idiot did review Cloudflare Secure.

And you are going to believe someone on Youtube? If so I have some prime land to sell you in Southern Florida....

Here is another test where Quad9 blocked the most malicious sites. That's my primary concern. If Cloudflare blocked more then I would use them. I don't owe any allegiance to Quad9.

Malicious site filters on DNS in 2020 – Skadligkod.se

 

[Intrepid]

I disagree with that. DNSSEC in that setup would validate that there is no DNS interception between Quad9 and the authoritative DNS. That does not mean there couldn't be DNS interception between Quad9 and you if you keep DNSSEC disabled on your end. Unless I'm missing something there, which is entirely possible.

That's what I thought. I wonder why this is an issue with Quad9. I'll ask them and let you know what they say.

 

[RMerlin]

That's what I thought. I wonder why this is an issue with Quad9. I'll ask them and let you know what they say.

One potential issue with DNSSEC is packet size. Some nameservers use a buffer that's too small to contain a full DNSSEC response, some resolvers also allocate a buffer that's too small. I believe both Asus and I should be using the buffer size with optimal compatibility at this point in dnsmasq, but I can't speak for Stubby.

 

[Intrepid]

One potential issue with DNSSEC is packet size. Some nameservers use a buffer that's too small to contain a full DNSSEC response, some resolvers also allocate a buffer that's too small. I believe both Asus and I should be using the buffer size with optimal compatibility at this point in dnsmasq, but I can't speak for Stubby.

Here is my question to Quad9 and their reply. I'm not a DNSSEC expert, so if you have a suggestion for a follow-up question, that would be nice. Thanks. BTW, I didn't notice any issues with other domains with DNSSEC enabled except business.comcast.com. I'm sure there must be others.

Question:
After doing some reading online, I have some questions. Other DNS providers have no issue with enabling DNSSEC at the router / forwarder. I’ve read that Quad9’s implementation validates there is no DNS interception between Quad9 and the authoritative DNS. But that does not mean there couldn't be DNS interception between Quad9 and the user if required to keep DNSSEC disabled on their end as specified by Quad9. Unless I'm missing something – I’m a bit confused. I hope you can shed some light on this confusion. Thank You.

Quad9 Response:
While, in theory, DNSSEC should work at the client/forwarder level, in practice, it causes subtle problems like this. While there are certainly some thousands, tens of thousands, or hundreds of thousands of devices using Quad9 with DNSSEC enabled on the forwarder or client level, there are always little issues like these that crop up. [Like breaking major websites such as business.comcast.com]

DNSSEC validation is critically time sensitive. Devices without a real-time clock (RTC), or rather, a CMOS/watch battery, to keep extremely accurate time, will experience clock drift, and even some amount of milliseconds can start throwing BOGUS responses. The most-common occurrence of this is running DNS forwarders, like Pi-Hole, on a Raspberry Pi, which has no real-time clock. I have no idea whether your router has a real-time clock; I assume yes, but I thought this was an important point anyway.

Another issue is very-low TTL resource records (RRs). Anything under 60 seconds. While the forwarder/client device is asking the recursive service (Quad9) to manually re-fetch the DNSSEC records, that the TTL of the original RR could've expired during that time, causing a SERVFAIL. If that were to happen on the Quad9 side, our recursive software would hold your query until a fresh DNSSEC validation is initiated. <60s TTLs are unfortunately more common, and with that, comes more problems. As it is, one of the CNAMEs in the DNS lookup in your use case has a 20s TTL: e26014.dscx.akamaiedge.net. That DNSSEC failure would then be cached for a small amount of time, and the cycle repeats. By not caching DNSSEC failures, with would reduce these issues, but would open ourselves to various attacks vectors.

The last issue is that, it's frankly a massive waste of resources. Forwarders or clients doing their own DNSSEC validation greatly increases the cost of the DNSSEC validation for each domain which is DNSSEC-signed. Quad9's stance is: "If you don't trust Quad9 to perform DNSSEC validation, you should be running your own recursor".

With regards to the security aspect of this (MITM), the answer is Encrypted DNS [like DoT or DoH ], not DNSSEC on the client/forwarder. Encrypted DNS cannot be tampered with (MITM), unless that device is part of a corporate network which uses MITM as a security mechanism, in which case, that's a completely different topic.

Also See: Costs and Benefits of Local DNSSEC Validation
https://cyounkins.medium.com/costs-and-benefits-of-local-dnssec-validation-53c72ca9059b

 

[Treadler]

When using Quad9 DNS on my router, if I turn ON DNSSEC support on my ASUS RT-AX86U Pro router business.comcast.com will not load properly. When I turn DNSSEC Off the problem is resolved. I'm working with Quad9 technical support currently. Has anyone else heard of DNSSEC on Asus Routers causing issues like this? Thanks!

Well spotted. Cloudflare here, & same problem seen with that site with DNSSEC enabled. Interesting…….

 

[Intrepid]

Well spotted. Cloudflare here, & same problem seen with that site with DNSSEC enabled. Interesting…….

Thanks for confirming this is not just a "Quad9 issue". I don't trust DNSSEC anymore - if business.comcast.com doesn't work there certainly are many others. I will leave it off on the router.

 

[SkippyP]

If I use Quad9 DoT or Cloudflare DoT, that site loads perfectly fine with or without DNSSEC support enabled on my AX86U Pro. The only time that site doesn’t load properly is when using my ISP’s DNS with DNSSEC support enabled on my router.

1. Quad9 DoT with DNSSEC enabled = site loads properly

2. Quad9 DoT with DNSSEC disabled = site loads properly

3. Cloudflare DoT with DNSSEC enabled = site loads properly

4. Cloudflare DoT with DNSSEC disabled = site loads properly

5. ISP DNS with DNSSEC enabled = site does not load properly

6. ISP DNS with DNSSEC disabled = site loads properly

 

[Intrepid]

If I use Quad9 DoT or Cloudflare DoT, that site loads perfectly fine with or without DNSSEC support enabled on my AX86U Pro. The only time that site doesn’t load properly is when using my ISP’s DNS with DNSSEC support enabled on my router.

1. Quad9 DoT with DNSSEC enabled = site loads properly

2. Quad9 DoT with DNSSEC disabled = site loads properly

3. Cloudflare DoT with DNSSEC enabled = site loads properly

4. Cloudflare DoT with DNSSEC disabled = site loads properly

5. ISP DNS with DNSSEC enabled = site does not load properly

6. ISP DNS with DNSSEC disabled = site loads properly

That's not the results I got. But I kept DoT off and only tested with DNSSEC and Quad9. I'll have to test with both enabled - don't know why that would make a difference. In addition, I flushed the DNS caches between tests!

DNS flush commands for those interested:

Windows Flush DNS: ipconfig /flushdns or reboot
Cache TTL: 24 hours (negative is 5 minutes)

Firefox Flush DNS: about:networking#dns
Cache TTL: 2 Minutes

Chrome Flush DNS: http://chrome//net-internals/#dns
Cache TTL: 1 Minute

Asus Router DNS flush: Reboot router, or SSH command killall -HUP dnsmasq or killall -1 dnsmasq or through Merlin GUI.
Cache TTL: 6 to 24 hours

Windows Server DNS Flush: dnscmd /clearcache
Cache TTL: 24 hours

 

[SkippyP]

I don’t see how DoT would make a difference either. But that’s just how I tested.

When configuring Quad9 or Cloudflare, I kept the IPv4 DNS fields in the WAN tab as “get DNS from ISP automatically” and the IPv6 DNS fields as “connect to DNS server automatically.” I only configured the Quad9 and Cloudflare addresses in the DoT table since whatever is in that table overrides the other settings anyway.

Each time I changed DNS providers, I verified that I was using the correct resolver before testing the Comcast Business site.