Why DNS over TLS is so important, and if you are not using it you should be

[dave14305]

For me, port 53 gets hijacked and port 853 does not.

I’d still like to see more dnsleak samples with other DNS providers configured besides Google DNS. If they all use Comcast Naples FL servers, then you have a problem with hijacking.

 

[Viktor Jaep]

Same here. Been running DoT since the first time it arrived on Merlin FW. Never fail to respond to queries. Make sure the servers are good since i tried a few back then and sometimes it times out and when I checked on stubby it seems like some servers sometimes just stops working for a few and then works again for some reason.

Believe me, @Morris... I have a fairly large family, over 50+ wifi devices, and a wife going for her BS online almost 24x7. I would hear it if things went to sh*t. . It's been working great. I ended up switching from cloudfare to Quad9 a while back. Haven't looked back.

 

[Matthew Patrick]

Believe me, @Morris... I have a fairly large family, over 50+ wifi devices, and a wife going for her BS online almost 24x7. I would hear it if things went to sh*t. . It's been working great. I ended up switching from cloudfare to Quad9 a while back. Haven't looked back.

Been wanting to use quad9 but I'm afraid that since it blocks malware or I guess suspected malware sites, it could have false positives and it's gonna be hard to whitelist them.

 

[Viktor Jaep]

Been wanting to use quad9 but I'm afraid that since it blocks malware or I guess suspected malware sites, it could have false positives and it's gonna be hard to whitelist them.

It's best to just try it, and revert back if you run into issues. I really have had zero issues with blocking or false positives. They seem pretty liberal for the most part. I even stepped it up and created my own skynet firewall blocklist in addition to this... which probably puts Quad9's blocklist to shame.

 

[Paliv]

Been wanting to use quad9 but I'm afraid that since it blocks malware or I guess suspected malware sites, it could have false positives and it's gonna be hard to whitelist them.

I’ve never seen a false positive with Quad9. I have however had regular issues with failed DoT lookups.

 

[Treadler]

I’ve never seen a false positive with Quad9. I have however had regular issues with failed DoT lookups.

I’ve had ‘some’ issues with Quad9 DoT lookups (just sits staring at a blank page sometimes), currently trying Cleanbrowsing DoT & so far, no issues seen.

Contributing factors might be Quad9 is ~1700km from me, Cleanbrowsing ~700km?Dunno……..

 

[Swistheater]

Intermittent, about every month or two. Now you have heard of it. By hang, I mean fail to respond.

One of the problems with DoT and stubby is, stubby likes to round robbin by default, unless you disable it. Some dns providers will not like round Robin queries. They prefer their secondary dns only being asked upon a request failure on part of the primary. Round Robin indiscriminately queries one after the other. Think of the "hang" being the dns provider ratelimiting how many times stubby round robins their secondary dns servers.

 

[Treadler]

One of the problems with DoT and stubby is, stubby likes to round robbin by default, unless you disable it. Some dns providers will not like round Robin queries. They prefer their secondary dns only being asked upon a request failure on part of the primary. Round Robin indiscriminately queries one after the other. Think of the "hang" being the dns provider ratelimiting how many times stubby round robins their secondary dns servers.

I hear you.
I will experiment, Quad9 DoT, only selecting one server. Round robin out of the equation then?

 

[Swistheater]

I hear you.
I will experiment, Quad9 DoT, only selecting one server. Round robin out of the equation then?

I would just use the primary servers if you don't have the understanding to disable round robin, but if you are adventurous you can disable stubby round robin with stubby.postconf. If @RMerlin were to ever add another DoT WebUI option, I would definitely think round robin enable/disable would be a good option.

 

[Treadler]

I would just use the primary servers if you don't have the understanding to disable round robin, but if you are adventurous you can disable stubby round robin with stubby.postconf. If @RMerlin were to ever add another DoT WebUI option, I would definitely think round robin enable/disable would be a good option.

I’ll see how I go with just a single Quad9 server nominated.

If it fixes my issue, then I guess stubby.postconf is my next port of call.

As you say, a simple radio button to toggle round robin on/off would make it easy for an amateur like me.

 

[drinkingbird]

One potential reason for an ISP to want to intercept DNS queries is to redirect you to caching servers within their network for certain services like Netflix or Youtube. There are better ways to do that (through EDNS extensions and partnering with those providers), but that's one lazy way to do it.

Some providers also want to provide you with a generic error page when querying for a non-existent domain, so they want to point you to an internal server rather than the client getting an NXDOMAIN response. Yes, it's a completely stupid idea that breaks a lot of things such as spam control for mail servers, but quite a few ISPs experimented with that over the years.

Logging queries is unlikely to be a reason tho. They only need to monitor port 53 traffic to do that.

The main reason they do it here is for advertising revenue, the page you get with a list of search results when your lookup fails brings them revenue if you click one of those links (plus some revenue from the banner ads they show). Verizon lets you disable this behavior by pointing to DNS servers that are 1 IP up from the default ones you get assigned.. I've never seen an ISP actively intercept a DNS query to an outside server though. If Comcast is now actively intercepting queries to servers that aren't theirs, that's a new behavior. Not sure I'd be terribly concerned with it though, DNS queries are the least of our issues when it comes to privacy. As you say it could end up hurting you, if they are using their own DNS to make sure you use their local Netflix (and other) connections, those are the ones you want to be using, Netflix (and others) have direct 10G and 100G connections right into the major ISPs and they perform much better.

I did a ton of testing using a utility, I think it was DNSperf - and my ISPs servers were the fastest by far for cached domains (makes sense) and even for uncached they were either faster or equal, so I use theirs (the ones that don't redirect you on NXDOMAIN) with no encryption and no concerns.

If they are intercepting, would be curious (as a network guy) as to how they're doing it. Could be as simple as NAT'ing all traffic with UDP 53 destination to their own server IPs, or static destination NATs for all the common public DNS servers, etc. Seems anything more than that would require more significant investment, i.e. additional hardware etc, where NAT can be done right on their existing routers without issue, I'm sure they have plenty of memory and probably had the NAT timeout set fairly low. Since it is UDP traffic you can NAT the outbound query but not the inbound response (which is a totally separate connection) which would explain why OP is seeing the comcast servers responding rather than appearing to come from the outside server.

Not aware of anything in Cisco that would allow intercept and redirect like this other than NAT, however Juniper does have more features as far as basic firewall etc. Then again the Cisco ASRs have a lot more features than previous routers and I haven't played with them al. I believe Comcast uses both brands as do many network providers (including the one I work for) for redundancy (some sort of vulnerability or bug that takes down one brand will hopefully not impact the other). However our Cisco gear is far more reliable and less buggy than Juniper. Cost is a factor too, Juniper is about 1/2 the price.

Cisco has turned into a combination of HP (the switch or router price doesn't seem too bad until you start buying SFPs for it which are outrageous, similar to ink in HP printers) and Intel (many of their processors are actually the top of the line one with a bit of code that limits it to slower speeds, cheaper to just make one processor). Though Intel hasn't caught up to Cisco where you can just pay for a license to unlock the full speed/capability. Cisco calls it "performance license" or 'security license" if you want to utilize the onboard (but default disabled) hardware encryption module.

We buy SFPs from Cisco in batches of $1M and that gets us a 92% discount - what does that tell you about the actual cost of SFPs for them? List price for a 10G SFP is nearly the price of a nice 48 port Nexus switch, and a 100G SFP costs more than an ASR 9K router. Juniper just charges you a fraction of the amount regardless of how many you buy (with some small discounts for bulk orders).

/Tangent
/Tangent

 

[RMerlin]

If they are intercepting, would be curious (as a network guy) as to how they're doing it. Could be as simple as NAT'ing all traffic with UDP 53 destination to their own server IPs, or static destination NATs for all the common public DNS servers, etc.

Probably a simple DNAT for all outbound port 53 traffic.

 

[bbunge]

I would just use the primary servers if you don't have the understanding to disable round robin, but if you are adventurous you can disable stubby round robin with stubby.postconf. If @RMerlin were to ever add another DoT WebUI option, I would definitely think round robin enable/disable would be a good option.

My stubby.postconf to change roundrobin and enable DNSSEC in Stubby:

#!/bin/sh
CONFIG=$1
source /usr/sbin/helper.sh
pc_replace "round_robin_upstreams: 1" "round_robin_upstreams: 0" $CONFIG
pc_insert "tls_authentication: GETDNS_AUTHENTICATION_REQUIRED" "dnssec_return_status: GETDNS_EXTENSION_TRUE" $CONFIG

Note: if you enable DNSSEC this way in Stubby do not enable it in the GUI.

 

[ludespeedny]

Is there a good tut out there on how to setup/configure? I have it set up, but not sure as it is working or not.

 

[dave14305]

I have it set up, but not sure as it is working or not.

Quad9 is WoodyNet. So that’s OK.

 

[ludespeedny]

Ok, so based on my results, it is working correctly?

 

[Paliv]

Ok, so based on my results, it is working correctly?

Yes

 

[ludespeedny]

Great! Thanks!

 

[Paliv]

Interestingly several comcast users aren't seeing a DNS hijack. It makes me wonder if something else is up, or if it is a regional test thing. It is definitely not company wide as I do not have this issue at the moment.

 

[Treadler]

Interestingly several comcast users aren't seeing a DNS hijack. It makes me wonder if something else is up, or if it is a regional test thing. It is definitely not company wide as I do not have this issue at the moment.

I’m not with Comcast, but DNS hijack not something I’ve had to deal with.