Stubby-Installer-Asuswrt-Merlin

[DonnyJohnny]

I believe overhead is inevitable if we need the extra cover of last mile from ISP. Like many articles already states, isp will still know domain u visit due to SNI sent in plain. We are just making it more troublesome for them. Still waiting for full implementation of ESNI (TLS1.3)
I will still use dot or doh for the extra security from hijack.

 

[kvic]

I believe overhead is inevitable if we need the extra cover of last mile from ISP. Like many articles already states, isp will still know domain u visit due to SNI sent in plain. We are just making it more troublesome for them. Still waiting for full implementation of ESNI (TLS1.3)
I will still use dot or doh for the extra security from hijack.

Or you can use Tor or other stealth tech...

I respect your choice but we have quite a different view and priority on technologies. I don't appreciate a simple statement to make a siding.

Some more digging..numbers etc.. in performance (e.g. as suggested by Merlin) perhaps can steer the discussion into a meaningful way. Or else we could stop here. That's equally good to end well.

 

[scjr]

I'd say the same applies (to a lesser degree) to anyone using one of the numerous filtering DNS services such as OpenDNS. A lot of these will break CDNs to various levels, depending whether or not they support EDNS (if I remember correctly, Cloudflare is one that does NOT support EDNS). They will always bypass any local ISP cache, which for services like Netflix or Youtube can provide a significant performance degradation in some cases.

It's a compromise, based on one's needs: performance versus security.

Hi Eric,

Could be uneducated questions, so bear with me. I’m running DNSSEC and DoT using Cloudflare. I use Netflix and YouTube quite a bit. What kind of performance degradation should I be looking for?

I haven’t really noticed anything, but maybe I’m having issues that I don’t know of? Should I just use my ISP dns servers, which are just marginally faster than Cloudflare’s?

Thank you.

 

[kvic]

Hi Eric,

Could be uneducated questions, so bear with me. I’m running DNSSEC and DoT using Cloudflare. I use Netflix and YouTube quite a bit. What kind of performance degradation should I be looking for?

I haven’t really noticed anything, but maybe I’m having issues that I don’t know of? Should I just use my ISP dns servers, which are just marginally faster than Cloudflare’s?

Thank you.

Sorry to step in. I'm fully aware that you aren't addressing the question to me.

You're asking a tricky question. Don't you..?

I think if you're happy with it you can continue to use it.

Perhaps we could wait for people working on DoT/DoH to do some performance measurement if there is time. Might not be in a few days I would say. But in weeks/months likely. Well, I don't know perhaps that it won't come.

Anyhow that won't change the fact that you're happy with it and continue to use it.

 

[scjr]

Sorry to step in. I'm fully aware that you aren't addressing the question to me.

You're asking a tricky question. Don't you..?

I think if you're happy with it you can continue to use it.

Perhaps we could wait for people working on DoT/DoH to do some performance measurement if there is time. Might not be in a few days I would say. But in weeks/months likely. Well, I don't know perhaps that it won't come.

Anyhow that won't change the fact that you're happy with it and continue to use it.

I appreciate your response. I’m pleased yes, but it doesn’t mean I wouldn’t change if there’s a performance advantage. I was curious about that.

Having said that, it’s very interesting reading all your opinions. You folks are experienced. I’m sure other folks like me, with far less experience, appreciate your posts very much.

Thanks.

 

[kvic]

I appreciate your response. I’m pleased yes, but it doesn’t mean I wouldn’t change if there’s a performance advantage. I was curious about that.

Having said that, it’s very interesting reading all your opinions. You folks are experienced and I’m sure folks like me with far less experience appreciate your posts very much.

Thanks.

Oh, I see. Let's be patient. Like you I'm also happy to change position about DoT/DoH. So I think we're reasonable people after all.

 

[RMerlin]

Could be uneducated questions, so bear with me. I’m running DNSSEC and DoT using Cloudflare. I use Netflix and YouTube quite a bit. What kind of performance degradation should I be looking for?

Hard to evaluate because it depends on your ISP, their peering agreements, where you are located in the world, etc... I'd say if you don't experience buffering problems, then you're fine.

If too many users start doing the same thing, then eventually it might generate congestion problems for ISPs, as their nearline caches wouldn't be used as extensively, requiring them to transmit more data through their peers. From a purely ideological point of view, one might say users aren't being "nice netizen" in doing so, by bypassing networking optimizations currently in place, which might impact performance for everyone else. But we're probably still a long way from reaching a critical mass where this might have an actual measurable impact on performance and transit costs. Let's just say that if someday by some magic 50% of an ISP's users were to start using DoT/DoH with a non-EDNS aware DNS (like Cloudflare), then it might become a problem.

Which is why personally I believe that DoT/DoH won't be long-term solutions, unless ISPs start running such servers within their local network, so they can provide security, while still being able to resolve queries to point their customers at their own nearline caches within their network.

DNSSEC is a different story, because supporting DNSSEC at the DNS level is fairly easy and inexpensive, so it would be fairly easy for ISPs to start supporting DNSSEC on their own servers (mine does).

 

[RMerlin]

In an ideal world, this is what should happen IMHO:

1) People need to stick with DoT rather than DoH (lower overhead, less network-intrusive)
2) The world needs to move to TLS 1.3, and take advantage of its optimized handshake capabilities
3) ISPs need to start offering DoT support on their own servers, so customers can still go through their local caches (or closest regional servers)

It will still have a performance impact on resolution and dramatically increase the resolution time. It will also be expensive for ISPs to support as they will need much more powerful servers to handle the crypto part (plus the increased network load), so this might seriously hamper its deployment.

At this point, we need someone to provide some data points as to the performance requirements of running a DoT-capable resolver (the link I posted only shared DoH-related numbers) to better determine if it's even remotely realistic for ISPs to eventually begin offering that service.

A middle ground option would be for resolvers to all support EDNS, which will at least help with CDNs. But the people being paranoid about their "privacy" won't like it.

 

[skeal]

Read the article. In this case the duty of dnssec validation is passed to our resolver (CF in this case) by using the proxy-dnssec.

Then we do not need to enable router/stubby/dnscrypt-proxy dnssec validation as it would be wasting of system resources because cloudflare already done the validation. (Not sure of proxy-dnssec will override the system dnssec validation if dnssec validation is also enabled)

You may experience yourself that with proxy-dnssec enable, off your dnssec validation. You will still get the ad flag.
If you disable both dnssec validation and proxy-dnssec, then you will not get the ad flag as there is not validation done.

Based on the articles, i assumed dot/dns has created a secured path for dns traffic. So Long as cloudflare verified the dns request (dnssec), mean the dns forward to us is definitely correct and clean since it is in a secured path?

After testing I would say this is true. I have noticed that this configuration passes all the tests (so it can be verified) and supports Cloudflares methodology. I would encourage people to try this script and enter dnssec-proxy in the dnsmasq.conf.add file. Restart stubby and you are golden. For now without further input from other entities it is hard not to agree with the @DonnyJohnny method.

 

[skeal]

@RMerlin This is Cloudflares position on EDNS support:

EDNS Client Subnet
1.1.1.1 is a privacy centric resolver so it does not send any client IP information and does not send the EDNS Client Subnet Header to authoritative servers.

Found this here:https://developers.cloudflare.com/1.1.1.1/nitty-gritty-details/

 

[kvic]

To echo Merlin's comment, I would add further the following (or at least my position..).

1. Currently the most convincing argument in favour of deploying DoT/DoH on home routers is to prevent eavesdropping/tampering DNS queries by your ISP. I still remember the guy (@lancethepants) who mentioned this to me a few years ago when asked about why people want to (or need to) use it.

When an ISP starts deploying its own DoT/DoH servers, it becomes a very interesting/embarrassing dilemma. That is if your last mile is clean, you don't need DoT/DoH. If your last mile is hostile, one greater evil is the ISP...

2. DNSSEC seems to be engineered by people with a greater vision. It'll be good to have and more widely deployed. However, it's not cheap in computation. From my brief tests, it adds noticeable delay. Hence, a better caching DNS server is needed on home routers.

3. DoT/DoH. The overhead between these two at protocol level is trivial and negligible. Performance difference or users' perception of it so far is highly dependent on client/server implementation. I think Dnscrypt-proxy V1 (the one I briefly tried years ago) was horrible. Hence, people perhaps see a noticeable improvement when Stubby showed up. Intrinsically DoH could be just as fast.

The comparison that developers should try to establish is between DoT/DoH and no DoT/DoH. There should two baselines (for comparison) too: one to your ISP's DNS server. another to now IMHO one of those over hyped public DNS servers.

I'm usually pretty good at this sort of analytical stuff. I would expect to see a non-trivial differences. I'm happy to be proved wrong though.

 

[RMerlin]

@RMerlin This is Cloudflares position on EDNS support:

EDNS Client Subnet
1.1.1.1 is a privacy centric resolver so it does not send any client IP information and does not send the EDNS Client Subnet Header to authoritative servers.

Found this here:https://developers.cloudflare.com/1.1.1.1/nitty-gritty-details/

Yes I know, that's why I mentionned Cloudflare as one that doesn't support it, meaning it will negatively impact your performance with pretty much every single CDN providers.

 

[skeal]

Yes I know, that's why I mentionned Cloudflare as one that doesn't support it, meaning it will negatively impact your performance with pretty much every single CDN providers.

Sorry for pursuing this but I'm trying to wrap my head around it. If I'm using the Cloudflare CDN, so you are saying that when it (CF CDN), interacts with another CDN (example Amazon), there would be a performance hit? Can you give me an example because I haven't seen a hit to performance on any one of my devices. My ping times increased overall by 1ms. I must admit my network is largely quiet, except for streaming video through a VPN.

 

[DonnyJohnny]

I assumed cloudflare, like Quad9 are using some method to improve geographic accuracy.
https://mobile.twitter.com/quad9dns/status/961282616572997633?lang=en
So far I have no issue in watching streaming while using cloudflare dot.

Only can say the more people using the particular resolver will improve caching and better performance. I see the difference me better non-dot vs dot is 8ms vs 60-70ms.

I can survive with that difference as I see my country (Singapore) getting from bad to worse in policing the Web content. It is getting scary. Haha. I know somehow they will still know sites we going but just trying to make it more difficult for them. Best solution of coz will still be off-shore VPN service.

 

[RMerlin]

Sorry for pursuing this but I'm trying to wrap my head around it. If I'm using the Cloudflare CDN, so you are saying that when it (CF CDN), interacts with another CDN (example Amazon), there would be a performance hit? Can you give me an example because I haven't seen a hit to performance on any one of my devices. My ping times increased overall by 1ms. I must admit my network is largely quiet, except for streaming video through a VPN.

This isn't about using the Cloudflare CDN, this is about using the Cloudflare DNS (or any non-ISP DNS that does not support EDNS, for that matter) to access any kind of CDN that relies on EDNS to point you to the closest mirror (or to your ISP's own caching server if they have one).

I can't give you any specific test procedure as I don't have any actual test hostname to test against. Not all CDN rely on EDNS either, some use routing-based rules to point you at the most optimal server regardless of the resolved IP. These would not be affected by lack of EDNS support, only a VPN would cause you to get routed to the wrong server.

 

[RMerlin]

I assumed cloudflare, like Quad9 are using some method to improve geographic accuracy.

No, Cloudflare specifically doesn't support EDNS, citing privacy reasons.

 

[RMerlin]

Only can say the more people using the particular resolver will improve caching and better performance. I see the difference me better non-dot vs dot is 8ms vs 60-70ms.

It's not about time-to-resolve, it's about resolving the IP that is optimally located relative to you network-wise. Same reason why I've been saying for years that the vast majority of those DNS benchmark tests are totally worthless. Time to resolve has next to no noticeable impact on performance since it's a one-time resolution that gets cached locally afterward, while downloading/streaming from a server at the other end of the country versus one that is in your home city can potentially have a very visible performance impact.

 

[kvic]

To put it simply. If your upstream public DNS server (e.g. CF or Google) is further away from your PC in terms of network latency (read 5ms vs 50ms vs 150ms for example), IP addresses of CDN servers resolved by the upstream DNS server will not be optimal in terms of network latency.

If people don't have issue in user experience, no problem continue with current setup. Just that from engineering perspective, it's sub-optimal and issues could arise at any time. Some people like myself just want the optimal solution..the best one could get.

Also note that video streaming apps have improved a lot when compared to say 10 years ago. Less prone to network latency. Hence, it quite difficult to tell if your network latency increase by five times, you're going to see video stuttering.

As an aside, one big goal with huge effort spent in 5G wireless network is to reduce network latency. So for people like me, there is little point advocating techniques/technologies that will increase that unnecessarily in the contrary.

 

[kvic]

I see the difference me better non-dot vs dot is 8ms vs 60-70ms.

It's not about time-to-resolve

I would bet that the measurement actually doesn't include the DNS resolve time. Taking it out is good in this discussion. So the difference indicates the overhead in no DoT/DoH vs DoT/DoH, both on the same network latency (I assume). I would expect the performance spread to grow wider for people living further away from big cities and in smaller towns as the network latency increases.

About resolve time in upstream DNS servers. I found my ISP is actually doing a better job than public DNS servers (checked CF and Google). I would say it matters, especially if home routers do not come with a good caching DNS server. On this (I hope I won't break people's heart and be flamed..), DNSmasq is not that great and scripts (like Diversion) make the situation even worse.

In a nutshell, you want an upstream DNS server that's close by and resolving fast as well as improving DNS caching on your home routers.

 

[DonnyJohnny]

I would bet that the measurement actually doesn't include the DNS resolve time. Taking it out is good in this discussion. So the difference indicates the overhead in no DoT/DoH vs DoT/DoH, both on the same network latency (I assume). I would expect the performance spread to grow wider for people living further away from big cities and in smaller towns as the network latency increases.

About resolve time in upstream DNS servers. I found my ISP is actually doing a better job than public DNS servers (checked CF and Google). I would say it matters, especially if home routers do not come with a good caching DNS server. On this (I hope I won't break people's heart and be flamed..), DNSmasq is not that great and scripts (like Diversion) make the situation even worse.

In a nutshell, you want an upstream DNS server that's close by and resolving fast as well as improving DNS caching on your home routers.

lol.. maybe that only apply to my small island country Singapore. most common CDN/DNS servers/nodes are located here. And thanks to that, maybe that's why I don't notice any significant performance difference.