Tutorial How to monitor DNS traffic in real-time

[sfx2000]

handy tool to keep in the bookmark list..

dnscheck.tools - check your dns resolvers

A tool to test for DNS leaks, DNSSEC validation, and more

dnscheck.tools

 

[ghc88]

My knowledge of DNS is pretty basic but your script really helped. I had followed Merlin's guide on setting up DoT on my AX88U, so I was surprised when the script showed UDP requests were not using DoT and were being transmitted in plaintext. It was exactly as you've said, it was using the DNS servers I had previously set on the WAN DNS Setting page. Setting them to blank has resolved the issue and now your tool shows all DNS requests are using DoT. This feels like something that should be corrected (or at least made clear) in the firmware, as it's really not obvious and undermines/defeats the purpose of setting DoT, if requests are still being sent in plaintext...

 

[SomeWhereOverTheRainBow]

My knowledge of DNS is pretty basic but your script really helped. I had followed Merlin's guide on setting up DoT on my AX88U, so I was surprised when the script showed UDP requests were not using DoT and were being transmitted in plaintext. It was exactly as you've said, it was using the DNS servers I had previously set on the WAN DNS Setting page. Setting them to blank has resolved the issue and now your tool shows all DNS requests are using DoT. This feels like something that should be corrected (or at least made clear) in the firmware, as it's really not obvious and undermines/defeats the purpose of setting DoT, if requests are still being sent in plaintext...

Keep in mind leaving your WAN dns blank has the potential to break many addon scripts, and also break your router's local dns resolution. The traffic you were seeing was the "routers" own traffic. It uses plaintext dns to perform various "router" operations. All client related traffic should have been traveling over DoT.

 

[ghc88]

Keep in mind leaving your WAN dns blank has the potential to break many addon scripts, and also break your router's local dns resolution. The traffic you were seeing was the "routers" own traffic. It uses plaintext dns to perform various "router" operations. All client related traffic should have been traveling over DoT.

Oh right, what are the noticeable effects to look out for? When I first changed the WAN DNS servers to blank the router had a melt down, however, following a reboot it was fine and has been for a few days. All of the approx 30 devices on my network are still connected and functioning ok for now. The script seemed to show that TCP requests were going over DoT but UDP was going over port 53 plain text. Online DNS Leak tests also showed both the DoT DNS servers and the WAN DNS servers (I had one set to Quad 9 and the other Cloudflare, so I was able to differentiate...)

p.s. my WAN connection is over PPPoE, I'm not sure if that is a factor, but seems to have been mentioned a few times in this thread.

 

[SomeWhereOverTheRainBow]

Oh right, what are the noticeable effects to look out for? When I first changed the WAN DNS servers to blank the router had a melt down, however, following a reboot it was fine and has been for a few days. All of the approx 30 devices on my network are still connected and functioning ok for now. The script seemed to show that TCP requests were going over DoT but UDP was going over port 53 plain text. Online DNS Leak tests also showed both the DoT DNS servers and the WAN DNS servers (I had one set to Quad 9 and the other Cloudflare, so I was able to differentiate...)

p.s. my WAN connection is over PPPoE, I'm not sure if that is a factor, but seems to have been mentioned a few times in this thread.

The request you saw going over port 53 were most likely router functions, for example the router checking the firmware update service, the router communicating with trendmicro ai protect, set the routers time for NTP services, or the routers built in connectivity check doing a dns request to make sure your modem was still active. These types of request would not go via DoT, instead they would most likely use the WANs dns servers, hence why clearing out WAN DNS 1 And 2 is a bad idea. You never know when it is going to break one of your router services or create a race condition at reboot time.

The traffic you do see traveling the TCP DoT servers would specifically be all your clients dns requests. None of these would be included in the udp traffic.

 

[ghc88]

The request you saw going over port 53 were most likely router functions, for example the router checking the firmware update service, the router communicating with trendmicro ai protect, or the routers built in connectivity check doing a dns request to make sure your modem was still active. These types of request would not go via DoT, instead they would most likely use the WANs dns servers. The traffic you do see traveling the TCP DoT servers would specifically be all your clients dns requests.

Hmmm, that would make sense as I do have the trendmicro features turned on. However, there's a couple of things that I don't understand if that's the case. Firstly, online/web page based DNS leak testers were able to identify the WAN DNS servers - which makes me think the router is allowing clients to use them. If it was only the router using those DNS servers directly, surely that should not be possible? Also, it seems that after setting the WAN DNS servers to blank, the requests are now showing as encrypted/DoT, which suggests even if those are DNS requests that support router services, they are now using DoT, which surely isn't a bad thing?

 

[SomeWhereOverTheRainBow]

Hmmm, that would make sense as I do have the trendmicro features turned on. However, there's a couple of things that I don't understand if that's the case. Firstly, online/web page based DNS leak testers were able to identify the WAN DNS servers - which makes me think the router is allowing clients to use them. If it was only the router using those DNS servers directly, surely that should not be possible? Also, it seems that after setting the WAN DNS servers to blank, the requests are now showing as encrypted/DoT, which suggests even if those are DNS requests that support router services, they are now using DoT, which surely isn't a bad thing?

Well let's say for example your wan dns was 1.1.1.1 and your DoT servers were also 1.1.1.1, then that might not be a fair assessment with the dnsleaktest, however, if your wan dns was your ISP and your DoT server was 8.8.8.8, and you saw both Google and your isp servers on the leak test, then you might be dealing with a leak. In that case, the correct thing to do is go to the tools page and enable local dns service.

Other places where possible leaks could be a concern is if we are talking about hardcoded dns servers on the client, or browser dns service such as Firefox, Chrome, or edge DoH services.


These odd behaviors only further change when you start using VPN servers. Then you have to adjust the accept dns policy according to your use case.

 

[ghc88]

Well let's say for example your wan dns was 1.1.1.1 and your DoT servers were also 1.1.1.1, then that might not be a fair assessment with the dnsleaktest, however, if your wan dns was your ISP and your DoT server was 8.8.8.8, and you saw both Google and your isp servers on the leak test, then you might be dealing with a leak. In that case, the correct thing to do is go to the tools page and enable local dns service.

Other places where possible leaks could be a concern is if we are talking about hardcoded dns servers on the client, or browser dns service such as Firefox, Chrome, or edge DoH services.


These odd behaviors only further change when you start using VPN servers. Then you have to adjust the accept dns policy according to your use case.

Yes, it was the latter... I had them set to two different DNS servers (Quad9 and Cloudflare), and they both showed up. I don't seem to have an option to enable a local DNS service in my Tools or Network Tools settings.

I am running a VPN server (Open VPN), I have IPv6 DNS set to my LAN IPv6 Link-Local Address, as per merlin's guidance. I also have a VPN client running (OpenVPN to Nord), and I have the DNS set to exclusive, so it should only be using Nord's VPN servers, but I haven't investigated further at this point...

As everything seems to be working with the WAN DNS servers set to blank, I think I'll keep this configuration for now. If I notice something stops working properly then I'll just set WAN DNS servers to Cloudflare or Quad 9 and live with the DNS leak. Unless you think setting up a local DNS server would be a better solution, but I can't seem to find that in the settings.

I'll make sure I report back if any issues with this config arise.

Cheers,

G.

 

[SomeWhereOverTheRainBow]

Yes, it was the latter... I had them set to two different DNS servers (Quad9 and Cloudflare), and they both showed up. I don't seem to have an option to enable a local DNS service in my Tools or Network Tools settings.

I am running a VPN server (Open VPN), I have IPv6 DNS set to my LAN IPv6 Link-Local Address, as per merlin's guidance. I also have a VPN client running (OpenVPN to Nord), and I have the DNS set to exclusive, so it should only be using Nord's VPN servers, but I haven't investigated further at this point...

As everything seems to be working with the WAN DNS servers set to blank, I think I'll keep this configuration for now. If I notice something stops working properly then I'll just set WAN DNS servers to Cloudflare or Quad 9 and live with the DNS leak. Unless you think setting up a local DNS server would be a better solution, but I can't seem to find that in the settings.

I'll make sure I report back if any issues with this config arise.

Cheers,

G.

Sorry, I should have been more specific about what local server I was talking about. It is under

Tools>Other settings>Advanced tweaks and hacks

 

[ghc88]

Got it, thanks for your help. If I have any issues I'll put the WAN DNS servers back in and enable that setting and retest for DNS leaks. I was just reading another thread where merlin said leaving them blank may cause DoT to break as the router won't be able to set time via NTP. However, it seems some people haven't had any issues running with them blank, so I don't know what to believe! I'm installing diversion which comes with entware now, and they are my first real addons, so I will see if they manage to break my current config shortly. The latter I understand should allow me to install tcpdump as well, so I'll take a closer look at what's going on with my DNS.

 

[SomeWhereOverTheRainBow]

Got it, thanks for your help. If I have any issues I'll put the WAN DNS servers back in and enable that setting and retest for DNS leaks. I was just reading another thread where merlin said leaving them blank may cause DoT to break as the router won't be able to set time via NTP. However, it seems some people haven't had any issues running with them blank, so I don't know what to believe! I'm installing diversion which comes with entware now, and they are my first real addons, so I will see if they manage to break my current config shortly. The latter I understand should allow me to install tcpdump as well, so I'll take a closer look at what's going on with my DNS.

Another thing you may notice is you might not be able to update your entware repository if the ntp is not set right on the router. Another example, assume the router dns is not set and stubby does not start due to ntp not set properly, then none of the router security features such as Ai protect or skynet will work properly either.; also, the router will assume the internet is down when it tries to use its network detection tools.

Having the WAN DNS blank doesn't mean what you are doing is necessarily "wrong", it is just important to understand that there are things that can go wrong; hell, you may even experience times when your vpn server won't connect if your dns does not work. Especially if the vpn provider uses hostnames as the gateway point instead of ip addresses. Without dns capabilities, the router would not be able to resolve the vpn gateways hostname to make adequate connection. These issues that "could" happen wont just occur at random. They may occur more as a ripple effect-one after the other.

 

[ghc88]

Another thing you may notice is you might not be able to update your entware repository if the ntp is not set right on the router. Another example, assume the router dns is not set and stubby does not start due to ntp not set properly, then none of the router security features such as Ai protect or skynet will work properly either.; also, the router will assume the internet is down when it tries to use its network detection tools.

Having the WAN DNS blank doesn't mean what you are doing is necessarily "wrong", it is just important to understand that there are things that can go wrong; hell, you may even experience times when your vpn server won't connect if your dns does not work. Especially if the vpn provider uses hostnames as the gateway point instead of ip addresses. Without dns capabilities, the router would not be able to resolve the vpn gateways hostname to make adequate connection. These issues that "could" happen wont just occur at random. They may occur more as a ripple effect-one after the other.

I've taken your advice and added back in the Quad9 WAN DNS servers.

I've installed tcpdump. What interface(s) should I be monitoring?

If I run:

tcpdump -i any port 53 [This shows a huge amount of traffic on various interfaces, mainly eth4, eth7 and br0. If I visited a website, I can see it show up in the output. Majority seems to be IPv6 but not all, there's IPV4 addresses too]

Noting my WAN connection is PPPoE, I thought perhaps I should only be looking at this interface, so I ran:

tcpdump -i ppp0 port 53 [not much shows at all, if I visit a website I haven't resolved in a while, I get trend micro showing up, using the WAN DNS servers (expected from what you've said) and also a couple related to my ISP (assume this is something to do with the PPPoE connection]

tcpdump -i ppp0 port 853 [I see some traffic (basically all to a domain/hostname related to my ISP). There's not really anything other than packets relating to my ISP and one.one.one.one.853:]


I'm not convinced it's working as it should. Basically, if I should only be concerned with the ppp0 interface then it's possibly working how it should. If I need to be concerned with the other interfaces then it almost certainly isn't using DoT properly. Especially for IPv6, I had read a guide that said to manually set IPv6 DNS server to the router IPv6 link-local address, however, I've just read someone saying to leave the IPv6 DNS setting to auto. Either doesn't seem to effect the above results...
\\
EDIT: I've just realised all of the traffic picked up on eth4/eth7/br0 over port 53 seems to be internal i.e. between my router and my devices (+ the occasional trendmicro/ISP traffic over ppp0). So I think (maybe) I can conclude it's all working as it should - key thing is other than the router based requests that are still going over port 53 (trend and router to ISP related), everything else on the ppp0 interface is going over 853/DoT.

 

[SomeWhereOverTheRainBow]

I've taken your advice and added back in the Quad9 WAN DNS servers.

I've installed tcpdump. What interface(s) should I be monitoring?

If I run:

tcpdump -i any port 53 [This shows a huge amount of traffic on various interfaces, mainly eth4, eth7 and br0. If I visited a website, I can see it show up in the output. Majority seems to be IPv6 but not all, there's IPV4 addresses too]

Noting my WAN connection is PPPoE, I thought perhaps I should only be looking at this interface, so I ran:

tcpdump -i ppp0 port 53 [not much shows at all, if I visit a website I haven't resolved in a while, I get trend micro showing up, using the WAN DNS servers (expected from what you've said) and also a couple related to my ISP (assume this is something to do with the PPPoE connection]

tcpdump -i ppp0 port 853 [I see some traffic (basically all to a domain/hostname related to my ISP). There's not really anything other than packets relating to my ISP and one.one.one.one.853:]


I'm not convinced it's working as it should. Basically, if I should only be concerned with the ppp0 interface then it's possibly working how it should. If I need to be concerned with the other interfaces then it almost certainly isn't using DoT properly. Especially for IPv6, I had read a guide that said to manually set IPv6 DNS server to the router IPv6 link-local address, however, I've just read someone saying to leave the IPv6 DNS setting to auto. Either doesn't seem to effect the above results...

So, what interfaces are you concerned with?


When pointing a client at a VPN, you need to make sure the correct DNS policy is selected to ensure that those clients connected to the VPN server only use the "DoT" servers, otherwise you will see potentially DoT dns servers, ISP dns servers, and/or VPN dns servers for those clients accessing the VPN.

Guest Networks introduce a whole new aspect to be concerned about, but in this case you can use YazFi guestnetworks setup to force clients on each guestnetwork to use the routers IP as DNS (which should point back to DoT servers).

If it helps, here is the "Official DoT Guide": https://github.com/RMerl/asuswrt-merlin.ng/wiki/DNS-Privacy

OpenVPN Clients​

This will mostly work as before. OpenVPN clients with "Accept DNS configuration" set to "Exclusive" will still use the DNS servers provided by the VPN server, bypassing DNS Privacy. Setting DNS configuration to "Disabled" on the OpenVPN client configuration will allow it to use DNS Privacy, however note that some VPN providers will block the use of DNS servers other than their own, to protect you against leaking information by sending DNS queries outside of the tunnel. If you trust the OpenVPN server you connect to, it's usually best to leave the setting to Exclusive mode - your DNS queries are already encrypted by the VPN tunnel anyway (for all clients configured to use the tunnel).

 

[SomeWhereOverTheRainBow]

If I remember correctly, when I used the DoT with my ipv6 enabled, I did not have to do this part.

IMPORTANT: for DNS Privacy to work in IPv6, you must set IPv6 DNS Server in IPv6 page (not equivalent to add IPv6 DoT servers on the WAN -> Internet Connection page) to your router's LAN IPv6 Link-Local Address. You can find your router's LAN IPv6 Link-Local Address in System Log -> IPv6 tab. Link-local address starts with fe80.

Because your system Dhcp already hands out your LAN IPV6 prefix via Dnsmasq DHCP server. In theory, placing your link-local address inside the dns slot here would then create a DNS loop. This may be causing you to see your DNS "Leak". Best practice would be to leave this DNS set to Auto (like the wan dns 1 and 2).

 

[ghc88]

Good point about the VPN client. I used tcpdump on tun15 and can see that it is correctly using Nord's DNS servers, which is good as it's how I understood it to be configured.

Sorry, I edited my previous post seconds before it notified me you had replied. I *think* everything is working as it should. All of the port 53 traffic on the eth* interfaces seems to be internal between my clients and the router. I'm guessing that these plaintext DNS requests go between the clients and the router, and then the router is sending the DNS requests via ppp0 using DoT/port 853 Therefore, I assume I can be solely concerned with only the ppp0 interface, and other than a small amount relating to router functions e.g. trend, everything is going over port 853 on the ppp0 interface.

Sorry for all the questions, but I hugely appreciate your help.

 

[SomeWhereOverTheRainBow]

Good point about the VPN client. I used tcpdump on tun15 and can see that it is correctly using Nord's DNS servers, which is good as it's how I understood it to be configured.

Sorry, I edited my previous post seconds before it notified me you had replied. I *think* everything is working as it should. All of the port 53 traffic on the eth* interfaces seems to be internal between my clients and the router. I'm guessing that these plaintext DNS requests go between the clients and the router, and then the router is sending the DNS requests via ppp0 using DoT/port 853 Therefore, I assume I can be solely concerned with only the ppp0 interface, and other than a small amount relating to router functions e.g. trend, everything is going over port 853 on the ppp0 interface.

Sorry for all the questions, but I hugely appreciate your help.

I had these concerns before myself... So I understand.

I even wrote a small tutorial on confirming encryption of DoT with wireshark.

Tutorial - [ DoT Study ] ######MBM DoT Study#######

I want to first of all thank my Loyal followings of #Nobody. Here is another @SomeWhereOverTheRainBow thread-cast #my-first-one-ever List of things needed. Asuswrt-Merlin(384.12 and up preferred) or with DoT entware setup putty (SSH client) entware tcp-dump wireshark Okay so this is a guide...

www.snbforums.com

But your final assumptions are correct!


Things go like this

Client port 53---<---->----Router port 53----<--Stubby-->-----port 853 DoT server

 

[Stephen Harrington]

@eibgrad sorry to trouble you but since the last batch of Entware updates a week or so ago I'm now getting lots of warning messages about egrep/grep when I run this script now?
If it takes your fancy would you mind casting your eyes over what might need to be done to fix it when you get a few moments please?

 

[Stephen Harrington]

@eibgrad

Actually, just replacing the 3 (I think?) instances of "egrep" with "grep -E" seems to do the trick.
But you should check it out and maybe publish an update if you feel so inclined?

 

[chongnt]

@eibgrad sorry to trouble you but since the last batch of Entware updates a week or so ago I'm now getting lots of warning messages about egrep/grep when I run this script now?
If it takes your fancy would you mind casting your eyes over what might need to be done to fix it when you get a few moments please?

View attachment 46609

I just noticed this yesterday. It happens when we use entware /opt/bin/egrep. If we use /bin/egrep there is no such error.

Found these lines in the script

    egrep 'tcp .* dport=(53|853) ' /proc/net/nf_conntrack | \
            egrep -q "(src|dst)=($wan0_ip|$wan1_ip)( |$)"; then
            egrep -q "(src|dst)=($wan0_ip|$wan1_ip)( |$)"; then

Quick fix is to either change to grep -E or /bin/egrep

<     grep -E 'tcp .* dport=(53|853) ' /proc/net/nf_conntrack | \
---
>     egrep 'tcp .* dport=(53|853) ' /proc/net/nf_conntrack | \
294c294
<             grep -Eq "(src|dst)=($wan0_ip|$wan1_ip)( |$)"; then
---
>             egrep -q "(src|dst)=($wan0_ip|$wan1_ip)( |$)"; then
306c306
<             grep -Eq "(src|dst)=($wan0_ip|$wan1_ip)( |$)"; then
---
>             egrep -q "(src|dst)=($wan0_ip|$wan1_ip)( |$)"; then

 

[Stephen Harrington]

Quick fix is to either change to grep -E or /bin/egrep

Far more elegantly illustrated than I did ...