Dnsmasq & Ab-solution/Diversion restarting over and over

[dugaduga]

Using the latest RT-AC68U 384.7-beta1,

I had this issue with both Ab-solution and Diversion. Dnsmasq restarting over and over... logs: https://pastebin.com/PRd7jJXj

This started after the latest entware and Diversion update. Hitting "apply" under lan/dhcp server typically fixed this issue, so long as it reassigned an ip. Disabling samba, doing sig checks, and diversion list updates would often trigger this bug, occasionally persisting after a reboot. I reverted to Ab-solution, for a few days everything was fine, with the updated entware packages.

3-4 days later it started again and went on for hours. In troubleshooting I had initiated factory defaults, reinstalled the latest firmware, re-applied all settings manually; hit apply lan multiple times, rebooted over and over, started and stopped ab solution to no avail, turned on and off automatic dns... withdrawing trendmicro privacy agreement, you name it.

It seems the DNScrypt servers were not working. I could access cached domains. Skynet blocked incoming packets. I tried multiple DNScrypt servers, including cloudflare as they previously worked when others were down or possibly blocked. I managed to get one or two domains loaded via browser... but that was about it. I'm wondering if the ISP was blocking my DNScrypt servers, or this dnsmasq/absolution restarting didn't allow for a proper DNS connectivity. Dnscrypt always successfully reported the latency of the dnscrypt servers.

Will dnsmasq/absolution do this if dnscrypt packets are being blocked or throttled?

I successfully started an openvpn client on the router, no changes. Finally, once I had assigned an IP to openvpn client and started browsing through the VPN, the problem suddenly stopped!

Summing up watchdog was restarting dnsmasq over and over again, ab-solution was doing the same, loading in and out of the ram. I was also getting icmp black nurse attacks from the loopback device while this was happening before, and immediately after factory default & firmware re-update; I had hardened all potential router vulnerabilities prior to plugging in wan and updated sigs immediately upon reboot. I suspect this was not an internal infection. Could this be a DNS ddos? Any Ideas?

@RMerlin, @thelonelycoder

 

[dugaduga]

Just updated to beta2, will see what comes of it. Updated the previous comment as well.

 

[pattiri]

I've had the same issue yesterday and to fix it I've disabled "dnsmasq settings" via Diversion. Just after that it was fixed and I've re-enabled "dnsmasq settings" and working fine now. I guess because of a reason Diversion couldn't configured the dnsmasq settings.

AiProtection is off on my end.

 

[dugaduga]

Thanks @pattiri, I will keep that in mind, they are considered "experimental" in ab-solution. I am also using them. Perhaps this has something to do with it. I will try disable them when I run into this problem again.

@thelonelycoder
I saw this with each reload:
Sep 26 05:39:31 dnsmasq[10560]: warning: interface tun22 does not currently exist
Sep 26 05:39:31 dnsmasq[10560]: warning: interface tun21 does not currently exist
Sep 26 05:39:31 dnsmasq[10560]: warning: interface pptp* does not currently exist

I thought I saw this many times before, not sure if it means anything.

 

[dugaduga]

@thelonelycoder @RMerlin Here are some more logs https://hastebin.com/finatidiko.js

 

[kvic]

I will try disable them when I run into this problem again.

Does it resolve the dnsmasq restarting issue for you?

@pattiri did you get to the bottom of the cause?

I believe a few more Diversion users are experiencing this issue than we heard on this board. Most users perhaps even not aware of its existence..

 

[pattiri]

@pattiri did you get to the bottom of the cause?

nope. I haven't faced this issue since my post in this threat.

 

[truglodite]

...I saw this with each reload:
Sep 26 05:39:31 dnsmasq[10560]: warning: interface tun22 does not currently exist
Sep 26 05:39:31 dnsmasq[10560]: warning: interface tun21 does not currently exist
Sep 26 05:39:31 dnsmasq[10560]: warning: interface pptp* does not currently exist

I thought I saw this many times before, not sure if it means anything.

I think those lines indicate that you have some vpn servers and/or vpn clients enabled. tun22 & tun21 show up on my router because I'm running 2 openvpn servers. pptp* might be similar (pptp client on router?).

 

[truglodite]

Could this be a DNS ddos? Any Ideas?

@RMerlin, @thelonelycoder

Well 127.0.0.1 is probably not a source of an outside attack... that's the local loopback interface (used by many things, including skynet for dns filtering). This is probably a side effect of some misconfig... likely has to do with your vpn server confusing dnsmasq and/or dnscrypt. I run a similar config on my router (diversion+skynet+dnscrypt+dnssec+ovpn server) and have no similar issues. If it happens again, try getting debug logs, and see if a hard reset straightens it out.

 

[john9527]

The log shows that the watchdog is restarting dnsmasq. Just guessing here, but it appears that the Diversion startup is exceeding the watchdog interval, so that the router is continually restarting dnsmasq before Diversion can complete it's work. Try using smaller blocking files if it happens again.

 

[jrmwvu04]

Try using smaller blocking files if it happens again.

I quoted you just so I could see it twice.

 

[thelonelycoder]

The log shows that the watchdog is restarting dnsmasq. Just guessing here, but it appears that the Diversion startup is exceeding the watchdog interval, so that the router is continually restarting dnsmasq before Diversion can complete it's work. Try using smaller blocking files if it happens again.

What's that interval time? Seconds or a minute?

Diversion offloads most of its operations into the post-conf.div file which is sourced in dnsmasq.postconf.
A large blocking file along with a slow USB device to read from might come close to or over the watchdog timer.
Will keep an eye on it.

 

[john9527]

What's that interval time? Seconds or a minute?

I'd need to check the code...but IIRC there's a 15sec and a 30sec timer.

I just implemented a dnsmasq watchdog on my fork (I admit I didn't check Merlin first). I was worried about false positives since dnsmasq gets restarted multiple times in a lot of scenarios. So my implementation is that it has to be gone for 3 consecutive 30sec intervals (or between 90 and 120 sec). For me, it's a failsafe when something is badly wrong.

 

[thelonelycoder]

I'd need to check the code...but IIRC there's a 15sec and a 30sec timer.

I just implemented a dnsmasq watchdog on my fork (I admit I didn't check Merlin first). I was worried about false positives since dnsmasq gets restarted multiple times in a lot of scenarios. So my implementation is that it has to be gone for 3 consecutive 30sec intervals (or between 90 and 120 sec). For me, it's a failsafe when something is badly wrong.

I restart Dnsmasq for any operation that involves Dnsmasq in Diversion. Naturally. Along with other scripts that add to dnsmasq.postconf and increase the time it takes to fully restart Dnsmasq it might get over this limit.
I'll do some timing tests for common scenarios as soon as (my) time allows.