Diversion Diversion 5.1.3 - the Router Ad-Blocker, May 09, 2024

[Treadler]

Or the device making the request is being very insistent. Here is what pihole says about NXDOMAIN blocking:

View attachment 59005

Diversion sets up DNSMASQ to use NXDOMAIN blocking, where as Pihole defaults to NULL blocking (e.g. address=/t.nineanalytics.io/#). hence my curiosity for investigating the behaviors. But without you doing additional testing following my inquires in the previous post, or access to any of your diversion files (e.g. the blocklist and allowlists which is generated by diversion), it may be impossible to identify why this behavior is happening. To determine what is happening, you have to investigate using dns tools (e.g dig), and try various blocking methods to determine if behaviors are specific to how your blocking is currently configured. It is like peeling the layers back on an onion. If you are concerned about your devices reaching out to this domain, you can always block the IP address (34.110.168.46) associated with it via the Skynet script.

Blocked it in Skynet, smashed it!
If I go to the site/s, Skynet registers hundreds of outward blocks in a minute or so, from that ip. [Edit: log shows one hit per second from the sites].
Maybe dnsmasq can only keep up with a finite frequency of requests? Dunno, above my pay grade.
Skynet has fixed my issue.

 

[dave14305]

Blocked it in Skynet, smashed it!
If I go to the site/s, Skynet registers hundreds of outward blocks in a minute or so, from that ip.
Maybe dnsmasq can only keep up with a finite frequency of requests? Dunno, above my pay grade.
Skynet has fixed my issue.

So we didn’t learn anything about why dnsmasq was behaving that way. What kind of client was making the requests? iOS, Android, Windows? The HTTPS DNS queries smell of iOS to me. What site triggers that domain lookup? What upstream DoT servers being used?

Maybe someone else can reproduce it and figure out what’s happening.

 

[Treadler]

So we didn’t learn anything about why dnsmasq was behaving that way. What kind of client was making the requests? iOS, Android, Windows? The HTTPS DNS queries smell of iOS to me. What site triggers that domain lookup? What upstream DoT servers being used?

Maybe someone else can reproduce it and figure out what’s happening.

iOS.

Both HTTPS, & AAAA queries ‘sometimes’ weren’t blocked.
Dot Quad9 servers used.

Site:

Australian Breaking News Headlines & World News Online | SMH.com.au

Breaking news from Sydney, Australia and the world. Features the latest business, sport, entertainment, travel, lifestyle, and technology news.

www.smh.com.au

It’s not that that domain was causing any grief, I was just puzzled by the now it’s blocked, now it’s not, behaviour. Haven’t seen that before.

Toggling DoT off, DNSSEC off, DNS rebind protection off, made no difference.

 

[dave14305]

iOS.

Both HTTPS, & AAAA queries ‘sometimes’ weren’t blocked.
Dot Quad9 servers used.

Site:

Australian Breaking News Headlines & World News Online | SMH.com.au

Breaking news from Sydney, Australia and the world. Features the latest business, sport, entertainment, travel, lifestyle, and technology news.

www.smh.com.au

It’s not that that domain was causing any grief, I was just puzzled by the now it’s blocked, now it’s not, behaviour. Haven’t seen that before.

Toggling DoT off, DNSSEC off, DNS rebind protection off, made no difference.

I can’t fully paint the picture yet, but I suspect it’s a conflict between dnsmasq using proxy-dnssec when Stubby is enabled but dnsmasq still having its own cache. I think if dnssec duties were fully done in dnsmasq instead of stubby, the results would be more reliable. I can’t get the SMH site to query that particular domain, so I can’t replicate your results.

This is a hypothesis to be proven or disproven.

 

[Treadler]

I can’t fully paint the picture yet, but I suspect it’s a conflict between dnsmasq using proxy-dnssec when Stubby is enabled but dnsmasq still having its own cache. I think if dnssec duties were fully done in dnsmasq instead of stubby, the results would be more reliable. I can’t get the SMH site to query that particular domain, so I can’t replicate your results.

This is a hypothesis to be proven or disproven.

Many thanks for your time & attention.

 

[SomeWhereOverTheRainBow]

iOS.

Both HTTPS, & AAAA queries ‘sometimes’ weren’t blocked.
Dot Quad9 servers used.

Site:

Australian Breaking News Headlines & World News Online | SMH.com.au

Breaking news from Sydney, Australia and the world. Features the latest business, sport, entertainment, travel, lifestyle, and technology news.

www.smh.com.au

It’s not that that domain was causing any grief, I was just puzzled by the now it’s blocked, now it’s not, behaviour. Haven’t seen that before.

Toggling DoT off, DNSSEC off, DNS rebind protection off, made no difference.

Many thanks for your time & attention.

If any of you have the time and ability, it would definitely be something worth documenting and submitting to the dnsmasq dev. It is possible that @dave14305 is right about the proxy-dnssec and the caching issue, but that still doesnt explain why the request is even being forwarded to stubby. If the domain is requested to dnsmasq, and dnsmasq has a local record for it being nxdomain, then it should never leave dnsmasq and should end right then and there with nxdomain being the only cached response, but it appears that dnsmasq is forwarding the request to stubby since it is only finding a no data record for it in its cache. This could be an issue with the newer capabilities of dnsmasq and the frequent nature of the querying device. A miss fired query to stubby if you will. Especially if the requests were being flooded to dnsmasq by an unsatisfied client. It is definitely something worth the dnsmasq dev taking a look at. It is a good thing it wasn't impacting your service though, and the only domain you saw with this behavior.

 

[bibikalka]

I am as frustrated and confused as you are about this and want it gone sooner than later.

Clearly something is grabbing that ad-blocking exclusion IP before Diversion can start the separate Dnsmasq instance.

Alright, had a episode during this morning's reboot! Here is the relevant section of the log:

An update on the situation. A few days ago I could not restore the working state of dnsmasq by powering the router off with the button for 10 seconds - tried 2 times. This usually worked for me OK in the past.

I actually had to turn off the router, unplug the power supply, push the router button on so it blinked and burnt the remaining capacitor charge. Only then it booted and worked properly!

Apparently, a full cold reboot of the router is a real thing!

P.S. I am suspecting it is the unholy interaction of the dnsmasq v2.90 and the other stuff - so I am almost tempted to bring the previous version of dnsmasq binary back.

 

[bibikalka]

...

Alright, after more issues I disabled the custom DNS instance in Diversion, sending excluded devices to Google - 8.8.8.8

And I ran into another issue! A Diversion initiated restart gave this error:

Jun  7 04:06:52 syslogd exiting
Jun  7 04:06:54 RT-AC86U-9988 Diversion: restarted Dnsmasq to apply settings
Jun  7 04:06:54 RT-AC86U-9988 (dnsmasq.postconf): Updating /etc/dnsmasq.conf for unbound.....
Jun  7 04:06:54 RT-AC86U-9988 uiDivStats: dnsmasq has restarted, restarting taildns
Jun  7 04:06:54 RT-AC86U-9988 dnsmasq[4391]: failed to create listening socket for 192.168.1.1: Address already in use
Jun  7 04:06:54 RT-AC86U-9988 dnsmasq[4391]: FAILED to start up
Jun  7 04:06:54 RT-AC86U-9988 Samba_Server: daemon is started

So I pulled out the prior dnsmasq 2.89 version from RT-AC86U_386.12_4, and added this in /jffs/scripts/init-start :

mount -o bind /jffs/scripts/289/dnsmasq /usr/sbin/dnsmasq

I also added back the Diversion exclusion at 192.168.1.16. Will monitor the situation.

admin@RT-AC86U-9988:/tmp/mnt/ac86u/entware/var/log# grep dnsmasq messages |grep vers
Jun  5 04:05:08 dnsmasq[1159]: started, version 2.90 cachesize 1500
...
Jun  7 09:44:48 RT-AC86U-9988 dnsmasq[42416]: started, version 2.89 cachesize 150
Jun  7 09:45:02 RT-AC86U-9988 dnsmasq[44742]: started, version 2.89 cachesize 150
Jun  7 09:53:04 dnsmasq[1160]: started, version 2.89 cachesize 1500

Update: things are still screwy in more subtle ways. With dnsmasq 2.89 & Diversion 5.1.3 on some reboots devices that are filtered don't get a working DNS. Custom DNS devices work fine. All devices can connect to the router itself - so at least that's working. Before I had to push the power button, now I can do that in software.

I would downgrade diversion to 5.1.1 that ran quite nicely for a while with 2.89, but I don't know how to do that

 

[thelonelycoder]

I would downgrade diversion to 5.1.1 that ran quite nicely for a while with 2.89, but I don't know how to do that

Not possible.

 

[bibikalka]

Not possible.

OK! But nothing in 5.1.3 that'd make it incompatible with dnsmasq 2.89? Did the Diversion script evolve or the config files?

 

[bibikalka]

Not possible.

Rolled back to 5.1.1 using one of my backups, and copying stuff selectively to /opt/share/diversion from the backup. Also replaced /opt/bin/diversion to 511. Running dnsmasq 2.90. Stuff seems to be working now. Will see how that goes over the days.

 

[bibikalka]

Not possible.

Rolled back to 5.1.1 using one of my backups, and copying stuff selectively to /opt/share/diversion from the backup. Also replaced /opt/bin/diversion to 511. Running dnsmasq 2.90. Stuff seems to be working now. Will see how that goes over the days.

Alright, good 10 days now. The issue with "dnsmasq[4391]: FAILED to start up" seems to have disappeared!

I am even using the latest 513 diversion script in /opt/bin since the changes from 511 in it are trivial. However, I replaced 3 directories from 5.1.1 backup in /opt/share/diversion/, specifically, ".conf", "webui", "file". So whatever created the issue in the first place lives in those directories.

Extra shout out to @Viktor Jaep with his BACKUPMON that allowed me to have backups to dig through for the older Diversion in the 1st place! I also went ahead and set up secondary backups, since I originally wanted to dig up Diversion 5.1.0 to run with dnsmasq 2.89, but that was already overwritten in my primary backups, and I did not have secondary backups running. Anyway, Diversion 5.1.1 appears to be working, so no 5.1.0 is needed.

admin@RT-AC86U-9988:/tmp/mnt/ac86u/entware/bin# cd /opt/bin/
admin@RT-AC86U-9988:/tmp/mnt/ac86u/entware/bin# ls -lta diversion*
-rwxr-xr-x    1 admin    root         66695 Jun  7 13:21 diversion
-rwxr-xr-x    1 admin    root         66695 Jun  7 13:21 diversion_513
-rwxr-xr-x    1 admin    root         66069 Mar  3 15:05 diversion_511
admin@RT-AC86U-9988:/tmp/mnt/ac86u/entware/bin# cd /opt/share/diversion/
admin@RT-AC86U-9988:/tmp/mnt/ac86u/entware/share/diversion# ls -lta
drwxrwxrwx    2 admin    root          4096 Jun 20 17:20 .conf
drwxrwxrwx   13 admin    root          4096 Jun 17 04:06 .
drwxrwxrwx    2 admin    root          4096 Jun 17 04:06 list
drwxrwxrwx    3 admin    root          4096 Jun 17 04:06 webui
drwxrwxrwx    2 admin    root          4096 Jun 14 02:00 backup
drwxrwxrwx    2 admin    root          4096 Jun 10 16:18 file
drwxrwxrwx    2 admin    root          4096 Jun 10 08:46 .conf513
drwxrwxrwx    3 admin    root          4096 Jun  7 13:21 webui513
drwxrwxrwx    2 admin    root          4096 Jun  7 13:21 file513
drwxrwxrwx    2 admin    root          4096 Apr 16 20:40 log
drwxr-xr-x   12 admin    root          4096 Feb 22 12:13 ..
drwxrwxrwx    2 admin    root          4096 Aug 18  2023 custom
drwxrwxrwx    2 admin    root          4096 Aug 18  2023 stats

 

[KGB7]

I gave up screwing around with Diversion and constant bugs that keep blocking all access to internet at random times of the day.

Today I uninstalled it and just use built in Adguard DNS settings to block ads. Zero issues, websites actually load faster, correctly and ad free.

 

[grichaud]

Hi,
Is there any way to keep using #Diversion with a Wireguard VPN? I've used Diversion with OpenVPN and it works fine. Wireguard is way faster, but I can't find a way to keep using Diversion with it.

 

[Igs]

Hi! I have one question. Diversion has an option to add custom lists.
Will they be added as an addition to the built-in lists? Or once I have added a custom list, all the buit-in are ignored?

 

[Treadler]

Hi! I have one question. Diversion has an option to add custom lists.
Will they be added as an addition to the built-in lists? Or once I have added a custom list, all the buit-in are ignored?

You can use whatever compatible lists you like, including, or excluding the suggested ones.

 

[Igs]

Edit: OK, i see, when you click "custom" it shows the built-in list on the top.

Hi, good! But my question was slightly different.

"Or once I have added a custom list, all the buit-in are ignored?"

 

[bluzfanmr1]

Hi,
Is there any way to keep using #Diversion with a Wireguard VPN? I've used Diversion with OpenVPN and it works fine. Wireguard is way faster, but I can't find a way to keep using Diversion with it.

Are you talking VPN Server? If so, under the advanced settings, toggle "Allow DNS" to on.

 

[grichaud]

Are you talking VPN Server? If so, under the advanced settings, toggle "Allow DNS" to on.

Hi,
Thanks for replying. Yes, I mean a VPN server. Do you mean Diversion advanced settings or Wireguard VPN settings? I guess you mean Wireguard VPN advanced settings, because I dont see any "Diversion" advanced settings or "Allow DNS" setting. But I don't see any advanced settings uder Wireguard VPN settings either. I'm using Asuswrt-Merlin and my VPN provider is Surfshark. It requires to set their servers IPs on the "DNS Server" setting. I've tried changing this setting to my router IP and then redirecting all DNS queries from the Dnsmasq server to those IPs. I don't know how Wireguard works, but this does not work because, when I do this, the IP belong to other servers, not to my VPN provider, and when I check for DNS leaks, it warns me that those servers are not Surfshark's. So, I don't know how to set it the right way.

 

[thelonelycoder]

I gave up screwing around with Diversion and constant bugs that keep blocking all access to internet at random times of the day.

Bugs?