Diversion Diversion 4.3.3 - the Router Ad-Blocker, released April 02 2023

[heysoundude]

Well... I had the large blocklist for a couple of days, but experienced sluggish router performance, so switched back to the default. For testing sake I just updated with large, no difference

"Many hands make light work"
while letting the router do the lion's share of the heavy lifting is the best way to go about it, you still have to consider/implement SOME client level backup, should something fail/break at the perimeter. I use Brave for my browser and i'm reassured by how much slips by diversion and gets caught by Brave. (I'm using Medium blocklist)
(If you DO choose Brave, stay away from the "Rewards" scheme to get crypto rich and you'll be fine)

 

[thelonelycoder]

"Many hands make light work"
while letting the router do the lion's share of the heavy lifting is the best way to go about it, you still have to consider/implement SOME client level backup, should something fail/break at the perimeter. I use Brave for my browser and i'm reassured by how much slips by diversion and gets caught by Brave. (I'm using Medium blocklist)
(If you DO choose Brave, stay away from the "Rewards" scheme to get crypto rich and you'll be fine)

View attachment 42909
View attachment 42911

Uhh, remember, Diversion blocks outgoing requests from, for example - and drumroll - a browser. Apps or browsers request ads from an ad domain which then Diversion blocks. Your 53 grand total would likely have been blocked by Diversion anyway.

 

[heysoundude]

Uhh, remember, Diversion blocks outgoing requests from, for example - and drumroll - a browser. Apps or browsers request ads from an ad domain which then Diversion blocks. Your 53 grand total would likely have been blocked by Diversion anyway.

I should've clarified, you're right: I've decided to use that browser for its other purported protection as well as a 2nd line of defence in case diversion isn't functioning or fails. (I assume the number is more tracking attempts than actual ads that diversion "missed")

 

[Ellenswamy]

If we have ipv6 enabled and want to use the ip to exclude some devices. I have the ipv4 Ip enabled as 192.168.50.3, but what do we put for ipv6?

 

[thelonelycoder]

If we have ipv6 enabled and want to use the ip to exclude some devices. I have the ipv4 Ip enabled as 192.168.50.3, but what do we put for ipv6?

That is not built into Diversion yet.

 

[Ellenswamy]

I am having a major issue with Diversion, I can't reboot my router with my USB installed. DHCP etc doesn't work and can't get IPs. I have tested to confirm it is Diversion slowly removing adding and rebooting. Not until I removed diversion did I get a successful reboot. Re-installed and same issue. Nothing shows in the logs so no clue what is happening.. :-(

edit: so I uninstalled everything and with just enterware installed the router boots with my usb plugged in. I reinstalled diversion standard but did not set up the exclusion ip. And rebooted and the router booted fine with my usb plugged in. Could that be the issue? I tried going through the system logs and found no reason why my router had issues with the usb drive in, unplug the drive and it comes right up.

 

[archiel]

Using secondary blocklist with, 386.7 IPv6 and DNSFilter

I had a secondary setup for Diversion (with Pixelserv) for my wife's work laptop, just using the minimal list (for other users set to standard) and applied this by adding the secondary IP as a custom DNS in DNSFilter and then adding the laptop to this. With the new DNSFilter in 386.7 (and with nothing in the IPv6 box) both dnsmasq.log and unbound.log were flooded with answered queries from this laptop (over 8Gb and 1.2Gb respectively).

The apparent knock-on was very slow loading after reboot, the swap file being brought into use and hanging on the System Log (with Scribe) page.

I ran the lograte process for diversion to strip down dnsmasq.log and scribe logrotate to do the same for unbound and for now I have changed the DNS filter for this device to Cloudflare.

The start-up process is still slow, presumably as dnsmasq1.log and dnsmasq2.log are still 8Gb and 12Gb respectively - is there any reason not to delete these files and should I disable diversion first?

 

[SomeWhereOverTheRainBow]

That is not built into Diversion yet.

It is possible to map all ipv6 request to the same ipv4 address by specifying ::ffff:192.168.50.3 for ipv6 replies instead of [::]. This will allow ipv4 to also handle ipv6 blocking. pixelserv-tls and dnsmasq understand these modifications. When pixelserv-tls recieves it , it assumes it is from ipv4, dnsmasq does all the magic. (this will only work with pixelserv-tls, generic diversion-lite blocking would have to be ::, since the main is 0.0.0.0 )

 

[jksmurf]

Hi, I have been happily using Diversion for quite a while (thank you to the Swiss Master) , just the light version to IP 0.0.0.0, no entware packages.

I am a relatively low-power user although I did manage to add a couple of modifications to the WhiteLists for sites I wanted to let through (including the ones below which said “forced” in any case). However (I believe) a recent update has somehow stopped me entering my web-based email (a repeatable issue).

When I turn off (disable) Diversion I can get to the Webmail Page fine every time; when Diversion is working the Login button simply does nothing.

The email login address is https://em.netvigator.com/mail#1 OR https://login.netvigator.com/ I added the domain netvigator.com to the Whitelist too.

Could someone please help explain what I can provide by way of logs or settings to enable others to help me check; or what steps I should go through to isolate the issue myself? Remember.. Low power user ... I can follow instructions though!

Cheers

k.

 

[archiel]

It is possible to map all ipv6 request to the same ipv4 address by specifying ::ffff:192.168.50.3 for ipv6 replies instead of [::]. This will allow ipv4 to also handle ipv6 blocking. pixelserv-tls and dnsmasq understand these modifications. When pixelserv-tls recieves it , it assumes it is from ipv4, dnsmasq does all the magic. (this will only work with pixelserv-tls, generic diversion-lite blocking would have to be ::, since the main is 0.0.0.0 )

I have just adjusted the custom DNS so it now reads IPv4: 10.50.60.11 and IPv6: ::ffff:10.50.60.11 and re-attached my wife's work laptop ..... and 12 hours later checking the dnsmasq and unbound logs all looks good. Thanks

 

[jksmurf]

Hi, I have been happily using Diversion for quite a while (thank you to the Swiss Master) , just the light version to IP 0.0.0.0, no entware packages.

I am a relatively low-power user although I did manage to add a couple of modifications to the WhiteLists for sites I wanted to let through (including the ones below which said “forced” in any case). However (I believe) a recent update has somehow stopped me entering my web-based email (a repeatable issue).

When I turn off (disable) Diversion I can get to the Webmail Page fine every time; when Diversion is working the Login button simply does nothing.

The email login address is https://em.netvigator.com/mail#1 OR https://login.netvigator.com/ I added the domain netvigator.com to the Whitelist too.

Could someone please help explain what I can provide by way of logs or settings to enable others to help me check; or what steps I should go through to isolate the issue myself? Remember.. Low power user ... I can follow instructions though!

Cheers

k.

Hi again,

My apologies in advance for the apparent bump, but I thought I'd try to revert to an ealier version of Diversion (no pun intended) to test out if the issue above was caused by a recent Diversion update or by something else. However I cannot find (on the Diversion WebPage) a mechanism to do this (revert to an earlier version)? Any pointers here please?

Thanks

k.

 

[Ellenswamy]

What does all of this mean. My number of SLU dropped requests seems pretty high. Want to make sure if I need to fix something or not.

slh	888	# of accepted HTTPS requests
slm	163	# of rejected HTTPS requests (missing certificate)
sle	0	# of rejected HTTPS requests (certificate available but not usable)
slc	159	# of dropped HTTPS requests (client disconnect without sending any request)
slu	4894	# of dropped HTTPS requests (other TLS handshake errors)

 

[thelonelycoder]

Hi, I have been happily using Diversion for quite a while (thank you to the Swiss Master) , just the light version to IP 0.0.0.0, no entware packages.

I am a relatively low-power user although I did manage to add a couple of modifications to the WhiteLists for sites I wanted to let through (including the ones below which said “forced” in any case). However (I believe) a recent update has somehow stopped me entering my web-based email (a repeatable issue).

When I turn off (disable) Diversion I can get to the Webmail Page fine every time; when Diversion is working the Login button simply does nothing.

The email login address is https://em.netvigator.com/mail#1 OR https://login.netvigator.com/ I added the domain netvigator.com to the Whitelist too.

Could someone please help explain what I can provide by way of logs or settings to enable others to help me check; or what steps I should go through to isolate the issue myself? Remember.. Low power user ... I can follow instructions though!

Cheers

k.

The best way to find out what's blocked and determine what needs to be whitelisted is to use the built in filter function in Diversion with f.

It is very likely that the domain netvigator.com itself does not need to be whitelisted as otherwise you would not be able to see any content from that website at all.

I'm sure what prevents your login button to work is because some third party link is blocked.
When I run f, 3 (blocked domains) the gist of what is blocked on that website are the following domains:

www.google-analytics.com
8666729.fls.doubleclick.net
stats.g.doubleclick.net
googleads.g.doubleclick.net
cm.g.doubleclick.net
collect.tealiumiq.com

With these blocked I can enter a fake email and password and it appears to work as it states that I used an "Invalid Login ID or Password".

Your mileage may vary, so you'll have to determine whats blocked with your Diversion setup.
In Diversion, enter f and select 4 to limit the filtered blocked domains to the device you want to log into your webmail.
You will have to enter the IP address of that device first.
Once done, open your browser on that device and go to the netvigator login page, enter your credentials and hit the Login button.
All blocked domains will be printed out in Diversion, after a slight delay.
Considering that the domains I posted above do not block your login attempt, I would imagine any other domain as being a candidate for being the one that needs to be whitelisted.

Either test it by entering each one separately into the whitelist and then try again or add them all, test test the login and if it works, remove one by one until login fails.

 

[jksmurf]

The best way to find out what's blocked and determine what needs to be whitelisted is to use the built in filter function in Diversion with f.

....

With these blocked I can enter a fake email and password and it appears to work as it states that I used an "Invalid Login ID or Password".

Your mileage may vary, so you'll have to determine whats blocked with your Diversion setup.

...

Either test it by entering each one separately into the whitelist and then try again or add them all, test test the login and if it works, remove one by one until login fails.

Just wanted to say a huge thank you @thelonelycoder for very concise, clear directions.

Whilst f,4 would certainly pinpoint more quickly I did try using f, 4 but oddly nothing (at all) come up for the IP address of my device. So I just used f, 1 and watched everything that was logged whilst trying to acess ONLY that site. I eventually compiled a list of possible culprits and came up with this (below) which worked, so hopefully that was the one.

If not then I have a few others to try but at least whitelisting this one seems to make it work for now and I know how to troubleshoot it.

tags.tiqcdn.com

Thanks once again, very happy.

cheers,

k.

 

[collations_interrena]

Only today I noticed the setting to opt out of hardcoded whitelist (setting introduced with 4.2) is gone. Since when? I'm checking the changelog with each update and I didn't find any mention of this change/revert. Not even after doing a recheck after noticing this change.

Even if I really appreciate your work I don't find really nice to remove this setting without saying anything (again, nothing in the changelogs). Yes, users can still add the hard coded domains into their blacklist but it still give a bad taste of discovering it this way.

My bad, I looked at the wrong section (blacklist instead of whitelist). My fault, please accept my apologies after my too quick statement.

 

[thelonelycoder]

Hey all.
I am planning on removing the experimental feature YouTube video ads blocking in Diversion with the next update.
If I go ahead with it, it will automatically remove the feature from your router (when installed) during the Diversion update.

This move is open to discussion, feel free to add your comments.

My reasoning to remove it is that YouTube video ads blocking does not and cannot work with a DNS based blocker like Diversion is.
The feature has always been experimental and was originally hastily coded for Diversion when it apparently seemed to work as an addon for Pi-Hole.
Although I improved the original code to be more precise over time and added levels of "forcefulness" to the Video ads blocking. In the end it was all in vain in my opinion. It had a placebo effect at best and/or non-loading content-videos at worst.

Let us know what you think: Keep or drop YouTube video ads blocking feature in Diversion.

 

[Kingp1n]

My vote goes to drop this feature. I had some of issues with the non-loading content of videos. But either way I'm down with whatever u decide. I'll just keep it off if needed.

 

[SomeWhereOverTheRainBow]

Hey all.
I am planning on removing the experimental feature YouTube video ads blocking in Diversion with the next update.
If I go ahead with it, it will automatically remove the feature from your router (when installed) during the Diversion update.

This move is open to discussion, feel free to add your comments.

My reasoning to remove it is that YouTube video ads blocking does not and cannot work with a DNS based blocker like Diversion is.
The feature has always been experimental and was originally hastily coded for Diversion when it apparently seemed to work as an addon for Pi-Hole.
Although I improved the original code to be more precise over time and added levels of "forcefulness" to the Video ads blocking. In the end it was all in vain in my opinion. It had a placebo effect at best and/or non-loading content-videos at worst.

Let us know what you think: Keep or drop YouTube video ads blocking feature in Diversion.

I have no qualms with keeping or dropping it. The reason why is because you coded it @thelonelycoder . To remove such beautiful code is a hard decision indeed, even if some say the effectiveness is "placebo" in nature. I am at the belief that it does minimize the ad load, but it may not be substantial enough to be considered an "effective" solution on all platforms concerned. Your beautiful coding ensures that the honeypot never gets too full, therefore it resets after awhile preventing or limiting the potential for youtube videos themselves to be blocked. Maybe if you keep it, then you need to lower the honeypot number to ensure the ad content collection resets sooner. To be honest though I am not truely sure it is worth the effort to keep it then.

 

[jsbeddow]

I'll add my vote to the "drop" count. It has long been agreed that this cannot be fully effective as a DNS based blocker of ads served from the same source as the content.
I applaud the effort that went into it of course.

 

[wbennett77]

Since it is a Diversion option and not part of a regular installation would be the only reason that I can think of that you might want to leave it. That being said, I agree that it is not very effective and I am ok either way. Thanks for all you that you do @thelonelycoder