Diversion Diversion 5.1.3 - the Router Ad-Blocker, May 09, 2024

[Amwjujo]

It should block all the extra domains , in total now 612581 ....
View attachment 55234

Thank you.
Is very strange. I just checked again. It does block some sites but not all of them .. Iike the main ones. The strange fact is that when i was testing the NSFW list on its own it was working.

 

[thelonelycoder]

Someone please correct me, but I think the blocked domain count on the Diversion 5.0 main screen counts the last blank line in the file blocklist.conf and file blocklist.fs_conf as a blocked domain. My blocked domain count appears to be one number higher than the list of domains in the primary and secondary blocking lists.

If so, that’s it. I quit coding. One off? Unbearable. What a miserable experience. I want my money back.
Can you at least post a screenshot so I can see it with my own eyes?

 

[martinr]

Thank you.
Is very strange. I just checked again. It does block some sites but not all of them .. Iike the main ones. The strange fact is that when i was testing the NSFW list on its own it was working.

Reboot the router, go to bed and approach it with a clear head tomorrow. it may well make more sense then.

 

[Viktor Jaep]

Reboot the router, go to bed and approach it with a clear head tomorrow. it may well make more sense then.

@Amwjujo may also want to clear his own local PC's DNS cache or any other DNS caches in use, as I'm sure it knows exactly how to get to playboy.com and whatnot.

 

[Amwjujo]

Reboot the router, go to bed and approach it with a clear head tomorrow. it may well make more sense then.

You're probably right. I'll just open another beer and start over tomorrow. Not going to bed that early
Cheers guys.

 

[nomore4u]

If so, that’s it. I quit coding. One off? Unbearable. What a miserable experience. I want my money back.
Can you at least post a screenshot so I can see it with my own eyes?

Sorry if I sounded negative. I do appreciate all the hard work. No fix required. All is working great. Thank You.

As requested, see attached for a screenshot of the last line in file blocklist.conf and the 5.0 main menu.

I was playing with these 3 host files to see the count differences as I added one at a time and noticed the main menu count appeared to be one higher than the unique total:

https://big.oisd.nl/dnsmasq2

https://nsfw.oisd.nl/dnsmasq2

https://raw.githubusercontent.com/hagezi/dns-blocklists/main/dnsmasq/pro.txt

 

[thelonelycoder]

@thelonelycoder ... just wanted to give you a huge thanks and congrats on both your stellar releases for both AMTM and Diversion here at the end/beginning of the new year! We appreciate you immensely!!

Thank you! Much appreciated.

 

[Amwjujo]

@Amwjujo may also want to clear his own local PC's DNS cache or any other DNS caches in use, as I'm sure it knows exactly how to get to playboy.com and whatnot.

That I sure do .. just trying to keep my son away from it .. at least for a while

 

[thelonelycoder]

Sorry if I sounded negative. I do appreciate all the hard work. No fix required. All is working great. Thank You.

As requested, see attached for a screenshot of the last line in file blocklist.conf and the 5.0 main menu.

I was playing with these 3 host files to see the count differences as I added one at a time and noticed the main menu count appeared to be one higher than the unique total:

https://big.oisd.nl/dnsmasq2

https://nsfw.oisd.nl/dnsmasq2

https://raw.githubusercontent.com/hagezi/dns-blocklists/main/dnsmasq/pro.txt

Yeah. Looks like I have to continue coding. See that one entry in the blacklist?
Good night.

 

[SomeWhereOverTheRainBow]

If so, that’s it. I quit coding. One off? Unbearable. What a miserable experience. I want my money back.
Can you at least post a screenshot so I can see it with my own eyes?

While I am not saying Diversion's code is doing this, I ran into such an occurrence before using grep -c in my own codings, my fix was to change to wc -l (or it might have been vice versa) or add alittle more logic to the grep -c to know that it was counting an unintended line. I honestly don't suspect your code of doing this either. I believe some more details are in order to actually indicate the issue. I have yet to run into any issues on my tests runs.

@thelonelycoder
Upon further review of diversion, the only way diversion script could be counting the lines wrong is if there are lines with whitespace characters , or characters which have not been properly dealt with either in the grep command logic, or filtered out of the list processing by some other means.

eg.
/opt/bin/grep "^[^#]" test.fs | wc -l

content of test.fs

somelist
and
  #list with space

output count

RT-AX88U_Pro:/tmp/home/root# /opt/bin/grep "^[^#]" test.fs | wc -l
3

something like /opt/bin/grep -E "^[^#[:space:]]+([[:space:]]|$)" would be more forgiving as the output in this case would only be 2 instead of 3; otherwise there are unintended characters finding their way into the blocklist if the users reporting the issue is correct. We still have to see more details to support their claims though. For example, what does their blocklist actually consist of? Are they manually editing the blocklist directly themselves? There are a lot of things that need to be investigated.

 

[Rhialto]

Does yours block adult?

Haven't installed 5 yet but for adult content, why not use DNS filtering in the WAN section? https://cleanbrowsing.org/filters/ is available or you can enter other ones manually.

 

[thelonelycoder]

While I am not saying Diversion's code is doing this, I ran into such an occurrence before using grep -c in my own codings, my fix was to change to wc -l (or it might have been vice versa) or add alittle more logic to the grep -c to know that it was counting an unintended line. I honestly don't suspect your code of doing this either. I believe some more details are in order to actually indicate the issue. I have yet to run into any issues on my tests runs.

@thelonelycoder
Upon further review of diversion, the only way diversion script could be counting the lines wrong is if there are lines with whitespace characters , or characters which have not been properly dealt with either in the grep command logic, or filtered out of the list processing by some other means.

eg.
/opt/bin/grep "^[^#]" test.fs | wc -l

content of test.fs

somelist
and
  #list with space

output count

RT-AX88U_Pro:/tmp/home/root# /opt/bin/grep "^[^#]" test.fs | wc -l
3

something like /opt/bin/grep -E "^[^#[:space:]]+([[:space:]]|$)" would be more forgiving as the output in this case would only be 2 instead of 3; otherwise there are unintended characters finding their way into the blocklist if the users reporting the issue is correct. We still have to see more details to support their claims though. For example, what does their blocklist actually consist of? Are they manually editing the blocklist directly themselves? There are a lot of things that need to be investigated.

I‘m using wc -l, and the count is correct. That number includes the one blacklist entry. This years mystery is solved.

 

[Milan]

I suppose this will not work together with Unbound as DNS resolver and disabled dnsmasq?

 

[SomeWhereOverTheRainBow]

I suppose this will not work together with Unbound as DNS resolver and disabled dnsmasq?

Nope, your dns requests will bypass dnsmasq(diversion).

 

[Milan]

Nope, your dns requests will bypass dnsmasq(diversion).

Then, is it possible to make it work with unbound? Just now I am using AdBlock there - same method (NXDOMAIN)

 

[Smokey]

I'm having an odd issue where only some domains on my whitelist are being unblocked, and I can't figure out why. Diversion and amtm were installed after the big update on a freshly formatted drive. I have tried hard resetting my router as well.

I have processed the whitelist, sort and verified the whitelist, and processed all lists. I have nothing manually blocked. These domains were correctly whitelisted on the previous version. I did not restore anything from a backup.

The blocklist I'm currently using is: https://raw.githubusercontent.com/hagezi/dns-blocklists/main/domains/pro.txt

The two domains that remain blocked are shown in the screenshot provided. I appreciate any help you can give.

 

[martinr]

I'm having an odd issue where only some domains on my whitelist are being unblocked, and I can't figure out why. Diversion and amtm were installed after the big update on a freshly formatted drive. I have tried hard resetting my router as well.

I have processed the whitelist, sort and verified the whitelist, and processed all lists. I have nothing manually blocked. These domains were correctly whitelisted on the previous version. I did not restore anything from a backup.

The blocklist I'm currently using is: https://raw.githubusercontent.com/hagezi/dns-blocklists/main/domains/pro.txt

The two domains that remain blocked are shown in the screenshot provided. I appreciate any help you can give.

Identical behaviour on all devices in the house?

and welcome to the forum, too.

 

[Smokey]

Identical behaviour on all devices in the house?

and welcome to the forum, too.

Yes indeed, all devices are blocking the same whitelisted domains.

.. and thank you.

 

[martinr]

Have a look at L&LD’s tutorials (see his signature block) on resetting and minimal and manual setup (M&M Config) as well as installing AMTM just to make sure you’re happy you done everything righ. For example, see here:

L&LD

www.snbforums.com

It’s good to get familiar with where to find these tutorials, if nothing else.

 

[Smokey]

No, sorry. I only meant that I power cycled the router. The extent of my resetting before reinstalling amtm and Diversion was wiping the drive and reformatting it, then reinstalling from scratch. I haven't touched anything as far as the router firmware is concerned.

Do you think that's a likely solution to the issue?