Diversion Need help with selecting new set of filter lists / block lists for Diversion 5.0

[thelonelycoder]

Hey all
I'm still feeling a bit under the weather after a complete breakdown last Sunday - from sheer work overload at my job over the last weeks. Things look much better and brighter now and I want to thank all members here that expressed their concerns. I am grateful for your moral support.

Now, Diversion 5.0 is due to be released soon and only a couple of items are left to clear from my list.
The very last item is to prepare the release (duh), so I need help with the second-to-last task.

If you have time to look into evaluating filter lists, black lists (whatever one calls them) I would greatly appreciate your help.
This is for the b, 1 option in Diversion, Select a predefined list

Lets start with what Diversion currently offers in v4.3.3:
Minimal list
https://winhelp2002.mvps.org/hosts.txt (obsolete, no longer maintained)

Small list
https://someonewhocares.org/hosts/zero/hosts (hosts only format available)
https://pgl.yoyo.org/adservers/serv...&showintro=0&mimetype=plaintext&useip=0.0.0.0 (Dnsmasq server variant available)
https://raw.githubusercontent.com/hoshsadiq/adblock-nocoin-list/master/hosts.txt (Blocking Web Browser Bitcoin Mining, will no longer be included)

Standard list
https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts (Unified hosts, only in hosts format available)

Medium list
https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts (Unified hosts, only in hosts format available)
https://hostfiles.frogeye.fr/firstparty-trackers-hosts.txt (first-party trackers, only in hosts format available, will no longer be included)

Large list
https://dbl.oisd.nl/ (Hosts format, will no longer be available on January 1st 2024, Dnsmasq local format available)

All the above are hosts or domains lists. They are not ideal format wise or are either obsolete or will no longer be available after years end.

With Diversion 5.0 the following formats to block domains are supported:
- Traditional hosts list, an IP-domain pair. Examples: 0.0.0.0 domain.com or 127.0.0.1 domain.com
- Traditional domains only list. Example: domain.com
- Dnsmasq address format. Example, note the number sign after last slash: address=/domain.com/#
- Dnsmasq local or server format. Examples: local=/domain.com/ or server=/domain.com/

Diversion 5.0 converts all these to the Dnsmasq local format: local=/domain.com/
So ideally a new list would be in local or server format. These lists are generally optimized for Dnsmasq, while traditional lists are generally not.

This is what I have so far for Diversion 5.0:
Minimal list, target ~10,000 blocked domains
<none>

Small list, target ~18,000 blocked domains

https://pgl.yoyo.org/adservers/serverlist.php?hostformat=dnsmasq-server&showintro=0&mimetype=plaintext

<needs more lists, or one list that fits target>

Standard list, target ~40,000 blocked domains. In my opinion this list ideal, it will be the default list for new installations

https://small.oisd.nl/dnsmasq2

<meets target>

Medium list, target ~60,000 - 100,000 blocked domains, In my opinion this list is ideal

https://raw.githubusercontent.com/pexcn/daily/gh-pages/adblock/adblock.conf

<meets target>

Large list, target BIG, In my opinion this list is ideal

https://big.oisd.nl/dnsmasq2

<meets target>

Ideally a single file would meet the target, as some already do. I will consider only well maintained, non-commercial and regularly updated lists.

Sources are many, but FilterLists is a great way to start. One can set filters, Diversion is a filter term but it will present hosts lists only at the moment. I will alert the maintainer of the change as soon as 5.0 is out the door.

Thanks for your time

 

[John Fitzgerald]

Hey all
I'm still feeling a bit under the weather after a complete breakdown last Sunday - from sheer work overload at my job over the last weeks. Things look much better and brighter now and I want to thank all members here that expressed their concerns. I am grateful for your moral support.

Now, Diversion 5.0 is due to be released soon and only a couple of items are left to clear from my list.
The very last item is to prepare the release (duh), so I need help with the second-to-last task.

If you have time to look into evaluating filter lists, black lists (whatever one calls them) I would greatly appreciate your help.
This is for the b, 1 option in Diversion, Select a predefined list

Lets start with what Diversion currently offers in v4.3.3:
Minimal list
https://winhelp2002.mvps.org/hosts.txt (obsolete, no longer maintained)

Small list
https://someonewhocares.org/hosts/zero/hosts (hosts only format available)
https://pgl.yoyo.org/adservers/serv...&showintro=0&mimetype=plaintext&useip=0.0.0.0 (Dnsmasq server variant available)
https://raw.githubusercontent.com/hoshsadiq/adblock-nocoin-list/master/hosts.txt (Blocking Web Browser Bitcoin Mining, will no longer be included)

Standard list
https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts (Unified hosts, only in hosts format available)

Medium list
https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts (Unified hosts, only in hosts format available)
https://hostfiles.frogeye.fr/firstparty-trackers-hosts.txt (first-party trackers, only in hosts format available, will no longer be included)

Large list
https://dbl.oisd.nl/ (Hosts format, will no longer be available on January 1st 2024, Dnsmasq local format available)

All the above are hosts or domains lists. They are not ideal format wise or are either obsolete or will no longer be available after years end.

With Diversion 5.0 the following formats to block domains are supported:
- Traditional hosts list, an IP-domain pair. Examples: 0.0.0.0 domain.com or 127.0.0.1 domain.com
- Traditional domains only list. Example: domain.com
- Dnsmasq address format. Example, note the number sign after last slash: address=/domain.com/#
- Dnsmasq local or server format. Examples: local=/domain.com/ or server=/domain.com/

Diversion 5.0 converts all these to the Dnsmasq local format: local=/domain.com/
So ideally a new list would be in local or server format. These lists are generally optimized for Dnsmasq, while traditional lists are generally not.

This is what I have so far for Diversion 5.0:
Minimal list, target ~10,000 blocked domains
<none>

Small list, target ~18,000 blocked domains

https://pgl.yoyo.org/adservers/serverlist.php?hostformat=dnsmasq-server&showintro=0&mimetype=plaintext

<needs more lists, or one list that fits target>

Standard list, target ~40,000 blocked domains. In my opinion this list ideal, it will be the default list for new installations

https://small.oisd.nl/dnsmasq2

<meets target>

Medium list, target ~60,000 - 100,000 blocked domains, In my opinion this list is ideal

https://raw.githubusercontent.com/pexcn/daily/gh-pages/adblock/adblock.conf

<meets target>

Large list, target BIG, In my opinion this list is ideal

https://big.oisd.nl/dnsmasq2

<meets target>

Ideally a single file would meet the target, as some already do. I will consider only well maintained, non-commercial and regularly updated lists.

Sources are many, but FilterLists is a great way to start. One can set filters, Diversion is a filter term but it will present hosts lists only at the moment. I will alert the maintainer of the change as soon as 5.0 is out the door.

Thanks for your time

Do you want to get rid of the “Minimal List” all together? It would seem practical as the next level is only 8000 more.
My gut says in todays world you could eliminate the”Small List” as well and just offer 3 levels starting with the “Standard List”.

Thank you and glad you are feeling better.

 

[bbunge]

I don't understand why the Standard list is not the Steve Black list. This is one reason I went back to Merlin's firmware.
FWIW stick with a standard list and dump the others. Diversion is easy enough to customize.

 

[jrichard326]

These are what I have been using. It is mostly a set and forget here. I am sure there is tons of duplication and a lot of parsing going on:

https://gitlab.com/hagezi/mirror/-/raw/main/dns-blocklists/hosts/multi.txt

https://hostfiles.frogeye.fr/firstparty-trackers-hosts.txt

https://o0.pages.dev/Lite/hosts.txt

https://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&mimetype=plaintext

https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts

https://raw.githubusercontent.com/r-a-y/mobile-hosts/master/AdguardDNS.txt

https://www.github.developerdan.com/hosts/lists/ads-and-tracking-extended.txt

If uiDivStats will no longer work with the new version, it will be sorely missed.

 

[kuki68ster]

Hey all
I'm still feeling a bit under the weather after a complete breakdown last Sunday - from sheer work overload at my job over the last weeks. Things look much better and brighter now and I want to thank all members here that expressed their concerns. I am grateful for your moral support.

Now, Diversion 5.0 is due to be released soon and only a couple of items are left to clear from my list.
The very last item is to prepare the release (duh), so I need help with the second-to-last task.

If you have time to look into evaluating filter lists, black lists (whatever one calls them) I would greatly appreciate your help.
This is for the b, 1 option in Diversion, Select a predefined list

Lets start with what Diversion currently offers in v4.3.3:
Minimal list
https://winhelp2002.mvps.org/hosts.txt (obsolete, no longer maintained)

Small list
https://someonewhocares.org/hosts/zero/hosts (hosts only format available)
https://pgl.yoyo.org/adservers/serv...&showintro=0&mimetype=plaintext&useip=0.0.0.0 (Dnsmasq server variant available)
https://raw.githubusercontent.com/hoshsadiq/adblock-nocoin-list/master/hosts.txt (Blocking Web Browser Bitcoin Mining, will no longer be included)

Standard list
https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts (Unified hosts, only in hosts format available)

Medium list
https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts (Unified hosts, only in hosts format available)
https://hostfiles.frogeye.fr/firstparty-trackers-hosts.txt (first-party trackers, only in hosts format available, will no longer be included)

Large list
https://dbl.oisd.nl/ (Hosts format, will no longer be available on January 1st 2024, Dnsmasq local format available)

All the above are hosts or domains lists. They are not ideal format wise or are either obsolete or will no longer be available after years end.

With Diversion 5.0 the following formats to block domains are supported:
- Traditional hosts list, an IP-domain pair. Examples: 0.0.0.0 domain.com or 127.0.0.1 domain.com
- Traditional domains only list. Example: domain.com
- Dnsmasq address format. Example, note the number sign after last slash: address=/domain.com/#
- Dnsmasq local or server format. Examples: local=/domain.com/ or server=/domain.com/

Diversion 5.0 converts all these to the Dnsmasq local format: local=/domain.com/
So ideally a new list would be in local or server format. These lists are generally optimized for Dnsmasq, while traditional lists are generally not.

This is what I have so far for Diversion 5.0:
Minimal list, target ~10,000 blocked domains
<none>

Small list, target ~18,000 blocked domains

https://pgl.yoyo.org/adservers/serverlist.php?hostformat=dnsmasq-server&showintro=0&mimetype=plaintext

<needs more lists, or one list that fits target>

Standard list, target ~40,000 blocked domains. In my opinion this list ideal, it will be the default list for new installations

https://small.oisd.nl/dnsmasq2

<meets target>

Medium list, target ~60,000 - 100,000 blocked domains, In my opinion this list is ideal

https://raw.githubusercontent.com/pexcn/daily/gh-pages/adblock/adblock.conf

<meets target>

Large list, target BIG, In my opinion this list is ideal

https://big.oisd.nl/dnsmasq2

<meets target>

Ideally a single file would meet the target, as some already do. I will consider only well maintained, non-commercial and regularly updated lists.

Sources are many, but FilterLists is a great way to start. One can set filters, Diversion is a filter term but it will present hosts lists only at the moment. I will alert the maintainer of the change as soon as 5.0 is out the door.

Thanks for your time

Firstly, feel better soon!

Health comes first!!!

Secondly, have you heard about Hagezis (https://github.com/hagezi/dns-blocklists) lists ? I use them on my devices, with diversion... It is well maintained, regularly updated and free to use.

My suggestion, using some of the lists created by Hagezis:

Minimal list (Entries: 21.2194 domains/hosts - 65.381 compressed domains)
Multi light
https://raw.githubusercontent.com/hagezi/dns-blocklists/main/domains/light.txt
Hand brush - Cleans the Internet and protects your privacy! Blocks Ads, Tracking, Metrics, some Malware and Fake.
Small list (Entries: 73.8642 domains/hosts - 169.339 compressed domains)

Multi normal

https://raw.githubusercontent.com/hagezi/dns-blocklists/main/domains/light.txt

Broom - Cleans the Internet and protects your privacy! Blocks Ads, Affiliate, Tracking, Metrics, Telemetry, Phishing, Malware, Scam, Fake, Coins and other "Crap".

Standard list (Entries: 97.9831 domains/hosts - 253.729 compressed domains) Recommended
Multi pro

https://raw.githubusercontent.com/hagezi/dns-blocklists/main/domains/pro.txt

Cleans the Internet and protects your privacy! Blocks Ads, Affiliate, Tracking, Metrics, Telemetry, Phishing, Malware, Scam, Fake, Coins and other "Crap".

Medium list (Entries: 117.5387 domains/hosts - 307.167 compressed domains)
Multi pro++

https://raw.githubusercontent.com/hagezi/dns-blocklists/main/domains/pro.plus.txt

Sweeper - Aggressive cleans the Internet and protects your privacy! Blocks Ads, Affiliate, Tracking, Metrics, Telemetry, Phishing, Malware, Scam, Fake, Coins and other "Crap".

More aggressive version of the Multi PRO blocklist. It may contain few false positive domains that limit functionality. Therefore it should only be used by experienced users. Furthermore, an admin should be available to unblock incorrectly blocked domains. Reported false positive domains will be removed from the list!

Large list (139.3719 domains/hosts - 387.651 compressed domains)
Multi ultimate

https://raw.githubusercontent.com/hagezi/dns-blocklists/main/domains/ultimate.txt

Ultimate Sweeper - Strictly cleans the Internet and protects your privacy! Blocks Ads, Affiliate, Tracking (+Referral), Metrics, Telemetry, Phishing, Malware, Scam, Free Hoster, Fake, Coins and other "Crap".

Stricter version of the Multi PRO++ blocklist. It may contain false positive domains that limit functionality. Therefore it should only be used by experienced users. Furthermore, an admin should be available to unblock incorrectly blocked domains. Reported false positive domains will be removed from the list!

NOTE: META graph trackers are blocked in Ultimate. This restricts the use of Facebook/Instagram and Facebook Messenger. To use Facebook/Instagram with Ultimate, unblock the following domains: META Tracker

 

[martinr]

I echo kuki68ster’s “health comes first”. ”The graveyards are full of people who were once indispensable.”

Whichever lists you settle on, I hope they also block malicious sites/domains. People talk about ad-blocking lists - and perhaps malicious-site blocking is implied within that phrase - but I’d far rather see ads getting through than a single malicious site, and I doubt I’m alone.

 

[ajp2k14]

Another vote for Hagezi, they are updated often and well maintained.

GitHub - hagezi/dns-blocklists: DNS-Blocklists: For a better internet - keep the internet clean!

DNS-Blocklists: For a better internet - keep the internet clean! - hagezi/dns-blocklists

github.com

 

[Mutzli]

You'll probably get as many recommendations as comments for this topic. For me steve blacklist https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts has served me well over the last few years. It's one of the best maintained lists out there.

I also use the small one from oisd.nl ( https://small.oisd.nl/domains ) and a few others with protection for streaming, country and android specific.

So, my vote for the medium list would also go to Steve Black List and for the large list the one from oisd.nl

 

[bluepoint]

I've been using large list with Diversion lite and happy with it so most likely I'll stay.

 

[Davidncali001]

Need any Beta Testers? I’d be happy to apply if Beta Testing is available. Thanks and glad things are getting better for you thelonelycoder.

 

[thelonelycoder]

If uiDivStats will no longer work with the new version, it will be sorely missed.

Diversion 5.0 will disable uiDivStats for the time being.
amtm will show uiDivStats, incompatible until an updated version is available.
There is no ETA for this version at this time, please do not bother @Jack Yaz, it will be done when it's done.

 

[kstamand]

Ditto the large list. Been using it all along with NO issues and plan to stay with it or it’s equivalent.

Thanks @thelonelycoder for all your efforts and commitment!!!

 

[^Tripper^]

Curious why you’re dropping the following list;

https://hostfiles.frogeye.fr/firstparty-trackers-hosts.txt (first-party trackers, only in hosts format available, will no longer be included)

And please take care and recharge yourself! Diversion 5.0 can wait. Your work and YOU are appreciated!

 

[John Fitzgerald]

Curious why you’re dropping the following list;

https://hostfiles.frogeye.fr/firstparty-trackers-hosts.txt (first-party trackers, only in hosts format available, will no longer be included)

And please take care and recharge yourself! Diversion 5.0 can wait. Your work and YOU are appreciated!

From above, 1st Post: "I will consider only well maintained, non-commercial and regularly updated lists."

 

[^Tripper^]

From above, 1st Post: "I will consider only well maintained, non-commercial and regularly updated lists."

Ah ok. Suppose it’s not being maintained any longer.

Thanks for pointing that out.

 

[aru]

Can Diversion's new filter list/block list be compatible with AdGuardHome's filter list/block list?

for example:

https://adguardteam.github.io/HostlistsRegistry/assets/filter_1.txt

https://adguardteam.github.io/HostlistsRegistry/assets/filter_2.txt

 

[John Fitzgerald]

Can Diversion's new filter list/block list be compatible with AdGuardHome's filter list/block list?

for example:

https://adguardteam.github.io/HostlistsRegistry/assets/filter_1.txt

https://adguardteam.github.io/HostlistsRegistry/assets/filter_2.txt

Does it meet this criteria???

From above, 1st Post: "I will consider only well maintained, non-commercial and regularly updated lists."

AND, what do you mean by compatible? Your private adjustments are not bearing on the the Developer/ Author.

 

[thelonelycoder]

Can Diversion's new filter list/block list be compatible with AdGuardHome's filter list/block list?

for example:

https://adguardteam.github.io/HostlistsRegistry/assets/filter_1.txt

https://adguardteam.github.io/HostlistsRegistry/assets/filter_2.txt

See my first post what compatible entries are for Diversion. The first list definitely is not.

 

[aru]

Thank you for your reminder. Next time, before posing a question, I will make sure to check the first post again. I appreciate your guidance.

 

[Gary_Dexter]

I don't understand why the Standard list is not the Steve Black list. This is one reason I went back to Merlin's firmware.
FWIW stick with a standard list and dump the others. Diversion is easy enough to customize.

Lists like OISD and Hagezi incorporate lists from many sources. I believe the Steve Black lists are included in both these list providers options.