The ‘full’ I believe.Now there is a "basic" and "full" oisd.nl list in six different categories.
Which one is Diversion using when choosing the "large" list?
For some reason, I am not able to use Large block list i.e. hosts.oisd.nl. Did a monitor of dnsmasq.log when switching to Large block list and it seems to stop processing DNS request after the list update is completed
Code:Mar 14 18:21:47 dnsmasq[32569]: query[A] dns.msftncsi.com from 127.0.0.1 Mar 14 18:21:47 dnsmasq[32569]: forwarded dns.msftncsi.com to 127.0.1.1 Mar 14 18:21:47 dnsmasq[32569]: query[AAAA] dns.msftncsi.com from 127.0.0.1 Mar 14 18:21:47 dnsmasq[32569]: cached dns.msftncsi.com is fd3e:4f5a:5b81::1 Mar 14 18:21:47 dnsmasq[32569]: validation result is INSECURE Mar 14 18:21:47 dnsmasq[32569]: reply dns.msftncsi.com is 131.107.255.255 Mar 14 18:21:51 dnsmasq[32569]: read //etc//hosts - 42 addresses
If I were to switch to another list e.g. Medium, processing of the DNS request resumes after the the list update is completed
Code:Mar 14 18:32:44 dnsmasq[8813]: read //etc//hosts - 42 addresses Mar 14 18:32:44 dnsmasq[8813]: using nameserver 127.0.1.1#53 Mar 14 18:32:44 dnsmasq[8813]: using only locally-known addresses for domain lan.kylim.net Mar 14 18:32:55 dnsmasq[8813]: query[NS] . from 192.168.1.13 Mar 14 18:32:55 dnsmasq[8813]: forwarded . to 127.0.1.1 Mar 14 18:33:04 dnsmasq[8813]: query[A] dns.msftncsi.com from 127.0.0.1 Mar 14 18:33:04 dnsmasq[8813]: forwarded dns.msftncsi.com to 127.0.1.1
Any suggestion on how to further troubleshoot or what could be the cause?
Not running any 3rd-party resolver e.g. Unbound; only using the built-in Stubby resolver with DoT that comes with Merlin FW
TIA
I ‘think’ the Oisd list has massively increased in size recently. (By some 400,000 entries?)
I could be wrong & maybe the problem lies elsewhere.
So what may have been handled ok by the router previously, is now too much?
Something for everyone here:Maybe @thelonelycoder should consider changing OISD full to OISD basic which should prove to be comprehensive enough for most users.
Only offer the full list as an extended option.
I was using pihole, but recently install diversion the other day. So far working very well! Seems to be working the same with less overheard and less to manage from my limited experience so far. And with the shortcuts extension on iOS makes it easy to use also. And easy enough to exclude any device from blocking, just requires a simple setup.I had removed diversion thinking of reloading it, having issues with DNS ad blockers. Adguard is in beta, Nextdns is flaky beyond a joke.
Thoughts please? Thanks.
Agree I am liking it better then running a separate piholeDespite there being pihole and adguardhome alternatives, every time I access diversion I am in awe at the amazing capabilities it brings to our limited yet small routers. It speaks volumes to the capabilities of @thelonelycoder to leverage this unique awesomeness with his astonishing coding abilities. When I test adblockers, I always use Diversion as the preferred Asuswrt-Merlin standard, everything else is second to me.
You should try it with AdGuardHome.This might interest at least someone. Optimized oisd.nl lists.
GitHub - cbuijs/oisd: Optimized version of OISD BlockLists
The dilemma I had was the full list was just what I needed, but unfortunately took up too much memory on the old RT-AC86U. It was fine on its own, but if I enabled something like AiProtection or IPv6, my swapfile usage would just go crazy (not great for the life of the flash drive I have plugged in). The optimized version reduces the amount of blocks and size to roughly a third of the vanilla full list. It does some tricks which may eventually lead to a false positive or two in theory, but so far I haven't had any problems with it.
I already tried the Developer Dan's list posted above but I had to whitelist various domains right out of the box. Otherwise that would have been perfect.
Welcome to the forum.I am new user of Diversion, and not able to update the Block list type, any suggestions how to do it? View attachment 40568
Even bigger list of something for everyone, in more formats than you can 'shake a stick at' !!!Something for everyone here:
Developer Dan's Hosts
Hosts formatted blocklist to block various things like ads and internet trackers.www.github.developerdan.com
‘Filterlists’ is the mother lode for sure!Even bigger list of something for everyone, in more formats than you can 'shake a stick at' !!!
filterlists.com
Question for thelonelycoder:
Currently the format of the wc_blacklist in diversion is simply <domain> .... i.e. bbc.co.uk
Would it be possible to accept files in the <*.bbc.co.uk> format which is a common format for wildcard domains.
Diversion would simply have to strip off the leading <*.> .
This would enable existing wildcard format files to be used directly without any 'File mangling'
[I know that I can 'filter' the file myself but it would enable the url to be used directly within Diversion ]
Example space saving if using wildcard filterlists:
[Full text version] oisd_hosts_full.txt = 40.3 MB
[Wildcard version] oisd_dblw_full.txt = 11.4 MB
[Full text version] oisd_hosts_full.txt = 40.3 MB
[Wildcard version] oisd_dblw_full.txt = 11.4 MB
I can live with the risk of more 'false positives' as the exceptions can be quite easily handled.While I see and like your arguement for this,
oisd_dblw_full.tx= more false positive potential, but smaller size list.
oisd_hosts_full.txt= albiet much bigger, means blocking everything directly.
A fair compromise-
while I get your point, one issue present is Diversion has a limited allowed whitelisting, 1000 domains. In my case of using wildcards and filters, my whitelist extends to 50,000 domains easy. While you may find it rectifiable in your current state of setup, I find it hard to believe every user will fall in this bracket. That is why I strongly suggest against such.I can live with the risk of more 'false positives' as the exceptions can be quite easily handled.
The main plus for me was the memory saving as this gives me more room to play with.
I am an inveterate tinkerer and keep changing configurations, filterlists etc etc
The change to the format accepted by wc_blacklist would enable direct usage of urls and Diversion would do all the 'grunt work' automagically !!!
Welcome To SNBForums
SNBForums is a community for anyone who wants to learn about or discuss the latest in wireless routers, network storage and the ins and outs of building and maintaining a small network.
If you'd like to post a question, simply register and have at it!
While you're at it, please check out SmallNetBuilder for product reviews and our famous Router Charts, Ranker and plenty more!