AB-Solution - The Ad Blocking Solution

[seattle]

It has been on my Asus router for more than 6 months. I want to block certain device at 2nd router. How to do that?
Thanks,

If your RT-AC66U runs on Asuswrt-Merlin then there is nothing stopping you from installing AB-Solution on it.
Consult the FAQ on how to exclude clients from ad-blocking here:
https://www.ab-solution.info/use/faq.html

 

[M@rco]

@DonnyJohnny: Seems to work fine here:

 running update-hosts.add 3.11 in verbose mode
 using temporary pgl.yoyo.org file to lower memory usage while updating
 preparing temporay whitelist
 updated /jffs/shared-AB-whitelist, for other scripts to use
 refreshing Skynet to whitelist domains in shared-AB-whitelist
 including /jffs/shared-Skynet-whitelist in AB-Solution
 including /jffs/shared-Skynet2-whitelist in AB-Solution
 downloading Custom+ hosts files, using 192.168.1.2 as blocking IP

 getting hosts file 1, preprocessing file:
 https://raw.githubusercontent.com/hoshsadiq/adblock-nocoin-list/master/hosts.tx                                                                                                             t
-                   100%[===================>]   1.37K  --.-KB/s    in 0s
 processing file 1 done

 file preprocessing took 14 seconds
 sorting, removing whitelist entries
 this will take about 10 seconds...
 your RT-AC68U did this in 0 seconds
 updating blocking file took 14 seconds

 

[thelonelycoder]

It has been on my Asus router for more than 6 months. I want to block certain device at 2nd router. How to do that?
Thanks,

Wait. What is it exactly you wat to do? AB-Solution is an ad-blocker, it does not prevent devices from accessing the web.
If you want to exclude a certain device from ad-blocking, then the FAQ article is how to do it. It does need to have an IP that is assigned by the Asus router. If it has an an I p assigned by the downstteam router it cannot be differentiated by the Asus router as it likely only sees the IP of the downstream router.

 

[DonnyJohnny]

@DonnyJohnny: Seems to work fine here:

 running update-hosts.add 3.11 in verbose mode
 using temporary pgl.yoyo.org file to lower memory usage while updating
 preparing temporay whitelist
 updated /jffs/shared-AB-whitelist, for other scripts to use
 refreshing Skynet to whitelist domains in shared-AB-whitelist
 including /jffs/shared-Skynet-whitelist in AB-Solution
 including /jffs/shared-Skynet2-whitelist in AB-Solution
 downloading Custom+ hosts files, using 192.168.1.2 as blocking IP

 getting hosts file 1, preprocessing file:
 https://raw.githubusercontent.com/hoshsadiq/adblock-nocoin-list/master/hosts.tx                                                                                                             t
-                   100%[===================>]   1.37K  --.-KB/s    in 0s
 processing file 1 done

 file preprocessing took 14 seconds
 sorting, removing whitelist entries
 this will take about 10 seconds...
 your RT-AC68U did this in 0 seconds
 updating blocking file took 14 seconds

I deleted the post.. I realised that Skynet blocking 8.8.8.8
Lol.. unable to resolve the domain... I used firehol level 1 and it will cause this problem.., whitelisted and now it is good.

 

[M@rco]

Ehm, maybe a stupid question, but I can't find the answer on the website (in the manual and in the info regarding host files). I'm using Medium+ currently. If I want to use Medium+ and add the nocoin list mentioned above, should I copy all the hosts files urls which combined are the Medium list, as listed on the website and the url to the nocoin hosts file into the custom hosts file in /adblocking/custom_files/custom_hosts_list.txt? I just tried Custom for @DonnyJohnny to see if could use the nocoin list, but noticed that the other lists in Medium(+) weren't used anymore, so I assume I have to assemble my own list then? And run 'u' afterwards to update all the lists in the custom_hosts_list.txt?

 

[DonnyJohnny]

Ehm, maybe a stupid question, but I can find the answer on the website (in the manual and in the info regarding host files). I'm using Medium+ currently. If I want to use Medium+ and add the nocoin list mentioned above, should I copy all the hosts files in Medium as listed on the website and the url to the nocoin hosts file into the custom hosts file in /adblocking/custom_files/custom_hosts_list.txt? I just tried Custom for @DonnyJohnny to see if could use the nocoin list, but noticed that the other lists in Medium(+) weren't used anymore, so I assume I have to assemble my own list then? And run 'u' afterwards to update all the lists in the custom_hosts_list.txt?

Don’t just press and press thru the option.. read them... haha..

When you choose use custom list, one of the option asked whether u want to use empty file or together with the existing option... choose the existing option 1-6/7. Then go to the custom list.txt and add in any addition host list u want to add in...

I would recommend also add in
https://github.com/StevenBlack/hosts
I used Unified hosts = (adware + malware)
https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts

 

[shooter40sw]

Some are and have been included in for a while in several of the used hosts files. Whichever blocking file type you use the coin-hive.com and more are already blocked.
For a more comprehensive list you could add this actively maintained hosts file to it:
https://github.com/hoshsadiq/adblock-nocoin-list.
Use the raw link to the file:
https://raw.githubusercontent.com/hoshsadiq/adblock-nocoin-list/master/hosts.txt

I officially inform you that the hosts file type named after you will be gone in AB-Solution 4.0, along with the AdsBeGone! name.
The name disappears but the memory lingers on .
Hosts files are handled differently in AB4, hence names are no longer of significance.

Thanks for the info, I got a R66U, that probably will have to go to johns fork for support and update because merlin will not support this router anymore, will ab-solution 4 work on johns fork?
Thanks

 

[M@rco]

Don’t just press and press thru the option.. read them... haha..

When you choose use custom list, one of the option asked whether u want to use empty file or together with the existing option... choose the existing option 1-6/7. Then go to the custom list.txt and add in any addition host list u want to add in...

I would recommend also add in
https://github.com/StevenBlack/hosts
I used Unified hosts = (adware + malware)
https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts

I think I missed that option, as I created the custom hosts list beforehand through ssh and only got the question whether I want to keep using that list

I'll walk through the steps again, thanks for the additional hosts files.

 

[DonnyJohnny]

I don’t know what this crypto mining attack is.. but I think firehol has a bunch of list regarding coin nodes...

you can use Skynet to add all these additional ipset
http://iplists.firehol.org/?ipset=coinbl_hosts
Use local COPY for the URL
You can use pastebin to create your own custom list. Then use the raw link for skynet

 

[M@rco]

I don’t know what this crypto mining attack is..

Browser-based mining, as its name suggests, is a method of cryptocurrency mining that happens inside a browser and is implemented using scripting language. This is different compared to the more widely known file-based cryptocurrency mining approach which involves downloading and running a dedicated executable file.

Source: Symantec Blog

 

[M@rco]

I would recommend also add in
https://github.com/StevenBlack/hosts
I used Unified hosts = (adware + malware)
https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts

Not sure if you're only using a custom list (none of the predefined lists, I mean), but if I'm not mistaken, this hosts file from Steven Black you mentioned is in option 4 (AdsBeGone!) as well as option 6 (AB-Maximum) according to the AB-Solution website. If you're using either of those, adding it as a custom list seems somewhat redundant. Not sure wether AB-Solution checks for duplicate hosts files, I believe it does check for duplicate hosts when processing an update.

 

[DonnyJohnny]

Oh ya, you are right.. ab does check for redundancy. Personally I removed those individual list like yoyo, mpvs malware domain and add in stevenblack coz it is already pre-populate. Coz seems more efficient and faster in getting the ad block up

 

[thelonelycoder]

(...) will ab-solution 4 work on johns fork?

Sure, AB4 runs as good on 374.43 LTS as on 380.69, 382.2 and 384.3 alpha2. I have no elusive RT-AC86U to test it on but I'm sure it'll work just as well once AB4 gets to see more than just the four routers to my right

 

[thelonelycoder]

Oh ya, you are right.. ab does check for redundancy. Personally I removed those individual list like yoyo, mpvs malware domain and add in stevenblack coz it is already pre-populate. Coz seems more efficient and faster in getting the ad block up

I don't check for hosts files doubles in AB3.x but if you happen to use stevenblacks and, say the malwaredomain hosts file the latter gets eliminated by the processor intensive sort by uniques because it is part of Stevens list. AB4 is handling that much better.

 

[thelonelycoder]

Not sure if you're only using a custom list (none of the predefined lists, I mean), but if I'm not mistaken, this hosts file from Steven Black you mentioned is in option 4 (AdsBeGone!) as well as option 6 (AB-Maximum) according to the AB-Solution website. If you're using either of those, adding it as a custom list seems somewhat redundant. Not sure wether AB-Solution checks for duplicate hosts files, I believe it does check for duplicate hosts when processing an update.

The predefined blocking files are assembled so that none of the lists is doubly used. When using one of the blocking files that includes the stevenblack list then several original hosts files are not separately downloaded.

 

[M@rco]

The predefined blocking files are assembled so that none of the lists is doubly used. When using one of the blocking files that includes the stevenblack list then several original hosts files are not separately downloaded.

I understand what you're saying. What I meant was the scenario where one uses a predefined list in AB-solution, switches to Custom list with the predefined list included and then adds another url to a hosts list which has an overlap with the hosts files in the predefined list. For example, I just selected AdsBeGone!, switched to custom with AdsBeGone! included and added the largests Steven Black hosts list on Github. When updating, it looks like all hosts files are being downloaded anyway. Does that cause lots of duplicates or does AB-Solution have some de-dupe functionality to limit the amount of lines/hosts in the accumulated list?

 

[thelonelycoder]

I understand what you're saying. What I meant was the scenario where one uses a predefined list in AB-solution, switches to Custom list with the predefined list included and then adds another url to a hosts list which has an overlap with the hosts files in the predefined list. For example, I just selected AdsBeGone!, switched to custom with AdsBeGone! included and added the largests Steven Black hosts list on Github. When updating, it looks like all hosts files are being downloaded anyway. Does that cause lots of duplicates or does AB-Solution have some de-dupe functionality to limit the amount of lines/hosts in the accumulated list?

I meant the same, more or less.
But a check if a downloaded hosts file is in itself an amalgamated list that contains other hosts files to download is in all likelyhood impossible to determine from the viewpoint of a shell script.
It is done the hard way by removing duplicate on the combined lists.

I believe this is a job for the human that assembles such a list to check.
I can only look at the URL and eliminate a duplicate. That's what I do in AB4 now.

 

[M@rco]

Sorry, not sure if I understand you correctly. Do you mean AB-solution does check for duplicates in blocking_file? (sorry in advance if I didn't understand it correctly, but my mind is fuzzy and I have trouble thinking straight...)

I played around a bit and as mentioned above, I purposely added a hosts file url to the custom_list to see what happened to duplicates. This is what the custom list looks like now:

marco@RT-AC68U:/tmp/mnt/USB/adblocking/custom_files# cat custom_hosts_list.txt
# Add direct dowload link to hosts file, one per line.
# Blocking IP must be 127.0.0.1 or 0.0.0.0
https://hosts-file.net/ad_servers.txt
https://hosts-file.net/grm.txt
https://adblock.mahakala.is
https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
https://hosts-file.net/emd.txt
https://hosts-file.net/exp.txt
https://hosts-file.net/hjk.txt
https://hosts-file.net/mmt.txt
https://hosts-file.net/psh.txt
https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/fakenews-gambling-porn-social/hosts
https://raw.githubusercontent.com/hoshsadiq/adblock-nocoin-list/master/hosts.txt

Both the Unified hosts list as well as the most extensive list by Steven Black are downloaded along with the other files.

But when I run

sort blocking_file | uniq -u

absolutely no duplicates are found, so it does seem to delete duplicate hosts in the blocking file?

Edit: this is what confuses me (whether it does or not):

It is done the hard way by removing duplicate on the combined lists.

I believe this is a job for the human that assembles such a list to check.
I can only look at the URL and eliminate a duplicate.

Edit 2: If I'm asking too much or if it annoys you that I don't understand it, just skip the post, it's not a big deal anyway. Looking forward to AB4 though.

 

[thelonelycoder]

I played around a bit and as mentioned above, I purposely added a hosts file url to the custom_list to see what happened to duplicates. This is what the custom list looks like now:

Both the Unified hosts list as well as the most extensive list by Steven Black are downloaded along with the other files.

That is what I mean with the router sorting it out the hard way. You purposely added hosts files that contain the same lists.
The sort -u in update-hosts.add then removes duplicates as you've noticed.

I can check that the same URL is not accidentally twice added to download. But I cannot check if
https://raw.githubusercontent.com/S...lternates/fakenews-gambling-porn-social/hosts
and
https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
already contain the someonewhocares.org hosts. Both do, so the sort -u has to sort it out.
This has to be the job of the user, do a little research what hosts files contain and add them in an informed way, not blindly adding hosts files.

But then, I don't care, the blocking file update just takes a lot longer and much more memory is used during the process. The user ends up with a blocking file that stripped all of what was added from some hosts files, down to unique host names.

 

[Xentrk]

Hey lonelycoder your AB-Solution is awesome! Just got it working on my RT-AC86U.

I had a few minor issues but was able to work them out thanks to your VERY through website and Xentrk's Torguard instructions.

Issue #1: I couldn't get the ad-blocker to work on clients going through my two VPNs (PIA & NordVPN). Xentrk's instructions explained how to set the OpenVPN Client's Accept VPN Configuration to Strict instead of Exclusive, that fixed the issue right away. My only concern is, as I understand it, Strict puts your VPN's DNS servers before the DNS setup for the WAN connections; whereas Exclusive puts your VPN's DNS servers as a replacement for the DNS setup for the WAN connections. There is a possibility (though highly unlikely) that the router wouldn't be able to connect to the VPN's DNS servers and use the DNS for the WAN connection while set for Strict and there could be a DNS leak. I'm planning on dropping NordVPN soon and I will set my WAN DNS servers to point to PIA's.

https://www.snbforums.com/threads/t...or-asus-merlin-380-65-380-65_2-part-ii.38282/

Issue #2: I had a problem setting up the sendmail feature until I remembered that I had two-factor authentication setup on my email. I just had to create an app password specifically for the router sendmail then it worked great!

It's an incredible amount of work to create, update and maintain a project like this. I suggest anyone using AB-Solution send a donation through www.ab-solution.info. I just did!

Thanks for all your hard work!

The scripts I will be sending you will allow you to set Accept DNS Configuration to Exclusive and have ABS work over the VPN tunnel. Please use the @ symbol in front of a members name so they get a notification. I am forking the terminal menu and will add menu options to automate some of the selective routing functions to make it more user friendly. I will have more free time starting tomorrow. Eventually, I will post on GitHub and provide a wiki on how to install and setup.