AB-Solution - The Ad Blocking Solution

[RMerlin]

Without a device to test it on myself, I'm not going to change any of the AB-Solution code. It would be guesswork with no end, even if only a small part of the thousands of lines of code need adjustment.

And things may still change, even tho I'm nearing the end of the alpha stage. Broadcom/Asus have caused me a few headaches because Broadcom now uses /opt to store some scripts in the flash, so /opt is no longer a symlink to /tmp/opt, but an actual folder (with a bunch of symlinks inside it pointing to the various /tmp/var/* folders). I had to add a bunch of them to fix Entware. I'm not totally dismissing yet the idea of just scrapping Broadcom's /opt/scripts folder, and reverting to a more natural /opt setup.

So I'd say don't invest too much time implementing solutions yet (exploratory work however is fine).

BTW, I also considered moving HND to Entware-NG-3x, which is more up-to-date and more optimized for newer kernels. Unfortunately, its maintainer decided on a few things that are deal-breakers for me, especially as it forces the installation of Busybox AND makes /opt/bin have priority over /bin, overriding any of the Asuswrt-Merlin own applet. This means any changes Asus and Broadcom made to busybox are then lost, potentially breaking some of the router's core functionalities. The maintainer didn't seem open into changing this, so I stuck with Entware-NG, and focused on getting it to work on HND.

 

[jimf]

A little off the topic. I'm trying to stream programs from NBC and it's complaining about adblocking. I tailed the logfile and updated the whitelist for domains shown as going to pixelserv. The website still complains. Tailing the logfile again showed different (or slightly different) domains. So it would appear this is a dynamic situation to say the least. Has anyone sucessfully whitelisted nbc.com so shows play? Turning off absolution completely fixes it but I'd rather do a whitelist.

Alternatively, is there a way to selectively exclude LAN IPs from participating in absolution?

 

[thelonelycoder]

Alternatively, is there a way to selectively exclude LAN IPs from participating in absolution?

https://www.ab-solution.info/faq-reader/how-to-exclude-a-client-from-ad-blocking.html

 

[thelonelycoder]

Broadcom/Asus have caused me a few headaches because Broadcom now uses /opt to store some scripts in the flash, so /opt is no longer a symlink to /tmp/opt, but an actual folder (with a bunch of symlinks inside it pointing to the various /tmp/var/* folders). I had to add a bunch of them to fix Entware.

And this is the reason why these brave early testers cannot install pixelserv-tls.

BTW, I also considered moving HND to Entware-NG-3x, which is more up-to-date and more optimized for newer kernels.

HND?
That seems to make things much more complicated for the average Asuswrt-Merlin user, from https://github.com/Entware-for-kernel-3x/Entware-ng-3x:

The main differences from the original project are

• 3.x kernels are used to build toolchain. 3.4.112 is used for mips(el) and 3.2.40 is used for arm and intel;
• glibc is now a system library for all architectures including mips(el);
• glibc has a patch that allows to use separate (from the original firmware) users and passwords;
• two different installations are possible for most devices: (1) standard, (2) alternative (with serarated from firmware users);
• busybox from Entware is forced instolled;

Glibc patch above moves files /et(c)/passwd, /et(c)/shaddow, /et(c)/group, /et(c)/gshadow, /et(c)/shells, /et(c)/localtime to /opt/et(c) folder. For most devices this allows two different instalation type.

We call standard installation - the installation where /opt/etc folder has symlinks to /et(c) files (passwd etc.).

 

[thelonelycoder]

Since Entware cannot be installed through AB-Solution in the new Asuswrt-Merlin 382.xx (currently only runs on the new RT-AC86U), I have removed the support in the addon pixelserv-tls.add.
The standard AB-Solution installation is still possible, only the automated install of Entware and pixelserv-tls is disabled.

In return and for those testing @kvic's latest pixelserv-tls KL-test1 version, the -O option has been added to the available switches in the ps menu.
There is no version change, enter 12 into the AB-Solution UI to re-download all addons.

 

[.TT.]

AB is up and running on my ac86u.
Thanks for all the help!! =)

 

[thelonelycoder]

AB is up and running on my ac86u.
Thanks for all the help!! =)

I wish I could do more but without a device to tinker I'll leave it at that.
A standard AB installation (without pixelserv-tls) is far better than no ad-blocking at all.

 

[eclp]

I have another question about possible duplicate entries in the syslog. The following is an excerpt after the router restart via AB-Solution entries. Is this behavior normal?

Oct 12 02:38:07 87u: AB-Solution added entries via ab_dnsmasq_postconf.sh
Oct 12 02:38:08 87u: AB-Solution linked ab_dnsmasq_postconf.sh via /jffs/scripts/dnsmasq.postconf
Oct 12 02:38:08 87u: AB-Solution created br0:pixelserv 192.168.1.2 via /jffs/scripts/wan-start
Oct 12 02:38:08 87u: dnscrypt-proxy started for boot services
Oct 12 02:38:09 87u: AB-Solution added entries via /jffs/scripts/post-mount
Oct 12 02:38:09 87u: AB-Solution started rc.unslung via /jffs/scripts/services-start
Oct 12 02:38:09 87u: AB-Solution added entries via ab_dnsmasq_postconf.sh
Oct 12 02:38:09 87u: AB-Solution linked ab_dnsmasq_postconf.sh via /jffs/scripts/dnsmasq.postconf
Oct 12 02:38:12 87u: Started ntpd from /jffs/scripts/services-start.
Oct 12 02:38:13 87u: Started pixelserv-tls (AB-Solution) from /jffs/scripts/services-start.
Oct 12 02:38:14 87u: Adaptive QOS: Modification Script Started
Oct 12 02:38:18 kernel: gro disabled
Oct 12 02:38:18 kernel: gro enabled with interval 2
Oct 12 02:38:21 87u: AB-Solution added entries via ab_dnsmasq_postconf.sh
Oct 12 02:38:21 87u: AB-Solution linked ab_dnsmasq_postconf.sh via /jffs/scripts/dnsmasq.postconf
Oct 12 02:40:05 kernel: * Make sure sizeof(struct sw_struct)=160 is consistent
Oct 12 02:40:07 87u: Start dnscrypt-proxy for normal operations
Oct 12 02:40:08 kernel: sizeof forward param = 160
Oct 12 02:40:21 87u: Adaptive QOS: No change required for Unidentified Traffic Container or Custom Rules
Oct 12 02:40:22 87u: Adaptive QOS: Changing minimum alloted bandwidth per QOS category to user defined percentages
Oct 12 05:20:05 87u: AB-Solution counted ads and rotated log files (daily cron job)
Oct 12 05:20:05 87u: AB-Solution blocked 100,860 total 11,400 week 8,100 new ads

 

[thelonelycoder]

I have another question about possible duplicate entries in the syslog. The following is an excerpt after the router restart via AB-Solution entries. Is this behavior normal?

Oct 12 02:38:07 87u: AB-Solution added entries via ab_dnsmasq_postconf.sh
Oct 12 02:38:08 87u: AB-Solution linked ab_dnsmasq_postconf.sh via /jffs/scripts/dnsmasq.postconf
Oct 12 02:38:08 87u: AB-Solution created br0:pixelserv 192.168.1.2 via /jffs/scripts/wan-start
Oct 12 02:38:08 87u: dnscrypt-proxy started for boot services
Oct 12 02:38:09 87u: AB-Solution added entries via /jffs/scripts/post-mount
Oct 12 02:38:09 87u: AB-Solution started rc.unslung via /jffs/scripts/services-start
Oct 12 02:38:09 87u: AB-Solution added entries via ab_dnsmasq_postconf.sh
Oct 12 02:38:09 87u: AB-Solution linked ab_dnsmasq_postconf.sh via /jffs/scripts/dnsmasq.postconf
Oct 12 02:38:12 87u: Started ntpd from /jffs/scripts/services-start.
Oct 12 02:38:13 87u: Started pixelserv-tls (AB-Solution) from /jffs/scripts/services-start.
Oct 12 02:38:14 87u: Adaptive QOS: Modification Script Started
Oct 12 02:38:18 kernel: gro disabled
Oct 12 02:38:18 kernel: gro enabled with interval 2
Oct 12 02:38:21 87u: AB-Solution added entries via ab_dnsmasq_postconf.sh
Oct 12 02:38:21 87u: AB-Solution linked ab_dnsmasq_postconf.sh via /jffs/scripts/dnsmasq.postconf
Oct 12 02:40:05 kernel: * Make sure sizeof(struct sw_struct)=160 is consistent
Oct 12 02:40:07 87u: Start dnscrypt-proxy for normal operations
Oct 12 02:40:08 kernel: sizeof forward param = 160
Oct 12 02:40:21 87u: Adaptive QOS: No change required for Unidentified Traffic Container or Custom Rules
Oct 12 02:40:22 87u: Adaptive QOS: Changing minimum alloted bandwidth per QOS category to user defined percentages
Oct 12 05:20:05 87u: AB-Solution counted ads and rotated log files (daily cron job)
Oct 12 05:20:05 87u: AB-Solution blocked 100,860 total 11,400 week 8,100 new ads

The two entries at 5:20 is the rotate-logs.add addon doing its scheduled daily job.

The 2:38 entries were probably triggered by QOS or dnscrypt restarting dnsmasq, possibly twice.
Restarting dnsmasq runs /jffs/scripts/dnsmasq.postconf and with it the linked file for AB.
From my (AB-Solutions) point this is all normal and has nothing directly to do with what AB does.

 

[eclp]

Thank you for your enlightening answer, a quick and great support here!

 

[2992]

Can AB-Solution be configured to run on the TUN interface (as well)?
I need it mostly for to block the ads over the OpenVPN connection.
Best would be to run on all interfaces and block the ads/trackers on every each internet connection.
Thank you!

 

[Xentrk]

Can AB-Solution be configured to run on the TUN interface (as well)?
I need it mostly for to block the ads over the OpenVPN connection.
Best would be to run on all interfaces and block the ads/trackers on every each internet connection.
Thank you!

OpenVPN Client or Server?

There are some settings on the client side that can cause ABS not to work over the VPN tunnel.

Are you using Policy Rules, All Traffic, None? What is your setting for Accept DNS Configuration?

 

[2992]

OpenVPN Client or Server?

There are some settings on the client side that can cause ABS not to work over the VPN tunnel.

Are you using Policy Rules, All Traffic, None? What is your setting for Accept DNS Configuration?

OpenVPN Client.
I am now considering switching from Tomato to XWRT (I have a R7000), but before doing it, I need to understand what's possible with ABS and what's not.
Now, Tomato can do adblocking, but not over the TUN VPN client, therefore, I need to understand whether ABS can block the ads over the TUN interface. (The plan is to have some IP clients routed to connect using the VPN TUN client, and some other not over TUN but directly to WAN/internet)
I am open to receive configuration recommendation for the VPN client for to make ABS working over TUN - as long as the TUN can stay up and route the traffic and get no DNS leak from ABS or something else..

 

[Xentrk]

OpenVPN Client.
I am now considering switching from Tomato to XWRT (I have a R7000), but before doing it, I need to understand what's possible with ABS and what's not.
Now, Tomato can do adblocking, but not over the TUN VPN client, therefore, I need to understand whether ABS can block the ads over the TUN interface. (The plan is to have some IP clients routed to connect using the VPN TUN client, and some other not over TUN but directly to WAN/internet)
I am open to receive configuration recommendation for the VPN client for to make ABS working over TUN - as long as the TUN can stay up and route the traffic and get no DNS leak from ABS or something else..

You don't need to specify interface to get ABS working over the vpn. The second link will have the settings needed to make ABS work over VPN tunnel.

https://www.snbforums.com/threads/t...for-asus-merlin-380-65-380-65_2-part-i.38281/

https://www.snbforums.com/threads/t...or-asus-merlin-380-65-380-65_2-part-ii.38282/

https://www.snbforums.com/threads/t...r-asus-merlin-380-65-380-65_2-part-iii.38283/

I have since found some other settings when combined using scripts to perform selective routing that allows some mods to these settings. See if the suggestions help and let me know.

 

[2992]

You don't need to specify interface to get ABS working over the vpn. The second link will have the settings needed to make ABS work over VPN tunnel.

https://www.snbforums.com/threads/t...for-asus-merlin-380-65-380-65_2-part-i.38281/

https://www.snbforums.com/threads/t...or-asus-merlin-380-65-380-65_2-part-ii.38282/

https://www.snbforums.com/threads/t...r-asus-merlin-380-65-380-65_2-part-iii.38283/

I have since found some other settings when combined using scripts to perform selective routing that allows some mods to these settings. See if the suggestions help and let me know.

Thank you, that looks promising!
But, isn't the Accept DNS Configuration set to "Strict" going to be a risk for DNS Leak? I've been reading about that here:
http://www.linksysinfo.org/index.php?threads/using-adblock-script-vpn-client-tunneling.72110/

 

[thelonelycoder]

You don't need to specify interface to get ABS working over the vpn. The second link will have the settings needed to make ABS work over VPN tunnel.

https://www.snbforums.com/threads/t...for-asus-merlin-380-65-380-65_2-part-i.38281/

https://www.snbforums.com/threads/t...or-asus-merlin-380-65-380-65_2-part-ii.38282/

https://www.snbforums.com/threads/t...r-asus-merlin-380-65-380-65_2-part-iii.38283/

I have since found some other settings when combined using scripts to perform selective routing that allows some mods to these settings. See if the suggestions help and let me know.

Should I add this to the AB-Solution FAQ?

 

[2992]

Should I add this to the AB-Solution FAQ?

I, as a regular user (noob), would say: definitely yes! It's a great guide!

 

[Xentrk]

Thank you, that looks promising!
But, isn't the Accept DNS Configuration set to "Strict" going to be a risk for DNS Leak? I've been reading about that here:
http://www.linksysinfo.org/index.php?threads/using-adblock-script-vpn-client-tunneling.72110/

Yes, the DNS will leak. I need to add that caveat to the guide on the next update. from the research I did, DNS leak with policy rules appears to be a common issue not only with Asuswrt-Merlin, but with other firmware as well. We had a discussion about this recently as to why this occurs. The relevant links are below.

Spoiler: DNS Leak

https://www.snbforums.com/threads/a...ing-solution-v3-9-1.37511/page-67#post-344423

https://www.snbforums.com/threads/a...ing-solution-v3-9-1.37511/page-67#post-344472

https://www.snbforums.com/threads/a...ing-solution-v3-9-1.37511/page-67#post-344498

I recently experimented with using ipset, fwmarks and iptables to perform selective routing rather than using the web GUI menu to do it. I find I can then set Accept DNS Configuration to exclusive and have ABS work over the VPN tunnel. But DNS can still leak!

I then found a hack. If I change from Policy Rules to No Traffic, my script will still route traffic to the tunnel and the DNS will not leak. However, being able to route traffic to the vpn client when No Traffic setting is turned on has not worked 100% of the time. I think it is a combination of settings and the number of clients in use.

How many vpn clients are you running?

 

[Xentrk]

Should I add this to the AB-Solution FAQ?

Probably a good idea as I have seen the question/issue come up several times recently.

 

[2992]

Yes, the DNS will leak. I need to add that caveat to the guide on the next update. from the research I did, DNS leak with policy rules appears to be a common issue not only with Asuswrt-Merlin, but with other firmware as well. We had a discussion about this recently as to why this occurs. The relevant links are below.

Spoiler: DNS Leak

https://www.snbforums.com/threads/a...ing-solution-v3-9-1.37511/page-67#post-344423

https://www.snbforums.com/threads/a...ing-solution-v3-9-1.37511/page-67#post-344472

https://www.snbforums.com/threads/a...ing-solution-v3-9-1.37511/page-67#post-344498

I recently experimented with using ipset, fwmarks and iptables to perform selective routing rather than using the web GUI menu to do it. I find I can then set Accept DNS Configuration to exclusive and have ABS work over the VPN tunnel. But DNS can still leak!

I then found a hack. If I change from Policy Rules to No Traffic, my script will still route traffic to the tunnel and the DNS will not leak. However, being able to route traffic to the vpn client when No Traffic setting is turned on has not worked 100% of the time. I think it is a combination of settings and the number of clients in use.

How many vpn clients are you running?

Just one VPN Client (on this router with this firmware: I now run Tomato Shibby on it, and this Firmware can support 2 VPN Clients to be configured, but only one can run at a time. Now I have on the OpenVPN Client configured the option "Accept DNS configuration" to "Disabled", and the DNS doesn't leak. As there is no Kill Switch on Tomato for VPN, I am doing that by blocking all the interfaces except TUN on the firewall.)

I am willing to try XWRT-Vortex + ABSolution on this R7000 as I would like to have HTTP&HTTPS ad-blocking on the TUN IF (Tomato doesn't support it), QOS on the TUN (Tomato seemingly doesn't support it on the TUN IF, but Lede does - I have Lede on another router), and Policy Rules (or something similar for to have some IP/Clients over VPN and some others routed to WAN/Internet - i.e. Netflix blocks my VPN provider DNS and I think it can detect the location as well and blocks my VPN's IP - such selective routing is possible with Lede, but not with Tomato or at least I do not know how to do it, unfortunately Lede doesn't work on my R7000), but as I have limited time to fiddle around and change firmwares back and forward, I am firstly trying to understand the possibilities and the options of XWRT/Merlin FW, before doing the switch and investing time in having a working configuration on the XWRT.