AB-Solution - The Ad Blocking Solution

[thelonelycoder]

Has anyone compiled a whitelist of domains that may be useful to people? e.g. popular gaming domains for PS3/4, Xbox etc etc?

Example: when my son says why is my EA sports game suddenly taking 5 minutes to load, then I will need to start search for domains to whitelist - Steam is another one etc etc

I can find blacklists, but a good whitelist would be infinitely useful as well.

thanks in advance
Rimmel

This "service type whitelist" you mention has come up in the last beta thread and will be part of the next major update.
The idea is to make these lists available to selectively add and remove, separately from the standard whitelist.

As such lists needs careful selection and curating of the whitelisted domains, they need some planning and assembling before I can actually implement it into AB.
I will start the first part as soon as time permits (and will need the help of volunteers).

 

[thelonelycoder]

Just move everything outside of the US. The DMCA is an American law.

The problem is that too much of the internet is controlled by american companies/organizations/interest groups/lobbyists and the US government.

 

[Rimmel]

This "service type whitelist" you mention has come up in the last beta thread and will be part of the next major update.
The idea is to make these lists available to selectively add and remove, separately from the standard whitelist.

As such lists needs careful selection and curating of the whitelisted domains, they need some planning and assembling before I can actually implement it into AB.
I will start the first part as soon as time permits (and will need the help of volunteers).

More than an actually official type whitelist I was thinking more of just a collection of well known whitelist domains in a document that people can use to unblock useful sites.

PS an hour ago I emailed you on your site - can AB-solution be used on a secondary router e,g not the main internet connected router?

 

[Makaveli]

The problem is that too much of the internet is controlled by american companies/organizations/interest groups/lobbyists and the US government.

Agreed but the Internet was created in America so not totally surprising.

 

[thelonelycoder]

PS an hour ago I emailed you on your site - can AB-solution be used on a secondary router e,g not the main internet connected router?

I'd rather discuss these things here so they can be seen by others.

I don't quite understand what you want to do from your text in the email.

Hi I already use pixelserv on a linksys device. It is not my main router though e.g. my main router is connected to the internet but my pixelserv router is static on the network and connects to the internet through the main router (on same network), I used it as a dedicated ad blocker. Can absolution be used in the same way?

Do you mean to replace the Linksys router with an Asus router and then run AB with pixelserv on that?
If so, just set your clients DNS Server to the router with AB installed on.

 

[Pir8pete]

having an issue using the Hotmail/Outlook notify feature !!!

 

[mstombs]

In Firefox, I have this old addon "Google search link fix" that removes the affiliate links from Google pages.
This will likely be obsolete with one of the coming Firefox updates where only Web extensions addons are allowed.
On iOS I simply scroll down a little further to the direct link, if I ever have to do that.
Not much I or the dev's of pixelserv-tls can do about it.

Since we all want secure and untraceable web usage, and we should, this is more and more enforced by the browsers to only trust trusted certificates.
Stupidity by users led the browser developers to remove the simple ability to add self-signed certificates to the trusted chain.
Even we power users are left out with this good trend.

What needs to be possible is to have a valid certificate for non-web accessible resources such as the pixelserv-tls and a router WebUI over https.
If ever, this will be solved in another life or is available right now in a parallel Universe nearby. Or some sleight of hand by kvic or @mstombs in this life.

I have given up on trying to do it with my limited understanding and knowledge to get it to work in a way that is reasonably secure and doable.
I'm calling on my parallel Universe thelonelycoder in his free flowing work lab to help us out and send a fix through the nearest wormhole.

Bit late to the party, but pixelserv includes code to strip urls and encourage the browser to go straight to the target url, not via the referrer. It was never 100% and probably now needs maintenance, but the option is on by default, counted as "rdr":-

pixelserv-tls version: v35.HZ12.Kj compiled: May 31 2017 04:30:23 options: 192.168.66.254 -p 80 -p 81 -p 8080 -p 8081 -k 443 -o 2

uts    66d 23:52    pixelserv uptime
log    0    logging access to syslog (0=disabled 1=enabled)
req    951739    total # of requests (HTTP, HTTPS, success, failure etc)
avg    1115 bytes    average length of request URL
rmx    9291 bytes    maximum length of request URL
tav    91 ms    average processing time (per request)
tmx    2748 ms    maximum processing time (per request)
slh    935259    # of accepted HTTPS requests
slm    201    # of rejected HTTPS requests (missing certificate)
sle    0    # of rejected HTTPS requests (certificate available but bad)
slu    1164    # of dropped HTTPS requests (unknown error)
nfe    2028    # of GET requests for server-side scripting
gif    70    # of GET requests for GIF
ico    3    # of GET requests for ICO
txt    2937    # of GET requests for Javascripts
jpg    0    # of GET requests for JPG
png    1    # of GET requests for PNG
swf    0    # of GET requests for SWF
sta    5    # of GET requests for HTML stats
stt    1    # of GET requests for plain text stats
ufe    432    # of GET requests /w unknown file extension
rdr    2350    # of GET requests resulted in REDIRECT response
...

The redirect code is here if anyone interested:-
https://github.com/kvic-z/pixelserv-tls/blob/master/socket_handler.c#L714

 

[thelonelycoder]

having an issue using the Hotmail/Outlook notify feature !!!

To use the Hotmail address, you will need to create an app password in the account settings and use it as password.
Then, to make it work edit /mnt/<your device>/adblocking/scripts/send-email.sh like this:
Existing line

curl --url smtps://$SMTP:$PORT \

Edited line:

curl --url smtp://$SMTP:$PORT \

(smtps is changed to smtp)

 

[Rimmel]

Hi got a rt-ac66u, when I run the install script I get:

The router is in AP mode (as it is not the main router) and the script would not connect to the website with the router in standard mode.

-----------------------------------------------------------
checking firmware capability: LAN IP port 80
your router's firmware (380.67) is too old
to run pixelserv-tls if you plan to use it
upgrade your routers firmware first
before attempting to install it.
------------------------------------------------------------


Yet 380.67 is the latest???????


Any ideas?

thanks

 

[thelonelycoder]

Hi got a rt-ac66u, when I run the install script I get:

The router is in AP mode (as it is not the main router) and the script would not connect to the website with the router in standard mode.

-----------------------------------------------------------
checking firmware capability: LAN IP port 80
your router's firmware (380.67) is too old
to run pixelserv-tls if you plan to use it
upgrade your routers firmware first
before attempting to install it.
------------------------------------------------------------


Yet 380.67 is the latest???????


Any ideas?

thanks

AB-Solution WILL NOT work and will not install unless the router is in Wireless router mode.
As for the firmware warning, this has actually nothing to do with your fw version installed.
The test checks if your router listens on port 80 on more than the router LAN IP.
Enter this into the SSH terminal and post the output:

netstat -tuln | grep ":80 " | grep  '0\.0\.0\.0:80 '

And also this is of interest:

netstat -tuln | grep ":80 "

 

[Rimmel]

AB-Solution WILL NOT work and will not install unless the router is in Wireless router mode.
As for the firmware warning, this has actually nothing to do with your fw version installed.
The test checks if your router listens on port 80 on more than the router LAN IP.
Enter this into the SSH terminal and post the output:

netstat -tuln | grep ":80 " | grep  '0\.0\.0\.0:80 '

And also this is of interest:

netstat -tuln | grep ":80 "

Ah, this is why I was asking if AB-Solution would work on a router that wasn't the main router in my previous post.

----------------------------------------------------
ASUSWRT-Merlin RT-AC66U 380.67-0 Sun Jul 16 16:56:20 UTC 2017
admin@RTAC66U:/tmp/home/root# netstat -tuln | grep ":80 " | grep '0\.0\.0\.0:80 '

admin@RTAC66U:/tmp/home/root# netstat -tuln | grep ":80 "
tcp 0 0 127.0.0.1:80 0.0.0.0:* LISTEN
tcp 0 0 192.168.1.140:80 0.0.0.0:* LISTEN

admin@RTAC66U:/tmp/home/root# curl -O ab-solution.info/releases/latest/ab-solution.sh && sh ab-solution.sh
curl: (6) Couldn't resolve host 'ab-solution.info'

admin@RTAC66U:/tmp/home/root#
----------------------------------------------------

When the router is not the main router (wan interface) disbled and DHCP server disabled then I get a could not resolve host message when trying to install the script.

thanks

 

[thelonelycoder]

Ah, this is why I was asking if AB-Solution would work on a router that wasn't the main router in my previous post.

"not the main router" has an ambiguous meaning.
I have 3 test routers that are not main routers but run in "wireless router mode" in my LAN.
Not being the main router says nothing about the router operation mode.
And AB only supports one operation mode: (have a guess).
Since yours is in AP mode, it cannot resolve my domain locally on the router, which does not matter, AB will not work.

As for your netstat output, they look OK if 192.168.1.140 is your routers IP, but then, this looks like an AP IP address.
Sorry AB will not work in that case.

 

[Rimmel]

"not the main router" has an ambiguous meaning.
I have 3 test routers that are not main routers but run in "wireless router mode" in my LAN.
Not being the main router says nothing about the router operation mode.
And AB only supports one operation mode: (have a guess).
Since yours is in AP mode, it cannot resolve my domain locally on the router, which does not matter, AB will not work.

As for your netstat output, they look OK if 192.168.1.140 is your routers IP, but then, this looks like an AP IP address.
Sorry AB will not work in that case.

I don't think main router is ambiguous, the main router you use to connect to the internet.

Anyway, I set the router back to wireless router mode and that was the output. So it was NOT in AP mode. However it is not the main router (connected to the internet). I wanted to use ab-solution as a pure ad blocker. I have this setup on DD-WRT at present and it works fine, but is dated and does not block HTTPS.

How have you got your 3 test routers set up in "wireless router mode"?

**** Edit: here is something wierd, when i connect to the router using wireless it works fine, but if i try the script through SSL the it fails on dns.

thanks

 

[thelonelycoder]

I don't think main router is ambiguous, the main router you use to connect to the internet.

Anyway, I set the router back to router mode and that was the output. So it was NOT in AP mode.

How have you got you other routers set up in "wireless router mode"?

thanks

Your Asus router has three or four operations modes, set in Administration.
Only the first one works, see also https://www.ab-solution.info/install/requirements.html

I don't think main router is ambiguous, the main router you use to connect to the internet.

That still does not mean the other routers are necessarily in Access Point Mode (AP), Repeater mode or a Media bridge.
As I said, my test routers are behind the main router, just running as separate routers, with their own IP range and WLAN settings.

 

[Rimmel]

Your Asus router has three or four operations modes, set in Administration.
Only the first one works, see also https://www.ab-solution.info/install/requirements.html

That still does not mean the other routers are necessarily in Access Point Mode (AP), Repeater mode or a Media bridge.
As I said, my test routers are behind the main router, just running as separate routers, with their own IP range and WLAN settings.

Ok got it, the DD-WRT allows you to do the same thing but you don't have to have their own separate IP range. Basically instead of having to connect the wan ports you can just use the LAN ports. Sorted it now though (with the aid of a static route to access the other subnet). I guess the DD-WRT firmware is a little more flexible than the Merlin firmware.

thanks

 

[thelonelycoder]

Ok got it, the DD-WRT allows you to do the same thing but you don't have to have their own separate IP range. Basically instead of having to connect the wan ports you can just use the LAN ports. Sorted it now though (with the aid of a static route to access the other subnet). I guess the DD-WRT firmware is a little more flexible than the Merlin firmware.

thanks

My test routers have only one purpose: I develop and test the scripts I code on them.
Them having their own IP range helps to simulate real world conditions.

 

[Alfred]

I ran into a conflict in a specific situation, and I am not sure where to post this question:

Running AB solution 3.9.1 with Pixelserv-tls on 192.168.#.2
Running Asuswrt-Merlin 380.67 on 192.168.#.1
Trying to start OpenVPN Server 2 on TCP port 443 instead of UDP port 1194 gives: OpenVPN server daemon failed to start.
System Log:

Aug 14 21:10:34 admin: Started pixelserv-tls (AB-Solution) from /tmp/mnt/entware/adblocking/addon/pixelserv-tls.add.
Aug 14 21:10:53 rc_service: httpds 292:notify_rc restart_chpass;restart_vpnserver2
Aug 14 21:10:54 kernel: device tun22 entered promiscuous mode
TCP/UDP: Socket bind failed on local address [AF_INET6][undef]:443: Address already in use
Aug 14 21:10:57 openvpn[31617]: Exiting due to fatal error

If Pixelserv-tls is stopped, OpenVPN is able to start.
But then Pixelserv-tls is not able to start (the address is responding but pixelserv is not up). AB-Solution recommends an install check (22) and detects errors.
If OpenVPN (on TCP443) is stopped, Pixelserv-tls is able to start and AB-Solution runs ok.

Is this expected behaviour?
Is this a Pixelserv-tls issue or an Asuswrt-Merlin issue?
Is there a workaround? (other than running AB-Solution without Pixelserv-tls ;-)

P.S.: Normally I use this OpenVPN server on TCP port 443 with the following custom configuration: port-share <NAS-IP> 443

 

[thelonelycoder]

I ran into a conflict in a specific situation, and I am not sure where to post this question:

Running AB solution 3.9.1 with Pixelserv-tls on 192.168.#.2
Running Asuswrt-Merlin 380.67 on 192.168.#.1
Trying to start OpenVPN Server 2 on TCP port 443 instead of UDP port 1194 gives: OpenVPN server daemon failed to start.
System Log:

Aug 14 21:10:34 admin: Started pixelserv-tls (AB-Solution) from /tmp/mnt/entware/adblocking/addon/pixelserv-tls.add.
Aug 14 21:10:53 rc_service: httpds 292:notify_rc restart_chpass;restart_vpnserver2
Aug 14 21:10:54 kernel: device tun22 entered promiscuous mode
TCP/UDP: Socket bind failed on local address [AF_INET6][undef]:443: Address already in use
Aug 14 21:10:57 openvpn[31617]: Exiting due to fatal error

If Pixelserv-tls is stopped, OpenVPN is able to start.
But then Pixelserv-tls is not able to start (the address is responding but pixelserv is not up). AB-Solution recommends an install check (22) and detects errors.
If OpenVPN (on TCP443) is stopped, Pixelserv-tls is able to start and AB-Solution runs ok.

Is this expected behaviour?
Is this a Pixelserv-tls issue or an Asuswrt-Merlin issue?
Is there a workaround? (other than running AB-Solution without Pixelserv-tls ;-)

P.S.: Normally I use this OpenVPN server on TCP port 443 with the following custom configuration: port-share <NAS-IP> 443

The answer is simple: Since pixelserv runs on port 443 and this is hard coded in my script, use another port for your VPN.

 

[Alfred]

The answer is simple: Since pixelserv runs on port 443 and this is hard coded in my script, use another port for your VPN.

Of course there is a reason why I would like to use TCP 443 for my OpenVPN. That way I can pass through firewalls which only allow outgoing TCP on ports 80 and 443.
Do I understand correctly that a listening process on TCP port 443 reserves this port on the hardware it runs on and not only on the IP address it uses?
Or to put it differently: It is not possible to make two processes on the same hardware listen to the same port but on separate IP addresses?
I that case I agree that I need to make a choice here, unfortunately.

 

[thelonelycoder]

Of course there is a reason why I would like to use TCP 443 for my OpenVPN. That way I can pass through firewalls which only allow outgoing TCP on ports 80 and 443.
Do I understand correctly that a listening process on TCP port 443 reserves this port on the hardware it runs on and not only on the IP address it uses?
Or to put it differently: It is not possible to make two processes on the same hardware listen to the same port but on separate IP addresses?
I that case I agree that I need to make a choice here, unfortunately.

Can't tell for sure ATM, but pixelserv should only listen on port 443 on the ps IP.
Make sure OpenVPN does the same in its IP and see how it goes.
In my head it works...
There is also the harder way of doing it: The pixelserv switches, you can change the https port (443) in the AB UI, but you'll also have to add a firewall forwarding rule to make it work.
See kvic's pixelserv thread for how to do it.