pixelserv pixelserv - A Better One-pixel Webserver for Adblock

[kvic]

In addition, this question has been asked a couple of times before. The answer is in the first post..."how to best run pixelserv-tls." It offers a better way to have all services working nicely together in the greed to grab all interfaces for one's own purpose. Steps may appear a bit involved depending on ppl's technical background. But hey it at least gets people started thinking...

 

[thelonelycoder]

Greetings all!
I'm using a Merlin-Asus RT-AC68. FWICT both ublockr and AB-Solution use pixelserv and dnsmasq to substitute a gif for blocked ads; and they both use port 443 (and 80?).

How can I change those ports? I'd like to use 443 for an OpenVPN server. (I'd guess I need to tweak a config file somewhere for each of them?)

TIA

Ab-Solution has a user friendly way to change the ports after the installation.
But while installing, make sure you have shut down the OpenVPN server, else installation/startup of pixelserv will not work if something else is listening on port 443.

 

[mike37]

Thank You jrmwvu04, kvic, and thelonelycoder for the quick, useful replies!

I did a fresh install of AB-Solution and tested it - before selecting the AB option of installing pixelserv.

Dang.... it blocked images within HTTPS pages quickly and effortlessly (no "lagging" or indication that the browser was waiting for something).

IIUC this shouldn't happen - the partially-loaded page should hang up. Maybe the browser (customized Chromium 57 on Android Copperhead) is expediting things!?

At any rate I presume I'll soon want to install pixelserv and then change the ports to free up 443. How would this be done? Perhaps using the AB "experimental settings" on the sub menu?

 

[thelonelycoder]

Thank You jrmwvu04, kvic, and thelonelycoder for the quick, useful replies!

I did a fresh install of AB-Solution and tested it - before selecting the AB option of installing pixelserv.

Dang.... it blocked images within HTTPS pages quickly and effortlessly (no "lagging" or indication that the browser was waiting for something).

IIUC this shouldn't happen - the partially-loaded page should hang up. Maybe the browser (customized Chromium 57 on Android Copperhead) is expediting things!?

At any rate I presume I'll soon want to install pixelserv and then change the ports to free up 443. How would this be done? Perhaps using the AB "experimental settings" on the sub menu?

Once installed, you find the ps menu has the option to set switches.

 

[elorimer]

Thank You jrmwvu04, kvic, and thelonelycoder for the quick, useful replies!

I did a fresh install of AB-Solution and tested it - before selecting the AB option of installing pixelserv.

Dang.... it blocked images within HTTPS pages quickly and effortlessly (no "lagging" or indication that the browser was waiting for something).

IIUC this shouldn't happen - the partially-loaded page should hang up. Maybe the browser (customized Chromium 57 on Android Copperhead) is expediting things!?

At any rate I presume I'll soon want to install pixelserv and then change the ports to free up 443. How would this be done? Perhaps using the AB "experimental settings" on the sub menu?

Not the ports. Follow the options within ab-solution to set another IP that pixelserv listens to outside your dhcp range.

 

[mike37]

Thanks for the suggestion, elorimer - AB-Solution does a good job of instructing you how to free up an address (192.168.1.2) and installing it there. (Heh...I freed up 9 addresses and tried to install it on 192.168.1.7...... no dice; it wants 192.168.1.2).

thelonelycoder, I followed the script, added the -k switch of port 93, the script acknowledged the new port setting, but could not successfully restart.
Turning pixelserv off resulted in changing the hosts file addresses from 192.168.1.2 back to 0.0.0.0 and a clean freeing up of addresses/ports - 443 is again available for OVPN and AB-Solution is blocking admirably.

(I wonder if, as hinted by jrmwvu04, another progy needed to be told to look at the new port? )

AB-Solution seems easy to use and has user control in mind; at this point I'll wait 'til the new pixelserv-tls server comes out and try it all again.

 

[mike37]

Aha! hold the presses; just tried setting to port 3000 and got different diagnostics:
*) requires additional parameter
**) iptables rules need to be set to redirect
the queries to new port(s).
Both ports need to be set, even if one
remains at default setting.

I'm using my own iptables so tweaking that will be easy. I'll be back....

 

[mike37]

tried changing both 80 and 443 to iana-unreserved ports (30000 and 20000) and it seems to work fine. AB-S ended with a confirmation:
"
Done added pixelserv-tls switches:
-k 30000 -p 20000
"
Iptables did not require changes.

'twas quick before and seems equally quick now - though I'd guess that serving a gif may prevent a hangup on some fussy (scripted) pages.

FWIW, There may be more requests for changing these default ports.
I'm doing all of this because OVPN 2.4 has a new ability to encrypt both the control and data channels - making the connection appear to be HTTPS - even to DPI (sometimes called a "stealth vpn"). Apparently some public/open WAPs allow only browsers; don't like VPNs; and this'll look like another browser - getting me safely out.

THANKS! everyone for the patient help!

 

[jrmwvu04]

Here you go:

https://github.com/kvic-z/pixelserv-tls/releases/download/v35.HZ12.Kj/pixelserv-tls.Kj.zip

I have been using the arm binary for a while. Mostly looks like things are working, but I am seeing messages in the syslog that I don't think I've had before.

May  7 06:53:51 pixelserv[678]: invalid file path 13
May  7 07:13:32 pixelserv[678]: Failed to create conn_handler thread
May  7 07:13:32 pixelserv[678]: Failed to create conn_handler thread
May  7 07:13:32 pixelserv[678]: Failed to create conn_handler thread
May  7 07:13:32 pixelserv[678]: Failed to create conn_handler thread
May  7 07:13:32 pixelserv[678]: Failed to create conn_handler thread

Anything I should concern myself with?

 

[kvic]

Anything I should concern myself with?

"Failed to create conn_handler thread" indicates pixelserv-tls cannot spawn a pthread to serve an incoming request. Possible causes: your router was simply under heavy workload and short of RAM when it happened. You may have hit a "malicious" web page which issued hundreds of ad requests in a short period of time. The latter is a bit more worrying. If you have logging on, the log should have registered a burst of requests near the timestamps of the errors, and it tells you which LAN client caused it. Worst case..your LAN has a malware but very unlikely. If you only see a few such errors once in a while, most likely you browse a page loaded with lots of ads and not too much to worry about.

"invalid file path" indicates the ad request has a URL not in a valid format.

Both are transient issues from the perspective of pixelserv.

 

[jrmwvu04]

Ok, good to know. I don't leave logging on unless I'm actively tracking something so I didn't have that available to look at. A malicious website seems the likeliest cause to me. The conn_handler thread errors were often only single events, with a day or two between instances. Only that one specific event on on the 7th was multiple entries. Also, the 7th was the only instance of the invalid file path error. That particular uptime was a little more than a week, so it's not a chronic issue.

 

[rromeroa]

tried changing both 80 and 443 to iana-unreserved ports (30000 and 20000) and it seems to work fine. AB-S ended with a confirmation:
"
Done added pixelserv-tls switches:
-k 30000 -p 20000
"
Iptables did not require changes.

'twas quick before and seems equally quick now - though I'd guess that serving a gif may prevent a hangup on some fussy (scripted) pages.

FWIW, There may be more requests for changing these default ports.
I'm doing all of this because OVPN 2.4 has a new ability to encrypt both the control and data channels - making the connection appear to be HTTPS - even to DPI (sometimes called a "stealth vpn"). Apparently some public/open WAPs allow only browsers; don't like VPNs; and this'll look like another browser - getting me safely out.

THANKS! everyone for the patient help!

Hi!

Did not need to change iptables? https requests are correctly redirected?
I needed to modify firewall script adding:

iptables -t nat -I PREROUTING -d 192.168.1.3 -p tcp --dport 443 -j DNAT --to 192.168.1.3:30000

 

[elorimer]

I'm stumbling a little bit on this port thing.

I moved an OpenVPN server instance to 443, hoping to improve my ability to access it from two places where I couldn't get through on 1194. So that it would start without error, I used AB-Solution to add a switch for the https bit, adding -k 2000. My pixelserv is listening on 192.168.0.3, with the router at 0.1.

The OpenVPN server worked fine, and pixelserv started, but https ads were not being handled correctly. My statistics showed no https requests in a good long time, no certificates were being generated, and max access times bloomed up to my -o max.

I turned off the OpenVPN server, changed the switch to -k 443, and pixelserv returned to normal.

Assuming I wanted to keep the server on 443, do I need to add something? Specifically this iptables command? And then, since AB-Solution keeps me on the straight and narrow, should I suggest to TLC a further treatment of the -k switch?

 

[rromeroa]

The OpenVPN server worked fine, and pixelserv started, but https ads were not being handled correctly. My statistics showed no https requests in a good long time, no certificates were being generated, and max access times bloomed up to my -o max.

This is what I meant. I needed to modify iptables.

 

[kvic]

This time I received email notification but do not get alerts after login...even for my own thread. Weird...

This is what I meant. I needed to modify iptables.

Oh..if that's what you meant, modification to iptables is not necessary. See below.

I'm stumbling a little bit on this port thing.

I moved an OpenVPN server instance to 443, hoping to improve my ability to access it from two places where I couldn't get through on 1194.

I recommend adding "local <your ddns>" to OpenVPN config. If it's Merlin build, put it as the first line in the custom config textbox.

This tells OpenVPN to listen only on the WAN interface instead of all interfaces.

EDIT: note that if your DDNS changes frequently, restart OpenVPN server in your DDNS update script or when you detect IP address changes on your WAN interface. @rromeroa suggestion to change iptables shall also work if you don't mind a bit extra processing there..

 

[elorimer]

I recommend adding "local <your ddns>" to OpenVPN config. If it's Merlin build, put it as the first line in the custom config textbox.

This tells OpenVPN to listen only on the WAN interface instead of all interfaces.

Thanks. This seems to work. I have pixelserv listening on .3:443. OpenVPN server won't start otherwise on 443, reporting the address in use. When I add this line to the custom config text box, it starts normally.

For my own education, how does the -k switch work? Is it that dnsmasq receives the https lookup on port 443, forwards that to .3 on that port, where it dies if pixelserv is listening on a different port? I follow that now OpenVPN is listening on the WAN interface at port 443, and pixelserv is listening on the LAN interface at port 443. It sounds like if I set -k to a different port, something else needs to be done for the switch to work.

Separately, do you know when build kj will propagate through entware-ng?

 

[kvic]

Thanks. This seems to work. I have pixelserv listening on .3:443. OpenVPN server won't start otherwise on 443, reporting the address in use. When I add this line to the custom config text box, it starts normally.

Because without "local" statement, OpenVPN binds to and listen on all interfaces (including your .3:443). Since your OpenVPN launches first and when pixelserv launches, it finds .3:443 is already occupied.

For my own education, how does the -k switch work?

It simply tells pixelserv-tls which ports to listen for HTTPS requests.

Is it that dnsmasq receives the https lookup on port 443, forwards that to .3 on that port, where it dies if pixelserv is listening on a different port?

Dnsmasq does a simple job. It translates an adserver domain (as defined in your hosts file) e.g. doubleclick.com to IP address. Take this as an example. Your browser first looks up the ip address of doubleclick.com. Dnsmasq replies with 192.168.0.3. Your browser then connects to 192.168.0.3:443. Pixelserv-tls receives the request and replies with an empty page.

So in any DNS based adblock implementations, Dnsmasq (or its more powerful counterparts such Unbound and BIND) is doing the heavy lifting.

Note that the port isn't decided by pixelserv but the ad URL embedded in webpages you visit. 443 is de facto for HTTPS traffic just like port 80 for HTTP.

@mstombs used to say he found some ad networks use non-standard ports. I believe that's why option -p was added to cover such edge cases. I added -k in the same spirit. I believe ad networks using non-standard ports are extremely rare today.

I follow that now OpenVPN is listening on the WAN interface at port 443, and pixelserv is listening on the LAN interface at port 443. It sounds like if I set -k to a different port, something else needs to be done for the switch to work.

If you decide to use -k to specify additional ports, you've to determine such a need first (most likely you don't). If ad URLs embedded in webpages you visit use non-standard ports, your browsers will automatically talk to pixelserv on these ports. You don't have to do anything.

-p/-k provide lots of flexibility where you might have to manipulate e.g. iptables rules for it to work. This may be for fun but not required. Just like iptables isn't the only way in Linux for such stunt to work.

Separately, do you know when build kj will propagate through entware-ng?

This is outside my control (perhaps @ryzhov_al or @zyxmon can check if their refresh schedule is near).

Alternatively you can download the statically built binary from pixelserv-tls GitHub. Extract and replace the Entware binary. It'll work. When Kj is available from Entware, do an update and it will overwrite this copy again (so no junk will be created in any case).

 

[elorimer]

Note that the port isn't decided by pixelserv but the ad URL embedded in webpages you visit. 443 is de facto for HTTPS traffic just like port 80 for HTTP.

@mstombs used to say he found some ad networks use non-standard ports. I believe that's why option -p was added to cover such edge cases. I added -k in the same spirit. I believe ad networks using non-standard ports are extremely rare today.

Thanks for this thorough explanation! This is the part I was missing.

 

[elorimer]

I switched to the static .kj build, which ran fine for a day and a half and then crashed. Restarting it lasted for just a few minutes on my 87u. This is similar to the .ki static build problem a reported in post #351. I've gone back to the entware .ki build, which ran for months on end without issue.

I think perhaps not an issue with pixelserv, but something about the static build?

 

[rromeroa]

I switched to the static .kj build, which ran fine for a day and a half and then crashed. Restarting it lasted for just a few minutes on my 87u. This is similar to the .ki static build problem a reported in post #351. I've gone back to the entware .ki build, which ran for months on end without issue.

I think perhaps not an issue with pixelserv, but something about the static build?

Same issue here, with RT-AC68U.