pixelserv pixelserv - A Better One-pixel Webserver for Adblock

[joe scian]

kazoo is down

 

[Clark Griswald]

kazoo is down

Issues for the past two days, yet Kvic seems to get addressed quickly.

 

[quant88]

kazoo is up and rc5 is also available for testing.

 

[loveleeyoungae]

Hi, I'd like to connect an AC66U to an existing network, i.e. AC66U will act as an adblocking server for devices on its WAN interface.

For your quick visualization:

ISP Router & PPPOE Modem
LAN IP: 192.168.1.1 (DHCP Subnet 1: 192.168.1.0/24)
         ||
         ||
         ||
(WAN port of AC66U - Router mode running Diversion & Pixelserv-tls)
WAN IP: 192.168.1.5
LAN IP: 192.168.50.1 (DHCP Subnet 2: 192.168.50.0/24)

I think I got it work by setting following configurations:
1. Adding custom config to dnsmasq.conf.add:

# dnsmasq will also listen for DNS requests on WAN side
interface=vlan2

# dnsmasq shouldn't act as DHCP server for the WAN side (obviously!)
no-dhcp-interface=vlan2

# dnsmasq will use Cloudflare as Upstream DNS or it would fall into an infinite loop (?!)
# can disable this option and set the DNS on WAN page of the webgui instead
server=1.1.1.1

2. Adding iptables rules in firewall-start:

#!/bin/sh
#Add iptables rules below third rule "--state INVALID" for more efficiency by Martineau
#https://www.snbforums.com/threads/how-to-open-a-port-on-the-router-itself-not-forward.46588/#post-405060
iptables -I INPUT 4 -p udp --dport 53 -j ACCEPT
iptables -I INPUT 5 -p tcp --dport 53 -j ACCEPT
iptables -I INPUT 6 -p tcp --dport 80 -j ACCEPT
iptables -I INPUT 7 -p tcp --dport 443 -j ACCEPT
iptables -I INPUT 8 -p udp --dport 443 -j ACCEPT

Everything seems to work fine. It seems to be a bit slower compared to blocking on LAN interface. I never import CA certs of pixelserv-tls, as the blocking on LAN side has always been ok to me. Now this WAN side blocking mode is still ok, but I don't know if the configurations I did were perfectly optimized.

So, it'd be great if someone can give some better advice here. Thanks in advance.

 

[kvic]

Everything seems to work fine. It seems to be a bit slower compared to blocking on LAN interface. I never import CA certs of pixelserv-tls, as the blocking on LAN side has always been ok to me. Now this WAN side blocking mode is still ok, but I don't know if the configurations I did were perfectly optimized.

A few thought:

• Import the CA cert to browser clients/devices. It'll give you a snappier experience.
• Always run pixelserv-tls on the FASTEST always-on server that you have on your LAN.
• Always run DNSmasq (or another DNS server such as Unbound) on the FASTEST always-on server that you have on your LAN.
• Limit number of blocked domains in DNSmasq to NOT more than a few hundred K.

 

[rgnldo]

I'm gonna go with a translation problem here. I suspect "not legal" is not what he meant, more like "not possible". It makes more sense that way - he was saying he couldn't simply whitelist all advertisers called by a given website, he had to whitelist them one at a time.

They tell me English is a very hard language to learn as a second language, and any language is tough if you're not able to use it every day.

It is true. I appreciate your help.

 

[rgnldo]

I agree with you, although I think he was saying that just whitelisting snbforums doesn't work to allow add through on the site, you have to whitelist them individually, and later he said he copied diversion's white list to do that.

Yes.

aax-eu.amazon-adsystem.com
aax-us-east.amazon-adsystem.com
ad2.netshelter.net
assets.omidoo.com
flashtalking.com
fls-na.amazon-adsystem.com
images-na.ssl-images-amazon.com
ir-na.amazon-adsystem.com
ir-uk.amazon-adsystem.com
pagead2.googlesyndication.com
servedby.flashtalking.com
tgdaily.com
tgdaily.net
vma.tgdaily.com
vma.tgdaily.net
wms-eu.amazon-adsystem.com
wms-na.amazon-adsystem.com
wms-na.assoc-amazon.com
ws-eu.amazon-adsystem.com
ws-na.amazon-adsystem.com
z-na.amazon-adsystem.com

I made my own script with Pixelserv-tls

 

[loveleeyoungae]

A few thought:

• Import the CA cert to browser clients/devices. It'll give you a snappier experience.
• Always run pixelserv-tls on the FASTEST always-on server that you have on your LAN.
• Always run DNSmasq (or another DNS server such as Unbound) on the FASTEST always-on server that you have on your LAN.
• Limit number of blocked domains in DNSmasq to NOT more than a few hundred K.

Thank you. To make it clear, I'm preparing to help my friend setting his network like that. We know this is a rare case which is not the best optimization to bring a good experience. Normally people would buy a pi-hole for its cheaper cost, but as he has a spare AC66U, doesn't really mind if browsing websites would take some more time to load, and doesn't want to mess up his existing network (i.e. reflashing the routers with Merlin firmwares), we think that setup is acceptable.

So, from your post, I guess the dnsmasq & iptables configurations are ok. He can't install the certs to all his 60 devices, but obviously he can install them on some most-used ones. Regarding the blocked domains, fortunately, there is a guy in our country managing a list of only 11k, but it works really great for us here, no more worrying about redundant stuffs in an oversize list from popular international sources.

Thank you for your advice, and obviously your great program!

 

[teleporter]

I tested pixelserv-tls on Windows 10 WSL (Ubuntu 18.04). It compiles and shows the help menu, but it wont run. I tried to run in foreground as well but it doesn't. Has anybody else tried using Linux on Windows? Thanks.

 

[kvic]

I tested pixelserv-tls on Windows 10 WSL (Ubuntu 18.04). It compiles and shows the help menu, but it wont run. I tried to run in foreground as well but it doesn't. Has anybody else tried using Linux on Windows? Thanks.

Look inside "/var/log/messages" for error messages. If nothing, then run with command line option '-l 5' again. I would make a guess likely directory permission issue.

You need to create "/var/cache/pixelserv" and then "chown nobody:root /var/cache/pixelserv". Perhaps the critical step is..remember to launch with "sudo" e.g. "sudo pixelserv-tls -f -l 5"

More CA cert related info on wiki

 

[teleporter]

Thank you for the pointer @kvic
I had to start rsyslogd manually. Anyway, I fixed sslctx_tbl_load: /var/cache/pixelserv/prefetch doesn't exist. by creating and chown-ing it. There's also a protocol error which yielded no google results..

Dec 17 12:16:47 MYDESKTOP root: hello00
Dec 17 12:18:39 MYDESKTOP pixelserv-tls[1357]: pixelserv-tls 2.2.0 (compiled: Dec 16 2018 08:18:46 flags: tfo no_tls1_3) options: -f -l 5
Dec 17 12:18:39 MYDESKTOP pixelserv-tls[1357]: sslctx_tbl_load: /var/cache/pixelserv/prefetch doesn't exist.
Dec 17 12:18:39 MYDESKTOP pixelserv-tls[1357]: Abort: Protocol not available - :*:443
Dec 17 12:20:50 MYDESKTOP pixelserv-tls[1365]: pixelserv-tls 2.2.0 (compiled: Dec 16 2018 08:18:46 flags: tfo no_tls1_3) options: -f -l 5
Dec 17 12:20:50 MYDESKTOP pixelserv-tls[1365]: Abort: Protocol not available - :*:443

I ran netstat -an in Windows and port 443 is not being used (445 is).

 

[kvic]

Thank you for the pointer @kvic
I had to start rsyslogd manually. Anyway, I fixed sslctx_tbl_load: /var/cache/pixelserv/prefetch doesn't exist. by creating and chown-ing it. There's also a protocol error which yielded no google results..

Dec 17 12:16:47 MYDESKTOP root: hello00
Dec 17 12:18:39 MYDESKTOP pixelserv-tls[1357]: pixelserv-tls 2.2.0 (compiled: Dec 16 2018 08:18:46 flags: tfo no_tls1_3) options: -f -l 5
Dec 17 12:18:39 MYDESKTOP pixelserv-tls[1357]: sslctx_tbl_load: /var/cache/pixelserv/prefetch doesn't exist.
Dec 17 12:18:39 MYDESKTOP pixelserv-tls[1357]: Abort: Protocol not available - :*:443
Dec 17 12:20:50 MYDESKTOP pixelserv-tls[1365]: pixelserv-tls 2.2.0 (compiled: Dec 16 2018 08:18:46 flags: tfo no_tls1_3) options: -f -l 5
Dec 17 12:20:50 MYDESKTOP pixelserv-tls[1365]: Abort: Protocol not available - :*:443

I ran netstat -an in Windows and port 443 is not being used (445 is).

Use "ifconfig" to find out the IP address, say <pixel ip>. Then try "sudo pixelserv-tls <pixel ip> -f -l 5"

This is obviously for testing only. For "production", you may want to create a systemd service. Take a look at pixelserv-tls package for Arch Linux. You may borrow the definition of systemd service from here.

 

[teleporter]

Use "ifconfig" to find out the IP address, say <pixel ip>. Then try "sudo pixelserv-tls <pixel ip> -f -l 5"

This is obviously for testing only. For "production", you may want to create a systemd service. Take a look at pixelserv-tls package for Arch Linux. You may borrow the definition of systemd service from here.

I'm testing now . I may test Arch Linux on Windows later on.

I find the same error (also with 127.0.0.1)

Dec 17 12:52:07 MYDESKTOP pixelserv-tls[1369]: pixelserv-tls 2.2.0 (compiled: Dec 16 2018 08:18:46 flags: tfo no_tls1_3) options: 192.168.2.24 -f -l 5
Dec 17 12:52:07 MYDESKTOP pixelserv-tls[1369]: Abort: Protocol not available - :192.168.2.24:443

 

[kvic]

Interesting...I run out of ideas at the moment. I thought it's just a Linux inside a fully blown VM..

One last thing you may try.. rebuild with "./configure CFLAGS='-UIF_MODE'".

In case, you try Arch later, simply run "yaourt -S pixelserv-tls" that will handle everything for you.

 

[kvic]

@teleporter

I just tried a refresh build of 2.2.0 under Ubuntu 18.04 (under Vbox with MacOS host) by using the default command lines. Linked against OpenSSL 1.1.0g. Everything runs well. Didn't see the abort error. I would guess your error possibly related to build process or VM environment.

Hope you could figure out something. Please do keep me posted if you get some progress.

 

[jrmwvu04]

I'm testing now . I may test Arch Linux on Windows later on.

I find the same error (also with 127.0.0.1)

Dec 17 12:52:07 MYDESKTOP pixelserv-tls[1369]: pixelserv-tls 2.2.0 (compiled: Dec 16 2018 08:18:46 flags: tfo no_tls1_3) options: 192.168.2.24 -f -l 5
Dec 17 12:52:07 MYDESKTOP pixelserv-tls[1369]: Abort: Protocol not available - :192.168.2.24:443

I am curious, your flags are "tfo no_tls1_3"
In my compiling adventures I never had a platform that would enable tcp fast open but not tls 1.3 - are you seeing any warnings or anything on compile? I am a total beginner in this and don't exactly know what I'm talking about but that just caught my attention. I am wondering if there's something in the WSL that is not supported, or not fully supported, protocol wise. It's not fresh in my mind at this point, just throwing ideas.

 

[jrmwvu04]

@teleporter to be clear, there were a handful of the functions in the pixelserv code that weren't portable outside linux that caused me all kinds of hangups.

 

[teleporter]

@kvic Yes ur right. I got it to run in an ubuntu 18.04 hyper-v vm no problems. After googling, I think the "Protocol not available" is a problem associated with the Windows WSL. I was hoping to "keep it simple" and have it run on WSL but that seems not to be the case. I'm guessing Arch linux under WSL may have the same issue but I'll try it another time.

I also tried compiling using "./configure CFLAGS='-UIF_MODE'" in WSL Ubuntu but no joy.

@jrmwvu04 Under the ubuntu vm also I have pixelserv-tls 2.2.0 (compiled: Dec 17 2018 15:56:49 flags: tfo no_tls1_3). I'm not sure how I can compile with TLS 1.3 and what the advantage would be. I didn't notice any compile warnings.

 

[jrmwvu04]

@kvic Yes ur right. I got it to run in an ubuntu 18.04 hyper-v vm no problems. After googling, I think the "Protocol not available" is a problem associated with the Windows WSL. I was hoping to "keep it simple" and have it run on WSL but that seems not to be the case. I'm guessing Arch linux under WSL may have the same issue but I'll try it another time.

I also tried compiling using "./configure CFLAGS='-UIF_MODE'" in WSL Ubuntu but no joy.

@jrmwvu04 Under the ubuntu vm also I have pixelserv-tls 2.2.0 (compiled: Dec 17 2018 15:56:49 flags: tfo no_tls1_3). I'm not sure how I can compile with TLS 1.3 and what the advantage would be. I didn't notice any compile warnings.

Could be due to the version of OpenSSL you’re linking. Beyond that I have no ideas.

 

[kvic]

Could be due to the version of OpenSSL you’re linking. Beyond that I have no ideas.

UBUNTU 18.04 still comes with 1.1.0g which doesn't have TLS 1.3