pixelserv pixelserv - A Better One-pixel Webserver for Adblock

[thelonelycoder]

but I dont know what lines on the ab solutions I need to change so that the final host file is 192.168.1.3 whatever.com and so on, Im looking on the lonelys code but Im , cant figure out yet what to change

For simplicity, get this modified beta2 version. Change IP on line 1584: lan_psadress=<your IP>.

curl -O www.ab-solution.info/releases/shooter40sw/ab-solution.sh

 

[shooter40sw]

For simplicity, get this modified beta2 version. Change IP on line 1584: lan_psadress=<your IP>.

curl -O www.ab-solution.info/releases/shooter40sw/ab-solution.sh

Thanks I will give it a shot, after lunch! I could manage to change the host file to my virtual IP, and it seems to be working ok, I kind of see some lag between the pixelserv and the non pixelserv solution.

I used the code:
To make the certificates, I dont know if Im missing a step but I use chrome and the padlocks turn from green to white almost immediately, says there is mixed content. Thanks for the help! and thanks for this effort to bring this solution to all. I dont even use an extension to ad block anymor.
Code:
cd /opt/var/cache/pixelserv
openssl genrsa -out ca.key 1024
openssl req -key ca.key -new -x509 -days 3650 -sha256 -extensions v3_ca -out ca.crt -subj "/CN=Pixelserv CA"

 

[kvic]

pixelserv-tls version: V35.HZ12.Kh compiled: May 11 2016 15:13:54 options: 192.168.66.254 -p 80 -p 81 -p 8080 -p 8081 -k 443 -o 2
4995871 uts, 5682573 req, 1284 avg, 34331 rmx, 109 tav, 4159 tmx, 0 err, 8562 tmo, 5625639 cls, 0 nou, 0 pth, 12704 nfe, 2263 ufe, 492 gif, 0 bad, 18294 txt, 60 jpg, 76 png, 18 swf, 18 ico, 5579788 slh, 84 slm, 0 sle, 2557 slu, 11 sta, 3 stt, 0 204, 14114 rdr, 277 pst, 0 hed, 0 log

Uptime 4995871 seconds? That's close to 60 days!

My router is a few hours shy of 60 days since last reboot. In between I restarted pixelserv a few times. I kept a few screenshots of my RT-AC56U's home-brew 'hardcore' dashboard:



The left one was the first 29 days. I was shocked to find out over 3 million requests (3,230,655 to be exact). My first instinct was that my machine was infected with malware! The one on right was the most recent 25 days. A more reasonable number...over 100 thousand requests.

Upon closer check, about 3 million requests are 'nfe' as well as being HTTPS. These were generated by watching some YouTube channels! I wonder what those channels trying to collect from users.

 

[kvic]

I kind of see some lag between the pixelserv and the non pixelserv solution.

I like to see some data from others to assure this thesis. My own experience and non-rigid verification show that I can't tell with bare eyes. Here are some load time numbers I saved:

• anandtech.com
  • /w pixelserv-tls: 1.9s
  • no pix: 1.85s
• theverge.com
  • /w pix: 3.04s
  • no pix: 2.96s
• bbc.co.uk/news
  • /w pix: 1.95s
  • no pix: 2.25s
• scmp.com/frontpage/hk
  • /w pix: 12.52s
  • no pix: 10.38s
• cnbc.com/world/?region=world
  • /w pix: 3.2s
  • no pix: 2.11s
• cnet.com/news
  • /w pix: 0.767s
  • no pix: 1.052s

4 out of 6 are faster without pixelserv-tls. The other two are actually faster with pixelserv-tls. The differences are certainly not perceivable with bare eyes. A caution: your result might differ from the above due to many reasons e.g. your adblock lists.

As I said at the early days of this thread, modern browsers are very efficient at hitting 127.0.0.1 or 0.0.0.0. Some of the original goals of pixelserv were left in dust with its time. Hence, I refined the project goal of pixelserv-tls on github.

 

[shooter40sw]

Good morning!, what do I have to do to give you this data?

I like to see some data from others to assure this thesis. My own experience and non-rigid verification show that I can't tell with bare eyes. Here are some load time numbers I saved:

• anandtech.com
  • /w pixelserv-tls: 1.9s
  • no pix: 1.85s
• theverge.com
  • /w pix: 3.04s
  • no pix: 2.96s
• bbc.co.uk/news
  • /w pix: 1.95s
  • no pix: 2.25s
• scmp.com/frontpage/hk
  • /w pix: 12.52s
  • no pix: 10.38s
• cnbc.com/world/?region=world
  • /w pix: 3.2s
  • no pix: 2.11s
• cnet.com/news
  • /w pix: 0.767s
  • no pix: 1.052s

4 out of 6 are faster without pixelserv-tls. The other two are actually faster with pixelserv-tls. The differences are certainly not perceivable with bare eyes. A caution: your result might differ from the above due to many reasons e.g. your adblock lists.

As I said at the early days of this thread, modern browsers are very efficient at hitting 127.0.0.1 or 0.0.0.0. Some of the original goals of pixelserv were left in dust with its time. Hence, I refined the project goal of pixelserv-tls on github.

 

[mstombs]

Uptime 4995871 seconds? That's close to 60 days!
....

Well I can get the html formatted stats:-

pixelserv-tls version: V35.HZ12.Kh compiled: May 11 2016 15:13:54 options: 192.168.66.254 -p 80 -p 81 -p 8080 -p 8081 -k 443 -o 2
uts: Uptime 58 days 19:55
req: Total # of requests 5688039
...
cls: # of client shutdown 5629514
...
slh: # of HTTPS /w a good cert 5583782
...

I have 135 certs stored on my router cache/pixleserv, collected over last 4 months. I only use the small pgl.yoyo.org dnsmasq domain block list.

This is result of being in normal use in domestic household, over 5 million requests most of which are https. My options include o 2 where I was last experimenting to see where all the cls come from (watching traffic using wireshark), but clearly never got anywhere,

I do continuously page load times using 'page load timer' add-on in Google Chrome, or the network load tab in developer options - but it is very difficult to compare with/without due to caching etc. In the early days of C pixelserv there were some dramatic improvements in some sites which would retry blocked items many times, but of course browsers and web pages developed a lot over the years. It may be better to block some connections with tcp-resetrather than carry out https conversation (just block not good due to timeouts and retries) , but so many combinations of broweser and web site no universal answer! Many more sites now more aware of adblocking, using server side dns, locally hosted adverts etc. Note the current public battle between facebook and adblockplus!

 

[thelonelycoder]

ifconfig br0:pixelserv 192.168.1.3 up
logger -t $tag "br0:pixelserv 192.168.1.3 created."

Then start pixelserv on 192.168.1.3 (that you will change in Entware's init script for pixelserv-tls), and have 192.168.1.3 in your host files.

That works all quite well, thanks.

To use IPv6 I am using this:
ifconfig br0ixelserv 2002:a00:6e::2 up
pixelserv-tls 2002:a00:6e::2

And that appears to work according to the dnsmasq log.
Is this the correct way to use IPv6 with pixelserv-tls?
Also, with the ARGS="" set to the IPv4 address, how would I add the IPv6 address as well in the same line?

 

[mstombs]

I'm surprised pixelserv-tls works with ipv6 - I thought some of the socket stuff was ipv4 specific, sure it can be upgraded, maybe kvic has already done it.

 

[kvic]

Good morning!, what do I have to do to give you this data?

Here describes a way to measure page load time. See if this helps. http://kazoo.ga/measure-page-load-time-in-google-chrome/

 

[kvic]

That works all quite well, thanks.

To use IPv6 I am using this:
ifconfig br0ixelserv 2002:a00:6e::2 up
pixelserv-tls 2002:a00:6e::2

And that appears to work according to the dnsmasq log.
Is this the correct way to use IPv6 with pixelserv-tls?
Also, with the ARGS="" set to the IPv4 address, how would I add the IPv6 address as well in the same line?

I just tested but it didn't work for me. My first time test against a IPv6 address frankly. I think pixelserv was never intended to work with IPv6 though some of the codes are compatible. Listen address isn't coded for other than IPv4.

 

[thelonelycoder]

I just tested but it didn't work for me. My first time test against a IPv6 address frankly. I think pixelserv was never intended to work with IPv6 though some of the codes are compatible. Listen address isn't coded for other than IPv4.

No problem, I works just fine with IPv4 requests to pixelserv and the IPv6 stuff to a null address.

 

[shooter40sw]

Hi guys, is there a way so I dont need to use the https blocking feature? after a few days I just dont want to hassle to install certs in all devices, I did it on the PC but for some reason the https went from green lock to a white one on chrome due to mixed content, and mostly I have mobile devices, and its annoying, I just reverted to the @thelonelycoder AB solutions with no pixelserv. Also when this was enabled I did not observe any change in layout of webpages when pixelserv was active, like i have read from other users.
Thanks!

 

[Goobi]

@kvic I just wanted to share my stats with you. I have been running your pixelserv along with the old 1.06 ad block solution on a RT-AC68U running Merlin 380.59. Although I love tinkering with my router, I have promised my family that I would only do updates once a year. That said, I have been running for 146 days straight and happy to report your solution combined with @thelonelycoder ad block has made using the Internet at home a much better experience. Thank you both!! Over 1 million blocks and counting!!!

pixelserv version: V35.HZ12.Kh compiled: Mar 13 2016 15:38:28 options: 10.0.100.1
uts: Uptime 146 days 21:13
req: Total # of requests 1045579
avg: Avg size of reqs 1314 bytes
rmx: Max size of reqs 67834 bytes
tav: Avg process time 101 ms
tmx: Max process time 18533 ms
err: # of error reqs 0
tmo: # of client timeout 517
cls: # of client shutdown 504315
nou: # of reqs w/o URL 0
pth: # of invalid URL 0
nfe: # of missing file ext 288422
ufe: # of unknown file ext 27266
gif: # of GIF reqs 2435
bad: # of unknown HTTP methods 51
txt: # of TXT reqs 54925
jpg: # of JPG reqs 857
png: # of PNG reqs 919
swf: # of SWF reqs 126
ico: # of ICO reqs 31
slh: # of HTTPS /w a good cert 491136
slm: # of HTTPS w/o a cert 633
sle: # of HTTPS /w a bad cert 204
slu: # of unrecognized HTTPS 11301
sta: # of HTML stats 236
stt: # of text stats 0
204: # of HTTP/204 (no content) 0
rdr: # of redirects 153994
pst: # of POST method 8211
hed: # of HEAD method 2
log: access log enabled (0=no 1=yes) 0

 

[thelonelycoder]

@Goobi When is your next yearly update due?

 

[Goobi]

@Goobi When is your next yearly update due?

I had to reboot back in March so come December it won't be quite the year. However December is my window for any changes, so I will either update then or maybe just go until December 2017. Everything has been rock solid so it is appealing to just leave it as is. I have had a couple of ISP outages during my uptime but the dual wan feature bailed me out and worked flawlessly. I also had a couple of brief power outages due to bad storms but everything including the router is attached to a UPS and that worked as expected. Thanks again for your awesome Adblock feature.

 

[kvic]

Interesting story, Goobi. Thanks for sharing.

Not touching a piece of sophisticated toy for many months requires non trivial self-discipline! I'm trying. My RT-AC56U is up for 65 days. Not bad but neither great. Looking at my memory chart, I know it's dying slowly. Let's check back in December. lol

 

[elorimer]

This is a minor point, but when pixelserv writes to the syslog it is four hours ahead of the syslog time. I'm -5 hours (East Coast US). Is there a way for the pixelserv entries to be on the same time?

Edit: 87U and .62A1.

Aug 28 10:26:30 rc_service: waitting "start_vpnserver1" via udhcpc ...
Aug 28 14:26:30 pixelserv[1025]: pixelserv-tls version: V35.HZ12.Kh compiled: Jul 20 2016 22:57:01 options: 192.168.0.3
Aug 28 14:26:30 pixelserv[1025]: Listening on :192.168.0.3:80
Aug 28 14:26:30 pixelserv[1025]: Listening on :192.168.0.3:443
Aug 28 10:26:31 elorimer: Started pixelserv-tls from .
Aug 28 10:26:31 rc_service: hotplug 822:notify_rc restart_nasapps

 

[mstombs]

What router and Merlin version? Perhaps I don't see the problem on my N66 (380.59) because UTC+1 is correct for the UK?

Aug 28 20:04:49 miniupnpd[24133]: upnp_event_recv: recv(): Connection reset by peer
Aug 28 20:16:08 pixelserv[14718]: ads2.contentabc.com _.contentabc.com missing
Aug 28 20:16:09 pixelserv[14719]: cert _.contentabc.com generated and saved

But all logs should really be in UTC and translated when viewed, but I see /tmp/syslog.log has the local time in plain text...

 

[Popov]

Thanks for making this, nice to get a lighter browsing experience.
Took me some hours but I was able to get it running on a tiny core linux VM.

By the way, I'm experiencing a problem with the certificates : while they are correctly generated, they are missing the certificate chain information, while the CA issuer information is present and valid.
As I'm using a CA cert for internal use, I was hoping to be able to reuse it, but in fact even with a generated one, this anomaly is present :
pixelserv generated cert

The issuer should appear at the first level in the hierarchy, then only after, the blocked site should be printed one level under. When manually made, they show this correctly, but not those from pixelserv.

As comparison, a valid google cert
note : the intermediate level isn't required.

This is problematic, as the certs are in fact not fully valids. And all browsers hide this most of the time because the RFC requires to silently close the connection on an invalid cert when it's an external ressource and not the main page. On the other hand, accessing the url directly will have both Fox and IE complain about it, even after adding an exclusion.
Seems something is missing or isn't applied when pixelserv-tls is creating the certs.

For reference, currently running on the Kh version, didn't check with a previous one.

 

[thelonelycoder]

@Popov, sorry for the off topic comment. But I wanted to thank you for the two excellent Ad-Blocking test sites you posted. Those are the reason we use adblockers in the first place.