pixelserv pixelserv - A Better One-pixel Webserver for Adblock

[Raphie]

on my network are:
- 4 iPads
- 3 iPhones
- 2 Windows 10 systems
- 1 printer
- PS4 Pro
- 1 smart TV
- 3 Chromecasts
- one Synology NAS
- one ac66u router in accespoint mode

I didn't visit 538

 

[kvic]

One item from the blog post of the SSL Labs tests is worth highlighting again and I bet most users might have overlooked. It's the compatibility list of clients that will work with v2.1:



This link from the blog post carries a bigger and clearer screenshot.

So when people are concerned about your client devices compatible with pixelserv-tls or not, simply go and have a check.

 

[kvic]

Thank you for your great work!

Also no problems with Opera (51.0.2830.55; x64)

@eclp allow me to re-use your screenshoot do a bit marketing..

If people are accessing WebGUI over HTTPS, as a pixelserv-tls user, there is a convenient way to re-use your Pixelserv CA cert. Details are in this post #1352.

The underlying helper script config-webgui.sh is updated today to v0.1.2. Fixing an issue of not persistent certificate after reboot discussed in this thread.

So for people had run the one-liner script from post #1352 before today, it's recommended to re-run it once to retain the settings after router reboot.

 

[kvic]

My two instances of 2.1.0-rc.2 over 33 hours...marching on its 2nd day.

n1 is using 8.2MB RAM on RT-56U. n2 is using 6.5MB on ER-X.

Showing a pleasant constraint of no RAM abuse. So far rc.2 puts pixelserv-tls in its best shape ever yet.

(I did no window dressing on the servstats numbers... all from usual and everyday browsing activities. LOL)

 

[jrmwvu04]

My two instances of 2.1.0-rc.2 over 33 hours...marching on its 2nd day.

n1 is using 8.2MB RAM on RT-56U. n2 is using 6.5MB on ER-X.

Showing a pleasant constraint of no RAM abuse. So far rc.2 puts pixelserv-tls in its best shape ever yet.

(I did no window dressing on the servstats numbers... all from usual and everyday browsing activities. LOL)

If you happen to have an affinity for Nate Silver, there’s not necessarily any window dressing. pixelserv requests well rocket well into the thousands inside of 30 seconds and one or two page loads in some cases. It’s unlike anything I’ve experienced before.
Has anyone enterprising enough enabled logging to see what’s getting requested? I have relatively modest block lists.

 

[Makaveli]

Thanks for posting.

slc is a bit too high. I wonder what Apps/clients might have caused that.

slc is after clients having passed TLS handshake. So that means these clients already have CA cert imported. But then somehow they decide to disconnect without sending any requests. And this behaviour repeats frequently in your environment based on the total requests, req and the slu numbers. *puzzled*

Right now on the network

These devices have the CA.

Desktop
Laptop (this device does have ad block installed but only for youtube playback everything else is whitelisted)
Andriod phone

Devices that do not have

HTPC running Kodi which I don't browse on
Samsung Smart TV

So i'm still not sure what is causing the high slc.

I've yet to do the logging so that may help to identify what.

here are the Stats for the newest release

pixelserv-tls 2.1.0-rc.2 (compiled: Mar 20 2018 21:11:49) options: 192.168.1.3

uts    1d 00:36    process uptime
log    1    critical (0) error (1) warning (2) notice (3) info (4) debug (5)
kcc    1    number of active service threads
kmx    15    maximum number of service threads
kvg    1.09    average number of requests per service thread
krq    7    max number of requests by one service thread
req    2182    total # of requests (HTTP, HTTPS, success, failure etc)
avg    557 bytes    average size of requests
rmx    3178 bytes    largest size of request(s)
tav    8 ms    average processing time (per request)
tmx    213 ms    longest processing time (per request)
slh    296    # of accepted HTTPS requests
slm    0    # of rejected HTTPS requests (missing certificate)
sle    0    # of rejected HTTPS requests (certificate available but bad)
slc    1352    # of dropped HTTPS requests (client disconnect without sending any request)
slu    479    # of dropped HTTPS requests (other TLS handshake errors)
sct    69    cert cache: # of certs in cache
sch    1977    cert cache: # of reuses of cached certs
scm    1    cert cache: # of misses to find a cert in cache
scp    0    cert cache: # of purges to give room for a new cert
sst    4    sess cache: # of cached TLS sessions (for older non-RFC5077 clients)
ssh    197    sess cache: # of reuses of cached TLS sessions
ssm    9    sess cache: # of misses to find a TLS session in cache
ssp    0    sess cache: # of purges to give room for a new TLS session

Memory Usage 3.6%

 

[heysoundude]

One item from the blog post of the SSL Labs tests is worth highlighting again and I bet most users might have overlooked. It's the compatibility list of clients that will work with v2.1:

View attachment 12404

This link from the blog post carries a bigger and clearer screenshot.

So when people are concerned about your client devices compatible with pixelserv-tls or not, simply go and have a check.

I don't see chromium v65 listed there...I use the brave browser. am I out of luck?

 

[Protik]

1 Day 10 hours on rc2. Using 12MB of 250MB RAM.

pixelserv-tls 2.1.0-rc.2 (compiled: Mar 20 2018 21:11:49) options: 192.168.2.2 -c 150 -l 2

uts    1d 10:19    process uptime
log    2    critical (0) error (1) warning (2) notice (3) info (4) debug (5)
kcc    2    number of active service threads
kmx    42    maximum number of service threads
kvg    1.32    average number of requests per service thread
krq    20    max number of requests by one service thread
req    5081    total # of requests (HTTP, HTTPS, success, failure etc)
avg    1107 bytes    average size of requests
rmx    59408 bytes    largest size of request(s)
tav    57 ms    average processing time (per request)
tmx    9553 ms    longest processing time (per request)
slh    1355    # of accepted HTTPS requests
slm    2    # of rejected HTTPS requests (missing certificate)
sle    0    # of rejected HTTPS requests (certificate available but bad)
slc    1625    # of dropped HTTPS requests (client disconnect without sending any request)
slu    1854    # of dropped HTTPS requests (other TLS handshake errors)
sct    150    cert cache: # of certs in cache
sch    4128    cert cache: # of reuses of cached certs
scm    46    cert cache: # of misses to find a cert in cache
scp    8    cert cache: # of purges to give room for a new cert
sst    10    sess cache: # of cached TLS sessions (for older non-RFC5077 clients)
ssh    179    sess cache: # of reuses of cached TLS sessions
ssm    361    sess cache: # of misses to find a TLS session in cache
ssp    0    sess cache: # of purges to give room for a new TLS session

 

[kvic]

So i'm still not sure what is causing the high slc.

Thanks for the detailed update. This one should be easy to spot. You have to install and run tcpdump on your router.

Install:
$ opkg install tcpdump

Then run. replace 192.168.1.3 with your <pixelserv ip>
$ tcpdump -i br0 "tcp[tcpflags] & (tcp-syn) != 0 and host 192.168.1.3 and tcp port 443"

Sample outputs:
12:20:42.493524 IP myclient.ukei.64599 > n1.pixelserv.tls.https: Flags , seq 449919520, win 65535, options [mss 1460,nop,wscale 5,nop,nop,TS val 3158194437 ecr 0,sackOK,eol], length 0

"myclient.ukei" from port 64599 is making the connection at timestamp 12:20:42.493524.

You can make a good guest if this connection is slc. When you see slc rapidly increasing during browsing activities, and many recurring outputs from the tcpdump at the same time. Must be it.

 

[kvic]

I don't see chromium v65 listed there...I use the brave browser. am I out of luck?

lol. I believe SSL Labs update their list of test clients a few times per year. Usually older clients have compatibility issues. Safari 11 and latest FF are not on the list either.

You have a point. I wonder how they could test latest TLSv1.3 that its draft standard gets updated quite frequently. I guess that's why they claimed to test a quite old draft of TLSv1.3 only.

btw, all latest Chrome/FF/Safari work with v2.1 perfectly.

 

[kvic]

1 Day 10 hours on rc2. Using 12MB of 250MB RAM.

pixelserv-tls 2.1.0-rc.2 (compiled: Mar 20 2018 21:11:49) options: 192.168.2.2 -c 150 -l 2

uts    1d 10:19    process uptime
log    2    critical (0) error (1) warning (2) notice (3) info (4) debug (5)
kcc    2    number of active service threads
kmx    42    maximum number of service threads
kvg    1.32    average number of requests per service thread
krq    20    max number of requests by one service thread
req    5081    total # of requests (HTTP, HTTPS, success, failure etc)
avg    1107 bytes    average size of requests
rmx    59408 bytes    largest size of request(s)
tav    57 ms    average processing time (per request)
tmx    9553 ms    longest processing time (per request)
slh    1355    # of accepted HTTPS requests
slm    2    # of rejected HTTPS requests (missing certificate)
sle    0    # of rejected HTTPS requests (certificate available but bad)
slc    1625    # of dropped HTTPS requests (client disconnect without sending any request)
slu    1854    # of dropped HTTPS requests (other TLS handshake errors)
sct    150    cert cache: # of certs in cache
sch    4128    cert cache: # of reuses of cached certs
scm    46    cert cache: # of misses to find a cert in cache
scp    8    cert cache: # of purges to give room for a new cert
sst    10    sess cache: # of cached TLS sessions (for older non-RFC5077 clients)
ssh    179    sess cache: # of reuses of cached TLS sessions
ssm    361    sess cache: # of misses to find a TLS session in cache
ssp    0    sess cache: # of purges to give room for a new TLS session

@Protik since you're running with log LEVEL 2, what are the most frequently seen reason code for the ~1800 slu?

 

[Protik]

@Protik since you're running with log LEVEL 2, what are the most frequently seen reason code for the ~1800 slu?

Mostly 1046. Rest are 1048.

admin@RT-AC68U-DF28:/tmp/home/root# grep pixelserv-tls /tmp/syslog.log|grep 'ssl error'|grep -c 'reason(1046)'
568
admin@RT-AC68U-DF28:/tmp/home/root# grep pixelserv-tls /tmp/syslog.log|grep 'ssl error'|grep -c 'reason(1048)'
198
admin@RT-AC68U-DF28:/tmp/home/root# grep pixelserv-tls /tmp/syslog.log-1|grep 'ssl error'|grep -c 'reason(1046)'
205
admin@RT-AC68U-DF28:/tmp/home/root# grep pixelserv-tls /tmp/syslog.log-1|grep 'ssl error'|grep -c 'reason(1048)'
54

 

[kvic]

@mstombs

I plan to add a copyright section to the manpage in the coming update.

The state before my current fork in 2015 looks right below? Did I miss any other major contributors in the past?

Copyright (C) 2013-2015 pixelserv by HunterZ@github
Copyright (C) 2009-2013 pixelserv by mstombs@github

 

[Raphie]

1 day 19h...

pixelserv-tls 2.1.0-rc.2 (compiled: Mar 20 2018 21:11:49) options: 192.168.1.2
155915 uts, 1 log, 2 kcc, 43 kmx, 2.87 kvg, 54319 krq, 176754 req, 1505 avg, 51903 rmx, 71 tav, 7056 tmx, 166285 slh, 135 slm, 0 sle, 518 slc, 9127 slu, 100 sct, 10769 sch, 71 scm, 46 scp, 11 sst, 3404 ssh, 11 ssm, 0 ssp, 161023 nfe, 24 gif, 2 ico, 1588 txt, 1 jpg, 18 png, 0 swf, 10 sta, 5 stt, 87 ufe, 76 opt, 1399 pst, 33 hed, 209 rdr, 1 nou, 1 pth, 0 204, 64 bad, 143 tmo, 520 cls, 2289 cly, 0 clt, 0 err

 

[elorimer]

Has anyone enterprising enough enabled logging to see what’s getting requested?

I think it is this: cdn-gl.imrworldwide.com
The full line is

Mar 18 09:25:59 pixelserv-tls[9055]: 192.168.0.104 cdn-gl.imrworldwide.com GET /novms/js/2/ggcmb510.js?xhr=1 HTTP/1.1 secure

That, I think, has something to do with AC-Neilson. I don't know whether it is an ad or some sort of metrics collection.
I emailed them a few days ago--not expecting to hear back.

 

[kvic]

@mstombs

I plan to add a copyright section to the manpage in the coming update.

The state before my current fork in 2015 looks right below? Did I miss any other major contributors in the past?

Copyright (C) 2013-2015 pixelserv by HunterZ@github
Copyright (C) 2009-2013 pixelserv by mstombs@github

I've prepared HISTORY and COPYRIGHT sections for addition to the man page. Tentative plan is to include it with v2.1 release.

People knowing more history than me may chime in.

 

[heysoundude]

lol. I believe SSL Labs update their list of test clients a few times per year. Usually older clients have compatibility issues. Safari 11 and latest FF are not on the list either.

You have a point. I wonder how they could test latest TLSv1.3 that its draft standard gets updated quite frequently. I guess that's why they claimed to test a quite old draft of TLSv1.3 only.

btw, all latest Chrome/FF/Safari work with v2.1 perfectly.

you know how difficult it is to test for every possible platform/permutation/combination - all becomes clear in time.
speaking of time, I'll believe you (more, implicitly) on the "perfectly" when the rc gets removed from the filename and @thelonelycoder lets us know of the upgrade

 

[kvic]

you know how difficult it is to test for every possible platform/permutation/combination - all becomes clear in time.
speaking of time, I'll believe you (more, implicitly) on the "perfectly" when the rc gets removed from the filename and @thelonelycoder lets us know of the upgrade

Thank you for your kind words. About v2.1...rc.2 is very stable and provides better performance. Won't be a wrong time to start using it.

 

[RMerlin]

You have a point. I wonder how they could test latest TLSv1.3 that its draft standard gets updated quite frequently. I guess that's why they claimed to test a quite old draft of TLSv1.3 only.

Cloudflare offers TLS 1.3 support now BTW.

 

[thelonelycoder]

With 2.1.0-rc.2 and -l 3 I suddenly see errors like these, the devices behind the IPs are Sonos speakers:

Mar 22 17:50:44 pixelserv-tls[9138]: client 192.168.2.51 ssl error:14094418:lib(20):func(148):reason(1048)
Mar 22 17:50:47 pixelserv-tls[9138]: client 192.168.2.54 ssl error:14094418:lib(20):func(148):reason(1048)

Any idea why?