pixelserv pixelserv - A Better One-pixel Webserver for Adblock

[quant88]

admin@RT-AC88U-D360:/tmp/home/root# pixelserv-tls -B                         CERT_PATH: /opt/var/cache/pixelserv
CERT_FILE: _.bing.com
 1. generate cert to disk: 544.342 ms   load from disk: 8.402 ms
 2. generate cert to disk: 641.233 ms   load from disk: 8.381 ms
 3. generate cert to disk: 314.647 ms   load from disk: 8.353 ms
 4. generate cert to disk: 333.169 ms   load from disk: 8.368 ms
 5. generate cert to disk: 426.173 ms   load from disk: 8.368 ms
 6. generate cert to disk: 415.442 ms   load from disk: 8.330 ms
 7. generate cert to disk: 413.584 ms   load from disk: 8.427 ms
 8. generate cert to disk: 440.865 ms   load from disk: 8.318 ms
 9. generate cert to disk: 422.609 ms   load from disk: 8.340 ms
10. generate cert to disk: 902.672 ms   load from disk: 8.367 ms
generate to disk average: 485.474 ms
  load from disk average: 8.365 ms
admin@RT-AC88U-D360:/tmp/home/root#

V2.1 is looking good.

 

[punkinduster]

I have the Segmentation fault also.

xxxx@RT-AC86U-91D8:/tmp/home/root# pixelserv-tls -B
CERT_PATH: /opt/var/cache/pixelserv
CERT_FILE: _.bing.com
1. generate cert to disk: 47.689 ms load from disk: 4.813 ms
2. generate cert to disk: 41.997 ms load from disk: 4.818 ms
3. generate cert to disk: 68.382 ms load from disk: 4.775 ms
4. generate cert to disk: 43.799 ms load from disk: 4.764 ms
5. generate cert to disk: 60.862 ms load from disk: 4.835 ms
6. generate cert to disk: 92.682 ms load from disk: 4.785 ms
7. generate cert to disk: 55.877 ms load from disk: 4.850 ms
8. generate cert to disk: 54.163 ms load from disk: 4.830 ms
9. generate cert to disk: 49.418 ms load from disk: 4.804 ms
10. generate cert to disk: 85.684 ms load from disk: 4.827 ms
generate to disk average: 60.055 ms
load from disk average: 4.810 ms
Segmentation fault

 

[kvic]

2.1.0 re-issued

I swear.. the downloads were intentionally made available.
The segfault bug, however, caught me by surprise.

It didn't happen on both of my devices.
But inspecting the code further, crash is expected. No crash perhaps your systems have slightly more resilient libraries or perhaps just lucky.

I've fixed the bug and re-issued 2.1.0. Please use the one-liner to install again.

edit:
armv7 will have the following timestamp. armv8 & mipsel +/- 30s perhaps.
pixelserv-tls 2.1.0 (compiled: Apr 12 2018 02:01:14)

 

[visortgw]

2.1.0 re-issued

I swear.. the downloads were intentionally made available.
The segfault bug, however, caught me by surprise.

It didn't happen on both of my devices.
But inspecting the code further, crash is expected. No crash perhaps your systems have slightly more resilient libraries or perhaps just lucky.

I've fixed the bug and re-issued 2.1.0. Please use the one-liner to install again.

"One-liner" being the same command used to install the betas?

 

[penguin22]

2.1.0 re-issued

I swear.. the downloads were intentionally made available.
The segfault bug, however, caught me by surprise.

It didn't happen on both of my devices.
But inspecting the code further, crash is expected. No crash perhaps your systems have slightly more resilient libraries or perhaps just lucky.

I've fixed the bug and re-issued 2.1.0. Please use the one-liner to install again.

Ah, the joy of time travel for us West of you :
pixelserv-tls 2.1.0 (compiled: Apr 12 2018 02:01:15 flags: tfo)
Looks good now:

CERT_PATH: /opt/var/cache/pixelserv
CERT_FILE: _.bing.com
 1. generate cert to disk: 42.923 ms    load from disk: 4.763 ms
 2. generate cert to disk: 67.512 ms    load from disk: 4.795 ms
 3. generate cert to disk: 68.637 ms    load from disk: 4.835 ms
 4. generate cert to disk: 52.101 ms    load from disk: 4.755 ms
 5. generate cert to disk: 34.323 ms    load from disk: 4.786 ms
 6. generate cert to disk: 54.220 ms    load from disk: 4.793 ms
 7. generate cert to disk: 47.275 ms    load from disk: 4.776 ms
 8. generate cert to disk: 60.634 ms    load from disk: 4.818 ms
 9. generate cert to disk: 62.794 ms    load from disk: 4.783 ms
10. generate cert to disk: 43.282 ms    load from disk: 4.784 ms
generate to disk average: 53.370 ms
  load from disk average: 4.789 ms

 

[kvic]

"One-liner" being the same command used to install the betas?

You're right. It's also mentioned on kazoo.ga/pixelserv-tls

When the new openssl lib as well as 2.1.0 available on Entware as packages. Simply proceed with upgrade.

edit:
@XIII also fyi

 

[kvic]

I just found this in the syslog.

This is stack traces on crash. It's actually good to have it in general. But for Asus, it means the platform isn't stable for them yet or else they'll have removed it like older models..

 

[punkinduster]

Thank you very much for the update. It is working just fine now.

 

[XIII]

Installed the re-issued 2.1:

t# pixelserv-tls -B
CERT_PATH: /opt/var/cache/pixelserv
CERT_FILE: _.bing.com
 1. generate cert to disk: 49.251 ms    load from disk: 4.888 ms
 2. generate cert to disk: 46.762 ms    load from disk: 4.882 ms
 3. generate cert to disk: 79.599 ms    load from disk: 4.940 ms
 4. generate cert to disk: 57.605 ms    load from disk: 4.815 ms
 5. generate cert to disk: 43.523 ms    load from disk: 4.788 ms
 6. generate cert to disk: 58.828 ms    load from disk: 4.781 ms
 7. generate cert to disk: 63.310 ms    load from disk: 4.852 ms
 8. generate cert to disk: 48.102 ms    load from disk: 4.813 ms
 9. generate cert to disk: 54.307 ms    load from disk: 4.999 ms
10. generate cert to disk: 42.392 ms    load from disk: 4.844 ms
generate to disk average: 54.368 ms
  load from disk average: 4.860 ms

No crash indeed.

However I did initially get errors like this one:

pixelserv-tls[20254]: create_child_sslctx: cannot find or use /opt/var/cache/pixelserv/_.typekit.net

I guess this is a user error?

(I deleted generated certs after updating, but did not restart pixelserv-tls after deleting those certs)

 

[Butterfly Bones]

Is that in response to benchmarking specifically or something else? I don't see anything like that in my logs.

Yes, and repeatable.

/tmp/home/root# pixelserv-tls -B
CERT_PATH: /opt/var/cache/pixelserv
CERT_FILE: _.bing.com
 1. generate cert to disk: 63.285 ms    load from disk: 5.047 ms
 2. generate cert to disk: 31.885 ms    load from disk: 4.917 ms
 3. generate cert to disk: 47.584 ms    load from disk: 4.987 ms
 4. generate cert to disk: 70.068 ms    load from disk: 4.945 ms
 5. generate cert to disk: 70.163 ms    load from disk: 4.797 ms
 6. generate cert to disk: 73.921 ms    load from disk: 4.846 ms
 7. generate cert to disk: 66.931 ms    load from disk: 5.358 ms
 8. generate cert to disk: 63.508 ms    load from disk: 4.859 ms
 9. generate cert to disk: 52.451 ms    load from disk: 4.839 ms
10. generate cert to disk: 48.086 ms    load from disk: 4.850 ms
generate to disk average: 58.788 ms
  load from disk average: 4.945 ms
Segmentation fault

Apr 11 15:20:04 kernel: pgd = ffffffc00b621000
Apr 11 15:20:04 kernel: [00000007] *pgd=000000000aed7003, *pud=000000000aed7003, *pmd=0000000000000000
Apr 11 15:20:04 kernel: CPU: 1 PID: 12101 Comm: pixelserv-tls Tainted: P           O    4.1.27 #2
Apr 11 15:20:04 kernel: Hardware name: Broadcom-v8A (DT)
Apr 11 15:20:04 kernel: task: ffffffc019326bc0 ti: ffffffc011698000 task.ti: ffffffc011698000
Apr 11 15:20:04 kernel: PC is at 0x7f7dffc344
Apr 11 15:20:04 kernel: LR is at 0x7f7dfeceb4
Apr 11 15:20:04 kernel: pc : [<0000007f7dffc344>] lr : [<0000007f7dfeceb4>] pstate: 00000000
Apr 11 15:20:04 kernel: sp : 0000007fc4591ec0
Apr 11 15:20:04 kernel: x29: 0000007fc45920f0 x28: 0000007f7dff1d70
Apr 11 15:20:04 kernel: x27: 0000007f7dff1d90 x26: 0000000000000003
Apr 11 15:20:04 kernel: x25: 0000007f7e0a1dd0 x24: 0000007f7e0a1da8
Apr 11 15:20:04 kernel: x23: 0000007f7e0a7000 x22: 0000000000000000
Apr 11 15:20:04 kernel: x21: 0000007f7e0a1dd0 x20: 0000007f7e0a7000
Apr 11 15:20:04 kernel: x19: ffffffffffffffff x18: 593643958ea9b96c
Apr 11 15:20:04 kernel: x17: 0000007f7df55080 x16: 000000000000003f
Apr 11 15:20:04 kernel: x15: 0000000000007fff x14: 0000000000000000
Apr 11 15:20:04 kernel: x13: 203a656761726576 x12: 0000000100000000
Apr 11 15:20:04 kernel: x11: 0000007fc4591748 x10: 0000000100000000
Apr 11 15:20:04 kernel: x9 : 0000000000000000 x8 : 0000000000000040
Apr 11 15:20:04 kernel: x7 : 000000000a736d20 x6 : 0000000000000001
Apr 11 15:20:04 kernel: x5 : 0000000000000002 x4 : 0000007f7dff1aa0
Apr 11 15:20:04 kernel: x3 : 0000000000000000 x2 : 0000007f7e09cb08
Apr 11 15:20:04 kernel: x1 : 0000000000000003 x0 : ffffffffffffffff

edit - ok fixed now, thank you. I was away at a long session with the dentist.
Re-installed and all smooth, no segfault in benchmark or AC86U syslog. :thumbsup:

 

[Makaveli]

Does anyone use twitch?

Can you view a twitch stream both in a browser and with the twitch app and monitor your slc?

 

[penguin22]

Just giving you a 22-hour update on use with 2.1.0; everything looks great with mem sitting at 1.5%:

pixelserv-tls 2.1.0 (compiled: Apr 12 2018 02:01:15 flags: tfo) options: 192.168.50.4

uts 0d 22:18 process uptime
log 1 critical (0) error (1) warning (2) notice (3) info (4) debug (5)
kcc 13 number of active service threads
kmx 37 maximum number of service threads
kvg 1.19 average number of requests per service thread
krq 12748 max number of requests by one service thread

req 162452 total # of requests (HTTP, HTTPS, success, failure etc)
avg 1778 bytes average size of requests
rmx 609949 bytes largest size of request(s)
tav 26 ms average processing time (per request)
tmx 7699 ms longest processing time (per request)

slh 152359 # of accepted HTTPS requests
slm 8 # of rejected HTTPS requests (missing certificate)
sle 0 # of rejected HTTPS requests (certificate available but bad)
slc 1909 # of dropped HTTPS requests (client disconnect without sending any request)
slu 6278 # of dropped HTTPS requests (other TLS handshake errors)
uca 20 slu break-down: # of unknown CA reported by clients
uce 2165 slu break-down: # of unknown cert reported by clients

sct 50 cert cache: # of certs in cache
sch 9467 cert cache: # of reuses of cached certs
scm 163 cert cache: # of misses to find a cert in cache
scp 150 cert cache: # of purges to give room for a new cert
sst 0 sess cache: # of cached TLS sessions (for older non-RFC5077 clients)
ssh 1062 sess cache: # of reuses of cached TLS sessions
ssm 218 sess cache: # of misses to find a TLS session in cache
ssp 0 sess cache: # of purges to give room for a new TLS session

nfe 146917 # of GET requests for server-side scripting
gif 8 # of GET requests for GIF
ico 1 # of GET requests for ICO
txt 238 # of GET requests for Javascripts
jpg 50 # of GET requests for JPG
png 2 # of GET requests for PNG
swf 0 # of GET requests for SWF
sta 16 # of GET requests for HTML stats
stt 0 # of GET requests for plain text stats
ufe 95 # of GET requests /w unknown file extension

opt 129 # of OPTIONS requests
pst 1943 # of POST requests
hed 0 # of HEAD requests (HTTP 501 response)
rdr 46 # of GET requests resulted in REDIRECT response
nou 0 # of GET requests /w empty URL
pth 6 # of GET requests /w malformed URL
204 0 # of GET requests (HTTP 204 response)
bad 1715 # of unknown HTTP requests (HTTP 501 response)

tmo 49 # of timeout requests (client connect w/o sending a request in 'select_timeout' secs)
cls 3544 # of dropped requests (client disconnect without sending any request)
cly 1408 # of dropped requests (client disconnect before response sent)
clt 0 # of dropped requests (reached maximum service threads)
err 0 # of dropped requests (unknown reason)

 

[kvic]

However I did initially get errors like this one:

pixelserv-tls[20254]: create_child_sslctx: cannot find or use /opt/var/cache/pixelserv/_.typekit.net

I guess this is a user error?

(I deleted generated certs after updating, but did not restart pixelserv-tls after deleting those certs)

"cannot find or use <a cert file>" mostly relates to file/directory permission issue.

chown -R nobody /opt/var/cache/pixelserv

could always resolve the problem without deleting &re-generating the certs.

However, I don't know how the permissions got changed from time to time. I'm sure it's not by pixelserv-tls nor in its control.

Can you view a twitch stream both in a browser and with the twitch app and monitor your slc?

slc is less worrying. A slc implies TLS handshakes already passed. Somehow clients decide to disconnect rather than send in a request.

A common cause is people browse away from an existing page. Clients simply abandon previous to-be-sent requests.

A less common one could be apps implement certain application level protocol, if both ends do not talk in the same protocol, clients refuse to send data. We can't do anything in such scenario.

 

[kvic]

Just giving you a 22-hour update on use with 2.1.0; everything looks great with mem sitting at 1.5%:

@penguin22's servstats is good for educational purpose. He has a busy LAN. If people enjoy network admin, that's certainly the home to be.

In v2.1, default "-c" is set to 50 which is not ideal for a busy LAN. People can usually tell from sct and scp. After running for a few days, both counters shall stabilise. Then, a better "-c" value to set will be equal or greater than sct + scp.

So that most of your commonly used certs will be cached in memory. That saves 5ms on RT86 and 11ms on RT56/68. With the new openssl lib, setting "-c" to a high value is of little concern, and doesn't use noticeably more memory. Each cached cert and its associated data structure uses about 2KB extra memory.

avg and rmx - we talked about these a few pages back.

avg looks right here. rmx is interesting. It indicates the maximum size of requests (could be a GET or a POST for example). ~600KB shall catch people's attention.

600KB is certainly going to be a POST request and its content is 600KB. I would make a guess that it's all sort of privacy data (critical or not is another issue) about the user.

A few years ago, people were talking about 40KB requests being huge and of concern. Nowadays, advert/tracker companies are just getting crazier.

Since you have the advert/tracker domains blocked, the 600KB data isn't actually got sent to the real receiver. pixelserv-tls just helps to expose such attempts of privacy breaches.

If you're curious, turn on log LEVEL 4 and inspect what's the actual data. 600KB fills up a few full screens and shall be easy to spot.

A final reminder, if you're not running syslog-ng from Entware, you won't want to leave log LEVEL 4 full time. It'll choke your syslog.log in no time.

 

[penguin22]

@penguin22's servstats is good for educational purpose. He has a busy LAN. If people enjoy network admin, that's certainly the home to be.

In v2.1, default "-c" is set to 50 which is not ideal for a busy LAN. People can usually tell from sct and scp. After running for a few days, both counters shall stabilise. Then, a better "-c" value to set will be equal or greater than sct + scp.

So that most of your commonly used certs will be cached in memory. That saves 5ms on RT86 and 11ms on RT56/68. With the new openssl lib, setting "-c" to a high value is of little concern, and doesn't use noticeably more memory. Each cached cert and its associated data structure uses about 2KB extra memory.

avg and rmx - we talked about these a few pages back.

avg looks right here. rmx is interesting. It indicates the maximum size of requests (could be a GET or a POST for example). ~600KB shall catch people's attention.

600KB is certainly going to be a POST request and its content is 600KB. I would make a guess that it's all sort of privacy data (critical or not is another issue) about the user.

A few years ago, people were talking about 40KB requests being huge and of concern. Nowadays, advert/tracker companies are just getting crazier.

Since you have the advert/tracker domains blocked, the 600KB data isn't actually got sent to the real receiver. pixelserv-tls just helps to expose such attempts of privacy breaches.

If you're curious, turn on log LEVEL 4 and inspect what's the actual data. 600KB fills up a few full screens and shall be easy to spot.

A final reminder, if you're not running syslog-ng from Entware, you won't want to leave log LEVEL 4 full time. It'll choke your syslog.log in no time.

I'm happy to be a study case for the good, bad, and ugly. Interestingly, this capture is not a good indicator of normal LAN use, but rather kids playing some games with discord and other streaming services over a 2-hour period. In the following 24 hour span from the logs above, I am now at req 174725 compared to the 162452 previously. I think that it would be necessary to have a larger sample size, but your log level 4 recommendation with syslog-ng will be my next foray. Using TCPView, I was able to see the process with significant post data and it was what I expected from their use, nothing nefarious at least, and the blocking of it is consistent with their complaint of what we block on our home network .

Considering the much smaller req value over time, using ~20000 over a 24-hour period, would you still think that an adjusted '-c' value from the default configuration is of value? Is there a guide to adjusting the variables for Pixelserv-tls or is it generally best to leave the defaults unless markers otherwise indicate?

 

[kvic]

I'm happy to be a study case for the good, bad, and ugly.

your log level 4 recommendation with syslog-ng will be my next foray.

It's not only fun but also have a much more capable syslog for everyday operation. If I recall correctly, you have a RT86. There is an issue that /tmp/syslog.log as a symbolic link gets misteriously truncated after half hour or so in operation according to @Butterfly Bones. If this issue could be understood and fleshed out, I guess more RT86 users could benefit.

Considering the much smaller req value over time, using ~20000 over a 24-hour period, would you still think that an adjusted '-c' value from the default configuration is of value?

The value for '-c' has more to do with people's browsing habit. After a month or two in operations, the number of certs in CERT_PATH shall stabilise.

There are not that many global advert/tracker networks. These days all are dominated by big players. Regionally there will be some variations on smaller players. But overall, given a fixed set of users and browsing habit, the number of certs in practical use are limited.

For example, in my environment, I only got 300 or so. Hence, I put 250 for '-c'. At the moment, I've only used up 160 slots.

Is there a guide to adjusting the variables for Pixelserv-tls or is it generally best to leave the defaults unless markers otherwise indicate?

The other defaults are pretty good. The default for '-c' was 100 but reduced to 50 in rc.4 and stays like that in v2.1. It's to better accommodate users not upgrading to new openssl lib in the short term. Just being responsibly use of memory not that I would think people's router will crash because pixelserv-tls uses 50MB.

 

[Butterfly Bones]

I think that it would be necessary to have a larger sample size, but your log level 4 recommendation with syslog-ng will be my next foray.

It's not only fun but also have a much more capable syslog for everyday operation. If I recall correctly, you have a RT86. There is an issue that /tmp/syslog.log as a symbolic link gets misteriously truncated after half hour or so in operation according to @Butterfly Bones. If this issue could be understood and fleshed out, I guess more RT86 users could benefit.

I look forward to a RT-AC86U owner getting syslog-ng running. I've locked up my router four times now requiring a hard reset and reconfigure, and no new ideas why. I've searched until my brain turns to jello with no results. Fingers crossed here.

 

[Davidncali001]

Hi, I'm running the latest version of Pixelserver and was trying to import a certificate from PixelserverIP/ca.crt but it gives me a 404 error. Can you let me know what I may have done wrong.

 

[Butterfly Bones]

Hi, I'm running the latest version of Pixelserver and was trying to import a certificate from PixelserverIP/ca.crt but it gives me a 404 error. Can you let me know what I may have done wrong.

Are you sure you are using the correct numerical IP used for your pixelserv-tls installation? Mine looks like this.

https://192.168.1.2/ca.crt

 

[jrmwvu04]

Hi, I'm running the latest version of Pixelserver and was trying to import a certificate from PixelserverIP/ca.crt but it gives me a 404 error. Can you let me know what I may have done wrong.

I just came here to ask this very question. This worked in the rc builds and in the build I'm running right now (pixelserv-tls 2.1.0 compiled: Apr 12 2018 02:01:13) it doesn't work, I'm getting the 404 error as well. Anyone else? @kvic