pixelserv pixelserv - A Better One-pixel Webserver for Adblock

[froggy666uk]

The build I was using on my n66u was v2.0.0-rc1, my raspberry pi build was based on the github master branch. Have your latest v2.0.1-rc1 changes been committed to github master?

Could it be a problem peculiar to android devices? Have you tested using android?

Anything else I could look at on my raspberry pi that might help next time it happens?

The build I was using on my n66u was v2.0.0-rc1, my raspberry pi build was based on the github master branch. Have your latest v2.0.1-rc1 changes been committed to github master?

Could it be a problem peculiar to android devices? Have you tested using android?

Anything else I could look at on my raspberry pi that might help next time it happens?

Would it be beneficial to run the pixelserv binary without being stripped? Would it offer any more insight into what is going on using gdb?

 

[kvic]

@froggy666uk

Hmm..maybe you can turn on core dump on Rasp Pi. When pixelserv-tls gets stuck, after the usual strace/lsof, "killall -SIGSEGV pixelserv-tls". This will get the core dump and we may analyse from there.

I think I know the place of stall since the main event loop only has a single place of read(). Good to get a confirmation.

 

[kvic]

I got another set of traces from @quant88 about an hour ago. Very similar to those from @Protik and @froggy666uk. So the issue lies in the main event loop. Seems this is certain for now..

Looking at the original pixelserv, I found a separate issue. If more than one file/socket descriptors are set in a single select() return, none of the descriptors will be handled except the very first one. Then it enters select() again which will return immediately. Repeat until all descriptors are handled that are set in the very first select() return. This could happen on a busy system e.g. many clients access pixelserv at the same time.

This is inefficient at least but I'm not sure it introduces other complications. So perhaps I shall straighten this up first.

 

[kvic]

Uploaded v2.0.1-rc2

Changes:

• changed main event loop to be correct & more efficient
• fixed a memory leak on fail to create service thread for HTTPS connection
• default select_timeout to 10s

@quant88 @baltosml @Protik @tom-

Can you try this one if "stuck" issue is resolved?

@froggy666uk I also uploaded source change. You may get from HEAD for latest.

I did a quick&dirty test against v2 benchmark. The speed improvement is NON-trivial against v2.0.0-rc1.

 

[kvic]

Please install v2.0.1-rc2 again if you did it before this post.

The order of FD_CLR() is incorrect in the build of #866. The latest should have timestamp ("compiled: Dec 8 2017 01:13:34").

Please use this build to test "stuck" issue. Sorry for the hassle.

 

[Protik]

Uploaded v2.0.1-rc2

Changes:

• changed main event loop to be correct & more efficient
• fixed a memory leak on fail to create service thread for HTTPS connection
• default select_timeout to 10s

@quant88 @baltosml @Protik @tom-

Can you try this one if "stuck" issue is resolved?

@froggy666uk I also uploaded source change. You may get from HEAD for latest.

I did a quick&dirty test against v2 benchmark. The speed improvement is NON-trivial against v2.0.0-rc1.

I am testing it right now. Will let you know of any development. Thanks!

 

[kvic]

I am testing it right now. Will let you know of any development. Thanks!

Thank you. Pls make sure your version has timestamp "compiled: Dec 8 2017 01:13:34"

 

[thelonelycoder]

Maybe we can add this to amtm ? @thelonelycoder ?

Let me think about that, it's not exactly fitting into the menu structure I have built.
Your request is added to the list.

 

[jrmwvu04]

Please use this build to test "stuck" issue. Sorry for the hassle.

I don't have the stuck issue, but updated nonetheless. One thought because it's the only flag I'm running.. is there any chance that running as admin is avoiding the problem? I might change it back to nobody to see...

 

[kvic]

I don't have the stuck issue, but updated nonetheless. One thought because it's the only flag I'm running.. is there any chance that running as admin is avoiding the problem? I might change it back to nobody to see...

Very unlikely. I run as default which is "nobody."

Also in my stress test, 800 clients make requests to pixelserv-tls at the same time - that's lot of concurrency! But I fail to reproduce the "stuck" issue.

• changed main event loop to be correct & more efficient

The change in 2.0.1-rc2 above refers to how the result of select() system call in original pixelserv is not processed correctly nor consistent with code comments. This has potential to create weird issues.

If I've to bet the cause of "stuck"...it's this incorrec select() handling + many concurrent client requests + the Linux kernel where pixelserv-tls runs on.

But I'll wait for longer to assert that as in the next minute a report might pop up and says 2.0.1-rc2 still stalls..

 

[kvic]

Maybe we can add this to amtm ? @thelonelycoder ?

Let me think about that, it's not exactly fitting into the menu structure I have built.
Your request is added to the list.

I would suggest otherwise. "killall -SIGUSR1 pixelserv-tls" could make it into a sub-menu item under PS inside AB-SOLUTION.

This logically fits there and could be used as a troubleshooting tool for pixelserv-tls in general.

If fancy a bit more, a simple script parser could be used to get the output from '.../servstats.txt', parse it and add a bit text description to each counter (e.g like the HTML version's descriptions). This could be another sub-menu item. That'll be a nice integration of pixelserv-tls inside AB-SOLUTION.

Let's fancy a bit more.. ABS can also add another sub-menu item to change LOG LEVEL on the fly with "http://<pixelserv ip>/log=<LEVEL>". This way no restart of pixelserv-tls is required. Good for troubleshooting, inspecting privacy breach, and spotting what ad domains to whitelist.

 

[Protik]

Running for almost 13 hours. So far the stats are,

pixelserv-tls: v2.0.1-rc2 compiled: Dec 8 2017 01:13:34 options: 192.168.2.2 -l 5

uts    0d 12:51    process uptime
log    5    critical (0) error (1) warning (2) notice (3) info (4) debug (5)
kcc    2    number of active service threads
kmx    28    maximum number of service threads
kvg    1.93    average number of requests per service thread
krq    26    max number of requests by one service thread
req    2258    total # of requests (HTTP, HTTPS, success, failure etc)
avg    770 bytes    average size of requests
rmx    8960 bytes    largest size of request(s)
tav    156 ms    average processing time (per request)
tmx    290719 ms    longest processing time (per request)
slh    539    # of accepted HTTPS requests
slm    2    # of rejected HTTPS requests (missing certificate)
sle    0    # of rejected HTTPS requests (certificate available but bad)
slc    450    # of dropped HTTPS requests (client disconnect without sending any request)
slu    17    # of dropped HTTPS requests (unknown error)

Only that tav and tmx seems much higher compared to the earlier version.

 

[kvic]

@Protik 'tav' can vary a lot from different sessions of pixelserv-tls. More depends on the websites you visit during the sessions. Your 'tmx' does look very long about 30s. Seems to me you may have visited a website that's challenging to pixelserv-tls

@rromeroa Does your pixelserv-tls still crash? I got a report from @quant88 that v2.0.1-rc2 crashed on his routers.

 

[rromeroa]

@Protik 'tav' can vary a lot from different sessions of pixelserv-tls. More depends on the websites you visit during the sessions. Your 'tmx' does look very long about 30s. Seems to me you may have visited a website that's challenging to pixelserv-tls

@rromeroa Does your pixelserv-tls still crash? I got a report from @quant88 that v2.0.1-rc2 crashed on his routers.

Hi @kvic,

I was testing the latest versions with the same results. Crashing just opening tapatalk....
I really want to try the v2 improvements... But it is not being possible [emoji24]

 

[kvic]

That's sad to hear...For any consistently reproducible crash, running the debug build of pixelserv-tls perhaps can easily figure out the cause. I started a conversation with @rromeroa to offer the debug build.

Anyone with a reproducible crash..if you help to run the debug build, we might be able to resolve it with a proper fix.

 

[thelonelycoder]

Let's fancy a bit more.. ABS can also add another sub-menu item to change LOG LEVEL on the fly with "http://<pixelserv ip>/log=<LEVEL>". This way no restart of pixelserv-tls is required. Good for troubleshooting, inspecting privacy breach, and spotting what ad domains to whitelist.

Sorry for being ignorant, but that does not work for me.
If opened in browser= blank page
If I wget http://<pixelserv ip>/log=<LEVEL> I get an empty file in $HOME
Enlighten me please.

 

[kvic]

@thelonelycoder

that's expected behaviour in current versions. The request always succeeds. If you check servstats page. 'log' will be equal to the new level. If you check syslog, new log level takes effect immediately.

For 'wget', you can try 'wget -O /dev/null <URL>' to sink any output to black hole.

 

[thelonelycoder]

@thelonelycoder

that's expected behaviour in current versions. The request always succeeds. If you check servstats page. 'log' will be equal to the new level. If you check syslog, new log level takes effect immediately.

For 'wget', you can try 'wget -O /dev/null <URL>' to sink any output to black hole.

No wonder, I was looking at the command line switch to check. Works as intended, thanks!

 

[pattiri]

@kvic @thelonelycoder

I can crash pixelserv-tls whenever I want with the following;

-from AB-Solution go to "ps" and "Add or remove pixelserv switches" and commit. (no change is needed) after that whenever I close the putty, pixelserv crashes. pinging is OK but can't access web stats.

to fix this I need to run "pixelserv-tls <listening-IP>

additional info;

These switches are in use (none if none listed):
1: -f
2: -l 4
3: -o 15

I can provide you team viewer session or anyting else if it is OK for you.

 

[kvic]

@pattiri I added you to the 'crash conversation'.

So far the debug build is very productive.

For @rromeroa's crash issue, I created a issue tracker #7. It'll be fixed in next RC test..