pixelserv pixelserv - A Better One-pixel Webserver for Adblock

[quant88]

I encountered similar problems where the pages at stuck and not fully loaded. However, the pages are fully loaded when AP isolated are set to false.

Hi,

I am running Pixelserv-tls for ad-blocking and followed this guide for how best to set it up on Merlin:

https://github.com/kvic-z/pixelserv-tls/wiki/How-to-best-run-pixelserv-tls-on-Asuswrt-Merlin

It works great on my normal network. However, on my guest WiFi, I can not access 192.168.2.3, even though my device on the guest WiFi is assigned a 192.168.2.X address. The connection just hangs forever. This renders my guest WiFi essentially useless as any web page with an ad never finishes loading and rendering.

Does anyone know how to fix this? Is this something in iptables?

Environment - Asus AC68U running Merlin 380.63_2, subnet: 192.168.2.xxx

Thanks for any help or suggestions.

 

[JaimeZX]

Well that’s about 94% of your requests. At only 40 clean serves out of 13k something is amiss.

Yeah, I agree with you there. But how would I get even 40?? Either an encryption works or it doesn't....

 

[Butterfly Bones]

Yeah, I agree with you there. But how would I get even 40?? Either an encryption works or it doesn't....

kvic explained it here, note the bold type near the bottom.
https://www.snbforums.com/threads/p...bserver-for-adblock.26114/page-85#post-395904
Post 1712 give you steps to control these.
https://www.snbforums.com/threads/p...bserver-for-adblock.26114/page-86#post-396073

 

[kvic]

But how would I get even 40?? Either an encryption works or it doesn't....

I believe you're still on rc.3. Upgrade to rc.4 that will tell you what your slu is ..be it mostly uca (unknown CA) or uce (unknown cert).

Once confirmed, then go through the two posts linked by @Butterfly Bones. That help to suppress known servers.

99% of my slu comes from one server. After I suppress it, my slu is very clean now.

 

[kvic]

It works great on my normal network. However, on my guest WiFi, I can not access 192.168.2.3, even though my device on the guest WiFi is assigned a 192.168.2.X address. The connection just hangs forever.

A connection hanging there forever indicates that clients on your guests network cannot reach 192.168.2.3 (assume that's your pixelserv ip) at all.

I haven't looked into guest networks in detail. Hence, cannot quickly tell how guest/normal network segregation is done.

I assume your guest network can access the DNS server on your router. If so, that will be a good hint how it's provisioned to guest network. Very likely it's a simple iptables rule.

Once confirmed, you could simply duplicate the iptables rule for IP address 192.168.2.3. That I believe shall solve your issue. If still can't please post more details, such as your DNS ip, iptables output...

 

[kvic]

i only saw the prompt once i've done 5 reboots of the phone since and it never came back.

I believe it only came up because I never rebooted the first time I did the cert import.

I'm thinking when the prompt doesn't appear on reboot, does it mean the system accepts your Pixelserv CA or denies it?

If you look at slu, uca and uce, numbers (and % compared to req), I guess we can quickly make a judgement what's the case.

If it's the case that your phone only requires you (perhaps due to the BES IT policy or other vendor customization) to consent to your Pixelserv CA once on reboot, then very good news for you.

 

[kvic]

I plan to try the "two instances" later. I'm needing to get syslog-ng working first (curse this AC86U ) with the updates in pixelserv-tls and now Skynet, my logs need sorting and management because they are getting purged too often. I get it working, but see nothing in the router syslog after a few minutes. Searching another report method that I can view from my Linux desktop.......

My two instances in load balance mode work very well. Looks like the first field report on "two instances in primary/secondary mode" will be from you..?

I did some further work to the OpenSSL library in EdgeRouter X as first mentioned in this post: https://www.snbforums.com/threads/optimised-openssl-library-for-edgerouter-x.44924/

Now pixelserv-tls memory footprint is lightly less on ER-X than RT-AC56. Previously it was way more on ER-X than on RT-AC56. I plan to update and publish the OpenSSL deb on another day on this blog post: https://kazoo.ga/optimised-openssl-library-erx/

Regarding syslog-ng, I recommend every serious users to give it a try. The vanilla syslog come with ASUSWRT is insufficient in handling more than bare minimum logging.

With syslog-ng, people can filter loggings per application, service or whatever way you like including filtering out logs you don't want to see.

So @Asad Ali, syslog-ng is the way for you in the longer term if you don't want to see some logs.

Also a recap on the pixelserv-tls logging on "...missing cert...xxxx", whenever a cert is missing, you'll see this in log LEVEL 2.

After a month or so into running pixelserv-tls, you will rarely see such loggs because most of your ad server/tracker certificates have been generated already, especially pixelserv-tls from day one using a technique known as wildcard certs, meaning one cert can serve multiple sub-domains.

Hence, to re-iterate myself, auto cert generations are *rare* events in pixelserv-tls. No need to go out and buy a new and shiny router just for it. pixelserv-tls is designed to work very well on old hardware. That's the way should be for quality software (?)

 

[Raphie]

Htop 0.5

pixelserv-tls 2.1.0-rc.4 (compiled: Apr 5 2018 21:02:08) options: 192.168.1.2
220054 uts, 1 log, 1 kcc, 35 kmx, 1.12 kvg, 8299 krq, 52957 req, 1799 avg, 833509 rmx, 34 tav, 7436 tmx, 26791 slh, 45 slm, 0 sle, 4700 slc, 20580 slu, 2514 uca, 0 uce, 50 sct, 17920 sch, 258 scm, 245 scp, 2 sst, 4444 ssh, 59 ssm, 0 ssp, 7570 nfe, 16 gif, 0 ico, 11388 txt, 6 jpg, 11 png, 0 swf, 4 sta, 1 stt, 245 ufe, 91 opt, 2182 pst, 258 hed, 3234 rdr, 2 nou, 0 pth, 0 204, 269 bad, 68 tmo, 4700 cls, 2288 cly, 0 clt, 0 err

 

[quant88]

With 2.1.0-rc.4 and updated libopenssl and -c 300.
After 30hrs with AC88U, MEM%=1.1

108658 uts, 2 log, 5 kcc, 51 kmx, 1.31 kvg, 128 krq, 8943 req, 1431 avg, 17202 rmx, 30 tav, 3805 tmx, 1654 slh, 37 slm, 0 sle, 3673 slc, 902 slu, 0 uca, 281 uce, 277 sct, 5038 sch, 52 scm, 0 scp, 7 sst, 188 ssh, 744 ssm, 0 ssp, 1110 nfe, 15 gif, 23 ico, 546 txt, 5 jpg, 21 png, 0 swf, 26 sta, 0 stt, 203 ufe, 2 opt, 1342 pst, 0 hed, 410 rdr, 0 nou, 0 pth, 0 204, 253 bad, 284 tmo, 3764 cls, 0 cly, 0 clt, 0 err

 

[Butterfly Bones]

My two instances in load balance mode work very well. Looks like the first field report on "two instances in primary/secondary mode" will be from you..?

Quite likely, hoping that AC86U changes don't sabotage that effort when I'm ready to tackle it.

Regarding syslog-ng, I recommend every serious users to give it a try. The vanilla syslog come with ASUSWRT is insufficient in handling more than bare minimum logging.

Trying to get syslog-ng working I managed to lock up the router entirely yesterday after many tries to get it working. I finally had to hard reset and restore everything (I always have backups of everything from the router I can possible save beforehand). I just lost four or five hours of time and gained too much frustration.

As much as I want to run syslog-ng, the clues are that something changed in the AC86U that causes issues. I've found only two posts on SNB from others stating as such, and scouring the 'Net with a couple search engines turns up nothing, sadly. So that project is in a time out.

 

[kvic]

Quite likely, hoping that AC86U changes don't sabotage that effort when I'm ready to tackle it.

Perhaps lower this priority a bit then. I've been thinking of a suppression feature in v2.2 or later. It's has to be intuitive to use, and super fast to execute.

On the other hand, also not much incentive to add it, considering how little (one server entry only!) I've to suppress, and lack of users' feedback regarding this. Maybe we need more time to flourish before a plan need to be drafted up.

Trying to get syslog-ng working I managed to lock up the router entirely yesterday after many tries to get it working. I finally had to hard reset and restore everything (I always have backups of everything from the router I can possible save beforehand). I just lost four or five hours of time and gained too much frustration.

RT86 is the first router on 4.x kernel. Perhaps still worth tinkering. With every frustration, you get stronger

 

[Butterfly Bones]

Perhaps lower this priority a bit then. I've been thinking of a suppression feature in v2.2 or later. It's has to be intuitive to use, and super fast to execute.

On the other hand, also not much incentive to add it, considering how little (one server entry only!) I've to suppress, and lack of users' feedback regarding this. Maybe we need more time to flourish before a plan need to be drafted up.



RT86 is the first router on 4.x kernel. Perhaps still worth tinkering. With every frustration, you get stronger

As you know, I'm using the "fallback approach" to suppress the handshake failures on my Android devices. I've added 23 items to the hosts.add file and have not seen a new entry now for nearly 24 hours. A more straight forward suppression technique might be better, as there are a number of us here experiencing that, though it only shows up with level 2 logging.

I'm watching the accumulation of certs in the pixelserv directory and noticed that many are there that I see in the handshake failures logged. I'm going to experiment and see if I can remove those domains from the hosts.add and see the results. Wondering if force stopping them has generated the certs. It appears that the hosts that show up are only accessed via mobile apps, and not from my Linux desktop or chromebook. Tinkering.

I'll come back to syslog-ng. Trying to find more info first, since I'm at a dead end for ideas.

 

[punkinduster]

I want to try the two server config and send my Roku certificate errors over to the other server. I'm a noob and may need a little hand holding. I am using amtm, AB-Solution, Skynet, and pixelserv-tls. I'm not familiar with Entware. So, if I can do it any one should be able to set it up with a little guidance. I first installed the amtm script and then AB-Solution. I then had AB-Solution install Entware and pixelserv-tls. Then installed Skynet.

Looking around I found the script "/jffs/scripts/wan-start" and also "/opt/etc/init.d/S80pixelserv-tls" that was generated/modified by AB-Solution to start pixelserv-tls.

Question is where or which file do I modify to launch an additional pixelserv on a different IP?

 

[punkinduster]

Here is my latest.

 

[Jack Yaz]

How bad are these numbers? Running rc.4 with rebuilt openssl lib.

pixelserv-tls 2.1.0-rc.4 (compiled: Apr 5 2018 21:02:08) options: 10.14.16.3 -c 250

uts    3d 22:54    process uptime
log    1    critical (0) error (1) warning (2) notice (3) info (4) debug (5)
kcc    1    number of active service threads
kmx    60    maximum number of service threads
kvg    1.48    average number of requests per service thread
krq    1228    max number of requests by one service thread
req    50750    total # of requests (HTTP, HTTPS, success, failure etc)
avg    14538 bytes    average size of requests
rmx    971804 bytes    largest size of request(s)
tav    4 ms    average processing time (per request)
tmx    4445 ms    longest processing time (per request)
slh    36695    # of accepted HTTPS requests
slm    4    # of rejected HTTPS requests (missing certificate)
sle    0    # of rejected HTTPS requests (certificate available but bad)
slc    11669    # of dropped HTTPS requests (client disconnect without sending any request)
slu    872    # of dropped HTTPS requests (other TLS handshake errors)
uca    0    slu break-down: # of unknown CA reported by clients
uce    722    slu break-down: # of unknown cert reported by clients
sct    189    cert cache: # of certs in cache
sch    14333    cert cache: # of reuses of cached certs
scm    2    cert cache: # of misses to find a cert in cache
scp    0    cert cache: # of purges to give room for a new cert
sst    0    sess cache: # of cached TLS sessions (for older non-RFC5077 clients)
ssh    1760    sess cache: # of reuses of cached TLS sessions
ssm    221    sess cache: # of misses to find a TLS session in cache
ssp    0    sess cache: # of purges to give room for a new TLS session
nfe    3449    # of GET requests for server-side scripting
gif    11    # of GET requests for GIF
ico    3    # of GET requests for ICO
txt    18035    # of GET requests for Javascripts
jpg    0    # of GET requests for JPG
png    0    # of GET requests for PNG
swf    0    # of GET requests for SWF
sta    2    # of GET requests for HTML stats
stt    1    # of GET requests for plain text stats
ufe    162    # of GET requests /w unknown file extension
opt    0    # of OPTIONS requests
pst    964    # of POST requests
hed    0    # of HEAD requests (HTTP 501 response)
rdr    15302    # of GET requests resulted in REDIRECT response
nou    0    # of GET requests /w empty URL
pth    2    # of GET requests /w malformed URL
204    0    # of GET requests (HTTP 204 response)
bad    2    # of unknown HTTP requests (HTTP 501 response)
tmo    256    # of timeout requests (client connect w/o sending a request in 'select_timeout' secs)
cls    11686    # of dropped requests (client disconnect without sending any request)
cly    0    # of dropped requests (client disconnect before response sent)
clt    0    # of dropped requests (reached maximum service threads)
err    0    # of dropped requests (unknown reason)

 

[kvic]

As you know, I'm using the "fallback approach" to suppress the handshake failures on my Android devices. I've added 23 items to the hosts.add file and have not seen a new entry now for nearly 24 hours. A more straight forward suppression technique might be better, as there are a number of us here experiencing that, though it only shows up with level 2 logging.

The downside of this approach (and two instances in primary/secondary mode) is that suppression is solely based on server names.

A better criterion will be (client ip, server name) tuple that is only doable from within pixelserv-tls with a new feature implemented. This will be more precise on what to suppress. Other clients accessing the same server can still go though.

However, little impact when the number of servers suppressed are small. I think 23 servers are okay.

 

[kvic]

I want to try the two server config and send my Roku certificate errors over to the other server.

Question is where or which file do I modify to launch an additional pixelserv on a different IP?

1. Create a new file named "/opt/etc/init.d/S80pixelserv-tls.n2"
2. Copy & paste the below content. Edit pixelserv_ip and pixelserv_opt to suit your need.

Spoiler: S80pixelserv-tls.n2

#!/bin/sh

pixelserv_ip=192.168.2.6
pixelserv_opt='-c 200 -O 300 -l 1'

# NOTHING REQUIRES CHANGES BELOW IF YOU ARE NOT FAMILIAR

ENABLED=yes
PROCS=pixelserv-tls.n2
ARGS="$pixelserv_ip $pixelserv_opt"
PREARGS=""
DESC=$PROCS
PATH=/opt/sbin:/opt/bin:/opt/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

[ "$1" = start -a "$(ifconfig br0:pixelserv2 | grep -c inet)" = 0 ] && ifconfig br0:pixelserv2 $pixelserv_ip
[ "$1" = stop -a "$(ifconfig br0:pixelserv2 | grep -c inet)" = 1 ] && ifconfig br0:pixelserv2 down

. /opt/etc/init.d/rc.func

3. Issue "chmod a+x /opt/etc/init.d/S80pixelserv-tls.n2"
4. Issue "ln -s /opt/bin/pixelserv-tls /opt/bin/pixelserv-tls.n2"
5. Reboot will start second instance automatically. Or you can manually start it by issuing "/opt/etc/init.d/S80pixelserv-tls.n2 start"

Now the rest is to add your server names to ABS' whitelist. Then add the same servers to /jffs/configs/hosts.add. Use your 2nd pixelserv ip instead of 0.0.0.0

 

[kvic]

How bad are these numbers? Running rc.4 with rebuilt openssl lib.

You meant how good, right? Most number are good actually.

A bit worrying are avg and rmx. Both aren't small. It could be due to you're visiting lots of ad heavy websites. The requests URLs are really long. On average is 14KB long(!!).

The rmx is alarmingly large, > 900KB. I suspect some websites were attempting to upload lots of data about you. You can inspect what websites and the actual data by turning on log LEVEL 4. And monitor.

edit:

btw, how much memory does it consume?

 

[jrmwvu04]

A better criterion will be (client ip, server name) tuple that is only doable from within pixelserv-tls with a new feature implemented. This will be more precise on what to suppress. Other clients accessing the same server can still go though.

I will appreciate that. I have set up syslog-ng and set up a filter to send pixelserv logging to a separate file. Really handy to leave -l 2 or full time. Anyway through that I’ve discovered that a lot of my uca errors come from an Xbox One that go to a wide variety of common ad servers that I’d prefer not redirect to 0.0.0.0 for all clients. It would, I think, largely defeat the whole point of pixelserv at that point. At the same time, the errors aren’t bothersome and the whole of the application runs smooth and creates a smooth experience on clients. Not that I don’t think that striving to be better has value. I would just remind everyone that the user experience matters more than obsessing over numbers. So if you feel like things are good for you, don’t lose sleep over some slu here and some extra milliseconds there. It’s no big deal, if you’re happy with the product.

 

[penguin22]

Not sure if the full specs are helpful, however I have had some intermittent issues lately with high internet latency or randomly loosing internet from devices for a few seconds (Android phones would show the '!' by the WiFi signal). In part of troubleshooting, reboot didn't seem to immediately help, however I've been hovering at around 91-95% memory usage on my AC86U, but also have an old EA-N66U acting as a media bridge plugged into an RT-AC66R in AP mode, so couldn't put it past something in my layout. Anyway, I opted to temporarily disable Skynet and run for a day; memory after 14:30 hours is steady at 93%, so not a likely culprit, however with Skynet off, my Pixelserv stats are significantly different with at least a 10 fold increase in overall req:

pixelserv-tls 2.1.0-rc.4 (compiled: Apr 5 2018 21:02:10 flags: tfo) options: 192.168.50.4

uts 0d 14:30 process uptime
log 1 critical (0) error (1) warning (2) notice (3) info (4) debug (5)
kcc 8 number of active service threads
kmx 50 maximum number of service threads
kvg 2.80 average number of requests per service thread
krq 10565 max number of requests by one service thread

req 104324 total # of requests (HTTP, HTTPS, success, failure etc)
avg 1904 bytes average size of requests
rmx 784104 bytes largest size of request(s)
tav 23 ms average processing time (per request)
tmx 3968 ms longest processing time (per request)

slh 96069 # of accepted HTTPS requests
slm 14 # of rejected HTTPS requests (missing certificate)
sle 0 # of rejected HTTPS requests (certificate available but bad)
slc 1345 # of dropped HTTPS requests (client disconnect without sending any request)
slu 5306 # of dropped HTTPS requests (other TLS handshake errors)
uca 519 slu break-down: # of unknown CA reported by clients
uce 3972 slu break-down: # of unknown cert reported by clients

sct 50 cert cache: # of certs in cache
sch 8428 cert cache: # of reuses of cached certs
scm 116 cert cache: # of misses to find a cert in cache
scp 103 cert cache: # of purges to give room for a new cert
sst 0 sess cache: # of cached TLS sessions (for older non-RFC5077 clients)
ssh 572 sess cache: # of reuses of cached TLS sessions
ssm 406 sess cache: # of misses to find a TLS session in cache
ssp 0 sess cache: # of purges to give room for a new TLS session

nfe 78469 # of GET requests for server-side scripting
gif 4 # of GET requests for GIF
ico 10 # of GET requests for ICO
txt 126 # of GET requests for Javascripts
jpg 14 # of GET requests for JPG
png 2 # of GET requests for PNG
swf 0 # of GET requests for SWF
sta 18 # of GET requests for HTML stats
stt 0 # of GET requests for plain text stats
ufe 58 # of GET requests /w unknown file extension

opt 0 # of OPTIONS requests
pst 1537 # of POST requests
hed 0 # of HEAD requests (HTTP 501 response)
rdr 21 # of GET requests resulted in REDIRECT response
nou 0 # of GET requests /w empty URL
pth 0 # of GET requests /w malformed URL
204 0 # of GET requests (HTTP 204 response)
bad 15883 # of unknown HTTP requests (HTTP 501 response)

tmo 58 # of timeout requests (client connect w/o sending a request in 'select_timeout' secs)
cls 2407 # of dropped requests (client disconnect without sending any request)
cly 398 # of dropped requests (client disconnect before response sent)
clt 0 # of dropped requests (reached maximum service threads)
err 0 # of dropped requests (unknown reason)

Per htop, Pixelserv-tls is sitting at 3.8% mem.