pixelserv pixelserv - A Better One-pixel Webserver for Adblock

[DonnyJohnny]

kvic, my friend, thank you again for updating pixelserv. I'm running 2.2.0-rc.4 and getting expected results (TLS 1.3 in chrome beta).

I'm still having trouble with android 'system clients', but I understand this is out of the scope of router scripts. Add to that, since my son started playing rainbow 6 siege on our xbox, I have one more source of random annoying traffic (high uce stats... pings up to ~50/sec... watson.telemetry.microsoft.com... gah!). At least this behavior hasn't crippled my network. The most common app seems to come in a pattern that pings at roughly 5hz for 5min. These are almost imperceptible other than logs. The occasional 50hz ping has mostly downed my network for a couple of minutes while active. This behavior seems unacceptable to me. Is this how companies punish those who refuse to comply with data mining practices?

Regardless, my son's games are working fine, as is my phone and the other clients that exhibit this behavior (pretty much all devices, but the apple phones are much less aggressive). As long as this is true, I've got no reason to whitelist watson.telemetry or other known spywares. I've seen some other webpages where folks suggest whitelisting these urls for 'full functionality' of games... also to allow window's updates to work, LOL! If my son gets games that require telemetry to register achievements, I'll have to tell him too bad... but I guess if microsoft requires it to download updates, that would really suck. I hope game makers & os designers don't go down this road, without significant public backlash that stops it cold.

@SMS786
For info, chrome beta android already have final version of tls 1.3. But it is disable by default. You need to go chrome://flags to enable to final version of tls 1.3

I think Firefox beta android also have final version but unlike chrome beta mobile and Firefox desktop, Firefox beta android can’t see cert info. So no way to verify.

 

[SMS786]

@SMS786
For info, chrome beta android already have final version of tls 1.3. But it is disable by default. You need to go chrome://flags to enable to final version of tls 1.3

I think Firefox beta android also have final version but unlike chrome beta mobile and Firefox desktop, Firefox beta android can’t see cert info. So no way to verify.

Thanks for the info. Personally, I try to stay away from Chrome as much as possible. And the FF betas tend to break some of my extensions. Not sure why Mozilla is taking so long regarding incorporating the final TLS 1.3 draft into their stable builds. Hopefully it's sooner than later.

Edit: typo

 

[truglodite]

It's different than a Chrome browser client. The browsers and apps operating under os's with trusted root certs work as expected. The spikes in uce and slc I think come from some built in os clients which I have no way to change. This happens with my unrooted android, iOS, and windows devices, despite having all exposed 'telemetry' settings turned off in the OS (like error reporting, ms experience feedback, etc.) Windows update still works, despite telemetry remaining blocked. I can only guess what the intended payload might be.

 

[DonnyJohnny]

I am surfing with my iPad mini iOS 12.1 beta 1 most of my time.
So the tls 1.3 don’t matter since client don’t support.

Still waiting for Apple to come out with the final version of TLS 1.3.
They are slow.....

 

[dugaduga]

Chrome 66 (tls draft 23) and 69 (tls draft 28) both work fine for me. After DonnyJonny unveild the secret code buried deeply on kvics homepage

_binfavor=static sh -c "$(wget -qO - https://kazoo.ga/pixelserv-tls/install-beta.sh)"

I was strictly using chromium for a time thinking Chrome was tracking everything. However Chrome can be even more secured with group policy. The majority of these settings can be disabled under settings, and via commandline switches " --disable-webgl --safe-plugins --enable-sandbox --cipher-suite-blacklist=0x000a,0x0035,0x002f,0x009d,0x009c,0x0002,0x0001", the rest can be done under about:flags and with files that can be imported into GroupPolicy, which can force minimum tls 1.2, disable chromes built in DNS, calling home services and most if not all privacy concerns. It takes some time but once its done its done. The only issue I have in contrast to FF is I have not found a way to disable 0-RTT, or session tickets. Sorry for O/T.

 

[kvic]

I noticed a reoccurring crash as well <snipped> Hopefully this will be the end of that.

The regular version of pixelserv-tls (built on top of OpenSSL 1.0.2) has been very stable for the past year. Very unlikely you'll see crash. If you're doing things right, make sure test properly and give me a reproducible case to fix any potential issue.

The recent (in the past day or two) hiccups on instability are associated with the statically linked version built on top of OpenSSL 1.1.1. If you weren't running it yet, there was no reason to believe it's caused by it or solved by rc.4.

Chrome 66 (tls draft 23) and 69 (tls draft 28) both work fine for me.

Clients supporting a specific DRAFT version of TLS 1.3 will work with servers supporting that specific DRAFT version of TLS 1.3.

To see TLS 1.3 on your client when connecting to pixelserv-tls, you'll need a client supporting the final version of TLS 1.3.

TLS 1.3 final standard was officially published in August. There is no longer merits looking back on the DRAFT versions.

After DonnyJonny unveild the secret code buried deeply on kvics homepage

As mentioned in my original proposal a few pages back, the statically linked version of pixelserv-tls is an opt-in option. What that means people done enough with muscle memory won't have this version installed. You need to take one little extra step. Anyhow, I don't think it's secret code nor "deeply buried" on the release page if you had spent one minute reading it.

 

[M@rco]

...the secret code buried deeply on kvics homepage

That sounds somewhat exaggerated. If you'd just followed the instructions posted by @kvic

2.2.0-rc.3 is available

Statically linked binaries that support TLS 1.3 are included. No other functional change from rc.2.

For details, pls read the release notes at https://kazoo.ga/pixelserv-tls/.

you would have landed exactly on the post with instructions (now updated for rc.4, but nevertheless)

2.2.0-rc.4 (2018-9-24)
Changes

• NEW indicator of TLS 1.3 support status on servstats page
• FIXED failed to log server name on unsuccessful handshakes. Garbage may be captured instead. When it happens it may lead to crash or a hung process.

Download

Regular pixelserv-tls (support TLS versions <= 1.2)

sh -c "$(wget -qO - https://kazoo.ga/pixelserv-tls/install-beta.sh)"

Statically linked pixelserv-tls (support TLS 1.3 and versions <= 1.2)

_binfavor=static sh -c "$(wget -qO - https://kazoo.ga/pixelserv-tls/install-beta.sh)"

NOTES If you upgrade from pixelserv-tls v2.1.2 or older versions to this beta for the very first time, pls consider purging all certs except ca.crt and ca.key. This will avoid possible cert errors due to an enhancement in 2.1.3-test.1.

so how hard can it be? I have a visual disability, and even I can read it

 

[kvic]

Not sure why Mozilla is taking so long regarding incorporating the final TLS 1.3 draft into their stable builds. Hopefully it's sooner than later.

FF v63 will be released in October. Actually Firefox is on the forefront of TLS 1.3 support as apparently they want to beat Google Chrome in every single way they could.

So the tls 1.3 don’t matter since client don’t support.

Browsers are apparently still one major category of clients. However, Apps are taking HTTP traffic away from browsers. Users have little control inside those apps what HTML rendering engines they use or even embedding inside.

For example, Instagram has been on the forefront of using TLS 1.3. Their production clients have one DRAFT version of TLS 1.3 baked in. What Facebook? Perhaps lots of libraries are embedded for better performance or sadly and more truly collecting ppl's data. Otherwise, I could explain why FB App size keeps growing at a rapid pace in the past few years. Last time checked the iOS App was around 120MB. I stopped using FB for a few years already..so not sure about the latest size.

Still waiting for Apple to come out with the final version of TLS 1.3.
They are slow.....

Interestingly OpenSSL library isn't that popular among browsers. Don't know why. Apple switched full house to its in-house SSL library a few years ago. Chrome uses different SSL library dependent on platforms. Android Chrome uses OpenSSL. Perhaps that's why it's already available in beta.

 

[Asad Ali]

@kvic Latest FB iOS app size is 519.4MB
Even I was shocked when I saw that.

 

[DonnyJohnny]

@kvic Latest FB iOS app size is 519.4MB
Even I was shocked when I saw that.

...
I didn’t notice that.. lol..
Better to uninstall and just use browser for it.

 

[kvic]

The occasional 50hz ping has mostly downed my network for a couple of minutes while active. This behavior seems unacceptable to me. Is this how companies punish those who refuse to comply with data mining practices?

Can't rule out some firms will do it on purpose. Most likely it's not intended though. But rather the artefact of freely available blocklists that your adblock script pulls in. I can't emphasise more that you aren't simply better off with a larger block list. It increases your chance of hitting a wrongly blocked domain that sometimes could make your client very upset.

The benefit of paid service (such as OpenDNS..still available?) is (hopefully) that some paid curators are taking care of the blocklists, mainly by lots of testing on different clients that rule out any wrongly blocked domains. A few popular and freely available block lists do have a feedback loop from users. But in general no surprised that enough people are motivated to actively doing so.

The spikes in uce and slc I think come from some built in os clients which I have no way to change.

This is very likely. This option naturally comes up to any reasonably good developers/firms who want to hide transferred data from competitors and/or their end-users. lol

 

[kvic]

@kvic Latest FB iOS app size is 519.4MB
Even I was shocked when I saw that.

No wonder Zuckerbird is getting so much unwanted attention from politicians in the recent years!

 

[dugaduga]

The regular version of pixelserv-tls (built on top of OpenSSL 1.0.2) has been very stable for the past year. Very unlikely you'll see crash. If you're doing things right, make sure test properly and give me a reproducible case to fix any potential issue.

The recent (in the past day or two) hiccups on instability are associated with the statically linked version built on top of OpenSSL 1.1.1. If you weren't running it yet, there was no reason to believe it's caused by it or solved by rc.4..

Well I had noticed over the last 6 months or so servstats was often not accessible... though I never checked the logs. Its been accessible for 18 hours non stop since your update. Yesterday it was hit and miss by the hour, day. However many issues did arise after updating to entware and diversion; most of them have been solved reverting to ab-solution and retaining new entware packages... your update to pixelserv seems to have fixed my hanging problem, but more time is needed to be certain.

Clients supporting a specific DRAFT version of TLS 1.3 will work with servers supporting that specific DRAFT version of TLS 1.3.
To see TLS 1.3 on your client when connecting to pixelserv-tls, you'll need a client supporting the final version of TLS 1.3.
TLS 1.3 final standard was officially published in August. There is no longer merits looking back on the DRAFT versions.

Ok I see now!

 

[kvic]

Well I had noticed over the last 6 months or so servstats was often not accessible... though I never checked the logs.

Utterly non-sense you didn't look into it back then and reported such an issue 6 months later..

 

[dugaduga]

Thank you for the kick, I'll do it next time. I didn't think it was a bug, saw it was running in ab-solution and thought nothing of it. I suspect blocking rgom10-en.url.trendmicro.com had something to do with it because that was the most active query in pixelserv's logs; I've since pointed it to 0.0.0.0 to clean the logs up.

 

[dugaduga]

No wonder Zuckerbird is getting so much unwanted attention from politicians in the recent years!

You might want to consider this; they use the data to know what commercials are on if you watch tv, to assist in advertising to you and god knows what else

 

[regae]

i have high ush number comes from my ios devices with the same address, is it normal?

 

[DonnyJohnny]

i have high ush number comes from my ios devices with the same address, is it normal?

Normal. Due to use of own app’s ssl lib, they may have strict verification process causing handshake to drop after ServerHello.
App like gmail, or even iOS update, etc.
You can always on the debug mode to see the domain that is causing the ush error.

 

[truglodite]

... I can't emphasise more that you aren't simply better off with a larger block list. It increases your chance of hitting a wrongly blocked domain that sometimes could make your client very upset.

I've been using Diversion's 'Standard' lists and Skynet as well (fairly minimal I think). I believe those lists have also been well behaved for other users on these forums, or if not I'm sure we'll hear more about it soon. I will experiment a bit before I ignore the problem. If I discover that white listing sketchy spyware sites like watson.telemetry.microsoft.com is the only way to fix these floods, I'll just live with the occasional floods.

I suspect my ush spikes may be attempts by 'uncontrollable apps' to find the same well known spyware domains I mentioned earlier. Often bursts of ush occur immediately after a client after has repeated slc handshake failures. It happens so consistently that I doubt it is a coincidence.

 

[kvic]

For people on rc.4, how is your runtime/stability so far? Here is mine. Note the "tls1_3" flag that indicates I'm running the static version with support for TLS 1.3

I added a paragraph "Indicator of TLS 1.3 support on servstats page" shortly after release. Some ppl might have missed and may want to have a look..

pixelserv-tls 2.2.0-rc.4 (compiled: Sep 23 2018 19:12:30 flags: tls1_3) options: 192.168.1.3 -A 344 -l 2 -c 350

uts 2d 05:48 process uptime
log 2 critical (0) error (1) warning (2) notice (3) info (4) debug (5)
kcc 1 number of active service threads
kmx 104 maximum number of service threads
kvg 1.30 average number of requests per service thread
krq 3286 max number of requests by one service thread
req 33736 total # of requests (HTTP, HTTPS, success, failure etc)
avg 10193 bytes average size of requests
rmx 79635 bytes largest size of request(s)
tav 43 ms average processing time (per request)
tmx 9831 ms longest processing time (per request)
slh 21534 # of accepted HTTPS requests
slm 11 # of rejected HTTPS requests (missing certificate)
sle 0 # of rejected HTTPS requests (certificate available but bad)
slc 2773 # of dropped HTTPS requests (client disconnect without sending any request)
slu 2764 # of dropped HTTPS requests (other TLS handshake errors)
uca 0 slu break-down: # of unknown CA reported by clients
uce 0 slu break-down: # of unknown cert reported by clients
ush 509 slu break-down: # of shutdown by clients after ServerHello