pixelserv pixelserv - A Better One-pixel Webserver for Adblock

[gattaca]

^^^ Ah, OK I was wondering where to properly ask. TY! Stay safe!

 

[Wishmaster1965]

0.0.0.0     addelivery-engine-api.voodoo-ads.io
0.0.0.0     ads.mopub.com
0.0.0.0     ads.samsungads.com
0.0.0.0     analytics.query.yahoo.com
0.0.0.0     api.gameanalytics.com
0.0.0.0     api2.branch.io
0.0.0.0     app.adjust.com
0.0.0.0     app-measurement.com
0.0.0.0     browser.pipe.aria.microsoft.com
0.0.0.0     cdn.optimizely.com
0.0.0.0     cdp.cloud.unity3d.com
0.0.0.0     cfg.cml.ksmobile.com
0.0.0.0     cmdts.ksmobile.com
0.0.0.0     config.uca.cloud.unity3d.com
0.0.0.0     cprd1.samsungcloudsolution.net
0.0.0.0     data.flurry.com
0.0.0.0     ds.samsungads.com
0.0.0.0     e.crashlytics.com
0.0.0.0     front-logs.voodoo-ads.io
0.0.0.0     gate.hockeyapp.net
0.0.0.0     googleads.g.doubleclick.net
0.0.0.0     lcprd1.samsungcloudsolution.net
0.0.0.0     m.yap.yahoo.com
0.0.0.0     mobile.launchdarkly.com
0.0.0.0     mobile.pipe.aria.microsoft.com
0.0.0.0     nexus.officeapps.live.com
0.0.0.0     nexusrules.officeapps.live.com
0.0.0.0     p.presage.io
0.0.0.0     pp-measurement.com
0.0.0.0     reports.crashlytics.com
0.0.0.0     rt.applovin.com
0.0.0.0     rt.applvn.com
0.0.0.0     rubick.gameanalytics.com
0.0.0.0     sb.scorecardresearch.com
0.0.0.0     sdk.hockeyapp.net
0.0.0.0     securepubads.g.doubleclick.net
0.0.0.0     settings.crashlytics.com
0.0.0.0     sourcepoint.vice.com
0.0.0.0     ssl.google-analytics.com
0.0.0.0     telemetry.dropbox.com
0.0.0.0     track.tenjin.io
0.0.0.0     ups.ksmobile.net
0.0.0.0     vortex.data.microsoft.com
0.0.0.0     www.googleadservices.com

So I tried all these sites added to my whitelist, I use open VPN to always connect to my router when out and about, I use this to block ads when mobile. But after I added these my phone gets the dreaded unhappy dog when using the amazon app on my phone. I am based in the UK if there are different sites in Amazon.

Let me know

 

[Khadanja]

Are these normal?
Jul 28 00:11:22 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: bad cert. client 192.168.1.26:58913 server mobile.pipe.aria.microsoft.com
Jul 28 00:12:04 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:50985 server web.vortex.data.microsoft.com
Jul 28 00:12:04 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:50986 server www.googletagmanager.com
Jul 28 00:12:04 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:50987 server www.googletagservices.com
Jul 28 00:12:04 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:50989 server c.amazon-adsystem.com
Jul 28 00:12:18 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:51012 server web.vortex.data.microsoft.com
Jul 28 00:12:18 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:51013 server www.googletagmanager.com
Jul 28 00:12:18 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:51014 server www.googletagservices.com
Jul 28 00:12:18 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:51016 server c.amazon-adsystem.com
Jul 28 00:12:23 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:56449 server www.googletagmanager.com
Jul 28 00:12:23 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:56450 server www.googletagservices.com
Jul 28 00:12:23 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:56451 server c.amazon-adsystem.com
Jul 28 00:12:23 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:56452 server web.vortex.data.microsoft.com
Jul 28 00:12:24 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:56454 server www.googletagmanager.com
Jul 28 00:12:24 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:56455 server www.googletagservices.com
Jul 28 00:12:24 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:56456 server c.amazon-adsystem.com
Jul 28 00:12:24 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:56457 server web.vortex.data.microsoft.com
Jul 28 00:12:25 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:56459 server web.vortex.data.microsoft.com
Jul 28 00:12:25 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:56461 server adservice.google.com
Jul 28 00:12:37 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:56482 server web.vortex.data.microsoft.com
Jul 28 00:12:38 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:56487 server www.googletagmanager.com
Jul 28 00:12:38 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:56488 server www.googletagservices.com
Jul 28 00:12:38 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:56490 server c.amazon-adsystem.com

 

[thelonelycoder]

Are these normal?
Jul 28 00:11:22 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: bad cert. client 192.168.1.26:58913 server mobile.pipe.aria.microsoft.com
Jul 28 00:12:04 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:50985 server web.vortex.data.microsoft.com
Jul 28 00:12:04 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:50986 server www.googletagmanager.com
Jul 28 00:12:04 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:50987 server www.googletagservices.com
Jul 28 00:12:04 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:50989 server c.amazon-adsystem.com
Jul 28 00:12:18 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:51012 server web.vortex.data.microsoft.com
Jul 28 00:12:18 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:51013 server www.googletagmanager.com
Jul 28 00:12:18 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:51014 server www.googletagservices.com
Jul 28 00:12:18 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:51016 server c.amazon-adsystem.com
Jul 28 00:12:23 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:56449 server www.googletagmanager.com
Jul 28 00:12:23 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:56450 server www.googletagservices.com
Jul 28 00:12:23 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:56451 server c.amazon-adsystem.com
Jul 28 00:12:23 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:56452 server web.vortex.data.microsoft.com
Jul 28 00:12:24 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:56454 server www.googletagmanager.com
Jul 28 00:12:24 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:56455 server www.googletagservices.com
Jul 28 00:12:24 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:56456 server c.amazon-adsystem.com
Jul 28 00:12:24 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:56457 server web.vortex.data.microsoft.com
Jul 28 00:12:25 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:56459 server web.vortex.data.microsoft.com
Jul 28 00:12:25 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:56461 server adservice.google.com
Jul 28 00:12:37 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:56482 server web.vortex.data.microsoft.com
Jul 28 00:12:38 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:56487 server www.googletagmanager.com
Jul 28 00:12:38 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:56488 server www.googletagservices.com
Jul 28 00:12:38 RT-AC68U-20E0 pixelserv-tls[17973]: handshake failed: unknown cert. client 192.168.1.26:56490 server c.amazon-adsystem.com

Purge your domain certificates. In Diversion this is in ep, 3, 1.

 

[aauto]

I'm new to diversion and appreciate very much for this ad-blocking script.

I also hit the issue on amazon app, and it said the dog image. The suggested list is added into whitelist, and then sort and process/

However, dnsmasq.log shows the ann.amazon.com is blocked. Any comment?

Aug 1 17:43:18 dnsmasq[18957]: blocked by blockinglist aan.amazon.com is 192.168.50.2

 

[CriticJay]

I'm new to diversion and appreciate very much for this ad-blocking script.

I also hit the issue on amazon app, and it said the dog image. The suggested list is added into whitelist, and then sort and process/

However, dnsmasq.log shows the ann.amazon.com is blocked. Any comment?

Aug 1 17:43:18 dnsmasq[18957]: blocked by blockinglist aan.amazon.com is 192.168.50.2

So try whitelisting it and see if that helps...

 

[brec]

On installing Diversion and opting for https coverage, I was told:
Remember to import the pixelserv-tls
certificate into browsers and devices.
Open 192.168.51.2/ca.crt in a browser.

I opened that in Firefox/Mac and was given the option to open the .crt with the Keychain Access application or to save it to disk. I chose the former. When I look at the cert in Keychain Access it says, among many other things about it, "This root certificate is not trusted."

I opened the URL in Firefox/iOS and was offered the option to download it. So now I have ca.crt in my downloads area there.

Later, based on some research, I found out how to install it in Firefox/Mac. I still don't know how to install it in Firefox iOS. More generally, I was looking forward to telling the person living in the house in which I'm installing an AC86U that the router now does ad blocking. I guess it does, for http:, but I don't know if it's going to be feasible for her to install a certificate for each of her devices. I don't know what nor how many devices she uses.

Am I making this harder than it needs to be?

 

[JacquesR]

On installing Diversion and opting for https coverage, I was told:
Remember to import the pixelserv-tls
...
Later, based on some research, I found out how to install it in Firefox/Mac. I still don't know how to install it in Firefox iOS. More generally, I was looking forward to telling the person living in the house in which I'm installing an AC86U that the router now does ad blocking. I guess it does, for http:, but I don't know if it's going to be feasible for her to install a certificate for each of her devices. I don't know what nor how many devices she uses.

Am I making this harder than it needs to be?

The wiki page has instructions for installing on various devices/browsers, and might be helpful for your iOS question. As for your friend, she'll still have an appreciably-improved browsing experience, whether or not she installs the cert (or you volunteer to do it for her), so I wouldn't worry about that.

 

[brec]

she'll still have an appreciably-improved browsing experience, whether or not she installs the cert

That's good to know! Now I'm curious as to what difference in the experience it makes after installation of the cert.

The wiki page has instructions for installing on various devices/browsers, and might be helpful for your iOS question.

Alas, not for Firefox/iOS. I noticed that it's been a couple of years since that wiki page has been edited.

But it provides a procedure for system-wide installation for iOS/Android, with this note appended:

Since iOS 10.3, a user-installed CA cert requires enabling trust explicitly.

• Go to Settings > General About > Certificate Trust Settings.
• Under Enable full trust for root certificates, turn on trust for Pixelserv CA.

This tip is provided by @jrmwvu04 on snbforums.

In the last step, i.e., after accessing the ca.crt UFL with Safari and responding positively to the ensuing popup, in Settings under "Enable full trust for root certificates" I see nothing about Pixelserv, only these two on/off buttons:

AddTrust External CA Root​

USERTrust RSA Certificatio...​

The first one was on, the second one off.

EDIT: I was able to successfully install on iOS. The wiki instructions are very abbreviated. A step-by-step would encompass a dozen or so "clicks" (iOS: touches) of different things. I don't think I'll attempt conveying that to my non-techie friend. But, reminder of what I wrote at the top of this reply: I'm curious as to what improvement cert installation provides to our browsing experience.

 

[PeterR]

That's good to know! Now I'm curious as to what difference in the experience it makes after installation of the cert.

Alas, not for Firefox/iOS. I noticed that it's been a couple of years since that wiki page has been edited.

But it provides a procedure for system-wide installation for iOS/Android, with this note appended:

In the last step, i.e., after accessing the ca.crt UFL with Safari and responding positively to the ensuing popup, in Settings under "Enable full trust for root certificates" I see nothing about Pixelserv, only these two on/off buttons:

AddTrust External CA Root​

USERTrust RSA Certificatio...​

The first one was on, the second one off.

EDIT: I was able to successfully install on iOS. The wiki instructions are very abbreviated. A step-by-step would encompass a dozen or "clicks" (iOS: touches) of different things. I don't think I'll attempt conveying that to my non-techie friend. But, reminder of what I wrote at the top of this reply: I'm curious as to what improvement cert installation provides to our browsing experience.

From kvic's github page 'pixelserv-tls is a tiny bespoke HTTP/1.1 webserver with HTTPS and SNI support. It acts on behalf of hundreds of thousands of advert/tracker servers and responds to all requests with nothing to speed up web browsing.'

Instead of your web browser waiting until it times out for a response from site blocked by diversion, pixelsrv serves a 1x1 pixel image. The ca.crt import is required so that served image appears to be coming from a legitimate website.

 

[elorimer]

The ca.crt import is required so that served image appears to be coming from a legitimate website.

I am finding that about 90% of the blocked sites are now doing other things that reject the image anyway, so that the improvement is only working for 10%. Still worthwhile.

 

[brec]

I am finding that about 90% of the blocked sites are now doing other things that reject the image anyway, so that the improvement is only working for 10%. Still worthwhile.

Questions from a relative newbie:
--How do you know this, i.e., what are you looking at that indicates the about 90%? **
--What would a naive user notice in an A/B test that makes pixelserv-tls worthwhile?

**Edit: I found the servstats page, which just now contained:

req

3146

total # of requests (HTTP, HTTPS, success, failure etc)

slh	113	# of accepted HTTPS requests
slm	132	# of rejected HTTPS requests (missing certificate)
sle	0	# of rejected HTTPS requests (certificate available but not usable)
slc	1895	# of dropped HTTPS requests (client disconnect without sending any request)
slu	995	# of dropped HTTPS requests (other TLS handshake errors)

 

[brec]

From the wiki:

You may want to backup ca.crt and ca.key. This will save you from generating and importing a new CA cert on client devices in the event that the original CA cert files in /opt/var/cache/pixelserv are damaged.

This brought to mind that... Yesterday I set up a second AC86U at a different location from my first AC86U. As indicated recently in this thread, I installed the second AC86U's Pixelserv ca.crt on my devices. The same devices will travel with me next week when I return to where my first AC86U is. But the ca.crt there is different. Should I just overwrite the ca.crt and ca.key in the first location with the contents of those files from the second?

 

[QuikSilver]

From the wiki:
This brought to mind that... Yesterday I set up a second AC86U at a different location from my first AC86U. As indicated recently in this thread, I installed the second AC86U's Pixelserv ca.crt on my devices. The same devices will travel with me next week when I return to where my first AC86U is. But the ca.crt there is different. Should I just overwrite the ca.crt and ca.key in the first location with the contents of those files from the second?

IP scheme the same? Are you using 192.168.1.XXX at location 1 as well as location two? I have two as well but I use 192.168.2.XXX at my second location.

 

[brec]

IP scheme the same? Are you using 192.168.1.XXX at location 1 as well as location two? I have two as well but I use 192.168.2.XXX at my second location.

My IPs are conceptually the same as yours: 192.168.50.1 and 192.168.51.1.

 

[QuikSilver]

My IPs are conceptually the same as yours: 192.168.50.1 and 192.168.51.1.

Then you should have no problem importing the secondary one. I have two imported on my devices. No need to remove one to add the other.

 

[brec]

Then you should have no problem importing the secondary one. I have two imported on my devices. No need to remove one to add the other.

What I was thinking might be a problem is that on my devices now, after having imported from one of the AC86Us, the name of the certificate is "Pixelserv CA". E.g., that's what I see in my iPhone's Certificate Trust Settings. Wouldn't the import from the second router have the same name, and if so would that cause any problem?

 

[thelonelycoder]

From the wiki:
This brought to mind that... Yesterday I set up a second AC86U at a different location from my first AC86U. As indicated recently in this thread, I installed the second AC86U's Pixelserv ca.crt on my devices. The same devices will travel with me next week when I return to where my first AC86U is. But the ca.crt there is different. Should I just overwrite the ca.crt and ca.key in the first location with the contents of those files from the second?

I use the same certificates on all my routers. Just replace the ca.* files, then purge the domain certificates in Diversion in ep and you're good to go.

 

[elorimer]

Wouldn't the import from the second router have the same name, and if so would that cause any problem?

This does cause a problem with Chromebooks, which will overwrite the first certificate. That's why it is essential for the VPN servers to have the same certificate, among other things, although that is a different context.

 

[brec]

it is essential for theVPN servers to have the same certificate, among other things, although that is a different context.

We're off-topic, so if there's anything wrong with this please point me at another thread: each of my two AC86Us has a LAN-only OpenVPN server to allow my remote access to it, and the .OVPN files are identical -- same keys -- except for the "remote" field, i.e., DDNS domain name specific to that router.