pixelserv pixelserv - A Better One-pixel Webserver for Adblock

[PeterR]

Actually, another question. Is there a way to use a paid certificate instead of the self signed one so it doesn't have to be installed on each device? I did it easily on macbook but for iPhone and other mobile devices it won't be that easy + guests.

It's not mandatory to import the ca.crt to every device, from the pixelsrv-tls github page:

"pixelserv-tls is a tiny bespoke HTTP/1.1 webserver with HTTPS and SNI support. It acts on behalf of hundreds of thousands of advert/tracker servers and responds to all requests with nothing to speed up web browsing."

there will just be a delay whilst the browser tries to download the blocked elements

 

[thelonelycoder]

Err...? Could I ask for a clarification here? Do you mean that you will personally drop (stop using) Safari, or are you considering dropping support for pixelserv (and the corresponding certificates) within Diversion?

I agree with the first possibility, but would be thoroughly shocked if you meant the second one.

This may he shocking to you, but to me it would be a relieve. The uncertain support situation combined with the refusal to communicate with me and/or the community is and has always been an unsatisfying situation.
Diversion and amtm are long term projects for me, I have to consider my choices accordingly. The fun part counts too.

 

[Centrifuge]

The fun part counts too.

Yeah it there's a handy replacement, or even if not, go for it. You've worked some magic, maybe this is holding you back. Thanks for all your work.

 

[Asad Ali]

Is there a way to use a paid certificate instead of the self signed one

Sure, if you can spend thousands of dollars to get a Root Certificate Authority for your own personal use, you can do that lol.

 

[terminator]

Sorry, I am new to this and also have been reading some recent comments about cert expiration date changing so apologize if this has been asked before -

Option 1: what if a new certificate was purchased for domain https://pixelserv.com (not the Root Certificate Authority, just a regular https cert). Then the pixelserv web server/service used this on all the instances (all users/router installations). The router maps the DNS of pixelserv.com to your local IP where the PixelServ is running.

Option 2: If option 1 seems a bit weird to use the same cert on different routers, how about a connection to letsencrypt to get a https cert (not Root Certificate Authority) and use that similar to Option 1 above. The only change is that each user can have a different one instead of the same https://pixelserv.com

I think my point is that instead of generating your own https cert, why not the use the one generated by an exiting Root Certificate signing authority?

That how I have been doing it in the IIS/Web Server boxes for years but I have little to no context for the router based solutions so I may be off base here.

 

[Asad Ali]

Sorry, I am new to this and also have been reading some recent comments about cert expiration date changing so apologize if this has been asked before -

Option 1: what if a new certificate was purchased for domain https://pixelserv.com (not the Root Certificate Authority, just a regular https cert). Then the pixelserv web server/service used this on all the instances (all users/router installations). The router maps the DNS of pixelserv.com to your local IP where the PixelServ is running.

Option 2: If option 1 seems a bit weird to use the same cert on different routers, how about a connection to letsencrypt to get a https cert (not Root Certificate Authority) and use that similar to Option 1 above. The only change is that each user can have a different one instead of the same https://pixelserv.com

I think my point is that instead of generating your own https cert, why not the use the one generated by an exiting Root Certificate signing authority?

That how I have been doing it in the IIS/Web Server boxes for years but I have little to no context for the router based solutions so I may be off base here.

Pixelserv-tls generate the client certificates on the fly for all the blocked domain entries and to complete the TLS handshake you need a proper Root Certificate Authority to sign the generated client certificates. You can't use the already generated certificates from a third party Root CA to sign your own client certificates because you don't have the matching private key.

Also we're not generating a simple HTTPS certificate, we make our own Root Certificate Authority.

 

[terminator]

Pixelserv-tls generate the client certificates on the fly for all the blocked domain entries and to complete the TLS handshake you need a proper Root Certificate Authority to sign the generated client certificates. You can't use the already generated certificates from a third party Root CA to sign your own client certificates because you don't have the matching private key.

Also we're not generating a simple HTTPS certificate, we make our own Root Certificate Authority.

ahh, got it. I misunderstood how it works. I thought https://ads.google.com would get redirected to https://routerip/.. and the certificate was for that.

 

[HairyA00]

Yeah it there's a handy replacement, or even if not, go for it. You've worked some magic, maybe this is holding you back. Thanks for all your work.

Are there viable alternatives? Are we locked into dnsmasq (in other words, is there a technical reason Diversion is using dnsmasq and an alternative is not an option)? In my experience, FTLDNS is just as fast as pixelserv-tls (FTLDNS 0.0.0.0 timeouts are quicker than 0.0.0.0 dnsmasq+HTTPS requests).

Not sure it's feasible but thought it was worth mentioning. FTLDNS 5.0 is slated to be released here soon, too.

EDIT: Digging up an old post: https://www.snbforums.com/threads/diversion-the-router-ad-blocker.48538/page-181#post-512567

 

[elorimer]

Are there viable alternatives? Are we locked into dnsmasq (in other words, is there a technical reason Diversion is using dnsmasq and an alternative is not an option)? In my experience, FTLDNS is just as fast as pixelserv-tls (FTLDNS 0.0.0.0 timeouts are quicker than 0.0.0.0 dnsmasq+HTTPS requests).

Not sure it's feasible but thought it was worth mentioning. FTLDNS 5.0 is slated to be released here soon, too.

EDIT: Digging up an old post: https://www.snbforums.com/threads/diversion-the-router-ad-blocker.48538/page-181#post-512567

Aren't we talking about two different things? Dnsmasq built into the firmware is sinkholing a blocked address, and diversion is managing that. Diversion can sinkhole blocked addresses to pixelserv-tls, which serves up a 1x1 pixel instead of an ad, and instead of a broken link, which is what sinkholing to the no-address address does (I think--its been years). Pixelserv-tls does this for https requests as well, by spoofing the certificate of the blocked address. That has worked well except for blocked addresses that have hard coded elements that can't get spoofed.

Our current problem is that Apple is changing the rules faster and frequently, and @kvic isn't very active in changing pixelserv-tls (at least, I guess that is what @thelonelycoder was alluding to) in response. But I think the bigger headache is in importing the certificate into clients' trusted root. It's bad enough that I have to copy one new pair to the other routers I run, so I only have to import that one certificate, but if I have to import it every year on 15 devices that's somewhat inconvenient. But that isn't pixelserv's fault, or dnsmasq or unbound, either.

I did wish @kvic would have carried the redir project a little further. That might have been an interesting addition to unbound. (Also, cascading hash tables, whatever that was).

 

[juched]

Aren't we talking about two different things? Dnsmasq built into the firmware is sinkholing a blocked address, and diversion is managing that. Diversion can sinkhole blocked addresses to pixelserv-tls, which serves up a 1x1 pixel instead of an ad, and instead of a broken link, which is what sinkholing to the no-address address does (I think--its been years). Pixelserv-tls does this for https requests as well, by spoofing the certificate of the blocked address. That has worked well except for blocked addresses that have hard coded elements that can't get spoofed.

Our current problem is that Apple is changing the rules faster and frequently, and @kvic isn't very active in changing pixelserv-tls (at least, I guess that is what @thelonelycoder was alluding to) in response. But I think the bigger headache is in importing the certificate into clients' trusted root. It's bad enough that I have to copy one new pair to the other routers I run, so I only have to import that one certificate, but if I have to import it every year on 15 devices that's somewhat inconvenient. But that isn't pixelserv's fault, or dnsmasq or unbound, either.

I did wish @kvic would have carried the redir project a little further. That might have been an interesting addition to unbound. (Also, cascading hash tables, whatever that was).

Unbound uses a return of NXDOMAIN instead of a redirection pixelsrv. I see a test by kvic which shows those two operate similarly in performance. With NXDOMAIN there is no follow up connection. Unsure which is less impactful on websites. However no more certificate to import.

 

[elorimer]

Unbound uses a return of NXDOMAIN instead of a redirection pixelsrv. I see a test by kvic which shows those two operate similarly in performance. With NXDOMAIN there is no follow up connection. Unsure which is less impactful on websites. However no more certificate to import.

Yes, but don't you end up then with a broken link icon?

 

[Treadler]

Yes, but don't you end up then with a broken link icon?

I never saw anything untoward when I was trialling the Unbound Adblock.

 

[dave14305]

This was my script to rewrite Diversion’s blocklist for Unbound with Pixelserv. In the end, Diversion was best for me.

#!/bin/sh

if [ /opt/share/diversion/list/blockinglist -nt /opt/var/lib/unbound/ads.conf ] || [ ! -f /opt/var/lib/unbound/ads.conf ]; then
        awk '{for (i=2; i<=NF; i++) print "local-data: \""$i". 0 A 192.168.1.2\""}' /opt/share/diversion/list/blockinglist > /opt/var/lib/unbound/ads.conf

        if $(grep -q "ads\.conf" /opt/var/lib/unbound/unbound.conf); then
                unbound-control reload
        fi
fi

 

[Asad Ali]

But I think the bigger headache is in importing the certificate into clients' trusted root. It's bad enough that I have to copy one new pair to the other routers I run, so I only have to import that one certificate, but if I have to import it every year on 15 devices that's somewhat inconvenient.

As far as I can see, Apple didn't change the Root Certificate Authority time duration and it can be still valid for 10 years, what's changed/being changed is the clients certificate time duration from 825 days to 398 days and you don't need to import client certificates to devices, Pixelserv-tls generates them on the fly.

 

[jrmwvu04]

As far as I can see, Apple didn't change the Root Certificate Authority time duration and it can be still valid for 10 years, what's changed/being changed is the clients certificate time duration from 825 days to 398 days and you don't need to import client certificates to devices, Pixelserv-tls generates them on the fly.

This is correct.

I did wish @kvic would have carried the redir project a little further. That might have been an interesting addition to unbound. (Also, cascading hash tables, whatever that was).

The last bit of communication I had was concerning that topic. I never did get around to implementing unbound/redir but it seemed like a promising avenue based on the reports we were getting at the time. The technical details are way, way over my head but the gist of it was that for the explicit purpose of blocking ads via domain name, unbound with redir was superior to dnsmasq from a performance standpoint. But for various reasons that has gone unexplored (by me) to this point, though I have been watching the big unbound thread on this board.

 

[XIII]

Looks like Google is limiting certificates even more (1 year, 365 days):

https://security.googleblog.com/2020/03/how-google-does-certificate-lifecycle.html

 

[dave14305]

So has anyone gone back to Diversion Lite (w/o Pixelserv-tls)? Not that it's necessary, but I'm wondering if anyone has measured any difference in performance since kvic's original tests.

 

[juched]

So has anyone gone back to Diversion Lite (w/o Pixelserv-tls)? Not that it's necessary, but I'm wondering if anyone has measured any difference in performance since kvic's original tests.

Or even compared to returning NXDOMAIN ? This is what adblock does in unbound and I am not sure if there is a big difference between the two. Kvic’s site did a comparison with NXDOMAIN and pixelserv and they were very close in performance, although the still leaned on the side of using pixelserv.

 

[HairyA00]

So has anyone gone back to Diversion Lite (w/o Pixelserv-tls)? Not that it's necessary, but I'm wondering if anyone has measured any difference in performance since kvic's original tests.

I use Diversion Standard with pixelserv-tls disabled, so essentially the same thing as Diversion Lite. No issues here other than gray boxes where ads used to be on Android. Noticed no changes with speed. Both HTTP and HTTPS domains are still blocked.

 

[dave14305]

I use Diversion Standard with pixelserv-tls disabled, so essentially the same thing as Diversion Lite. No issues here other than gray boxes where ads used to be on Android. Noticed no changes with speed. Both HTTP and HTTPS domains are still blocked.

So is your blocking IP still the pixelserv IP or 0.0.0.0? I reinstalled with Diversion Lite and my blocking IP is now 0.0.0.0. So far so good.