Diversion https ads being allowed

[Ellenswamy]

I installed diversion with pixel tls and all seems to be working, in the logs I see A record getting blocked and then https right after going to the same site being allowed. I have DNS filter set to router, Also have DNS over TLS enabled with DNS rebind protection set to off ( I saw this might be an issue). Anything I should be checking?

 

[SomeWhereOverTheRainBow]

I installed diversion with pixel tls and all seems to be working, in the logs I see A record getting blocked and then https right after going to the same site being allowed. I have DNS filter set to router, Also have DNS over TLS enabled with DNS rebind protection set to off ( I saw this might be an issue). Anything I should be checking?

Well with pixelserv-tls in the mix you should still be able to see the https being served(it would appear returning as the blocked domain), but it should return as a pixel. It would still be normal to see the A record being blocked.

If you were not using pixelservtls, you would not see the https pixel being served you would only see the A blocked. The https would time out before being able to return a response.

Conclusion: this is a normal behavior when using pixelserv-tls. It's job is to return a pixel size image in response to https ads, upon observation this would appear as if the https ad was being served, however you would still observe the A query being blocked.

 

[Ellenswamy]

Well with pixelserv-tls in the mix you should still be able to see the https being served(it would appear returning as the blocked domain), but it should return as a pixel. It would still be normal to see the A record being blocked.

If you were not using pixelservtls, you would not see the https pixel being served you would only see the A blocked. The https would time out before being able to return a response.

Conclusion: this is a normal behavior when using pixelserv-tls. It's job is to return a pixel size image in response to https ads, upon observation this would appear as if the https ad was being served, however you would still observe the A query being blocked.

I am using pixel tls and the up is 192.168.50.2. I turned in dnsfilter, and set it to router. Do I need to do anything else? I see blocks for non https

 

[SomeWhereOverTheRainBow]

I am using pixel tls and the up is 192.168.50.2. I turned in dnsfilter, and set it to router. Do I need to do anything else? I see blocks for non https

You should be doing fine. The https traffic you see being returned for the "blocked" domains should be pixelservtls responding with its pixel obviously it would be as if you did not see an ad appearance wise because where the ad would have been you would have a tiny pixel. Any non https should be blocked as A.

With pixelserv tls disabled , and Diversion lite, your https will just time out, and you would see blank ad space where the ad normally would be, but fails to load.

 

[elorimer]

With pixelserv tls disabled , and Diversion lite, your https will just time out, and you would see blank ad space where the ad normally would be, but fails to load.

I think pixelserv is a combo of two things. Only 1-2% of my https requests pointed to pixelserv complete with the single pixel; the rest I think are being rejected by the page ad code and result in a blank ad space. The difference is that pixelserv responds immediately to the request rather than the request timing out.

 

[SomeWhereOverTheRainBow]

I think pixelserv is a combo of two things. Only 1-2% of my https requests pointed to pixelserv complete with the single pixel; the rest I think are being rejected by the page ad code and result in a blank ad space. The difference is that pixelserv responds immediately to the request rather than the request timing out.

That is exactly what I have been saying. However in my case pixelserv tls has majority of the time supplied its pixel. Seldomly it returns no-data, which means nothing gets loaded. I have yet to see the blank ad space, those must be from AAAA responses since dnsmasq blocks with standard :: for AAAA responses. However, if diversion mapped AAAA (eg. ::ffff:192.168.1.2) responses to the ipv4 block address, you wouldn't see any random blank ad space.

 

[Ellenswamy]

I think pixelserv is a combo of two things. Only 1-2% of my https requests pointed to pixelserv complete with the single pixel; the rest I think are being rejected by the page ad code and result in a blank ad space. The difference is that pixelserv responds immediately to the request rather than the request timing out.

Gotcha, the reason I asked is because I have yet to see that pixel that it was blocked. I didn't know that if that doesn't block it then diversion will with a blank space

 

[elorimer]

Well, not exactly. The web page has code that says "go to this https adserver and get an ad and place it here". Diversion intercepts the adserver lookup and instead of responding with the real one, responds with the pixelserv address. The web page goes to the pixelserv address and pixelserv immediately responds with a single pixel. But the web page also says, in effect, check that the certificate of the adserver is "xxx", sees that pixelserv isn't that, and drops the pixelserv response and puts, usually, a broken link box instead of the invisible pixel. Diversion without pixelserv sends the adserver lookup to oblivion, the webpage times out and presents the broken link box instead of the ad. The difference is measurable: the NYT home page loads in a second or so for me with pixelserv, and 6 or 7 seconds without.

Also, without diversion, never mind ads, the NYT home page loads for a long time, pulling video and chewing up data. It can be loading for on the order of 30 seconds.

 

[SomeWhereOverTheRainBow]

Well, not exactly. The web page has code that says "go to this https adserver and get an ad and place it here". Diversion intercepts the adserver lookup and instead of responding with the real one, responds with the pixelserv address. The web page goes to the pixelserv address and pixelserv immediately responds with a single pixel. But the web page also says, in effect, check that the certificate of the adserver is "xxx", sees that pixelserv isn't that, and drops the pixelserv response and puts, usually, a broken link box instead of the invisible pixel. Diversion without pixelserv sends the adserver lookup to oblivion, the webpage times out and presents the broken link box instead of the ad. The difference is measurable: the NYT home page loads in a second or so for me with pixelserv, and 6 or 7 seconds without.

I wonder if we can convince @thelonelycoder to map the AAAA responses for blocked domains to A (eg. ::ffff:192.168.1.2) so they can fully leverage pixelserv-tls, instead of using :: for ipv6 blocks. (call it an experimental feature kinda like youtube adblock.)

 

[Ellenswamy]

So I have been running diversion and A record logs show things are being blocked but https shows allowed for the exact same url and right after the block. Is this normal?

or did I have to do something else to set it up? I have dns filter set to on and tried no filter and router. The ip of my router is my dns etc. any help will be appreciated!

 

[SomeWhereOverTheRainBow]

So I have been running diversion and A record logs show things are being blocked but https shows allowed for the exact same url and right after the block. Is this normal?

or did I have to do something else to set it up? I have dns filter set to on and tried no filter and router. The ip of my router is my dns etc. any help will be appreciated!

The https would show allowed, since the https is loading the pixel of the blocked domain. pixelserv-tls acts as a middle man, using pixel certificates. it serves the blocked https domain back as a pixel response. therefore you would see the https response. As long as you are not seeing the https response returning as a big giant advertisement when loading a web page, then pixelserv-tls is doing its job.

 

[SomeWhereOverTheRainBow]

Gotcha, the reason I asked is because I have yet to see that pixel that it was blocked. I didn't know that if that doesn't block it then diversion will with a blank space

The pixel is so tiny it will appear as a empty space where a ad would normally reside on the webpage..

 

[thelonelycoder]

I wonder if we can convince @thelonelycoder to map the AAAA responses for blocked domains to A (eg. ::ffff:192.168.1.2) so they can fully leverage pixelserv-tls, instead of using :: for ipv6 blocks. (call it an experimental feature kinda like youtube adblock.)

Let me check that, sounds like a good idea in theory.

 

[Ellenswamy]

The https would show allowed, since the https is loading the pixel of the blocked domain. pixelserv-tls acts as a middle man, using pixel certificates. it serves the blocked https domain back as a pixel response. therefore you would see the https response. As long as you are not seeing the https response returning as a big giant advertisement when loading a web page, then pixelserv-tls is doing its job.

That helps. But I am seeing ads where there shouldn’t be. Stuff is getting blocked, but I am also seeing ads that I know are on the block list also. That is why I am confused

 

[dave14305]

That helps. But I am seeing ads where there shouldn’t be. Stuff is getting blocked, but I am also seeing ads that I know are on the block list also. That is why I am confused

Post some examples of what you see in the logs and in the browser.

 

[SomeWhereOverTheRainBow]

That helps. But I am seeing ads where there shouldn’t be. Stuff is getting blocked, but I am also seeing ads that I know are on the block list also. That is why I am confused

As @dave14305 has pointed out, it is not only important to be seeing ads, but it is also important to tie those ads to the log itself, one of the best way of probably doing that would be screenshots, logs, and if possible actual times. Also, it would be important to reproduce the issue. What block list are you using, maybe share whitelisted entries as well. Also what browser are you using? is your traffic being diverted by a browser DNS server? Do a www.dnsleaktest.com to confirm your traffic is actually traversing the proper DNS servers. Also make sure you are not using alternative ad blocking scripts that might hinder how Diversion+pixelservtls process's the blocked domain.

 

[Ellenswamy]

I made a change last night to force doh through the router instead of having set to auto. Doing some testing and went to some sites like cnn and nytimes and making sure my ad blockers are turned off on my web browser.
dns leak shows quad9 which is correct, all of my devices point to 192.168.50.1 as dns. Dns filter is set to router. Just making sure I am not missing anything else while I test?

 

[SomeWhereOverTheRainBow]

I made a change last night to force doh through the router instead of having set to auto. Doing some testing and went to some sites like cnn and nytimes and making sure my ad blockers are turned off on my web browser.
dns leak shows quad9 which is correct, all of my devices point to 192.168.50.1 as dns. Dns filter is set to router. Just making sure I am not missing anything else while I test?

Should be good, if you are using firefox, make sure these settings look like this.

 

[Ellenswamy]

Should be good, if you are using firefox, make sure these settings look like this.
View attachment 40841

View attachment 40842

I am using Safari. I checked and I don't see this option for Safari unless I missed it

 

[SomeWhereOverTheRainBow]

I am using Safari. I checked and I don't see this option for Safari unless I missed it

This could also be an issue specific to iOS and their policies for handling certificates (specifically pixelserv-tls uses user made certificates.).