Diversion Diversion Installed - Lots of Ads Getting through

[NetowrkingNewb]

Hello, and thanks so much for all the work and help on this site and by the teams behind Diversion, Skynet, Unbound, etc.

I just got an RT-AX88U, almost entirely because of what I read about the capabilites of Merlin and amtm for network-wide adblocking. I'll be the first to admit that I have very little clue what I'm doing when it come to networking, so any help would be greatly appreciated. I don't even know where to find or how to view the dnsmasq log, other than following it live through the diversion cli.

I installed Diversion Standard, using pixelserv-tls and the Medium blocklist. I have type 65 blocking enabled. I set the routers DNS Filtering to "On", with global filter set to "router" and thought I was done.

When I started using my phone (Android), I noticed a bunch of ads, along with many ads that were blocked but with the giant white space where they would have been (pixelserv problem?). Hoping back to my computer, I fired up Edge since it's the only browser I had never used before and it doesn't have an adblocking extension installed. Nearly every adblock test site I visit fails on multiple items.

https://canyoublockit.com fails almost every item, https://adblock-tester.com fails: flash banners file loading, visibility (as it has the giant blocks where the ad would be), Gif image file loading and visibility, Static image file loading and visibility.

I kept throwing more ideas at this, and installed Unbound, Skynet (which broke reddit), and nothing seems to solve it. I'm going to uninstall everything but Diversion to give myself a clean slate, and once I get it working I'll look into adding other services back on.

Does anyone have any ideas what I'm screwing up? I have WAN DNS set to cloudflare, and DoT set to the cloudflare IPs at port 853. Surely I've configured something incorectly!

Thanks in advance for your ideas and help.

-NetworkingNewb

 

[Treadler]

Hello, and thanks so much for all the work and help on this site and by the teams behind Diversion, Skynet, Unbound, etc.

I just got an RT-AX88U, almost entirely because of what I read about the capabilites of Merlin and amtm for network-wide adblocking. I'll be the first to admit that I have very little clue what I'm doing when it come to networking, so any help would be greatly appreciated. I don't even know where to find or how to view the dnsmasq log, other than following it live through the diversion cli.

I installed Diversion Standard, using pixelserv-tls and the Medium blocklist. I have type 65 blocking enabled. I set the routers DNS Filtering to "On", with global filter set to "router" and thought I was done.

When I started using my phone (Android), I noticed a bunch of ads, along with many ads that were blocked but with the giant white space where they would have been (pixelserv problem?). Hoping back to my computer, I fired up Edge since it's the only browser I had never used before and it doesn't have an adblocking extension installed. Nearly every adblock test site I visit fails on multiple items.

https://canyoublockit.com fails almost every item, https://adblock-tester.com fails: flash banners file loading, visibility (as it has the giant blocks where the ad would be), Gif image file loading and visibility, Static image file loading and visibility.

I kept throwing more ideas at this, and installed Unbound, Skynet (which broke reddit), and nothing seems to solve it. I'm going to uninstall everything but Diversion to give myself a clean slate, and once I get it working I'll look into adding other services back on.

Does anyone have any ideas what I'm screwing up? I have WAN DNS set to cloudflare, and DoT set to the cloudflare IPs at port 853. Surely I've configured something incorectly!

Thanks in advance for your ideas and help.

-NetworkingNewb

Welcome to the forum.

All I can suggest is you make sure your client/s (phones, tablets, pc’s, whatever) when connecting via your network, are using the router as their DNS server. Otherwise, they will just be bypassing all the good work you’ve done with Diversion etc.

 

[NetowrkingNewb]

Welcome to the forum.

All I can suggest is you make sure your client/s (phones, tablets, pc’s, whatever) when connecting via your network, are using the router as their DNS server. Otherwise, they will just be bypassing all the good work you’ve done with Diversion etc.

Thanks for the tip. I have the router DNS filtering turned on, and I verified that my devices are indeed using the router's IP as their DNS server. If I monitor the Diversion log while loading one of those testing sites, I see some requests get blocked.

 

[Treadler]

Thanks for the tip. I have the router DNS filtering turned on, and I verified that my devices are indeed using the router's IP as their DNS server. If I monitor the Diversion log while loading one of those testing sites, I see some requests get blocked.

Maybe try installing the excellent uiDivStats script.
Makes available within the router’s GUI what’s going on with Diversion, blocked/not blocked as the case may be, without having to SSH.
Diversion as an ad blocking solution IMHO is bullet proof. There is something else going on here.

 

[dave14305]

Try turning off the hardcoded whitelist for snbforums.com as a test.

• NEW: Option to opt out to support smallnetbuilder.com ads in el, 1, Hard coded whitelist setting.

 

[NetowrkingNewb]

Try turning off the hardcoded whitelist for snbforums.com as a test.

Thanks. I tried that and I'm getting the same results. I just did a NVRAM reset of the router to start from scratch, and did the following:
-Setup SSID/password
-Set router IP to 192.168.1.1
-Set DHCP IP Pool to start at 192.168.1.70 to allow for some manual local IPs (including pixelserv)
-Set WAN DNS to the preset "Privacy-respecting Quad 9"
-Turned on DNS Filtering and set global to "Router"
-Enabled JFFS custom scripts
-Enabled SSH (LAN only)
-Installed Disc Check
-Formatted thumb drive as EXT4 with journaling and set label to "MERLIN_USB"
-Created 2GB Swap file on the thumb drive
-Installed nsrum
-Installed Diversion Standard
-Installed ntpMerlin
-Enabled type 65 blocking in Diversion
-Updated blocking list to "Medium"
-opted out of hardcoded whitelist using el, 1, 8
-Verified my devices are using 192.168.1.1 as their DNS server

Everything is as before. Still seeing some ads, and getting broken image placeholders for many blocked ads.

The placeholders showing up makes me think it could be something wrong with my pixelserv setup? I see the following in my pixelserv-tls stats, which looks suspicious to me (although I don't really know what I'm doing):

slh	0	# of accepted HTTPS requests
slm	45	# of rejected HTTPS requests (missing certificate)
sle	0	# of rejected HTTPS requests (certificate available but not usable)
slc	11	# of dropped HTTPS requests (client disconnect without sending any request)
slu	148	# of dropped HTTPS requests (other TLS handshake errors)

uca	0	slu break-down: # of unknown CA reported by clients
ucb	0	slu break-down: # of bad certificate reported by clients
uce	148	slu break-down: # of unknown cert reported by clients
ush	0	slu break-down: # of shutdown by clients after ServerHello
sct	7	cert cache: # of certs in cache
sch	154	cert cache: # of reuses of cached certs
scm	7	cert cache: # of misses to find a cert in cache
scp	0	cert cache: # of purges to give room for a new cert
ssh	26	sess cache: # of reuses of cached TLS sessions
ssm	0	sess cache: # of misses to find a TLS session in cache
ssp	0	sess cache: # of purges to give room for a new TLS session

 

[visortgw]

Thanks. I tried that and I'm getting the same results. I just did a NVRAM reset of the router to start from scratch, and did the following:
-Setup SSID/password
-Set router IP to 192.168.1.1
-Set DHCP IP Pool to start at 192.168.1.70 to allow for some manual local IPs (including pixelserv)
-Set WAN DNS to the preset "Privacy-respecting Quad 9"
-Turned on DNS Filtering and set global to "Router"
-Enabled JFFS custom scripts
-Enabled SSH (LAN only)
-Installed Disc Check
-Formatted thumb drive as EXT4 with journaling and set label to "MERLIN_USB"
-Created 2GB Swap file on the thumb drive
-Installed nsrum
-Installed Diversion Standard
-Installed ntpMerlin
-Enabled type 65 blocking in Diversion
-Updated blocking list to "Medium"
-opted out of hardcoded whitelist using el, 1, 8
-Verified my devices are using 192.168.1.1 as their DNS server

Everything is as before. Still seeing some ads, and getting broken image placeholders for many blocked ads.

The placeholders showing up makes me think it could be something wrong with my pixelserv setup? I see the following in my pixelserv-tls stats, which looks suspicious to me (although I don't really know what I'm doing):

slh	0	# of accepted HTTPS requests
slm	45	# of rejected HTTPS requests (missing certificate)
sle	0	# of rejected HTTPS requests (certificate available but not usable)
slc	11	# of dropped HTTPS requests (client disconnect without sending any request)
slu	148	# of dropped HTTPS requests (other TLS handshake errors)

uca	0	slu break-down: # of unknown CA reported by clients
ucb	0	slu break-down: # of bad certificate reported by clients
uce	148	slu break-down: # of unknown cert reported by clients
ush	0	slu break-down: # of shutdown by clients after ServerHello
sct	7	cert cache: # of certs in cache
sch	154	cert cache: # of reuses of cached certs
scm	7	cert cache: # of misses to find a cert in cache
scp	0	cert cache: # of purges to give room for a new cert
ssh	26	sess cache: # of reuses of cached TLS sessions
ssm	0	sess cache: # of misses to find a TLS session in cache
ssp	0	sess cache: # of purges to give room for a new TLS session

Did you install pixelserv certificate on clients?

 

[NetowrkingNewb]

ahhh, I actually just noticed that while exploring the ep menu. Trying to figure out how to install the cert now. Going to 192.168.1.2/ca.cert on my android (chrome) doesn't seem to do anything... commencing Google search

-edit: Found directions in the Diversion changelog to follow this guide: https://github.com/kvic-z/pixelserv-tls/wiki/Create-and-Import-the-CA-Certificate
that says it's as simple as visiting http://pixelserv ip/ca.crt, but when I direct any browser (chrome, brave, edge, mobile chrome) to the correct address for my setup (http://192.168.1.2/ca.cert) I get nothing but a blank screen.

 

[visortgw]

ahhh, I actually just noticed that while exploring the ep menu. Trying to figure out how to install the cert now. Going to 192.168.1.2/ca.cert on my android (chrome) doesn't seem to do anything... commencing Google search

-edit: Found directions in the Diversion changelog to follow this guide: https://github.com/kvic-z/pixelserv-tls/wiki/Create-and-Import-the-CA-Certificate
that says it's as simple as visiting http://pixelserv ip/ca.crt, but when I direct any browser (chrome, brave, edge, mobile chrome) to the correct address for my setup (http://192.168.1.2/ca.cert) I get nothing but a blank screen.

Download cert to Android phone. In device Settings, navigate to Biometrics and security, and then to Other security settings. Click on Install from phone storage.

 

[dave14305]

setup (http://192.168.1.2/ca.cert) I get nothing but a blank screen.

The extension is .crt

 

[NetowrkingNewb]

The extension is .crt

LOL! What a simple mistake. I saw crt and thought "of course, it's a certificate file" and I typed in "cert". Thanks for catching that!

I've properly installed the certificate now, but adblocking performance is still the same

 

[bluepoint]

LOL! What a simple mistake. I saw crt and thought "of course, it's a certificate file" and I typed in "cert". Thanks for catching that!

I've properly installed the certificate now, but adblocking performance is still the same

Make sure your clients doesn't use DOH.

 

[NetowrkingNewb]

Make sure your clients doesn't use DOH.

Thanks for the idea. I verified and disabled it in Edge to re-run tests. Same results ;-(

 

[bluepoint]

Thanks for the idea. I verified and disabled it in Edge to re-run tests. Same results ;-(

Edge have DOH settings?

Update: Yes if you choose to use service provider in the edge dns settings and elected to use google.

 

[NetowrkingNewb]

I haven't had any luck. Still resorting to using an adblocker in my browser on PC, and Blokada on the phone.

To verify if I should keep trying to troubleshoot this: everyone else passes the tests on sites like canublockit with nothing but Diversion? No additional ad-blocking? My Diversion stats show that it is blocking some ads, but a lot of them get through.