Diversion Diversion - the Router Ad-Blocker

[dave14305]

I mean perhaps Amazon is doing something unorthodox?

Maybe they’re not using ports 80 or 443?

 

[HairyA00]

Maybe they’re not using ports 80 or 443?

Not sure; haven't packet sniffed other than the DNS records, which is going over port 853 in my case. Will keep an eye out for any updates to pixelserv-tls, but I'm not hopeful; sounds like this issue has been going on for years. Dumping to 0.0.0.0 it is for me!

 

[Solex]

Hello everyone

I'm new to the forum as a user, but I've been lurking and using Diversion/ab-solution for a number of years. It's a great piece of software which I really like to have on my network. You always appreciate others' work even more when you know how much time goes into these sorts of projects, not just from the original author(s), but also from the communities which help test and report issues on them too. So first of all - thanks.

I wanted to post here today because I found a windows bug which might be secretly affecting others so I wanted to share my symptoms, system and os info and also the solution I found which appears to have done the trick. There might even be a better solution you guys/girls can share so here goes - please feel free to school me if I've missed anything obvious.

System info

Diversion: 4.1.3 / pixelserv-tls 2.2.1 / Standard / Medium List
Router: Asus RT-AC68U / Asuswrt-Merlin 384.12
PC OS: Microsoft Windows 10 Pro / 10.0.18362 Build 18362
PC Spec: i5-9600K CPU @ 4.8GHz / 16GB DDR4

A month or so ago my PC was running out of memory over a period of time, and due to windows liking to hide what's-actually-doing-what in obscure locations it took me a while to work out that it appeared to be related to Diversion. I've been a bit busy recently so I disabled Diversion and the problem went away. To be clear, this is the PC suffering these issues, NOT the router (like I've seen in the forum that others have had with some older versions of Diversion).

What I was seeing is when Diversion was on, "Connected Devices Platform User Service" would constantly use up 10-15% of the CPU when it should be idling at around 1-3%. This in turn results in a memory leak which eventually consumes all 16GB of the system, then the System process then jumps in and desperately tries to garbage collect and grinds the whole thing to a halt. After a bit of googling, this Connected Devices Platform User Service is something to do with Bluetooth and printers and other BS. I presume (because there's not a whole lot out there on it, but plenty of people with problems with it in various scenarios) it's trying to do something which Diversion is somehow obstructing. I followed the dnsmasq log and added everything which appeared in the block list to see if I could narrow down which domain (if any) was causing the issue. Eventually I ran out of domains which were coming up (the usual microsoft stats stuff, bitwarden, nvidia, nothing unexpected) but the problem was still there.

It turns out you can't disable automatic startup of the Connected Devices Platform User Service through windows' services.msc as it just throws a "This parameter is incorrect" error - even through safe-mode and run as administrator. I found a way to corrupt it, but that did not feel like the right way to go so I looked for whatever called it. The similarly named "Connected Devices Platform Service" (notice no user). Disable this through the usual method and this kills the problem dead. So far I've noticed no adverse effects, and with some further research, this service is actually disabled by default with many versions of Windows.

So I guess my question are - how could Diversion be tripping this Windows bug? Should I be looking in one of the other logs? Does the app divert items other than DNS? Is there something I've missed? (Most certainly possible).

It appears to be solved for me, but if others here on this forum have had suspicions of something like this going on, this could be related.

Here's some links to some of the bits I uncovered:

CDPSVC Defaults in various versions of Windows
http://servicedefaults.com/10/cdpsvc/

Connected Devices Platform Service - Windows 10 Service
http://batcmd.com/windows/10/services/cdpsvc/

What does it even do?
https://social.technet.microsoft.co...vice-platform-service?forum=win10itprogeneral

 

[Ozioh]

I imported pixelserv certificate using https://support.comodo.com/index.php?/comodo/Knowledgebase/Article/View/636/17/
This took care of the certificate error that I was getting.
I followed it on dnsmasq per a recommendation from snb user.
Now I am getting this from dnsmasq.
I checked blocking list (which I am using just the standart list installed with diversion. So, no extra lists or additions.)
I can't find zerohedge.com on the list.
19:59:40 dnsmasq[29321]: 16 192.168.1.45/53350 query[A] www.zerohedge.com from 192.168.1.45
19:59:40 dnsmasq[29321]: 16 192.168.1.45/53350 blocked by blockinglist www.zerohedge.com is 192.168.1.2
19:59:40 dnsmasq[29321]: 17 192.168.1.45/63402 query[A] www.zerohedge.com from 192.168.1.45
19:59:40 dnsmasq[29321]: 17 192.168.1.45/63402 blocked by blockinglist www.zerohedge.com is 192.168.1.2


192.168.1.2 is pixelserv ip
192.168.1.45 is my laptop ip. Same results if I use any other device.

I only have amtm and diversion installed. No skynet or any others.

Similarly,
express.co.uk
20:03:52 dnsmasq[29321]: 115 192.168.1.45/50306 query[A] www.express.co.uk from 192.168.1.45
20:03:52 dnsmasq[29321]: 115 192.168.1.45/50306 blocked by blockinglist www.express.co.uk is 192.168.1.2

Getting the same result.

I am getting a blank page without any errors or explanation.

Any help is appreciated.

thank you.

 

[dave14305]

I imported pixelserv certificate using https://support.comodo.com/index.php?/comodo/Knowledgebase/Article/View/636/17/
This took care of the certificate error that I was getting.
I followed it on dnsmasq per a recommendation from snb user.
Now I am getting this from dnsmasq.
I checked blocking list (which I am using just the standart list installed with diversion. So, no extra lists or additions.)
I can't find zerohedge.com on the list.
19:59:40 dnsmasq[29321]: 16 192.168.1.45/53350 query[A] www.zerohedge.com from 192.168.1.45
19:59:40 dnsmasq[29321]: 16 192.168.1.45/53350 blocked by blockinglist www.zerohedge.com is 192.168.1.2
19:59:40 dnsmasq[29321]: 17 192.168.1.45/63402 query[A] www.zerohedge.com from 192.168.1.45
19:59:40 dnsmasq[29321]: 17 192.168.1.45/63402 blocked by blockinglist www.zerohedge.com is 192.168.1.2


192.168.1.2 is pixelserv ip
192.168.1.45 is my laptop ip. Same results if I use any other device.

I only have amtm and diversion installed. No skynet or any others.

Similarly,
express.co.uk
20:03:52 dnsmasq[29321]: 115 192.168.1.45/50306 query[A] www.express.co.uk from 192.168.1.45
20:03:52 dnsmasq[29321]: 115 192.168.1.45/50306 blocked by blockinglist www.express.co.uk is 192.168.1.2

Getting the same result.

I am getting a blank page without any errors or explanation.

Any help is appreciated.

thank you.

Those sites are in the StevenBlack fakenews hosts file. https://github.com/StevenBlack/hosts/blob/master/extensions/fakenews/hosts

What blocking lists do you use in Diversion?

 

[Ozioh]

Thank you very much.
I was using standard which includes, as you have pointed, fakenews.
So, I changed it to small blocking list.
I can access these sites now.

 

[HairyA00]

Thank you very much.
I was using standard which includes, as you have pointed, fakenews.
So, I changed it to small blocking list.
I can access these sites now.

I brought that up here. While the fakenews filter likely blocks a lot of malicious domains as well, they belong in a malicious blocklist. "fakenews" is arbitrary:
https://www.snbforums.com/threads/recommendation-to-change-default-diversion-blocklists.57741/

 

[thelonelycoder]

Minor update for Diversion, no version change

- Reverting Stephen Blacks hosts list to default, without the blocked fake news and gambling domains. I realize this causes more grief than necessary for some.
This change only takes effect when changing or re-selecting a pre-defined blocking list in b, followed by a manual update of the blocking list(s)
- Diversion update notification: Changed update URL for amtm to the Diversion server since amtm is now hosted there

Edit: The StevenBlack hosts file change affect the Standard, Medium and Large pre-defined blocking lists in Diversion.

Use u to update to this latest version

 

[HairyA00]

Minor update for Diversion, no version change

- Reverting Stephen Blacks hosts list to default, without the blocked fake news and gambling domains. I realize this causes more grief than necessary for some.
This change only takes effect when changing or re-selecting a pre-defined blocking list in b, followed by a manual update of the blocking list(s)
- Diversion update notification: Changed update URL for amtm to the Diversion server since amtm is now hosted there

Use u to update to this latest version

Thank you @thelonelycoder!

 

[thelonelycoder]

So, no known solution at this time short of bailing from WiFi when using the Android Amazon App? Are there other apps and/or domains that won't work with pixelserv-tls installed?

It's not pixelserv-tls that is the problem. It's one or more of the domains that are blocked. Use f to find out which one(s) are causing it for you. This is location dependent, so not everyone needs to whitelist the same domains.

With Diversion Lite installed or pixelserv-tls disabled, Diversion only blocks http domains. When pixelserv-tls is enabled https (secure) domains are blocked as well.
In your case the blocked domain is not blocked because it runs on the https protocol. The vast majority of ad servers run on https these days.

 

[thelonelycoder]

Thank you @thelonelycoder!

Sorry for my belated reply but things have been hot around here (and still are). Working on the PC in a non-air-conditioned room is not high on my list of favorite things to do in my spare time.

 

[HairyA00]

It's not pixelserv-tls that is the problem. It's one or more of the domains that are blocked. Use f to find out which one(s) are causing it for you. This is location dependent, so not everyone needs to whitelist the same domains.

With Diversion Lite installed or pixelserv-tls disabled, Diversion only blocks http domains. When pixelserv-tls is enabled https (secure) domains are blocked as well.
In your case the blocked domain is not blocked because it runs on the https protocol. The vast majority of ad servers run on https these days.

I've tailed the log; no domains popup when that error appears. My last test is to have a blank blocklist and see if it happens. That'll kind of narrow it down absolutely.

I ran privoxy at home for about 1 month to test it out. Realizing that it was only http requests, I figured it would be useless. Turned out to be wrong; almost 80% of 'blocked' domains were still http requests. I found that only 1 in 5 ads were served over https. That was with approx a month's worth of data points.

I ordered a new USB drive because the one I have is shot; I am going to setup everything from scratch and give pixelserv-tls a try again (probably this weekend). Will share my findings...

Thanks for being so responsive and helpful @thelonelycoder

 

[Mutzli]

Is there an option that still includes the fakenews + gambling blacklists?

 

[HairyA00]

Is there an option that still includes the fakenews + gambling blacklists?

https://github.com/StevenBlack/hosts

 

[thelonelycoder]

Is there an option that still includes the fakenews + gambling blacklists?

Customize your blocking list by removing the existing URL and pick and add your URL from the RAW Hosts column here: https://github.com/StevenBlack/hosts

 

[Mutzli]

https://github.com/StevenBlack/hosts

thanks

Customize your blocking list by removing the existing URL and pick and add your URL from the RAW Hosts column here: https://github.com/StevenBlack/hosts

Thank you that worked, added a few more. Btw. happy Swiss National Day

 

[HawkInOz]

Minor update for Diversion, no version change

- Reverting Stephen Blacks hosts list to default, without the blocked fake news and gambling domains. I realize this causes more grief than necessary for some.
This change only takes effect when changing or re-selecting a pre-defined blocking list in b, followed by a manual update of the blocking list(s)
- Diversion update notification: Changed update URL for amtm to the Diversion server since amtm is now hosted there

Edit: The StevenBlack hosts file change affect the Standard, Medium and Large pre-defined blocking lists in Diversion.

Use u to update to this latest version

I updated Diversion, but when I go to re-select the Blocking List to use, it appears that the descriptions of those lists still need updating:

Select predefined blocking list to use.
Your current type is Medium

1. Small
someonewhocares.org, pgl.yoyo.org,
github.com/hoshsadiq/adblock-nocoin-list,
zerodot1.gitlab.io/CoinBlockerLists/hosts.
Restricted blocking of Ads. If unsure, start here.
Filesize: ~840 KB, ~42,000 blocked hosts.

2. Standard
github.com/StevenBlack fakenews + gambling.
Good mix of blocked domains.
Filesize: ~850 KB, ~44,000 blocked hosts.

3. Medium
github.com/StevenBlack fakenews, gambling, porn.
adblock.mahakala.is.
Comprehensive list of blocked domains.
Filesize: ~4.9 MB, ~270,000 blocked hosts.

4. Large
github.com/StevenBlack fakenews, gambling, porn.
hosts-file.net, hphosts-partial (always latest),
ad_servers and grm.
Be careful, this blocks a lot! Use only if you
know how to use the whitelist. You have been warned!
Filesize: ~20.1 MB, ~953,000 blocked hosts.

Set new type: [1-4 e=Exit]

 

[gattaca]

Customize your blocking list by removing the existing URL and pick and add your URL from the RAW Hosts column here: https://github.com/StevenBlack/hosts

Can someone please provide an example of exactly HOW to add these back in or point the folks following this thread? I'm sorry I just don't know and I do not want to mess up what's working great!. Thank You.

 

[HairyA00]

Can someone please provide an example of exactly HOW to add these back in. I'm sorry I just dont know. Thanks.

amtm > diversion (1) > blocking list (b) > Change composition Standard (1) > Customize hosts list (2) > Add hosts list (1) > Add your list

Example list: https://raw.githubusercontent.com/StevenBlack/hosts/master/alternates/fakenews-gambling/hosts

 

[gattaca]

^^^ TY!