Diversion Diversion - the Router Ad-Blocker

[HairyA00]

If you have Skynet installed, it has an IoT Blocking setting.

Also a great idea!

 

[dugaduga]

Why incoming CNAME filtering is now, and will become more of a necessity over time:

uBlock Origin for Firefox addresses new first-party tracking method
gHacks Technology News / Martin Brinkmann

The latest version of the content blocker uBlock Origin for the Mozilla Firefox web browser includes a new feature to detect a new first-party tracking method that some sites have started to use recently.

The issue was first reported ten days ago by user Aeris on the project's official GitHub page. Some sites started to use canonical name records (CNAMEs) to bypass filters used in content blockers. First-party resources, e.g. a subdomain, are not blocked usuall unless they are known to only serve advertisement.

The main issue from a content blocking perspective is that identification and detection is difficult. The extensions would have to uncloak alias hostnames in order to provide the user with information and the ability to do something about it.

Raymond Hill, the developer of uBlock Origin, found a way to address the new first-party tracking method in Mozilla Firefox.

Side-note: Why only Firefox? Because Mozilla has created DNS APIs that may be used to expose the CNAME while Google has not. For now, it is not possible to protect against this form of tracking in Google Chrome. Hill writes "Best to assume it can't be fixed on Chromium if it does not support the proper API".

Firefox users who upgrade to the latest version of uBlock Origin, may notice a new permission request (Access IP address and hostname information). This is required to unlock access to the DNS API in the browser extension.

Firefox users who run the extension need to do the following to set things up properly on their end:

1. Open the Settings of the extension, e.g. from about:addons or by clicking on the dashboard icon in the uBlock Origin interface.
2. Check the "I am an advanced user" box on the first page that opens.
3. Activate the settings icon next to the option to open the advanced settings.
4. Change the value of the parameter cnameAliasList to *.

The change runs the actual hostnames through the filtering that uBlock Origin applies again. The log highlights these in blue.

Network requests for which the actual hostname differs from the original hostname will be replayed through uBO's filtering engine using the actual hostname. [..] Regardless, uBO is now equipped to deal with 3rd-party disguised as 1st-party as far as Firefox's browser.dns allows it.

The setting of the wildcard means that the process is done for any hostname that differs; this works but it means that a certain number of network requests are processed twice by uBlock Origin.

The next step is for me to pick a cogent way for filter list maintainers to be able to tell uBO to uncloak specific hostnames, as doing this by default for all hostnames is not a good idea -- as this could cause a huge amount of network requests to be evaluated twice with no benefit for basic users (default settings/lists) while having to incur a pointless overhead -- for example when it concerned CDNs which are often aliased to the site using them.

Hill wants to switch to using a maintained list of known offenders that uBlock Origin (UMatrix will support this as well) will process while leaving any other hostname untouched.

Closing Words
Firefox users may change the configuration to make sure that they are protected against this new form of tracking. Chromium users cannot because the browser's APIs for extensions does not have the capabilities at the time of writing.

Thank you for being a Ghacks reader. The post uBlock Origin for Firefox addresses new first-party tracking method appeared first on gHacks Technology News.



Original Article: https://www.ghacks.net/2019/11/20/u...ox-addresses-new-first-party-tracking-method/

 

[dugaduga]

CNAME Cloaking, the dangerous disguise of third-party trackers
https://medium.com/nextdns/cname-cl...disguise-of-third-party-trackers-195205dc522a

"This [cname filtering] feature has already been implemented [in dnscrypt] 2 days ago and you can expect it to be in the next release. See issue #1067"

https://github.com/DNSCrypt/dnscrypt-proxy/issues/1071#issuecomment-558447586

To get the same effect in diversion, dnsmasq will have to adopt this feature. It was reported earlier in here, that it already has been, for A queries, but not AAAA queries.

 

[Grisu]

You only forgot to mention that you will need beta version of uBlock to get this new feature!
Note: The version is currently available as a beta version. It may take some time before it becomes available to stable users (version 1.24 required)

 

[Jumpstarter]

I think i have noticed a speed bump in diversion's DNSMASQ setting section....

it mainly concerns bogus-priv and domain-needed functions
When it gets enabled it allows for it to be enabled twice in DNSMASQ.conf, note mine is automatically on in DNSMASQ, but when I use diversion to enable it, it allows for it to make two entries inside DNSMASQ.conf

the first place they appear listed is under the expanded-hosts option inside dnsmasq.

the second place it is listed is inside the diversion directives under add-hosts list....

Would this cause a problem for DNSMASQ if it is listed twice?
I can see that if you select No inside Diversion, that it should remove those entries from dnsmasq, but I don't think selected Yes should allow the entries to appear twice in the .conf file.

 

[Twiglets]

Re: Address 1st-party tracker blocking

This also picked up by SANS NewsBites Vol. 21 Num. 093.
If you follow the link to the uBlock issue it references a 1st Party Tracker Host list that can be imported directly into pihole & Diversion.

https://github.com/uBlockOrigin/uBlock-issues/issues/780#issuecomment-559340435

Regularly updated list is:
https://hostfiles.frogeye.fr/firstparty-trackers-hosts.txt

This may be of interest to others.

 

[Twiglets]

I think i have noticed a speed bump in diversion's DNSMASQ setting section....
View attachment 20036
it mainly concerns bogus-priv and domain-needed functions
When it gets enabled it allows for it to be enabled twice in DNSMASQ.conf, note mine is automatically on in DNSMASQ, but when I use diversion to enable it, it allows for it to make two entries inside DNSMASQ.conf
View attachment 20037
the first place they appear listed is under the expanded-hosts option inside dnsmasq.
View attachment 20038
the second place it is listed is inside the diversion directives under add-hosts list....

Would this cause a problem for DNSMASQ if it is listed twice?
I can see that if you select No inside Diversion, that it should remove those entries from dnsmasq, but I don't think selected Yes should allow the entries to appear twice in the .conf file.

As far as I am aware, you can have multiple entries in the dnsmasq.conf for the same parameter and it simply uses the last entry.
[I have 'played' with the dnsmasq.conf file and found this out the hard way when I 'accidentally' set a parameter twice, once at the top of the file and once at the bottom !!! ]

 

[Jumpstarter]

As far as I am aware, you can have multiple entries in the dnsmasq.conf for the same parameter and it simply uses the last entry.
[I have 'played' with the dnsmasq.conf file and found this out the hard way when I 'accidentally' set a parameter twice, once at the top of the file and once at the bottom !!! ]

yes I understand DNSMASQ will still function, I want to know is there any performance impact or will it try to run the option "twice" using more system resources, etc etc.

 

[bitmonster]

Could Diversion possibly knock out the WAN? I get intermittent WAN dropouts every few days or so which rebooting the router fixes. I notice a WiFi symbol on my phone (Samsung S10) with an exclamation mark when it happens.

So I looked in my traffic graph and noticed a big download died around 2am, and looked in to the system logs and could only see Diversion updating stuff.. Is there any way this could be linked? Seems odd but never know.

I have DNS set to Cloudflare too, so could be something funky anywhere. But at least I now know the dcp related kernel errors in the logs are hardware related so that's an issue

Nov 29 02:00:00 Central uiDivStats: Starting Diversion statistic generation...
Nov 29 02:00:01 Central kernel: echo (26625): drop_caches: 3
Nov 29 02:01:50 Central uiDivStats: Diversion statistic generation completed successfully!
Nov 29 02:03:52 Central Diversion: pgl.yoyo.org is not hosts file, keeping blocking list, from /opt/share/diversion/file/update-bl.div
Nov 29 02:17:08 Central Diversion: https://hosts-file.net/emd.txt not hosts file, using backup file, from /opt/share/diversion/file/update-bl.div
Nov 29 02:21:05 Central Diversion: https://hosts-file.net/exp.txt not hosts file, using backup file, from /opt/share/diversion/file/update-bl.div
Nov 29 02:24:56 Central Diversion: https://hosts-file.net/hjk.txt not hosts file, using backup file, from /opt/share/diversion/file/update-bl.div
Nov 29 02:28:47 Central Diversion: https://hosts-file.net/mmt.txt not hosts file, using backup file, from /opt/share/diversion/file/update-bl.div
Nov 29 02:32:38 Central Diversion: https://hosts-file.net/psh.txt not hosts file, using backup file, from /opt/share/diversion/file/update-bl.div
Nov 29 02:36:33 Central Diversion: https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts not hosts file, using backup file, from /opt/share/diversion/file/update-bl.div
ov 29 02:36:54 Central Diversion: updated Standard+ blocking list from 6 hosts files, 1071238 domains are now blocked, from /opt/share/diversion/file/update-bl.div
Nov 29 02:36:59 Central Diversion: updated and reset weekly ads counter: 92,989 total, 21,558 this week, 59 new since last count, from /opt/bin/diversion
Nov 29 02:36:59 Central Diversion: reset dnsmasq log files (weekly cron job), from /opt/share/diversion/file/update-bl.div

 

[Treadler]

Re: Address 1st-party tracker blocking

This also picked up by SANS NewsBites Vol. 21 Num. 093.
If you follow the link to the uBlock issue it references a 1st Party Tracker Host list that can be imported directly into pihole & Diversion.

https://github.com/uBlockOrigin/uBlock-issues/issues/780#issuecomment-559340435

Regularly updated list is:
https://hostfiles.frogeye.fr/firstparty-trackers-hosts.txt

This may be of interest to others.

Nice work!

 

[joe scian]

Hi Lonely coder
When I try and install Diversion from amtm 3.0 I get
Diversion installation failed.

I updated firmware on AC5300 to latest beta and ended up with diversion option missing in amtm. When I try and reinstall i get Diversion installation failed message above. Any ideas?

 

[loady]

Hi,

Got this installed. Im not looking to block ads on every single website in the www and seems to work directly connected to my router (going to test remote VPN connection next), Ads were not getting blocked so i updated the host file to the large one, i have a 16gb USB plugged in back, do i need to enable the swap file like it recommended ?..also on the PC that is directly connected to router, i have NOD32 running, it is CONSTANTLY throwing windows out since the Ads have been blocked, i looked closer at one and it referenced pixelserv when i clicked the link, im guessing thats it doing its job ?, what do i do about the constant complaints from NOD32, screenshot below. I keep trying to select 'remember action for this cert' but NOD is saying
"protocol filtering problem, failed to remember cert"

 

[dave14305]

View attachment 20058 Hi,

Got this installed. Im not looking to block ads on every single website in the www and seems to work directly connected to my router (going to test remote VPN connection next), Ads were not getting blocked so i updated the host file to the large one, i have a 16gb USB plugged in back, do i need to enable the swap file like it recommended ?..also on the PC that is directly connected to router, i have NOD32 running, it is CONSTANTLY throwing windows out since the Ads have been blocked, i looked closer at one and it referenced pixelserv when i clicked the link, im guessing thats it doing its job ?, what do i do about the constant complaints from NOD32, screenshot below. I keep trying to select 'remember action for this cert' but NOD is saying
"protocol filtering problem, failed to remember cert"

Import the pixelserv CA into Windows (and all your devices where possible) following the Wiki instructions.

https://github.com/kvic-z/pixelserv...ificate#import-pixelserv-ca-on-client-devices

http://192.168.11.1/ca.crt

 

[Twiglets]

yes I understand DNSMASQ will still function, I want to know is there any performance impact or will it try to run the option "twice" using more system resources, etc etc.

The .conf file is a list of instructions to dnsmasq to configure certain attributes when dnsmasq starts.
Having repeated options on the .conf file will set/reset those attributes *each* time BUT the .conf file is read only once ..... at start up.
The use of resources will be virtually the same as the attribute will set/reset the same memory 'structures' each time ..... therefore apart from transient memory allocation/deallocation etc it is the same.
[If you are at the level where a small number of bytes makes a 'difference' ..... you have greater problems than 'resource use' and need to look at your whole setup ... i.e. everything you are running in the background & their configuration. ]

 

[loady]

Import the pixelserv CA into Windows (and all your devices where possible) following the Wiki instructions.

https://github.com/kvic-z/pixelserv...ificate#import-pixelserv-ca-on-client-devices

http://192.168.11.1/ca.crt

I have done that, works fine on my android but not on the laptop, well it works, ads are blocked but nod 32 keeps hammering me with those warnings

 

[loady]

I use an app called shpock, it's a boot sale app, diversion doesn't seem to work with it, is it because it's an app as opposed to a browser interface

 

[Treadler]

I have done that, works fine on my android but not on the laptop, well it works, ads are blocked but nod 32 keeps hammering me with those warnings

Maybe the steps in post #4325 might be the go?

 

[dave14305]

I have done that, works fine on my android but not on the laptop, well it works, ads are blocked but nod 32 keeps hammering me with those warnings

Post a clear screenshot of the Certification Path tab of the certificate showing if the Pixelserv CA is trusted or untrusted. Maybe you imported into the wrong certificate store in Windows.

 

[loady]

Maybe the steps in post #4325 might be the go?

i tried to install the newer pixel crt.....

 This updates installed Entware packages.

 Entware version: Entware (armv7sf-k3.2)
 Installed from: bin.entware.net

 1. show pixelserv-tls info
 2. show installed packages
 3. update or upgrade pixelserv-tls
 4. update list of available packages
 5. update and upgrade installed packages

 Enter selection [1-5 e=Exit] 3
____________________________________________________

 This updates or upgrades pixelserv-tls v2.2.1

 1. Update pixelserv-tls, regular Entware version
 2. Upgrade pixelserv-tls, regular Entware version
 3. Upgrade pixelserv-tls to v2.3.0, Jack Yaz version

 Enter your selection [1-3 e=Exit] 3
____________________________________________________

 This upgrades pixelserv-tls to v2.3.0, Jack Yaz version.
 This version is compliant with the new required
 security settings enforced by Apple and other Companies.
 See https://github.com/jackyaz/pixelserv-tls/releases/tag/2.3.0

 It will install the  appropriate version for your
 Entware (armv7sf-k3.2) installation.

 After successful upgrade, purge and re-generate the
 CA certificate in  ep , 3, 2.

 Continue? [1=Yes e=Exit] 1
____________________________________________________

  i  Downloading pixelserv-tls

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   629    0   629    0     0   2428      0 --:--:-- --:--:-- --:--:--  2526
100 26936  100 26936    0     0  34139      0 --:--:-- --:--:-- --:--:-- 34139

  i  Download successful, installing...

Package pixelserv-tls (2.2.1-1) installed in root is up to date.

 Diversion 4.1.6                  by thelonelycoder

 RT-AC86U (aarch64) FW-384.13 @ 192.168.11.1

 1.213M  blocked domains by  1  hosts file(s)
 649 t  649 w  649 n ads since Nov 29 17:20
____________________________________________________

 d   Diversion Standard   enabled
 c   communication        DivUn stats backup FWun

 a   ad-blocking          to IP 192.168.11.2
 l   logging              dnsmasq.log 32.0K

 ep  pixelserv-tls        192.168.11.2 v2.2.1

 b   blocking list        Large Thu @ 2:00
 el  edit lists            0 w  0 b  0 wb

 f   follow dnsmasq.log

 e   exit Diversion                 more options  o
____________________________________________________

 Done  Failed to update pixelserv-tls

 What do you want to do?

 

[loady]

Post a clear screenshot of the Certification Path tab of the certificate showing if the Pixelserv CA is trusted or untrusted. Maybe you imported into the wrong certificate store in Windows.