Diversion Diversion - the Router Ad-Blocker

[ScenicView]

I have a security / privacy question. If I upgrade to Diversion standard to enable the https filtering, does that mean that Diversion can now read / filter all of my https traffic?

 

[thelonelycoder]

I have a security / privacy question. If I upgrade to Diversion standard to enable the https filtering, does that mean that Diversion can now read / filter all of my https traffic?

No. Diversion does not see more than the router does: The requested domain. And if the domain is blocked, it's blocked just as with an http request.

 

[dave14305]

I have a security / privacy question. If I upgrade to Diversion standard to enable the https filtering, does that mean that Diversion can now read / filter all of my https traffic?

No, Diversion is not a proxy of any kind. It will only manage hosts entries to be added to dnsmasq to prevent “undesirable” sites from working. Through Diversion logging, dnsmasq will log all your dns queries for troubleshooting and stats reporting, but both are optional.

If you use Pixelserv with Diversion, it can see the full http/https request that was made to the blocked host, but only the blocked hosts. Normal traffic never gets to Pixelserv.

 

[JJohnson1988]

As @dave14305 says, the IPv6 entries are not added when the nvram get ipv6_service is set to disabled. In all other cases, the IPv6 entry "::" is added for each IPv4 domain entry.

IPv6 is enabled for me, so I suppose those entries do make sense after all.

However, the issue still remains where the blacklist is being flooded with duplicate IPv6 entries upon update of blocking lists. I can provide an example if necessary.

Sent from my A0001 using Tapatalk

 

[thelonelycoder]

IPv6 is enabled for me, so I suppose those entries do make sense after all.

However, the issue still remains where the blacklist is being flooded with duplicate IPv6 entries upon update of blocking lists. I can provide an example if necessary.

Sent from my A0001 using Tapatalk

Please post the the content of diversion.conf, use sf to show the file.
Also, some lines of duplicate entries would be helpful.
Be aware that for each domain a IPv4 AND an IPv6 entry is how this works.

This example is normal when IPv6 is enabled in the blacklist and the blocking list.
With 0.0.0.0 being whatever your blocking IP is set to:

0.0.0.0.0 domain.com
:: domain.com
0.0.0.0.0 otherdomain.com
:: otherdomain.com

 

[JJohnson1988]

diversion.conf:

START FILE, --- lines are not part of file
 ---------------------------------------------------
 ### DO NOT EDIT THIS FILE ###

 ## General settings ##
 NAME=Diversion
 thisVERSION=4.0
 thisM_VERSION=6
 EDITION=Standard
 THEME=basic
 INSTALL_URL=https://diversion.ch/diversion
 DIVERSION_STATUS=enabled
 adblocking=on
 logging=on
 editorColor=on
 editorPaginate=20
 editorAutowww=on
 shHistory=on

 ## Communication settings ##

 ## Blocking file settings ##
 bfType=Standard
 bfPlusHosts=on
 bfUpdateDay=Monday
 bfUpdateDOW=Mon
 bfUpdateHour=6
 bfUpdateLastRun="Jan 12 00:10:21"
 bfUpdatePrevRun=""
 blockingIP="192.168.50.52"

 ## Entware settings ##
 entPath="/tmp/mnt/BarryAllen/entware"
 entVersion="Entware (aarch64-k3.10)"
 psState=on
 prevPsState=
 psIP="192.168.50.52"

 ## Ad-blocking counters ##
 blockedDomains=876552
 adsBlocked=809
 adsWeek=809
 adsNew=809
 adsPrevCount="Jan 12 05:20"
 adsLastCount="Jan 12 05:20"
 adsCounter=off

 ## Messaging ##

 ---------------------------------------------------
 END FILE

So here's the IPv6 versions of my entries in the blacklist (only 15 custom entries, plus IPv6 versions = total of 30 lines):

 16: :: activeupdate.trendmicro.co.jp
 17: :: backup21.url.trendmicro.com
 18: :: dlcdnets-ds.asus.com.edgekey.net
 19: :: dlcdnets.asus.com
 20: :: e5110.dscd.akamaiedge.net
 21: :: fbsv1.trendmicro.com
 22: :: fbsv2.trendmicro.com
 23: :: gslb1.fbs.trendmicro.com.akadns.net
 24: :: manifest.googlevideo.com
 25: :: ntd-asus-2014b-en.fbs20.trendmicro.com
 26: :: rgom10-en.url.trendmicro.com
 27: :: slb1.fbs.trendmicro.com.akadns.net
 28: :: trendmicro.com.edgesuite.net
 29: :: wideip-dlcdnets.isoi.asia
 30: :: wrs.trendmicro.com

Everything looks good so far. I then go to update the Standard lists and the plus hosts. I check the blacklist again. IPv4 addresses are all the same (no dupes, and they all correctly point to 192.168.50.52), but now look at the IPv6 list:

 16: :: activeupdate.trendmicro.co.jp
 17: :: backup21.url.trendmicro.com
 18: :: dlcdnets-ds.asus.com.edgekey.net
 19: :: dlcdnets.asus.com
 20: :: e5110.dscd.akamaiedge.net
 21: :: fbsv1.trendmicro.com
 22: :: fbsv2.trendmicro.com
 23: :: gslb1.fbs.trendmicro.com.akadns.net
 24: :: manifest.googlevideo.com
 25: :: ntd-asus-2014b-en.fbs20.trendmicro.com
 26: :: rgom10-en.url.trendmicro.com
 27: :: slb1.fbs.trendmicro.com.akadns.net
 28: :: trendmicro.com.edgesuite.net
 29: :: wideip-dlcdnets.isoi.asia
 30: :: wrs.trendmicro.com
 31: :: activeupdate.trendmicro.co.jp
 32: :: backup21.url.trendmicro.com
 33: :: dlcdnets-ds.asus.com.edgekey.net
 34: :: dlcdnets.asus.com
 35: :: e5110.dscd.akamaiedge.net
 36: :: fbsv1.trendmicro.com
 37: :: fbsv2.trendmicro.com
 38: :: gslb1.fbs.trendmicro.com.akadns.net
 39: :: manifest.googlevideo.com
 40: :: ntd-asus-2014b-en.fbs20.trendmicro.com

That doesn't look right. Let's say I update the Standard lists again and the plus hosts. Now I check the blacklist once more:

 16: :: activeupdate.trendmicro.co.jp
 17: :: backup21.url.trendmicro.com
 18: :: dlcdnets-ds.asus.com.edgekey.net
 19: :: dlcdnets.asus.com
 20: :: e5110.dscd.akamaiedge.net
 21: :: fbsv1.trendmicro.com
 22: :: fbsv2.trendmicro.com
 23: :: gslb1.fbs.trendmicro.com.akadns.net
 24: :: manifest.googlevideo.com
 25: :: ntd-asus-2014b-en.fbs20.trendmicro.com
 26: :: rgom10-en.url.trendmicro.com
 27: :: slb1.fbs.trendmicro.com.akadns.net
 28: :: trendmicro.com.edgesuite.net
 29: :: wideip-dlcdnets.isoi.asia
 30: :: wrs.trendmicro.com
 31: :: activeupdate.trendmicro.co.jp
 32: :: backup21.url.trendmicro.com
 33: :: dlcdnets-ds.asus.com.edgekey.net
 34: :: dlcdnets.asus.com
 35: :: e5110.dscd.akamaiedge.net
 36: :: fbsv1.trendmicro.com
 37: :: fbsv2.trendmicro.com
 38: :: gslb1.fbs.trendmicro.com.akadns.net
 39: :: manifest.googlevideo.com
 40: :: ntd-asus-2014b-en.fbs20.trendmicro.com
 41: :: rgom10-en.url.trendmicro.com
 42: :: slb1.fbs.trendmicro.com.akadns.net
 43: :: trendmicro.com.edgesuite.net
 44: :: wideip-dlcdnets.isoi.asia
 45: :: wrs.trendmicro.com
 46: :: activeupdate.trendmicro.co.jp
 47: :: backup21.url.trendmicro.com
 48: :: dlcdnets-ds.asus.com.edgekey.net
 49: :: dlcdnets.asus.com
 50: :: e5110.dscd.akamaiedge.net
 51: :: fbsv1.trendmicro.com
 52: :: fbsv2.trendmicro.com
 53: :: gslb1.fbs.trendmicro.com.akadns.net
 54: :: manifest.googlevideo.com
 55: :: ntd-asus-2014b-en.fbs20.trendmicro.com
 56: :: rgom10-en.url.trendmicro.com
 57: :: slb1.fbs.trendmicro.com.akadns.net
 58: :: trendmicro.com.edgesuite.net
 59: :: wideip-dlcdnets.isoi.asia
 60: :: wrs.trendmicro.com
 61: :: activeupdate.trendmicro.co.jp
 62: :: backup21.url.trendmicro.com
 63: :: dlcdnets-ds.asus.com.edgekey.net
 64: :: dlcdnets.asus.com
 65: :: e5110.dscd.akamaiedge.net
 66: :: fbsv1.trendmicro.com
 67: :: fbsv2.trendmicro.com
 68: :: gslb1.fbs.trendmicro.com.akadns.net
 69: :: manifest.googlevideo.com
 70: :: ntd-asus-2014b-en.fbs20.trendmicro.com
 71: :: rgom10-en.url.trendmicro.com
 72: :: slb1.fbs.trendmicro.com.akadns.net
 73: :: trendmicro.com.edgesuite.net
 74: :: wideip-dlcdnets.isoi.asia
 75: :: wrs.trendmicro.com
 76: :: activeupdate.trendmicro.co.jp
 77: :: backup21.url.trendmicro.com
 78: :: dlcdnets-ds.asus.com.edgekey.net
 79: :: dlcdnets.asus.com
 80: :: e5110.dscd.akamaiedge.net
 81: :: fbsv1.trendmicro.com
 82: :: fbsv2.trendmicro.com
 83: :: gslb1.fbs.trendmicro.com.akadns.net
 84: :: manifest.googlevideo.com
 85: :: ntd-asus-2014b-en.fbs20.trendmicro.com
 86: :: rgom10-en.url.trendmicro.com
 87: :: slb1.fbs.trendmicro.com.akadns.net
 88: :: trendmicro.com.edgesuite.net
 89: :: wideip-dlcdnets.isoi.asia
 90: :: wrs.trendmicro.com
 91: :: activeupdate.trendmicro.co.jp
 92: :: backup21.url.trendmicro.com
 93: :: dlcdnets-ds.asus.com.edgekey.net
 94: :: dlcdnets.asus.com
 95: :: e5110.dscd.akamaiedge.net
 96: :: fbsv1.trendmicro.com
 97: :: fbsv2.trendmicro.com
 98: :: gslb1.fbs.trendmicro.com.akadns.net
 99: :: manifest.googlevideo.com
 100: :: ntd-asus-2014b-en.fbs20.trendmicro.com
 101: :: rgom10-en.url.trendmicro.com
 102: :: slb1.fbs.trendmicro.com.akadns.net
 103: :: trendmicro.com.edgesuite.net
 104: :: wideip-dlcdnets.isoi.asia
 105: :: wrs.trendmicro.com
 106: :: activeupdate.trendmicro.co.jp
 107: :: backup21.url.trendmicro.com
 108: :: dlcdnets-ds.asus.com.edgekey.net
 109: :: dlcdnets.asus.com
 110: :: e5110.dscd.akamaiedge.net
 111: :: fbsv1.trendmicro.com
 112: :: fbsv2.trendmicro.com
 113: :: gslb1.fbs.trendmicro.com.akadns.net
 114: :: manifest.googlevideo.com
 115: :: ntd-asus-2014b-en.fbs20.trendmicro.com
 116: :: rgom10-en.url.trendmicro.com
 117: :: slb1.fbs.trendmicro.com.akadns.net
 118: :: trendmicro.com.edgesuite.net
 119: :: wideip-dlcdnets.isoi.asia
 120: :: wrs.trendmicro.com

So the IPv6 entries are being multiplied every time I update the lists. I eventually get to a point where the router hangs because there are thousands of dupe entries in the blacklist.

 

[thelonelycoder]

That doesn't look right. Let's say I update the Standard lists again and the plus hosts. Now I check the blacklist once more:

I see, that really does not look right. Thanks for reporting.
This might be the reason some users get out of memory problems when IPv6 is enabled.
Will get to it ASAP.

 

[thelonelycoder]

So the IPv6 entries are being multiplied every time I update the lists. I eventually get to a point where the router hangs because there are thousands of dupe entries in the blacklist.

Found the error.
During the blocking list update, a temporary blocking list is used to lower the memory usage (https://pgl.yoyo.org/adservers/serv...&showintro=0&mimetype=plaintext&useip=0.0.0.0).

For this temp list I do not add the IPv6 entries, since it's only in use for a few seconds, a minute at most. To make that list active I restart Dnsmasq, which triggers post-conf.div. post-conf.div adds the IPv6 entries if missing and since they indeed are missing in the temporary blocking list, they get added to it, and due to a simplified if/else clause, to the blacklist as well, regardless if they are already there.
Hence your double/triple :: entries in the blacklist.

Thanks again for reporting, I'll sort this out right away and post a hotfix.

 

[thelonelycoder]

@JJohnson1988 and everyone having IPv6 enabled

I have uploaded hot fixes for both Diversion 4.0.6 and the 4.0.7_beta.
This fixes the multiple IPv6 duplicates in the blacklist during the manual or scheduled blocking list(s) updates.

There's no version change, enter 12 in the Diversion UI to re-download the changed files.

If you see IPv6 duplicates in the blacklist, run el, edit the blacklist and select 4. Sort and verify blacklist.
This will remove the duplicates.

 

[JJohnson1988]

Thank you for the very fast time taken to diagnose and correct the problem!

 

[Miles2019]

Greetings, I have a newly purchased AC86U, flashed to Merlin 384.8_2.
JFFS custom scripts enabled with an appropriately formatted USB stick, tried multiple sticks (both ext2 and ext4).
I am able to access the router with putty however I end up with the following message when trying to install Diversion:

"if [ -f /usr/sbin/curl ]; then
cd "$HOME"
/usr/sbin/curl -Os https://diversion.ch/diversion/4.0/diversion
chmod 0755 "$HOME/diversion"
rm -f "$0"
exec "$HOME/diversion"
else
echo
echo " Sorry, wrong platform."
echo " Diversion cannot be installed on this device."
echo " Goodbye"
echo
rm -f "$0"
exit 1
fi

#eofsh: can't open 'install'
******@RT-AC86U-6A38:/tmp/home/root# admin@192.168.50.1
-sh: admin@192.168.50.1: not found
******@RT-AC86U-6A38:/tmp/home/root#"


I try to install amtm and the process goes a bit further however I end up with the following message and no progress:

"# asks to install, show menu, pass along positional parameter
if [ ! -f "/jffs/scripts/$thisScript" ]; then
echo
print_info_line
echo " Do you want to install $thisScript, the"
echo " $title?"
continue_dialog
install_amtm
elif [ -z "$1" ]; then
recheck_opt
show_amtm menu
else
show_amtm "$1"
fi
#eofsh: can't open 'amtm'"

Thanks for looking in advance, Miles

 

[Skeptical.me]

Hi!

Nothing wrong, you just need an usb stick formatted with one of these file systems for swap file, plugged into router permanently (think of it as extended ram...)

Find an usb stick at home you don't need, at least 4 GB will do. Usb 2 or 3 does not matter...
Then format it with ext2 or ext3 file system (if you have linux installed on your comp then it is no problem, if windows, then you will have to find a program to do that, windows does not support ext file system)

Plug it into your router, if not recognized, reboot the router and you are ready to go!

You can use the router to format the usb thumb drive (but I cannot remember the command I was given when I setup Diversion/Skynet). I didn't use a Desktop to format it.


Sent from my iPhone using Tapatalk Pro

 

[XIII]

Updated 4.0.7 Beta 1 and ran “el”, “4”. That got stuck on this:

refreshing Skynet to whitelist domains in shared-Diversion-whitelist

 

[martinr]

You can use the router to format the usb thumb drive (but I cannot remember the command I was given when I setup Diversion/Skynet). I didn't use a Desktop to format it.


Sent from my iPhone using Tapatalk Pro

A guide can be found here
https://www.snbforums.com/threads/e...ptions-on-the-router.48302/page-2#post-455723

 

[Skeptical.me]

A guide can be found here
https://www.snbforums.com/threads/e...ptions-on-the-router.48302/page-2#post-455723

Yeah, I actually read the rest of the thread and there were links to posts about it. I have copied the commands into my Standard Notes app. But after I did that I discovered I had already created a note from the time @Marco directed me how to do it. I forgot about it. I try to keep every command/script etc I've used in Standard Notes (excellent App for notes and code). Thanks!!


Sent from my iPhone using Tapatalk Pro

 

[thelonelycoder]

Greetings, I have a newly purchased AC86U, flashed to Merlin 384.8_2.
JFFS custom scripts enabled with an appropriately formatted USB stick, tried multiple sticks (both ext2 and ext4).
I am able to access the router with putty however I end up with the following message when trying to install Diversion:

"if [ -f /usr/sbin/curl ]; then
cd "$HOME"
/usr/sbin/curl -Os https://diversion.ch/diversion/4.0/diversion
chmod 0755 "$HOME/diversion"
rm -f "$0"
exec "$HOME/diversion"
else
echo
echo " Sorry, wrong platform."
echo " Diversion cannot be installed on this device."
echo " Goodbye"
echo
rm -f "$0"
exit 1
fi

#eofsh: can't open 'install'
******@RT-AC86U-6A38:/tmp/home/root# admin@192.168.50.1
-sh: admin@192.168.50.1: not found
******@RT-AC86U-6A38:/tmp/home/root#"


I try to install amtm and the process goes a bit further however I end up with the following message and no progress:

"# asks to install, show menu, pass along positional parameter
if [ ! -f "/jffs/scripts/$thisScript" ]; then
echo
print_info_line
echo " Do you want to install $thisScript, the"
echo " $title?"
continue_dialog
install_amtm
elif [ -z "$1" ]; then
recheck_opt
show_amtm menu
else
show_amtm "$1"
fi
#eofsh: can't open 'amtm'"

Thanks for looking in advance, Miles

This looks like a copy/paste error. Copy the complete command and paste the complete command into the SSH terminal, then press Enter.
Every bit of that install command counts.

 

[thelonelycoder]

You can use the router to format the usb thumb drive (but I cannot remember the command I was given when I setup Diversion/Skynet). I didn't use a Desktop to format it.

I went ahead and automated the disk formatting. It's in beta stage but so far seems reliable:
https://www.snbforums.com/threads/beta-amtm-v1-6_beta-now-with-disk-formatting-automated.54490/
@sl4fko this might be something for you to try. Read the warnings given when running the Format Disk function in amtm.

 

[thelonelycoder]

Updated 4.0.7 Beta 1 and ran “el”, “4”. That got stuck on this:

refreshing Skynet to whitelist domains in shared-Diversion-whitelist

It's likely that Skynet was busy doing something else and had placed its lockfile, preventing Diversion to run the refresh command.
I'll add a check for Skynet's lockfile before the command is run in the next update.

 

[Skeptical.me]

I went ahead and automated the disk formatting. It's in beta stage but so far seems reliable:
https://www.snbforums.com/threads/beta-amtm-v1-6_beta-now-with-disk-formatting-automated.54490/
@sl4fko this might be something for you to try. Read the warnings given when running the Format Disk function in amtm.

Ahh, that will be very handy if I ever need to reformat and reinstall amtm, diversion and so forth. Excellent.


Sent from my iPhone using Tapatalk Pro

 

[Skeptical.me]

Does anyone know if google analytics, and other analytics/crashlytics services are blocked in Diversion Standard?