Diversion Diversion - the Router Ad-Blocker

[heysoundude]

Your Data and Entware partitions are only 22MB each in size!

Houston, we’ve found the problem.
How big is their USB drive again? Is it the 128Mb size that was common back in the day the router was new/current?


Sent from my iPhone using Tapatalk

 

[L&LD]

Sorry 1 more question,

Once I flash the asuswrt Merlin would it still maintain it's old configs or should I do a reset and configure once again

Yes, after flashing to another firmware, particularly when going from RMerlin to Asus or vice versa, you need to do a full reset to factory defaults followed by a minimal and manual configuration to secure the router and connect to your ISP. You should also format the JFFS partition too (either manually or with using the 'Initialize all settings...' checkbox in the GUI) to fully reset the router.

Please see my signature below for the link to the M&M Config to properly get your router to a good/known state.

 

[dave14305]

Houston, we’ve found the problem.
How big is their USB drive again? Is it the 128Mb size that was common back in the day the router was new/current?


Sent from my iPhone using Tapatalk

But if you look at his “top” screenshot the grep of the whitelist is using over 400MB virtual memory. So it’s all 10 pounds of stuff in 5 pound bags...

 

[heysoundude]

Right. It seems he’s using too much of a blocking filter.


Sent from my iPhone using Tapatalk

 

[HairyA00]

Interesting discussion over on the Pi-hole subreddit:
https://www.reddit.com/r/pihole/comments/e00rar/bad_news_unblockable_web_trackers_emerge_good/

 

[RMerlin]

One interesting thing I observed lately, as I had to temporarily try a few things with Diversion as well as Pi-Hole: if I have my Shield TV go through either of these, then the usual "you can skip this ad in 5 secs" fronter on Youtube videos will turn into "This ad will end in 15 secs" unstoppable fronters instead. Looks like if they detect an ad blocker, then they push even more annoying ads instead.

 

[Mircica]

Your Data and Entware partitions are only 22MB each in size!

Well, both USB drives are 4GB!
That's very strange, isn't it?
Ext3 formatted (maximum supported by router) and I use Medium+ list with no custom list! All custom are disabled.
Sounds very strange to me!
All I did was replacing old one 2GB + 2GB with 4GB + 4GB
All reinstalled clean...
Still a mystery why space is so small snce in MC is reported correctly!

 

[Mircica]

Right. It seems he’s using too much of a blocking filter.


Sent from my iPhone using Tapatalk

Medium+ with no custom list, all custom are disabled!
Perhaps the firmware I am using...

 

[Mircica]

What are you blocking and whitelisting that consumes so much memory and space? Your situation seems unique, since Diversion works smoothly for most users with a reasonable blocking list.

Medium+ list!

 

[Mircica]

Your Data and Entware partitions are only 22MB each in size!

Well, that's a mystery for me too, same size reported on 2GB USB stick, and now on 4GB, but I will try to see if 8GB reports the same, if yes, 16/32GB...

 

[heysoundude]

Perhaps the firmware I am using...

I would scroll back in this thread and verify what @thelonelycoder says about his testing rig...


Sent from my iPhone using Tapatalk

 

[Mircica]

Almost the same firmware, 2 months difference!

"My own testing includes running amtm and Diversion on two RT-AC66U routers. One runs 380.70, the other johns LTS fork firmware. They're not dead yet."

And I have 380.68_4...

 

[Asad Ali]

Well, that's a mystery for me too, same size reported on 2GB USB stick, and now on 4GB, but I will try to see if 8GB reports the same, if yes, 16/32GB...

Since your both drives gives the same space issue, I think you should try to use a different method to format the drives now. Give amtm format method a try if you haven't done it or you can use Linux native format commands in the terminal for that as well. Also if that doesn't help then try with a single partition as well.

 

[Mircica]

I used amtm to format both USB sticks Ext3!
One partition per USB.

Previously I have had Ext2/2xUSB 2GB/one partition also!
Same report regarding space!
That's a big problem for me...

 

[HairyA00]

One interesting thing I observed lately, as I had to temporarily try a few things with Diversion as well as Pi-Hole: if I have my Shield TV go through either of these, then the usual "you can skip this ad in 5 secs" fronter on Youtube videos will turn into "This ad will end in 15 secs" unstoppable fronters instead. Looks like if they detect an ad blocker, then they push even more annoying ads instead.

Funny, when I am running pixelserv-tls, I seem to get shorter, quicker 3-5 seconds ads advertising YouTube Red - or whatever it's called now (in Spanish no-less, no idea why that's the case). I think blocking ads on YouTube has been cluttered with all kinds of anecdotal evidence. I remember the only thing that EVER worked for a hot second there was this (and I think this was implemented in Pixelserv-tls if my memory serves me right): https://www.reddit.com/r/pihole/comments/9w5swx/i_think_ive_managed_to_block_youtube_ads_with/

And Google closed that hole in less than 24 hours. In 2018, Google's ad revenue amounted to almost 116.3 billion US dollars. Breaking their logic requires a content blocker (a la uBlock Origin). Even then, I'm sure you can assume they have dozens of engineers looking at a way around that (a cynical, exaggerated statement; just stating that their revenue comes from YOU being the product).

 

[heysoundude]

Almost the same firmware, 2 months difference!

"My own testing includes running amtm and Diversion on two RT-AC66U routers. One runs 380.70, the other johns LTS fork firmware. They're not dead yet."

And I have 380.68_4...

It seems a Merlin upgrade is in your future. I would do a factory reset at this point, and re-do the amtm/diversion installation etc.


Sent from my iPhone using Tapatalk

 

[Mircica]

Funny thing, Midnight Commander (mc) reports correctly!
Which makes me think something is wrong somewhere...

 

[Asad Ali]

I used amtm to format both USB sticks Ext3!
One partition per USB.

Previously I have had Ext2/2xUSB 2GB/one partition also!
Same report regarding space!
That's a big problem for me...

Then just do a clean format of the drives again, perhaps use ext2 this time and check the space first without installing Entware and Diversion on it.

You can try using a different USB port as well ( assuming your router have more than one port )

 

[thelonelycoder]

@Mircica I have upladed a modified update-bl.div file that eliminates the necessity to duplicate the whitelist entries while updating the blocking list.
Since your whitelist is rather large this might now work.
Use command 12 in Diversion to download all addon files, then try again.

As a side note, your large whitelist actually only removes about 50 or so entries in the blocking file. Consider cutting it down to about a fourth of the current size.

 

[thelonelycoder]

Hi,

New user here, many months reading, first time posting (actually just made an account).
I'm using an Asus RT-AC68U HW rev. A1 router using firmware AsusWRT-Merlin 384.14_beta1 with AMTM, Diversion, Skynet, uiDivStats and uiScribe scripts, all installed from amtm. Thank you to all of the coders whose code I've been using!

After installing Diversion, I've configured its email settings for it to be able to send notifications and the file backups. I've used a gmail account, but hit a few problems along the way (one of which was to enable Less secure app access in the gmail account settings).

The other however was that I am using a password with many non alphanumeric characters in it, one of them being $ . However, this is a special character in the linux shell, thus the password did not get stored correctly (it only stored the password up to this character). After escaping the character with \ , it worked. But this still bothered me, along with the message that the character " cannot be used, but it should with the same method of escaping the character.

I've then started to check which other non alphanumeric characters should be escaped (from the 32 ones easily accessible with an US style keyboard) and proceeded to modify the script to escape the characters by itself. The result is that the password is stored with the special characters escaped and also works in sending the emails. I've then decided to share it here, so here we go:

This is the file that should be edited:

/opt/share/diversion/file/functions.div

1. Add a new procedure (it doesn't really matter where as long as it's between #BOF and #EOF and not wihin another procedure or the main procedure itself; a good place would be to add it before the first procedure, which for the current 4.1.6 version is au_function() at line 63):

escape_password_special_characters(){
   i=0
   PASSWORD=
   while [ $i -lt `expr length "$value"` ]
   do
       case ${value:$i:1} in
           [\`\'\$\\\"])
               PASSWORD="$PASSWORD\\"
               ;;
           %)
               PASSWORD="$PASSWORD%"
               ;;
       esac
       PASSWORD="$PASSWORD${value:$i:1}"
       i=`expr $i + 1`
   done
}

2. Modify line where the password is read directly in the PASSWORD variable by reading it in the value variable and calling the newly created procedure, from:

       printf "\\n${RED_BG} Enter Password: ${NC} ";read -r PASSWORD

to

       printf "\\n${RED_BG} Enter Password: ${NC} ";read -r value;escape_password_special_characters

3. Modify lines where the password is stored in the PASSWORD variable by calling the newly created procedure, from:

               6)   printf "\\n Note: Password may NOT contain \" character.\\n"
                   printf "\\n $PASSWORD${RED} <-- current password${NC}\\n"
                   printf "${RED_BG} Enter Password: ${NC} [e=Exit] ";read -r value
                   PASSWORD=$value;break;;

to

               6)   printf "\\n $PASSWORD${RED} <-- current password${NC}\\n"
                   printf "${RED_BG} Enter Password: ${NC} [e=Exit] ";read -r value
                   escape_password_special_characters;break;;

Save the changes made to the file and this should be it until a reinstall or upgrade of Diversion which will overwrite the file.

Maybe @thelonelycoder could add something similar in the next releases if he sees value in it or maybe not if there's something wrong in doing this that I'm not seeing.

Thanks, that looks just what's missing for the proper PW special char escaping. I'll add your proposal to the next release version.