Diversion Diversion - the Router Ad-Blocker

[rromeroa]

Nowhere in Skynet, amtm or Diversion/AB-Solution do we require for the output to be sent to /dev/null.

thanks for your answer, and sorry for confusion, the output sent to null was put by me (avoid error calling swapon twice) .. I was referrring to the swapon command, Skynet requires it in post-mount script.

 

[bengalih]

Consumer router are originally designed not for custom firmware and additional daemon/app to run with in.
You do no want to have swap file despite people advising you that you should have it and you complain daemon crushing.
What you want others to advise or help?

Clearly nothing we are doing with our routers are what they are designed for...isn't that kind of the point?

Also, I have not stated once that I don't want a swap file. I've stated that the issues seem to happen more when a swap file isn't present. Since memory utilization on my system without a swap file seems low (< 50%) and since I never see any swapping take place, I'm unsure why the presence of a swap file in these instances is necessary.

Please read that carefully. I'm not stating that a swap file isn't necessary, or may not be necessary under certain loads. I'm stating that I'm having strange issues that do not seem to stem from the absence of a swap file (due to the above reasons).

In any event, I continue to have dnscrypt instability issues even with a swap file. I've discussed these a few posts back. So, clearly while a swap file may have helped at certain points, the lack (or presence) of it is not what is causing the issues.

The point of my post was just to warn people that *I* have seen dnscrypt crash a lot and when it does it takes the internet with it (or at least DNS, which mean no internet resolution for all intents and purposes). The installation issues people are having with diversion due to this is not something @thelonelycoder can rectify because the issue is not with his software.

 

[bengalih]

**** UPDATE ****
THIS PATCH/FUNCTIONALITY IS INCLUDED NOW IN DIVERSION 4.0.1.
THERE SHOULD BE NO NEED TO MANUALLY INSTALL THIS.
***************

Welcome
This is Diversion - the Router Adblocker for Asuswrt-Merlin

AB-Solution 4.0 is now Diversion 4.0

Hey @thelonelycoder (and other interested parties...).
I'm not sure how you handle this, but I've done a couple of custom patches to add some functionality.
I'll provide some info and links to them here, but what you do with them is ultimately up to you.

I've added the following 2 functions to whitelist edits:
1) Ability to add all "near-matches" when specifying a domain.
2) Ability to remove more than one entry from the whitelist file at a time (i.e. add a "range" to delete).

Here are some screen shots:

1) Near match functionality:

2) Batch remove (delete range) functionality:

Files:

I've posted this to pastebin here:
https://pastebin.com/TS39VuJ6

You can look at the raw code and then click on the "download" link on pastebin to download it in file format.
The downloaded file needs to be renamed to "functions.div" and put in the following directory:

/opt/share/diversion/file

don't forget to backup your original:

mv functions.div functions.div.original

For those of you who prefer, here is a diff patch file (you can download patch via entware opkg):
https://pastebin.com/m1cFFc3y

Hope someone else finds it useful!
------------------------------------------------------
UPDATE 8/31/18 8:50pm CST - Updated links to newer version - minor color coding correction!
UPDATE 9/1/18 12:55 pm CST - Updated links to newer version - minor code correction!

 

[thelonelycoder]

/tmp/mnt/your_usb_drive_here/entware/share/diversion/file

Entware is installed for all Diversion installations.
This means we have a systemwide path available to access it:

/opt/share/diversion

Please use that in future posts.

 

[bengalih]

Entware is installed for all Diversion installations.
This means we have a systemwide path available to access it:

/opt/share/diversion

Please use that in future posts.

updated.

 

[thelonelycoder]

I've added the following 2 functions to whitelist edits:
1) Ability to add all "near-matches" when specifying a domain.
2) Ability to remove more than one entry from the whitelist file at a time (i.e. add a "range" to delete).

I'll look into it, there are reasons why I left the option of removing near maches out at the time.

Note that in d, when running Re-install Diversion <version>, Diversion is reinstalled from local files, not overwriting (user modified) local files.
When running Update Diversion <version>, all files are pulled from my server again, overwriting (user modified) local files.

 

[bengalih]

I'll look into it, there are reasons why I left the option of removing near maches out at the time.

reason being?

 

[thelonelycoder]

reason being?

It's late. I'll get back to you on that.

 

[bengalih]

It's late. I'll get back to you on that.

No problem. I assume (based on probability) that my changes might have an impact somewhere else in the code.
I did not have time to read the entire code-base (and I'm not sure I would have understood it all on the first read anyway ).

I tried my best to keep with your established coding conventions and flow and not do a "hack job."
Based on my testing it works and doesn't seem to cause any instabilities/problems either within those code branches or the large script as a whole. Based on what I observe the script doing I don't think these changes will have negative impact.
However, there is a possibility with your wildcard changes there might be issues.
But, in the end all these changes are doing is allowing a user to add or remove more than one domain at a time, but the end result is the same whether they run this patch or they choose to "add" or "delete" multiple times manually.

Of course, I would love for you to review it and give it a seal of approval at least for those others wanting to try it and hopefully incorporate my changes (or at least replicate the functionality) into a future release.

This was a good exercise for me as I haven't done any shell coding in close to 2 years, so it was nice to at least try solve a problem.

And - of course - thanks for all your work and the new release!

Oh - might I just add that I think the biggest benefit to these functions is for testing. When you are trying to figure out what domains to whitelist for a problem, etc. I had another crazy idea while I was finishing up the edits to have a "capture" function. It would be similar to the log file tailing, but additionally it would pull out every domain it shows blocked over the time it was run then present them in a list for the user to confirm import. This way you could run this option, go to your blocked site, then stop the option and import all the blocked sites. You could then try whittling down to only the bare essentials.

 

[FadgewackeR]

@thelonelycoder - I previously used AdsBeGone, and on my update, i was advised that this was no longer supported. I now have a custom +.

I assume, that the adsbegone "base" has been replaced with something similarly sized, and this will be updated with new blocks regularly, or do I need to start again with a blocking file of my choosing, and rebuild my whitelist?

Nice work on the update and UI by the way. Very slick.

 

[Xentrk]

@thelonelycoder - I previously used AdsBeGone, and on my update, i was advised that this was no longer supported. I now have a custom +.

I assume, that the adsbegone "base" has been replaced with something similarly sized, and this will be updated with new blocks regularly, or do I need to start again with a blocking file of my choosing, and rebuild my whitelist?

Nice work on the update and UI by the way. Very slick.

Not exactly the same lists. Diversion will preserve your current whitelist when you change from custom to one of the provided lists.

 

[thelonelycoder]

@thelonelycoder - I previously used AdsBeGone, and on my update, i was advised that this was no longer supported. I now have a custom +.

I assume, that the adsbegone "base" has been replaced with something similarly sized, and this will be updated with new blocks regularly, or do I need to start again with a blocking file of my choosing, and rebuild my whitelist?

Nice work on the update and UI by the way. Very slick.

I have nt made an actual comparison from AB to Diversion blocking lists, I just used the available sources better.
You can keep your formerly-known-as-AdsBeGone list that is now renamed to Custom.
It will update at the set day just as before in AB.

 

[thelonelycoder]

No problem. I assume (based on probability) that my changes might have an impact somewhere else in the code.
I did not have time to read the entire code-base (and I'm not sure I would have understood it all on the first read anyway ).

I tried my best to keep with your established coding conventions and flow and not do a "hack job."
Based on my testing it works and doesn't seem to cause any instabilities/problems either within those code branches or the large script as a whole. Based on what I observe the script doing I don't think these changes will have negative impact.
However, there is a possibility with your wildcard changes there might be issues.
But, in the end all these changes are doing is allowing a user to add or remove more than one domain at a time, but the end result is the same whether they run this patch or they choose to "add" or "delete" multiple times manually.

Of course, I would love for you to review it and give it a seal of approval at least for those others wanting to try it and hopefully incorporate my changes (or at least replicate the functionality) into a future release.

This was a good exercise for me as I haven't done any shell coding in close to 2 years, so it was nice to at least try solve a problem.

And - of course - thanks for all your work and the new release!

Oh - might I just add that I think the biggest benefit to these functions is for testing. When you are trying to figure out what domains to whitelist for a problem, etc. I had another crazy idea while I was finishing up the edits to have a "capture" function. It would be similar to the log file tailing, but additionally it would pull out every domain it shows blocked over the time it was run then present them in a list for the user to confirm import. This way you could run this option, go to your blocked site, then stop the option and import all the blocked sites. You could then try whittling down to only the bare essentials.

Heads up: Code looks good, going to add it to the next release.
Your code needs some error corrections and format changes which I'm doing right now.
One error is if you delete only one line in a file. It is already fixed in my code.

 

[episyron]

I updated AB-Solution to Diversion with port option and after boot noticed that link to /tmp/mnt/usbstick/ab-solution.sh was created into root's home. This comes from /jffs/scripts/post-mount where there is still code block from AB-Solution 3.11. I suppose this block can be removed now?

By the way what is the difference between lite and standard version of Diversion?

Thanks for your outstanding work with AB-Solution and Diversion!

 

[thelonelycoder]

I updated AB-Solution to Diversion with port option and after boot noticed that link to /tmp/mnt/usbstick/ab-solution.sh was created into root's home. This comes from /jffs/scripts/post-mount where there is still code block from AB-Solution 3.11. I suppose this block can be removed now?

Can you post the content of that file? AB content should have been removed and I wonder why.

By the way what is the difference between lite and standard version of Diversion?

Diversion Standard installs and uses pixelserv-tls, Diversion Lite does not.

 

[episyron]

Can you post the content of that file? AB content should have been removed and I wonder why.

Here it is:

#!/bin/sh

# DO NOT EDIT this part of the file #
# generated by AB-Solution 3.11
if [ -d "/tmp/mnt/usbstick" ];then
    service restart_dnsmasq
        logger "AB-Solution added entries via $0"
    ln -s /tmp/mnt/usbstick/ab-solution.sh /root/ab-solution.sh
    cp -f /jffs/.ash_history /root/.ash_history
fi
# end of DO NOT EDIT #
. /jffs/scripts/post-mount.div # Added by Diversion

Diversion Standard installs and uses pixelserv-tls, Diversion Lite does not.

OK, thanks.

 

[thelonelycoder]

Here it is:

#!/bin/sh

# DO NOT EDIT this part of the file #
# generated by AB-Solution 3.11
if [ -d "/tmp/mnt/usbstick" ];then
    service restart_dnsmasq
        logger "AB-Solution added entries via $0"
    ln -s /tmp/mnt/usbstick/ab-solution.sh /root/ab-solution.sh
    cp -f /jffs/.ash_history /root/.ash_history
fi
# end of DO NOT EDIT #
. /jffs/scripts/post-mount.div # Added by Diversion

OK, thanks.

Looks like you did not install the pixelserv-tls option in AB-Solution. Is that correct?

These two lines will do, remove the rest:

#!/bin/sh
. /jffs/scripts/post-mount.div # Added by Diversion

 

[episyron]

Looks like you did not install the pixelserv-tls option in AB-Solution. Is that correct?

Yes, that is correct.

 

[chewy74]

Adding these to the white list did not not work and the Amazon app still cannot be used

I had the same problem, and had to whitelist some domains (in addition to the default whitelisted domains to support SNB forums). Unfortunately, I didn't record which of the following made the difference, but with these whitelisted (in Diversion and Skynet), the app works for me.

device-metrics-us-2.amazon.com
fls-eu.amazon.com
fls-eu.amazon.de
fls-na.amazon.com
mads.amazon-adsystem.com
s.amazon-adsystem.com
z-na.amazon-adsystem.com

 

[thelonelycoder]

Yes, that is correct.

I see where it failed now. Thanks for reporting.