Diversion Diversion - the Router Ad-Blocker

[thelonelycoder]

Understandable. Re-importing the code manually is really not a big deal. Just curious why it hangs while porting, does it get stuck trying connect to the internet immediately after restarting dnsmasq during the porting phase?

Apparently, since it restarts, or wants to restart dnsmasq, your WAN and perhaps LAN connection might be down.
This prevents Diversion from downloading the files from my server.
You'll figure it out, I'm sure.

 

[Martineau]

Very Minor issue during install:

1. I have several '.txt' files for notes etc. and whilst non-executable, many have spaces in their name.

Spoiler: 'dos2unix: can't open' and Busybox cmp error

[/COLOR][/FONT][/LEFT]
[FONT=Georgia][COLOR=rgb(20, 20, 20)]
[LEFT]  ✔  update.div          integrated
  ✔  update-bf.div       integrated
  ✔  update-notification integrated
  ✔  write-config.div    integrated

dos2unix: can't open '/jffs/scripts/$DNS': No such file or directory

BusyBox v1.25.1 (2018-08-07 14:58:25 EDT) multi-call binary.
Usage: cmp [-l] [-s] FILE1 [FILE2]
Compare FILE1 with FILE2 (or stdin)
    -l  Write the byte numbers (decimal) and values (octal)
        for all differing bytes
    -s  Quiet

dos2unix: can't open '/jffs/scripts/$Flip': No such file or directory

<snip>

​

2. Subsequently reviewing installation log, the message '(no file(s) found)' appears against a few of my custom files..... e.g. 'BlockWiFiDevices' but contents are displayed when requested:

Spoiler: (no file(s) found)

 What do you want to do?  sf
____________________________________________________
 Select file content to show:
     Diversion config file:
  1. diversion.conf
     Diversion custom files:
     (no file(s) found)
     Diversion log files:
  2. Diversion-install.log
     pixelserv-tls start file:
  3. S80pixelserv-tls
     dnsmasq config file:
  4. dnsmasq.conf
     jffs config files:
  5. BlockWiFiDevices
     (no file(s) found)
  6. IPGroups
  7. IoT_Alexa.apps
  8. IoT_Google.apps
  9. IoT_Hive.apps

  <snip>
 
     shell history file:
419. . ash_history


 Select file to show [1-419 e=Exit] 5
____________________________________________________
 /jffs/configs/BlockWiFiDevices has this content:
 START FILE, --- lines are not part of file
 ---------------------------------------------------
 # Specify DROP or ACCEPT statements for this device using CSV notation (see /jffs/scripts/BlockWiFiDevices)
 #
 # DON'T use TAB characters!!!!
 
 # e.g.
 #
 #       DROP    WAN
 #       DROP    DS-416
 #       DROP    10.88.8.120-10.88.8.125
 
 # /jffs/configs/BlockWiFiDevices is designed as a multi-Client config file and supports additional in-line directives:
 #
 # Directives
 #>FLUSH        
 #>PROCESS       [ACCEPT | DROP]
 #>IP            [ip_address | hostname] [ hostname_alt]
 #>EXIT
 
 # e.g. Peer / LAN / WAN rules for WiFI Client Nexus-7
 >FLUSH
 >IP       Nexus-7
 #DROP      DS-416
 DROP      CAMERAS
 #DROP      RaspberryPiB
 ACCEPT    ANDROID
 DROP      WAN
 
 ---------------------------------------------------
 END FILE

 

[DonnyJohnny]

Same here unfortunately. All outgoing queries were blocked and it hung on 'Porting all settings....' (or something alike) all night. Decided to start fresh with a different USB drive. If there's anything I can supply to troubleshoot this, let me know.

Mine also struck. I supposed it stopped ab-solution and pixelserv but somehow didn’t revert the Pixelserv IP and hence stop queries ( no internet access).
I have to force stop the installation.
Did some unnecessary step to uninstall pixelserv-tls (entware) and reboot. It didn’t help. Still no internet access.
Finally change my dns server from manual to automatic. Internet access back. Try the diversion installation again. It work this time.
@thelonelycoder hope this helps to improve the installation issue.

 

[M@rco]

@thelonelycoder,

Here are links to the requested files:

/opt/share/diversion/log
Diversion-install.log: https://pastebin.com/NfdADhux

/old-jffs_scripts
dnsmasq.postconf: https://pastebin.com/fzvyYLe1
services-stop: https://pastebin.com/ZGcicWnb
wan-start: https://pastebin.com/6kgyAe3R

There are no other scripts in the /old-jffs_scripts folder, so I assume Diversion backups the ones it modifies?

After modification by the Diversion installer, the same scripts (in the backup I made before starting fresh) look like:

/jffs/scripts
dnsmasq.postconf: https://pastebin.com/yeCXsQT9
services-stop: https://pastebin.com/yLiKDGBw
wan-start: https://pastebin.com/rdRCinFU

I took the liberty of sending you a PM with a link to another file with more info.

Hopefully this can shed some light on why updating failed. As far as I'm aware, nothing exotic or out of the ordinary with my setup.

If there's anything else that can be helpful, just let me know

Thanks for all your efforts!

 

[M@rco]

Mine also struck. I supposed it stopped ab-solution and pixelserv but somehow didn’t revert the Pixelserv IP and hence stop queries ( no internet access).
I have to force stop the installation.
Did some unnecessary step to uninstall pixelserv-tls (entware) and reboot. It didn’t help. Still no internet access.
Finally change my dns server from manual to automatic. Internet access back. Try the diversion installation again. It work this time.

After a factory reset, I imported settings.cfg to save me some time, figuring the issue wouldn't be there. Changed the empty WAN DNS fields to CloudFlare DNS to make sure I wouldn't loose (outgoing) WAN connectivity again. Looking at the logs Skynet produced, incoming WAN was still up and running. Somehow I have a feeling this is rather about dnscrypt-proxy then pixelserv-tls... Can't say why, just a gut feeling. Fortunately, after a fresh install, all seems fine now.

 

[thelonelycoder]

1. I have several '.txt' files for notes etc. and whilst non-executable, many have spaces in their name.

Looks scary when replicating file names with spaces. You of all the members on this board uses filenames with spaces in this environment?
I'm not sure how best to tackle that best, suggestion how to code such a variable?
The relevant code is in /opt/share/diversion/file/functions.div in function sf_function() starting on line 2468.

Since sf helpfully lists all files found in /jffs/scripts and /jffs/configs there might be some drawbacks. This listing is not new in Diversion. AB used similar code to list them.

 

[thelonelycoder]

/opt/share/diversion/log
Diversion-install.log: https://pastebin.com/NfdADhux

Diversion install exited or aborted says it all, probably curl timeout to my server since Dnsmasq was down or Internet not reachable.

I took the liberty of sending you a PM with a link to another file with more info.

This suggests a dnscrypt issue. I will do more testing with 'dnscrypt installer' when time allows to improve the porting.
For now, do a fresh Diversion install first, then install the other scripts one at a time.

 

[dugaduga]

Yes only cached dns queries allowed successful connections to the net. Lan/Wan was up. I removed all custom mods to dnsmasq,postconf and rebooted, this fixed nothing. I also replaced some of the code your installer removed and this fixed nothing either. Restarting dnsmasq manually fixed nothing. Thank you @DonnyJonny, this is what I suspected as well, was Pixelserve ip not properly changed?; I set DNS to auto prior to installing and it worked from the onset.

Had to manually re-apply dnsmasq.postconf mods, also added missing dnscrypt/manager line back into service-stop

#!/bin/sh
sh /jffs/scripts/firewall save # Skynet Firewall Addition
[ -x /jffs/dnscrypt/manager ] && /jffs/dnscrypt/manager services-stop
/opt/etc/init.d/rc.unslung stop # Added by Diversion

Services-start, was also deleted entirely, had to add the file back.

Hopefully this won't have to be done with each subsequent Diversion update.

First thing I noticed, possibly just placebo due to my extra critical eye in testing, was speed changes in webpage loading; I will have to run an actual tests to confirm.

 

[M@rco]

For now, do a fresh Diversion install first, then install the other scripts one at a time.

Exactly what I did, works like a charm now. Just configured e-mail settings in Diversion, now off to explore the rest. Any guidance (if possible) on how to re-import my AB3 black- and whitelist would be most welcome.

 

[thelonelycoder]

Any guidance (if possible) on how to re-import my AB3 black- and whitelist would be most welcome.

During porting it copies over /adblocking/whitelist.txt and /adblocking/blacklist.txt to the /opt/share/diversion/list folder, renaming them to whitelist and blacklist and sort them. You can manually copy/paste the content into the new files. Then sort the lists in el, this will remove the AB comments at the top of the old files. Diversion does no longer add headers to these files.

 

[dugaduga]

OK confirmed, speeds are snappier again after reboot. Prior to Diversion it was noticeably faster, but I suspect the dip was because I had disabled this postconf & rebooted prior to installing diversion:

dnsmasq.conf

#Experimental, Disable all Arpa: https://www.iana.org/domains/arpa
server=/arpa/
bogus-priv

This, when enabled, gives me surprising noticeable speed improvements in web browsing. 30-40%. Different websites vary. For whatever reason, these settings are best applied on boot. Applying them after booting with a clean dnsmasq.conf/postconf [which I did here] seems to result in no noticeable speed improvements. After re-applying these settings with reboot, browsing is now snappy as before, with Diversion running perfectly! This suggests to me certain features on the AsusWRT, perhaps having to do with TrendMicro, or some other AsusWRT feature, are disabled upon boot by blocking arpa during the initial dnsmasq initialization, which cause performance enhancements.

 

[XIII]

Porting from AB-Solution 3 to Diversion 4 went without a problem.

(since Diversion warned me that the shooter40sw list is no longer supported I changed it to "Standard")

 

[^Tripper^]

Congratulations on the release! Looking forward to playing with it (after the wife goes to sleep that is.)

 

[Zentachi]

I tried to update from AB-Solution to Diversion. It seemed to go just fine but this is what I get

Any ideas?

UPDATE: In any case I removed Diversion (loosing previous settings) re-installed and it works.

Many thanks thelonelycoder for your work!

 

[Protik]

I can't replicate your problem with pixelserv-tls. My best guess is you run a very old Entware version on your router.
Try to update opkg to the new Entware repo (formerly known as Entware-NG).
All installations I did with beta versions of pixelserv-tls installed run as smooth as I intended through the Diversion installer.

If you manage to install Diversion, just copy over your saved files in /diversion/list to /opt/share/diversion/list.
Only copy these: blacklist, whitelist and maybe hostslist (with the ported hosts lists from AB3).
The ps switches file goes into /opt/share/diversion/.conf

Thanks. Did a fresh install of diversion. Is there any way to retain my stats numbers from AB3?

 

[elorimer]

Porting went fine here, with skynet and pixelserv-tls test 4.

I did notice:
1. My pixelserv options reset to default and I had to readd.
2. My syslog-ng start line disappeared and had to be readded.
3. I had a couple of odd scripts that had to be restored.

But the backup files showed what I had to do. All good.

 

[elorimer]

(I had to temporarily close OpenVPN server 2 on Port 443 so that I could install pixelserv-tls,)

I didn't have to do that. Are you using the local variable in your OpenVPN config so it only listens on the WAN side?

 

[DonnyJohnny]

Diversion install exited or aborted says it all, probably curl timeout to my server since Dnsmasq was down or Internet not reachable.

This suggests a dnscrypt issue. I will do more testing with 'dnscrypt installer' when time allows to improve the porting.
For now, do a fresh Diversion install first, then install the other scripts one at a time.

I assume it is due to restart of dnsmasq causing dnscrypt-proxy to mal-function.
Meanwhile I suggest putting a remark at first post to inform people with dnscrypt-proxy to change Wan dns to auto before doing the upgrade (porting). I think that will save some trouble for people who likely will encounter the same issue of struck at porting installation.

 

[skeal]

I assume it is due to restart of dnsmasq causing dnscrypt-proxy to mal-function.
Meanwhile I suggest putting a remark at first post to inform people with dnscrypt-proxy to change Wan dns to auto before doing the upgrade (porting). I think that will save some trouble for people who likely will encounter the same issue of struck at porting installation.

When I ported it advised that a line was removed from dnsmasq. It took down my internet but the program advised me to set dns to connect automatically. I inserted 1.1.1.1 as second DNS in WAN settings and restarted the install. Worked perfect. I had to run:

/jffs/scripts/FreshJR_QOS install
From amtm ran the dnscrypt installer again.
Should have upgraded amtm to 1.3 first this would have helped.

I removed the WAN DNS server 1.1.1.1 I added when the install mentioned, and added back the DNS setting I had in DHCP settings (it was removed as well during install). The install script told me everything I needed to know in green print...

 

[DonnyJohnny]

When I ported it advised that a line was removed from dnsmasq. It took down my internet but the program advised me to set dns to connect automatically. I inserted 1.1.1.1 as second DNS in WAN settings and restarted the install. Worked perfect. I had to run:

/jffs/scripts/FreshJR_QOS install
From amtm ran the dnscrypt installer again.
Should have upgraded amtm to 1.3 first this would have helped.

I removed the WAN DNS server 1.1.1.1 I added when the install mentioned, and added back the DNS setting I had in DHCP settings (it was removed as well during install). The install script told me everything I needed to know in green print...

Lol.. must have missed that line.