Diversion Diversion - the Router Ad-Blocker

[martinr]

Was using pi-hole before and had various issues. Changed to Diversion + SkyNet on my AC68U and everything just works. Looking forward to looking at things a little closer now... Thank you lonely coder!

Two questions though:

I notice that when I connect to my router via VPN server from an external location, the Diversion ad-blocking is not active. It's only active from within the LAN.

Not sure if that's got something to do with sharing port 443 between Diversion and VPN server on the router.

I have the custom configration line 'local [ddns-name used on router]' setup on the 443 VPN server. Obviously I replace [ddns-name used on router] with my own personal ddns name! Is this the correct approach for sharing the port? I would use another port but it's the only one that works to get through for most firewalls.

Also is there an ETA for the Diversion GUI? Any time soon?

Cheers and thanks again!

Will.

Works for me and I also use port 443 as you do.

Can you try playing with the following 2 settings and see if they make a difference:

1. On the General settings of the OpenVPN server page (you didn’t say it was OpenVPN - I’m assuming), under “client will use VPN to access”, if you have “Internet only”, set it to both.

2. Under the Advanced settings for the server, second from the bottom, “Allow client <> client”, try setting to Yes.

Does 1 and/or 2. sort the problem? If so, do you need both 1. and 2. or is one of them the critical one?

 

[M@rco]

My problem is that I have 4 asus routers in diffrent locations over the country.
I dont allow SSH thru WAN as it will be unsafe.

Not necessarily, you could use SSH key pairs, deny SSH password login and run the SSH server on a non default port. Or you could run a VPN server on your router, preferably on a non default port as well. If you don't want any of these two, the only way is to deploy a custom made solution at each location, but you won't be able to check remotely if everything is functioning as it should.

I need an option under el (edit list) "custom diversion hostlists address/Set sync with custom list server " for the whitelist/blacklist/hostslist/plus-hostslist/wc_blacklist/customlist so it will synk to my other routers automatically - when I do changes on list in my server at: http://hostfile.unaux.com/whitelist
Will be ok if it only update/sync once per day.

It's up to @thelonelycoder ultimately, but to me it sounds like it's beyond the scope of Diversion. It sounds like a custom tailored solution to suit your needs, but I doubt there a lot of users in need of similar functionality.

I would like to have an option to specify a custom host file server source under the installation of Diversion -becouse I dont want to use the "steven host file" or the "yoyo file" by default. I want it to use http://hostfile.unaux.com/hosts <- from my own server and I want of course diversion to check for updates on that address. I know that you have so that I can specify a custom source of host file but then I need to enable the +list future that get like 4 other host files sources extra -that I also dont want to use.

This can easily be done by using a custom list. It can only contain the url to your selfhosted blockinglist, without any of the predefined lists. The list would have to be in the exact same format otherwise Diversion can't process it. As for the + , this option is used to share whitelisting with Skynet. If you don't use Skynet, there's no use for the + option, as far as I'm aware.

Will this be possible to do?

Yes. At least I think so, but my knowledge is limited when it comes to scripting. A script can be written to upload from one router for example and retrieve and replace all the files you mentioned on the other three routers, which can be run by cron jobs at the desired interval. But you'll have to find someone to write such a script and test it for you. Having said that, when running a script like that I doubt you will be eligible for support on Diversion...

 

[willsy]

Works for me and I also use port 443 as you do.

Can you try playing with the following 2 settings and see if they make a difference:

1. On the General settings of the OpenVPN server page (you didn’t say it was OpenVPN - I’m assuming), under “client will use VPN to access”, if you have “Internet only”, set it to both.

2. Under the Advanced settings for the server, second from the bottom, “Allow client <> client”, try setting to Yes.

Does 1 and/or 2. sort the problem? If so, do you need both 1. and 2. or is one of them the critical one?

Hi Martin

Thank you very much for the info. I had 1 set already. And I tried 2, but this didn't help.

I then noticed that I had 'Advertise DNS to clients' in the OpenVPN advanced settings set to No. I set this to Yes and it's working OK now!

Thanks for setting me on the right track... I must have changed it whilst trying to fix various issues that I had with Pi-Hole...

Thank you.

Will.

 

[M@rco]

@thelonelycoder: Just to report back: scheduled weekly backup with attachment works fine now.

I did notice a few things about the stats, which I can't recall seeing before.

In the 'noisiest client' overview as well as in the 'Top x domains for x clients' overview, I now see dhcp requests in between the regular stats:

 563    192.168.x.xx     xxxxxxxxxxxxxx:             155    DHCPREQUEST(br0)                         - 27.53%
 70     192.168.x.xx     xxxxx:                      13     DHCPREQUEST(br0)                         - 18.57%

Probably related, in the 'Top x domains for x clients overview':

192.168.aaa.aaa, iphone:
 --------------------------------------------------------
 155    DHCPREQUEST(br0)                        
 155    192.168.aaa.aaa

Also, I noticed this in the 'Top x domains for x clients', in the stats for my AVR to be exact:

 192.168.bbb.bbb, denon:
 --------------------------------------------------------
 314    denon.lan                               
 63     .                                         whitelisted
 37     esdk-ffl.spotify.com                    
 33     firmware.denon.jp                       
 12     DHCPREQUEST(br0)                        
 12     192.168.bbb.bbb
 9      DHCPOFFER(br0)

Besides the dhcp requests and offers, the second line wonders me. I don't know what it is. The only whitelisted domains are the ones whitelisted for snbforums.com by default, I haven't added any other domains to the whitelist. And I know for a fact that my Denon AVR is not connecting to any of those domains in the whitelist. Any thoughts on this?

Let me know if you need any additional info...

 

[martinr]

Hi Martin

Thank you very much for the info. I had 1 set already. And I tried 2, but this didn't help.

I then noticed that I had 'Advertise DNS to clients' in the OpenVPN advanced settings set to No. I set this to Yes and it's working OK now!

Thanks for setting me on the right track... I must have changed it whilst trying to fix various issues that I had with Pi-Hole...

Thank you.

Will.

Thanks for that. I didn’t think Advertise DNS to clients would affect it so I didn’t mention it, so that’s good to know for the future. Thanks for the feedback. Glad I was able to help, even if for the wrong reasons!

And welcome to this excellent forum, too.

 

[juched]

@thelonelycoder: Just to report back: scheduled weekly backup with attachment works fine now.

I did notice a few things about the stats, which I can't recall seeing before.

In the 'noisiest client' overview as well as in the 'Top x domains for x clients' overview, I now see dhcp requests in between the regular stats:

 563    192.168.x.xx     xxxxxxxxxxxxxx:             155    DHCPREQUEST(br0)                         - 27.53%
 70     192.168.x.xx     xxxxx:                      13     DHCPREQUEST(br0)                         - 18.57%

Probably related, in the 'Top x domains for x clients overview':

192.168.aaa.aaa, iphone:
 --------------------------------------------------------
 155    DHCPREQUEST(br0)                        
 155    192.168.aaa.aaa

Also, I noticed this in the 'Top x domains for x clients', in the stats for my AVR to be exact:

 192.168.bbb.bbb, denon:
 --------------------------------------------------------
 314    denon.lan                               
 63     .                                         whitelisted
 37     esdk-ffl.spotify.com                    
 33     firmware.denon.jp                       
 12     DHCPREQUEST(br0)                        
 12     192.168.bbb.bbb
 9      DHCPOFFER(br0)

Besides the dhcp requests and offers, the second line wonders me. I don't know what it is. The only whitelisted domains are the ones whitelisted for snbforums.com by default, I haven't added any other domains to the whitelist. And I know for a fact that my Denon AVR is not connecting to any of those domains in the whitelist. Any thoughts on this?

Let me know if you need any additional info...

I recently see weird things as the DNS name like:
DHCPREQUEST(br0)
DHCPACK(br0)

 

[Rhialto]

Can someone write a comparison between someone using uBlock Origin vs Diversion alone, vs Diversion + pixelserv-tls?

I was using AdBlockPlus before switching to uBlock over a yeare ago and I keep following Diversion posts (and previously AB-Solution) because I'm curious on this cool old-school looking program. Until now uBlock Origin is a set and forget solution for my computer. I understand Diversion would also block ads on iPad and other peripheral using WiFi but what else a neophyte should know? An extensive comparison would be welcome.

Since last summer we use a new router, the RT-AC1900P and I know I could run Diversion on it but will CPU heat increase a lot because of extra work? Current temps as I write this are 2.4 GHz: 52°C - 5 GHz: 55°C - CPU: 81°C and there is nothing running on it, no QoS or extra stuff. A few clients on 2.4 GHz and a single one on 5 GHz. Also, is a USB stick mandatory? I know there are a few 'blocking' levels to choose from, the small block seems not enough and the medium block require some whitelisting. I would very like a quick set and forget. I'm thinking the small block along with uBlock Origin on my 2 PCs would be the best combination? Then iPhone, iPad and Android phone would benefit from Diversion?

If such a comparison ever come, it could come handy if it was posted on the Diversion website.

Thanks!

 

[Idios]

100% CPU usage, any ideas? I'll try unplugging USB and restarting. Internet speed has slowed to 1MB/s from 30MB/s, and for the first time in 3 weeks my Overwatch ping went to 500ms for a few minutes.

Edit:
Unplugging USB and restarting gives about 5% CPU usages. However, I'm still getting really slow speeds (on my PC). Must have moved a wrong ethernet cable as I'm pulling 10Mbps max according to the adapter properties, and my phone is working fine.
Potential issue just seems to be CPU usage locked at 100%, then.

 

[Makaveli]

View attachment 14387

100% CPU usage, any ideas? I'll try unplugging USB and restarting. Internet speed has slowed to 1MB/s from 30MB/s, and for the first time in 3 weeks my Overwatch ping went to 500ms for a few minutes.

Edit:
Unplugging USB and restarting gives about 5% CPU usages. However, I'm still getting really slow speeds (on my PC). Must have moved a wrong ethernet cable as I'm pulling 10Mbps max according to the adapter properties, and my phone is working fine.
Potential issue just seems to be CPU usage locked at 100%, then.

SSH into the router and then write top and hit enter.

You should be able to see what is using the cpu.

 

[shark]

Hello,

I've updated pixelserver using diversion, from version 2.1.1 to 2.1.2, but now im getting this:

Sep 12 02:03:24 pixelserv-tls[12611]: Listening on :*:443
Sep 12 02:03:24 pixelserv-tls[12611]: Abort: Address already in use - :*:80

PS: I've also did an upgrade on the entware packages, using diversions after i did the upgrade on pixelserver.

I've tried changing the ip address from 192.168.50.2, that was being used by pixelserver to others like 192.168.50.3 and 192.168.50.4, but i still get the same thing on my router log! My dhcp starts at 192.168.50.10 - 254.

Edit:

Did a reinstall for diversion, and now everything works again

 

[Xentrk]

Hello,

I've updated pixelserver using diversion, from version 2.1.1 to 2.1.2, but now im getting this:

Sep 12 02:03:24 pixelserv-tls[12611]: Listening on :*:443
Sep 12 02:03:24 pixelserv-tls[12611]: Abort: Address already in use - :*:80

PS: I've also did an upgrade on the entware packages, using diversions after i did the upgrade on pixelserver.

I've tried changing the ip address from 192.168.50.2, that was being used by pixelserver to others like 192.168.50.3 and 192.168.50.4, but i still get the same thing on my router log! My dhcp starts at 192.168.50.10 - 254.

Edit:

Did a reinstall for diversion, and now everything works again

I am able to duplicate the issue when trying to update pixelserv-tls using diversion from version 2.1.1 to 2.1.2.

Sep 12 05:32:07 RT-AC88U-8248 pixelserv-tls[26884]: pixelserv-tls 2.1.2 (compiled: Sep  8 2018 20:33:38) options: <none>
Sep 12 05:32:07 RT-AC88U-8248 pixelserv-tls[26884]: Listening on :*:443
Sep 12 05:32:07 RT-AC88U-8248 pixelserv-tls[26884]: Abort: Address already in use - :*:80

I tried stopping/restarting diversion and still get the error message. Last time I had this issue, I tried the reinstall option with no luck. I had to uninstall and do a new install to get pixelserv-tls working. This time, the reinstall option fixed it.

 

[shark]

I am able to duplicate the issue when trying to update pixelserv-tls using diversion from version 2.1.1 to 2.1.2.

Sep 12 05:32:07 RT-AC88U-8248 pixelserv-tls[26884]: pixelserv-tls 2.1.2 (compiled: Sep  8 2018 20:33:38) options: <none>
Sep 12 05:32:07 RT-AC88U-8248 pixelserv-tls[26884]: Listening on :*:443
Sep 12 05:32:07 RT-AC88U-8248 pixelserv-tls[26884]: Abort: Address already in use - :*:80

I tried stopping/restarting diversion and still get the error message. Last time I had this issue, I tried the reinstall option with no luck. I had to uninstall and do a new install to get pixelserv-tls working. This time, the reinstall option fixed it.

On the bright side, at least i'm not alone in this hehe

 

[Xentrk]

On the bright side, at least i'm not alone in this hehe

@thelonelycoder is very good about updating the code based on the feedback in this thread. I had similar problem after migrating from AB-Solution. Based on the feedback, he updated the main page to check for pixelserv-tls running properly. Did you see an error or warning message on the Diversion main menu? I did not. Probably needs another look as the main page did not display a warning message for the condition of one port already in use:

Sep 12 05:32:07 RT-AC88U-8248 pixelserv-tls[26884]: pixelserv-tls 2.1.2 (compiled: Sep  8 2018 20:33:38) options: <none>
Sep 12 05:32:07 RT-AC88U-8248 pixelserv-tls[26884]: Listening on :*:443
Sep 12 05:32:07 RT-AC88U-8248 pixelserv-tls[26884]: Abort: Address already in use - :*:80

 

[shark]

@thelonelycoder is very good about updating the code based on the feedback in this thread. I had similar problem after migrating from AB-Solution. Based on the feedback, he updated the main page to check for pixelserv-tls running properly. Did you see an error or warning message on the Diversion main menu? I did not. Probably needs another look as the main page did not display a warning message for the condition of one port already in use:

Sep 12 05:32:07 RT-AC88U-8248 pixelserv-tls[26884]: pixelserv-tls 2.1.2 (compiled: Sep  8 2018 20:33:38) options: <none>
Sep 12 05:32:07 RT-AC88U-8248 pixelserv-tls[26884]: Listening on :*:443
Sep 12 05:32:07 RT-AC88U-8248 pixelserv-tls[26884]: Abort: Address already in use - :*:80

The only thing that i saw in the main menu of diversion, was that it was reporting that pixelserver was not running after i got that error on the log.

 

[Xentrk]

The only thing that i saw in the main menu of diversion, was that it was reporting that pixelserver was not running after i got that error on the log.

Interesting! I didn't have a warning message about pixelserv-tls not running. What router model and firmware version are you using?

 

[thelonelycoder]

Also is there an ETA for the Diversion GUI? Any time soon?

No, and no.

 

[thelonelycoder]

My problem is that I have 4 asus routers in diffrent locations over the country.
I dont allow SSH thru WAN as it will be unsafe.

So when I need to change any of these lists: whitelist/blacklist/hostslist/plus-hostslist/wc_blacklist/customlist

At my office then I need to physically SSH to do it, I also need to do it on the router at home and on my summer house and so on. its not that efficiant as you see.

I need an option under el (edit list) "custom diversion hostlists address/Set sync with custom list server " for the whitelist/blacklist/hostslist/plus-hostslist/wc_blacklist/customlist so it will synk to my other routers automatically - when I do changes on list in my server at: http://hostfile.unaux.com/whitelist
Will be ok if it only update/sync once per day.


The "blockinglist" is generated by host file source and updated automatically from Diversions default host file source. I would like to have an option to specify a custom host file server source under the installation of Diversion -becouse I dont want to use the "steven host file" or the "yoyo file" by default. I want it to use http://hostfile.unaux.com/hosts <- from my own server and I want of course diversion to check for updates on that address. I know that you have so that I can specify a custom source of host file but then I need to enable the +list future that get like 4 other host files sources extra -that I also dont want to use. You probably know by now what I meen. Would be grejt if we get an option "[1]Use default host source [2]Add your own host address:" after we choose "install diversion Lite or Standard"

Will this be possible to do?

This all will be a feature of the Diversion Pro Edition. While Diversion Light and Standard Edition are meant for single routers, Diversion Pro has features to manage multiple routers from one location.
This is how I manage my several test routers and the main router at the moment locally.
Diversion Pro is in development but time to code is scarce. No ETA, as with the WebUI for Diversion.

 

[thelonelycoder]

I dont allow SSH thru WAN as it will be unsafe.

I use a VPN to connect. The best and preferred option. I'm doing it right now, managing my main router from afar in a safe and secure way.

 

[thelonelycoder]

I did notice a few things about the stats, which I can't recall seeing before.

In the 'noisiest client' overview as well as in the 'Top x domains for x clients' overview, I now see dhcp requests in between the regular stats:

I'll look into it and likely simple to fix. Those DHCPREQUEST(br0) were filtered out in AB-Solution stats, so they should also not be included in Diversion. In c Diversion stats, did you set "Filter local client names" to on? This has immediate affect on the stats if you run it.
Also, try setting "domain-needed" in the Dnsmasq settings in ds. This, however, will only be fully effective with a new set of dnsmasq.log files which is after the stats and the blocking file update have run through the weekly cron job scheduler.

 

[thelonelycoder]

Can someone write a comparison between someone using uBlock Origin vs Diversion alone, vs Diversion + pixelserv-tls?

Diversion is, as the thread title says, router based. It cannot be as granular and effective as a Browser based ad-blocker. Browsers see the source code of the loading website and can act upon strings like "ad.jpg" from its sources. Diversion only blocks complete domains (or hosts) and therefore has to block doubleclick.com completely instead of only blocking certain content such as doubleclick.com/ads/images but letting doubleclick.com/ads/goodstuff trough like browser based solutions do.
Despite the simple and crude method Diversion is blocking ads, together with pixelserv-tls it does a remarkable job at it. I use no other browser or device based ad-blocker ever since I came up with a way to block them on the router. When on the road, I use a VPN to still be behind my main routers ad-blocker.

Since last summer we use a new router, the RT-AC1900P and I know I could run Diversion on it but will CPU heat increase a lot because of extra work?

The RT-AC1900P is my main testrouter. The one a good part of AB-Solution and all of Diversion was coded and first tested on. Diversion is not CPU intensive in it's normal operation. It's more a question of how much memory is used. The larger the blocking file, the more memory Dnsmasq needs.