Skynet SkyNet and Diversion have conflicts when using user defined DNS servers - SOLVED

[grahamgo]

Running Asus Merlin RT-AX88U 3004.388.7

I have found that when using SkyNet and Diversion that in my case I seemingly have to choose for one or the other. Running both at the same time does not seem to be possible.

1. I install Skynet, I have my DNS servers settings in the LAN DHCP server section. Result. Perfect, Skynet runs. If I use menu [3] Malware blacklist. [1] Update I see :-

Downloading filter.list | [0s]
Refreshing Whitelists | [5s]
Consolidating Blacklist | [4s]
Filtering IPv4 Addresses | [0s]
Filtering IPv4 Ranges | [0s]
Applying New Blacklist | [1s]
Refreshing AiProtect Bans | [11s]
Saving Changes | [0s]

However. If I move my DNS settings to the WAN, WAN DNS Settings, in DNS server - Assign section. If I now use menu [3] Malware blacklist. [1] Update I see :-

Downloading filter.list | [15s]
Refreshing Whitelists | [165s]
Consolidating Blacklist | curl: (2) no URL specified
curl: try 'curl --help' for more information [4s]
[*] List Content Error Detected - Stopping Banmalware

So conclusion. Diversion works great with my DNS settings in the LAN section, but not when in the WAN section.

Now we come to Diversion.

It is the exact opposite. Diversion works great with the my DNS settings in the WAN section, but does not work with the DNS settings in the LAN section ( this is explicitly mentioned when installing Diversion)

So final conclusion. It appears that Skynet and Diversion cannot be used together when a custom DNS is installed, but both do work when an IP such as googles 8.8.8.8 is used. So is there a workaround?, or do I have to choose which one to use?

 

[dave14305]

What’s in /jffs/addons/shared-whitelists/shared-Skynet-whitelist?

 

[Ripshod]

I have quad9 DoT servers defined in my WAN settings, and have DNS Director set to divert all DNS to Router. I've never seen such an issue, even when I was running 388.7. I don't doubt you have an issue but I do doubt it's directly related to Diversion.
If you define DNS servers in WAN, set DNS Director to Router and clear all the DNS settings in LAN, I'm sure the problem will be resolved

 

[grahamgo]

What’s in /jffs/addons/shared-whitelists/shared-Skynet-whitelist?

Hi, thanks for the interest, here are the contents. See my note at the bottom of the msg.
golly@RT-AX88U-1228:/jffs/addons/shared-whitelists# ls
shared-Skynet-whitelist shared-Skynet2-whitelist shared-amtm-whitelist

This is the contents of shared-Skynet-whitelist

https://iplists.firehol.org/files/bds_atif.ipset

https://iplists.firehol.org/files/cybercrime.ipset

https://iplists.firehol.org/files/dyndns_ponmocup.ipset

https://iplists.firehol.org/files/et_block.netset

https://iplists.firehol.org/files/et_compromised.ipset

https://iplists.firehol.org/files/firehol_level2.netset

https://iplists.firehol.org/files/firehol_level3.netset

https://iplists.firehol.org/files/spamhaus_edrop.netset

https://iplists.firehol.org/files/urlvir.ipset

This is the contents of shared-Skynet2-whitelist
ipdeny.com
ipapi.co
api.db-ip.com
api.bgpview.io
asn.ipinfo.app
speedguide.net
otx.alienvault.com
github.com
raw.githubusercontent.com
iplists.firehol.org
astrill.com
strongpath.net
snbforums.com
bin.entware.net
nwsrv-ns1.asus.com

pool.ntp.org


Note. This is with my custom DNS values in the LAN - DHCP Server - DNS and WINS Server Setting, i.e. DNS Server1 & 2 Therefore Skynet is running fine. Diversion is not installed. If you want me to install and run Diversion then let me know. Thanks!

 

[grahamgo]

I have quad9 DoT servers defined in my WAN settings, and have DNS Director set to divert all DNS to Router. I've never seen such an issue, even when I was running 388.7. I don't doubt you have an issue but I do doubt it's directly related to Diversion.
If you define DNS servers in WAN, set DNS Director to Router and clear all the DNS settings in LAN, I'm sure the problem will be resolved

I'm sure it's a self-inflicted problem. But I have just replied to @dave14305. First, I will wait and see what he says/requests me to do. Then I will take your guideline and see if that cures the issue. But I am pretty sure that I had the settings the same as you have suggested, but I will double-check. - thanks

 

[dave14305]

This is the contents of shared-Skynet-whitelist

Are there any extra blank lines in the file?

 

[dave14305]

Try nslookup raw.githubusercontent.com in both scenarios. It must resolve correctly.

 

[grahamgo]

Are there any extra blank lines in the file?

No, I dont think so. I am using MobaXterm to go to the location and notepad++ to view. It just shows 9 lines of entries with no blank lines.

 

[grahamgo]

Try nslookup raw.githubusercontent.com in both scenarios. It must resolve correctly.

Ok will try.

TEST1

This is the results with my DNS in the Merlin "WAN DNS Setting" section set to Default status : Get the DNS IP from your ISP automatically.
Then in the LAN section DNS server1 =46.166.189.67, DNS server2 =212.71.249.225 and DNS Director=ON Global Redirection = Router

TEST1 using nslookup raw.githubusercontent.com was very fast (instantaneous).
Also Tried amtm - Skynet menu [3],[1] (Blacklist update). This completes correctly (taking a few seconds)


golly@RT-AX88U-1228:/tmp/home/root# nslookup raw.githubusercontent.com
Server: 37.143.84.228
Address 1: 37.143.84.228 public-dns-primary.glasoperator.nl

Name: raw.githubusercontent.com
Address 1: 185.199.108.133 cdn-185-199-108-133.github.com
Address 2: 185.199.110.133 cdn-185-199-110-133.github.com
Address 3: 185.199.111.133 cdn-185-199-111-133.github.com
Address 4: 185.199.109.133 cdn-185-199-109-133.github.com
Address 5: 2606:50c0:8001::154
Address 6: 2606:50c0:8002::154
Address 7: 2606:50c0:8003::154
Address 8: 2606:50c0:8000::154
golly@RT-AX88U-1228:/tmp/home/root#

TEST2

This is the results with my DNS in the Merlin "WAN DNS Setting" section set to DNS 44.166.189.67,212.71.249.225
Then in the LAN section DNS server1 =blank, DNS server2 =blank and DNS Director=ON Global Redirection = Router

TEST1 using nslookup raw.githubusercontent.com was very slow (taking 51 seconds).
Also Tried amtm - Skynet menu [3],[1] (Blacklist update). This fails after 20 seconds or so.

golly@RT-AX88U-1228:/tmp/home/root# nslookup raw.githubusercontent.com
Server: 46.166.189.67
Address 1: 46.166.189.67

Name: raw.githubusercontent.com
Address 1: 185.199.108.133 cdn-185-199-108-133.github.com
Address 2: 185.199.110.133 cdn-185-199-110-133.github.com
Address 3: 185.199.109.133 cdn-185-199-109-133.github.com
Address 4: 185.199.111.133 cdn-185-199-111-133.github.com
Address 5: 2606:50c0:8003::154
Address 6: 2606:50c0:8002::154
Address 7: 2606:50c0:8001::154
Address 8: 2606:50c0:8000::154
golly@RT-AX88U-1228:/tmp/home/root#


Conclusion. Both nslookup's do seem to resolve in both situations, but with a big difference in response time.
My DNS1 = 44.166.189.67 DNS2= 212.71.249.225 provided by getflix.com

 

[dave14305]

Skynet’s curl download of the lists file is probably timing out waiting for the DNS to resolve. Try running this in both scenarios:

time curl -L --retry 3 --connect-timeout 3 --max-time 6 --retry-delay 1 --retry-all-errors https://raw.githubusercontent.com/Adamm00/IPSet_ASUS/master/filter.list

 

[grahamgo]

Skynet’s curl download of the lists file is probably timing out waiting for the DNS to resolve. Try running this in both scenarios:

time curl -L --retry 3 --connect-timeout 3 --max-time 6 --retry-delay 1 --retry-all-errors https://raw.githubusercontent.com/Adamm00/IPSet_ASUS/master/filter.list

Dave, thanks. Here are the results. The settings are as described in the previous post.
Test1

time curl -L --retry 3 --connect-timeout 3 --max-time 6 --retry-delay 1 --retry-all-errors https://raw.githubusercontent.com/Adamm00/IPSet_ASUS/master/filter.list

https://iplists.firehol.org/files/bds_atif.ipset
https://iplists.firehol.org/files/cybercrime.ipset
https://iplists.firehol.org/files/dyndns_ponmocup.ipset
https://iplists.firehol.org/files/et_block.netset
https://iplists.firehol.org/files/et_compromised.ipset
https://iplists.firehol.org/files/firehol_level2.netset
https://iplists.firehol.org/files/firehol_level3.netset
https://iplists.firehol.org/files/spamhaus_edrop.netset
https://iplists.firehol.org/files/urlvir.ipset
real      0m 0.07s
user    0m 0.04s
sys     0m 0.00s

Test2

time curl -L --retry 3 --connect-timeout 3 --max-time 6 --retry-delay 1 --retry-all-errors https://raw.githubusercontent.com/Adamm00/IPSet_ASUS/master/filter.list

curl: (6) name lookup timed out
Warning: Problem : timeout. Will retry in 1 seconds. 3 retries left.
curl: (6) name lookup timed out
Warning: Problem : timeout. Will retry in 1 seconds. 2 retries left.
curl: (6) name lookup timed out
Warning: Problem : timeout. Will retry in 1 seconds. 1 retries left.
curl: (6) name lookup timed out
Command exited with non-zero status 6
real    0m 15.01s
user    0m 0.00s
sys     0m 0.00s

Retested with an increased timeout:-

time curl -L --retry 3 --connect-timeout 10 --max-time 6 --retry-delay 1 --retry-all-errors https://raw.githubusercontent.com/Adamm00/IPSet_ASUS/master/filter.list

https://iplists.firehol.org/files/bds_atif.ipset
https://iplists.firehol.org/files/cybercrime.ipset
https://iplists.firehol.org/files/dyndns_ponmocup.ipset
https://iplists.firehol.org/files/et_block.netset
https://iplists.firehol.org/files/et_compromised.ipset
https://iplists.firehol.org/files/firehol_level2.netset
https://iplists.firehol.org/files/firehol_level3.netset
https://iplists.firehol.org/files/spamhaus_edrop.netset
https://iplists.firehol.org/files/urlvir.ipset
real      0m 5.23s
user    0m 0.05s
sys     0m 0.00s

 

[dave14305]

The settings are as described in the previous post.

Is the 44 DNS IP a typo? The getflix site lists 46.166.189.67, not 44.166.189.67. And so does TEST1. Please redo TEST2 with the correct IP.

 

[grahamgo]

Is the 44 DNS IP a typo? The getflix site lists 46.166.189.67, not 44.166.189.67. And so does TEST1. Please redo TEST2 with the correct IP.

Yes, sorry typo. I'm using 46.166.189.67 I have reported to getflix that I am experiencing problems with this DNS and have asked them to check. They are generally quite quick in responding. As you can guess, this is to provide the house with UK TV, we are in NL

More..... getflix have just responded. They confirm that there is a problem with 46.166.189.67 and suggest that I use another DNS until it is resolved. This DNS just happens to be the nearest in Amsterdam. So this whole thread may have been in vain.
I will re-test later today and report back. Thank you for all the help BTW

 

[dave14305]

Yes, sorry typo. I'm using 46.166.189.67 I have reported to getflix that I am experiencing problems with this DNS and have asked them to check. They are generally quite quick in responding. As you can guess, this is to provide the house with UK TV, we are in NL

Just run from the router:

time nslookup raw.githubusercontent.com 46.166.189.67

 

[grahamgo]

Just run from the router:

time nslookup raw.githubusercontent.com 46.166.189.67

Ok, I got have just got an update from getflix. They confirm this DNS is now fully operational. I have just run the test. The result is much better.


golly@RT-AX88U-1228:/tmp/home/root# time nslookup raw.githubusercontent.com 46.1
66.189.67
Server: 46.166.189.67
Address 1: 46.166.189.67

Name: raw.githubusercontent.com
Address 1: 185.199.109.133 cdn-185-199-109-133.github.com
Address 2: 185.199.111.133 cdn-185-199-111-133.github.com
Address 3: 185.199.110.133 cdn-185-199-110-133.github.com
Address 4: 185.199.108.133 cdn-185-199-108-133.github.com
Address 5: 2606:50c0:8000::154
Address 6: 2606:50c0:8002::154
Address 7: 2606:50c0:8001::154
Address 8: 2606:50c0:8003::154
real 0m 0.12s
user 0m 0.00s
sys 0m 0.00s

I have also run the amtm - Skynet test [3]+[1] its working !

Downloading filter.list | [0s]
Refreshing Whitelists | [6s]
Consolidating Blacklist | [1s]
Filtering IPv4 Addresses | [1s]
Filtering IPv4 Ranges | [0s]
Applying New Blacklist | [1s]
Refreshing AiProtect Bans | [15s]
Saving Changes | [0s]

 

[grahamgo]

Just run from the router:

time nslookup raw.githubusercontent.com 46.166.189.67

 

[grahamgo]

@dave14305 A special thank-you. This morning, rechecked Skynet using the repaired DNS, now it works great. I then installed Diversion and confirm both are now working in unison. Your assistance enabled me to focus in finding the problem. Thanks again!