Skynet Skynet - Router Firewall & Security Enhancements

[yk101]

Hi

Looking for some help We are having problem with onedrive not loading our files, I have now whitelisted 3 IPs and it works for a couple of hours but later it blocked again

Nov 19 20:43:16 kernel: [BLOCKED - OUTBOUND] IN=br0 OUT= MAC= SRC=10.*.*.* DST=157.55.109.224 LEN=52 TOS=0x00 PREC=0x00 TTL=128 ID=28533 DF PROTO=TCP SPT=64288 DPT=443 SEQ=2572823490 ACK=0 WINDOW=64240 RES=0x00 SYN URGP=0 OPT (020405B40103030801010402)

Interesting, this is not a onedrive IP!

Name:      157.55.109.224
Address 1: 157.55.109.224 msnbot-157-55-109-224.search.msn.com

ARe the rest of IP's you whitelisted are in the same range?

 

[Adamm]

Interesting, this is not a onedrive IP!

Name: 157.55.109.224
Address 1: 157.55.109.224 msnbot-157-55-109-224.search.msn.com

ARe the rest of IP's you whitelisted are in the same range?

It was part of Microsofts umbrella of servers, there were 2 conflicting /24 blocks I found that onedrive use and removed them, there might be more (probably use multiple servers for load balancing) but hopefully thats the last of them.

 

[iManuB]

I don't think pixel-server specifically but maybe something of that nature. I wouldn't loose too much sleep over it unless it happens again.

Okay, @Adamm!

Thanks so much!

 

[Butterfly Bones]

Man, some seriously aggressive blocking going now. Look at Outbound just this morning doing my morning web rounds with kickstart coffee.

Nov 21 02:25:37 Skynet: [Complete] 149763 IPs / 1849 Ranges Banned. 12660 New IPs / 25 New Ranges Banned. 301 Inbound / 0 Outbound Connections Blocked! [37s]
Nov 21 07:00:08 Skynet: [Complete] 149762 IPs / 1849 Ranges Banned. 0 New IPs / 0 New Ranges Banned. 683 Inbound / 24 Outbound Connections Blocked! [8s]

In the last two days I've had to whitelist over 20 websites I use regularly, Android development sites like Android File Host, my grocery stores, local coffee roasters, and more. Can we selectively choose the blocking lists that Skynet uses?

 

[Adamm]

Man, some seriously aggressive blocking going now. Look at Outbound just this morning doing my morning web rounds with kickstart coffee.

A good portion will be from our updated telemetry list, microsoft collects a massive amount of data in a lot of their windows applications, even silly things such as ms paint amounting to hundreds if not thousands of probes per day.

In the last two days I've had to whitelist over 20 websites I use regularly, Android development sites like Android File Host, my grocery stores, local coffee roasters, and more. Can we selectively choose the blocking lists that Skynet uses?

Banmalware uses public feeds from reputable providers, the downside is that due to the nature of shared hosting, if there is lets say 1000 websites on a webserver, it only takes one site distributing malware to get an entire server blacklisted (thus blocking 999 legitimate websites). In future when whitelisting websites, run the following command also;

sh /jffs/scripts/firewall stats search malware xxx.xxx.xxx.xxx

That will show you exactly what lists the IP in question appears on. If a certain list has a high number of reported false positives, I'll definitely consider removing it.

 

[Butterfly Bones]

A good portion will be from our updated telemetry list, microsoft collects a massive amount of data in a lot of their windows applications, even silly things such as ms paint amounting to hundreds if not thousands of probes per day.



Banmalware uses public feeds from reputable providers, the downside is that due to the nature of shared hosting, if there is lets say 1000 websites on a webserver, it only takes one site distributing malware to get an entire server blacklisted (thus blocking 999 legitimate websites). In future when whitelisting websites, run the following command also;

sh /jffs/scripts/firewall stats search malware xxx.xxx.xxx.xxx

That will show you exactly what lists the IP in question appears on. If a certain list has a high number of reported false positives, I'll definitely consider removing it.

No windows here, only Linux, ChromeOS and Android.

It is the shared hosting. When I get an outbound block, I copy the IP and check OTX. If OTX lists it as not malicious, and I check the URLS for the site I use, then I whitelist it. Here is one this morning that blocked my local coffee roaster, lots of shared sites!
https://otx.alienvault.com/indicator/ip/23.227.38.32

Thanks for the CLI reminder. I have your page one commands in a quick text file linked on my desktop that I use often, but overlooked that one. Here is the return of the above IP.

Exact Matches;
https://iplists.firehol.org/files/hphosts_hjk.ipset - 23.227.38.32
https://iplists.firehol.org/files/hphosts_mmt.ipset - 23.227.38.32
https://iplists.firehol.org/files/ransomware_feed.ipset - 23.227.38.32

The ransomware list gives me serious pause, but I know that one site I use is safe, so does whitelisting that IP leave one vulnerable to the other shared hosting sites that are malicious?

I had the Merlin firewall active of course, but never logged drops. Seeing how many port probes I get hour by hour, day in and day out, my web paranoia has gone up quite a few points since installing Skynet.

 

[matthew_eli]

Hi, I tried to install Skynet on my RT-AC86U with 382.1 fw, but at the ends of the install, it told me there was no SWAP install, even if I have a 3Gb SWAP partition currently used by the router. Is that a known problem on RT-AC86U? How can I fix it?

I already tried the command with the debug, but it does not detect the SWAP partition

 

[Adamm]

It is the shared hosting. When I get an outbound block, I copy the IP and check OTX. If OTX lists it as not malicious, and I check the URLS for the site I use, then I whitelist it. Here is one this morning that blocked my local coffee roaster, lots of shared sites!
https://otx.alienvault.com/indicator/ip/23.227.38.32

Yeah that's the catch 22 of cheap shared hosting. The server in question hosts around 500+ websites, but 4 are linked to malware, so unfortunately they got the whole server blacklisted by two list providers even though 99% are legitimate websites.

The ransomware list gives me serious pause, but I know that one site I use is safe, so does whitelisting that IP leave one vulnerable to the other shared hosting sites that are malicious?

As long as your not visiting the 4 blacklisted domains mentioned on the alienvault website it should be fine, and considering you seem like a competent individual I'm sure you are smart enough not to click on the dodgy websites in the first place.

I had the Merlin firewall active of course, but never logged drops. Seeing how many port probes I get hour by hour, day in and day out, my web paranoia has gone up quite a few points since installing Skynet.

Its a never-ending battle, with computing power growing exponentially and a limited amount of IPv4 addresses, botnets just probe the internet constantly looking for vulnerable devices. Until something like IPv6 is widespread and there are 340 undecillion unique addresses to scan, the problem is only going to get worse.

 

[Adamm]

Hi, I tried to install Skynet on my RT-AC86U with 382.1 fw, but at the ends of the install, it told me there was no SWAP install, even if I have a 3Gb SWAP partition currently used by the router. Is that a known problem on RT-AC86U? How can I fix it?

I already tried the command with the debug, but it does not detect the SWAP partition

Skynet scans post-mount for swap entries, if it doesn't detect one it gives you that error. Do you know how you installed the swap file (did you follow a certain guide or know where you added the startup entires)?

 

[matthew_eli]

Skynet scans post-mount for swap entries, if it doesn't detect one it gives you that error. Do you know how you installed the swap file (did you follow a certain guide or know where you added the startup entires)?

Yes, the startup entry is on init-start:

#!/bin/sh

swapon UUID="6dd974bd-f709-4095-974c-0bc65c5353b4"
echo "UUID=6dd974bd-f709-4095-974c-0bc65c5353b4 none swap sw 0 0" >> /etc/fstab

Do I need to place it in a different script?

 

[Adamm]

Yes, the startup entry is on init-start:

Two options here. personally I think you're better off installing swap the same way Skynet/entware do which is the recommended way for this platform (Skynet can automate this process). The result is one simple entry which looks similar to;

swapon /tmp/mnt/Main/skynet/myswap.swp # Skynet Firewall Addition

The second option is moving those entries to the post-mount file (init-start is executed upon startup, not necessarily when the USB is mounted) and Skynet will recognise them.

 

[matthew_eli]

Two options here. personally I think you're better off installing swap the same way Skynet/entware do which is the recommended way for this platform (Skynet can automate this process). The result is one simple entry which looks similar to;

swapon /tmp/mnt/Main/skynet/myswap.swp # Skynet Firewall Addition

The second option is moving those entries to the post-mount file (init-start is executed upon startup, not necessarily when the USB is mounted) and Skynet will recognise them.

Many thanks Adam! I followed your suggestion and I removed the init-start script and placed those entries on the post-mount. Now the Skynet script works flawlessly on RT-AC86U

 

[skeal]

Made a humble contribution to your cause. Thanks again man!

 

[Adamm]

Made a humble contribution to your cause. Thanks again man!

Highly appreciated, sent you a pm

 

[Martin_D]

Hi, All

We are having problems connecting to parts of the Microsoft Azure network

Nov 23 18:02:05 kernel: [BLOCKED - OUTBOUND] IN=br0 OUT= MAC= SRC=10.*.*.*DST=40.114.149.220 LEN=52 TOS=0x00 PREC=0x00 TTL=128 ID=3771 DF PROTO=TCP SPT=50391 DPT=443 SEQ=3514711273 ACK=0 WINDOW=64240 RES=0x00 SYN URGP=0 OPT (020405B40103030801010402)

 

[pattiri]

@Adamm is this something to worry about?

[1-13]: 4

Select Whitelist Option:
[1]  --> IP/Range
[2]  --> Domain
[3]  --> Port
[4]  --> Refresh VPN Whitelist
[5]  --> Remove Entries
[6]  --> Refresh Entries
[7]  --> List Entries

[1-7]: 6

Refreshing Shared Whitelist Files
Whitelisting Shared Domains
ipset v6.32: Error in line 1: The set with the given name does not exist
/opt/bin/firewall: line 2327: can't fork
/opt/bin/firewall: line 2327: can't fork
/opt/bin/firewall: line 2327: can't fork
/opt/bin/firewall: line 2327: can't fork

AC88U with 382.1_1

 

[Adamm]

Hi, All

We are having problems connecting to parts of the Microsoft Azure network

Nov 23 18:02:05 kernel: [BLOCKED - OUTBOUND] IN=br0 OUT= MAC= SRC=10.*.*.*DST=40.114.149.220 LEN=52 TOS=0x00 PREC=0x00 TTL=128 ID=3771 DF PROTO=TCP SPT=50391 DPT=443 SEQ=3514711273 ACK=0 WINDOW=64240 RES=0x00 SYN URGP=0 OPT (020405B40103030801010402)

I've removed the IP from the telemetry list, let me know if you run into other issues with Microsoft services. (Run banmalware again to apply the changes on your end)

AC88U with 382.1_1

Okay that's interesting, its the first time I've seen the error on any other device but the AC86U (which may be a good thing if we can narrow the issue down to the .382 codebase in general). If you continue to run into this error, try install a swap file. If after doing that it still continues, let me know and I will apply the same temporary fix I did for the AC86U to all devices on the 382 codebase.



@RMerlin might be worth looking into again, the error above is being triggered by the following, except this time on a new model;

                    grep -hvF "#" /jffs/shared-*-whitelist | sed 's~http[s]*://~~;s~/.*~~' | awk '!x[$0]++' | while IFS= read -r "domain"; do
                        for ip in $(Domain_Lookup "$domain" 2> /dev/null); do
                            ipset -q -A Whitelist "$ip" comment "Shared-Whitelist: $domain"
                        done &
                    done

The part that seems to be the issue is IPSet adding 35 or so entries to the IPSet in parallel. I don't have a device on the 382 codebase yet (this is confirmed to only be an issue with the new codebase) but I assume this can be reproduced using the following;

#!/bin/sh
ipset -q create Testlist hash:net comment
for ip in $(cat /tmp/iplist.txt); do
    ipset -q -A Testlist "$ip" comment "Testlist-Entry"
done &

With /tmp/iplist.txt being a random IP list. I tested on my AC68U and can add thousands of entries with no issue.

 

[pattiri]

If you continue to run into this error, try install a swap file. If after doing that it still continues, let me know and I will apply the same temporary fix I did for the AC86U to all devices on the 382 codebase.

Installed 512 MB swap file and it seems to be solved now.

[1-13]: 4

Select Whitelist Option:
[1]  --> IP/Range
[2]  --> Domain
[3]  --> Port
[4]  --> Refresh VPN Whitelist
[5]  --> Remove Entries
[6]  --> Refresh Entries
[7]  --> List Entries

[1-7]: 6

Refreshing Shared Whitelist Files
Whitelisting Shared Domains
Saving Changes

Skynet: [Complete] 160177 IPs / 1992 Ranges Banned. 0 New IPs / 0 New Ranges Banned. 17 Inbound / 7 Outbound Connections Blocked! [24s]

 

[RMerlin]

@RMerlin might be worth looking into again, the error above is being triggered by the following, except this time on a new model;

Just don't fork too many things in parallel, it's not gonna have any real performance improvement, and you run the risk of collisions anyway.

I can't do anything about it, Asus says the fork() issues are caused by Broadcom's toolchain.

 

[Adamm]

Installed 512 MB swap file and it seems to be solved now.

Keep an eye on if it comes back, I've yet to 100% confirm a swap file fixes the issue as its intermittent. But if it does I'll apply the 86U changes to all devices.

Just don't fork too many things in parallel, it's not gonna have any real performance improvement, and you run the risk of collisions anyway.

On an AC68U this shaves about 30-40s off the runtime of the overall function which accounts for about half the total time, so its quite noticeable. Its also the only method of parallel operations without a full xargs or GNU Parallel binary.

With

Downloading filter.list     [0s]
Whitelisting Shared Domains     [2s]
Consolidating Blacklist     [18s]
Saving Changes             [6s]
Removing Previous Malware Bans  [2s]
Filtering IPv4 Addresses     [5s]
Filtering IPv4 Ranges         [1s]
Applying Blacklists         [8s]
[45s Overall]

Without

Downloading filter.list     [0s]
Whitelisting Shared Domains     [11s]
Consolidating Blacklist     [37s]
Saving Changes             [5s]
Removing Previous Malware Bans  [1s]
Filtering IPv4 Addresses     [6s]
Filtering IPv4 Ranges         [0s]
Applying Blacklists         [11s]
[73s Overall]

I can't do anything about it, Asus says the fork() issues are caused by Broadcom's toolchain.

Is this something Broadcom is aware of and plan to fix? Seems like a pretty significant flaw introduced.