Skynet Skynet - Router Firewall & Security Enhancements

[returntrip]

I suggest users update to v4.9.3 and re-generate their whitelists via;

sh /jffs/scripts/firewall whitelist remove

There was a bug where the script was detecting some outgoing connections as Private IP's and incorrectly whitelisting them. While there is no harm in this it will prevent them from being blocked in future, so its best to purge the whitelist.

Done! Btw how do I check the whitelist content?

Sent from my Nexus 6P using Tapatalk

 

[Adamm]

Done! Btw how do I check the whitelist content?

Sent from my Nexus 6P using Tapatalk

ipset -L Whitelist

By default this includes;
WAN IP
DNS IP's
Lan /24
Github update server
AB-Solution host file provider (if it detects it installed)

The only time the script automatically adds to this list is if it detects a PrivateIP being banned so it doesn't happen in future or cause any unhappy devices.

 

[Kim Puleston]

This is fab, though I think I am encoutering an Error, Skynet seemed to install just fine but notice it is not banning any IP's so I started the firewall as per the command and I get this output.

Enabling Firewall Logging
Skynet: [INFO] Startup Initiated ... ... ...
ipset v6.29: Error in line 2: Hash is full, cannot add more elements
Skynet: [Complete] 0 IPs / 0 Ranges banned. 0 New IPs / 0 New Ranges Banned. 0 IP / 0 Range Connections Blocked! [2s]

Any idea ? I installed this without installing anything else, just Merlin Fimrware.

Thanks,

 

[Adamm]

This is fab, though I think I am encoutering an Error, Skynet seemed to install just fine but notice it is not banning any IP's so I started the firewall as per the command and I get this output.

Enabling Firewall Logging
Skynet: [INFO] Startup Initiated ... ... ...
ipset v6.29: Error in line 2: Hash is full, cannot add more elements
Skynet: [Complete] 0 IPs / 0 Ranges banned. 0 New IPs / 0 New Ranges Banned. 0 IP / 0 Range Connections Blocked! [2s]

Any idea ? I installed this without installing anything else, just Merlin Fimrware.

Thanks,

Skynet should automatically run after installation (or a restart of the firewall) so its best to let the script manage its-self in that aspect.

As for the error, I can't say I've run into that before (not sure how you can run into that limit on a fresh setup). Please run the following;

sh /jffs/scripts/firewall debug restart

Then post the output of the following;

sh /jffs/scripts/firewall debug info
ipset -L Blacklist | wc -l
ipset -L BlockedRanges | wc -l
ipset -L Whitelist | wc -l

Thanks

 

[Kim Puleston]

Skynet should automatically run after installation (or a restart of the firewall) so its best to let the script manage its-self in that aspect.

As for the error, I can't say I've run into that before (not sure how you can run into that limit on a fresh setup). Please run the following;

sh /jffs/scripts/firewall debug restart

Then post the output of the following;

sh /jffs/scripts/firewall debug info
ipset -L Blacklist | wc -l
ipset -L BlockedRanges | wc -l
ipset -L Whitelist | wc -l

Thanks

Odd, when I use the command ''sh /jffs/scripts/firewall debug restart'' I do not get an output as described, I get the below.

''Restarting Firewall Service

Done.''

If I run the debug info command I get the below

Router Model: RT-AC88U
Skynet Version: v4.9.3 (11/06/2017)
iptables v1.4.14 - (eth0)
ipset v6.29, protocol version: 6
FW Version: 380.66_4
Install Dir; /jffs
Startup Entry Detected
Cronjobs Detected
Autobanning Enabled
Debug Mode Enabled
Whitelist IPTable Detected
BlockedRanges IPTable Detected
Blacklist IPTable Detected
Whitelist IPSet Detected
BlockedRanges IPSet Detected
Blacklist IPSet Detected
Skynet: [Complete] IPs / Ranges banned. 0 New IPs / 0 New Ranges Banned. 0 IP / 0 Range Connections Blocked! [1s]

 

[yk101]

Did you ran "firewall banmalware"?

 

[Kim Puleston]

Did you ran "firewall banmalware"?

Im a fool, that sorted it,

Though it looks like it is conflicting with another script I use from here: https://www.snbforums.com/threads/yet-another-malware-block-script-using-ipset-v4-and-v6.38935/

''
Skynet: [ERROR] Conflicting Malware Script Detected; /jffs/scripts/ya-malware-block.sh
For Custom Filter Importing Use; ( sh /jffs/scripts/firewall banmalware URL )
Downloading Lists
Filtering IPv4 Addresses
Filtering IPv4 Ranges
Applying Blacklists
Warning; This May Have Blocked Your Favorite Website
For Whitelisting Domains Use; ( sh /jffs/scripts/firewall whitelist domain URL )
Saving Changes
Skynet: [Complete] 138658 IPs / 5238 Ranges banned. 138658 New IPs / 5238 New Ranges Banned. 24 IP / 0 Range Connections Blocked! [29s]
enterprise@RT-AC88U:/tmp/home/root#

 

[Adamm]

Though it looks like it is conflicting with another script I use from here: https://www.snbforums.com/threads/yet-another-malware-block-script-using-ipset-v4-and-v6.38935/

That explains the error (although I'd have to investigate further to get the exact reason why). That being said, the banmalware function of this script and ya-malware-block do pretty much the exact same thing as they both source from almost identical filter lists. There is no real need to run both as you will be blocking the same things twice. But this is just one of Skynets features so its up to the user on what script suits their needs better.

 

[Kim Puleston]

That explains the error (although I'd have to investigate further to get the exact reason why). That being said, the banmalware function of this script and ya-malware-block do pretty much the exact same thing as they both source from almost identical filter lists. There is no real need to run both as you will be blocking the same things twice. But this is just one of Skynets features so its up to the user on what script suits their needs better.

Thanks for that, I have gone with yours as I find the script more intuitive Thanks for the help

 

[MarCoMLXXV]

Hi, I have a few questions: why, when I run

marco@RT-AC68U:/tmp/home/root# sh /jffs/scripts/firewall stats

are the top 10 in- and outbound hosts targeting / blocked similar to @Adamm 's? I've posted the output on pastebin: https://pastebin.com/aid2kn3v

My questions: why is alienvault being blocked? Isn't it used specifically for verifying malicious IP-addresses? And second: what is speedguide doing in both our stats? I haven't used it, as far as I'm aware... Is the router firmware using it for something? And if both are serving a legit purpose: shouldn't they be permanently whitelisted?

Thanks in advance!

 

[Adamm]

My questions: why is alienvault being blocked? Isn't it used specifically for verifying malicious IP-addresses? And second: what is speedguide doing in both our stats?

Neither are being blocked, they are simply there as links to a IP reputation database and a port usage database for whats actually being blocked.

As for your bans, a lot look like false positives (most of your top10 outbound are google or apple IPs). I believe this is because you imported your old ipset list which was collecting data before I implemented this change which prevented mail ports from being blacklisted and only dropped. I suggest now the rules are mature you run an "unban all" and start fresh. Beyond that everything else looks fine.

 

[MarCoMLXXV]

Neither are being blocked, they are simply there as links to a IP reputation database and a port usage database for whats actually being blocked.

Ah, I see. I guess...

Last 10 Unique Connections Blocked (Inbound)
Last 10 Unique Connections Blocked (Outbound)
Last 10 Autobans
Last 10 Unique HTTP(s) Blocks

...gave me the wrong impression How is one (read: a noob) supposed to know these are just references? Did I miss something?

As for your bans, a lot look like false positives (most of your top10 outbound are google or apple IPs). I believe this is because you imported your old ipset list which was collecting data before I implemented this change which prevented mail ports from being blacklisted and only dropped. I suggest now the rules are mature you run an "unban all" and start fresh. Beyond that everything else looks fine.

Okay, will do that. Thanks again.

Ps. Told ya I couldn't keep up with your pace

Edit:

Skynet: [INFO] Removing All 121806 Entries From Blacklist ... ... ...
Saving Changes
Skynet: [Complete] 0 IPs / 0 Ranges Banned. -121806 New IPs / 0 New Ranges Banned. 0 Inbound                                                                                                 / 0 Outbound Connections Blocked! [1s]

Done... Now wondering why I restored them from my backup in the first place...

 

[Adamm]

...gave me the wrong impression How is one (read: a noob) supposed to know these are just references? Did I miss something?

My assumption was that its easier to provide information as to what an IP is rather then just post it so a user can see "hey this ip from a shady country has a previous history of questionable activity". I guess now you know

As for the pace of the project, I feel like we have hit maturity in the script/rules and are ready to mark it as stable. There are no outstanding bugs (that I'm aware of) and functionality is at a point I am happy with that can work around most users setups, along with things like smart lock file/usb detection.

 

[MarCoMLXXV]

Okay, thanks for clarifying, but please keep in mind that I'm not a true novice and yet so far, far from an expert, so I think it could come in handy to document it somewhere (if it hasn't already been), to prevent more questions... Just my two cents.

As for the pace of the project, I feel like we have hit maturity in the script/rules and are ready to mark it as stable. There are no outstanding bugs (that I'm aware of) and functionality is at a point I am happy with that can work around most users setups, along with things like smart lock file/usb detection.

It has definitely made a huge progress, no doubt about it and I truly appreciate and respect your effort and commitment in helping others out. Hat's off to you.

Next up: a sh*tload of data... Might be just a coincidence, but after following your instructions (unban all), I did a clean (?) reboot, initiated from the WebUI, and noticed this in my logfiles after the restart:

Jun 14 07:45:53 dMP17 Skynet: [ERROR] USB Not Found After 6 Retries - Please Fix Immediately!
Jun 14 07:45:53 dMP17 Skynet: [ERROR] When Fixed Run ( sh /jffs/scripts/firewall debug restart )

/dev/sdb1 has already been mounted at this point, which is a small 4 Gb mini USB 2.0 thumb drive used only for nvram-save (backups are stored on the stick too, but copied to a NAS when finished). Swap on /dev/sda1 (on a Kingston 64Gb USB 3.0 DataTraveler, with two partitions: linux-swap and data (=ext4)) has already been mounted and activated too, prior to the error mentioned above. So the USB stick is there, just the partition Skynet is looking for (datat) appears to not have been mounted yet. These are the loglines prior to the error above:

<snip>
Jun 14 07:45:37 dMP17 kernel: scsi 0:0:0:0: Direct-Access     Kingston DataTraveler 3.0 PMAP PQ: 0 ANSI: 6
Jun 14 07:45:37 dMP17 kernel: sd 0:0:0:0: Attached scsi generic sg0 type 0
Jun 14 07:45:37 dMP17 kernel: scsi 1:0:0:0: Direct-Access     Generic  STORAGE DEVICE   0250 PQ: 0 ANSI: 0
Jun 14 07:45:38 dMP17 kernel: sd 1:0:0:0: Attached scsi generic sg1 type 0
Jun 14 07:45:38 dMP17 kernel: sd 1:0:0:0: [sdb] 7698432 512-byte logical blocks: (3.94 GB/3.67 GiB)
Jun 14 07:45:38 dMP17 kernel: sd 1:0:0:0: [sdb] Write Protect is off
Jun 14 07:45:38 dMP17 kernel: sd 1:0:0:0: [sdb] Mode Sense: 0b 00 00 08
Jun 14 07:45:38 dMP17 kernel: sd 1:0:0:0: [sdb] Assuming drive cache: write through
Jun 14 07:45:38 dMP17 kernel: sd 1:0:0:0: [sdb] Assuming drive cache: write through
Jun 14 07:45:38 dMP17 kernel:  sdb: sdb1
Jun 14 07:45:38 dMP17 kernel: sd 1:0:0:0: [sdb] Assuming drive cache: write through
Jun 14 07:45:38 dMP17 kernel: sd 1:0:0:0: [sdb] Attached SCSI removable disk
Jun 14 07:45:38 dMP17 kernel: sd 0:0:0:0: [sda] 122945536 512-byte logical blocks: (62.9 GB/58.6 GiB)
Jun 14 07:45:38 dMP17 kernel: sd 0:0:0:0: [sda] Write Protect is off
Jun 14 07:45:38 dMP17 kernel: sd 0:0:0:0: [sda] Mode Sense: 23 00 00 00
Jun 14 07:45:38 dMP17 kernel: sd 0:0:0:0: [sda] Assuming drive cache: write through
Jun 14 07:45:38 dMP17 kernel: sd 0:0:0:0: [sda] Assuming drive cache: write through
Jun 14 07:45:38 dMP17 kernel:  sda: sda1 sda2
Jun 14 07:45:38 dMP17 kernel: sd 0:0:0:0: [sda] Assuming drive cache: write through
Jun 14 07:45:38 dMP17 kernel: sd 0:0:0:0: [sda] Attached SCSI removable disk
Jun 14 07:45:38 dMP17 start_nat_rules: apply the nat_rules(/tmp/nat_rules_eth0_eth0)!
Jun 14 07:45:39 dMP17 kernel: EXT2-fs (sdb1): warning: mounting unchecked fs, running e2fsck is recommended
Jun 14 07:45:39 dMP17 hotplug: USB ext2 fs at /dev/sdb1 mounted on /tmp/mnt/ASUS
Jun 14 07:45:39 dMP17 usb: USB ext2 fs at /dev/sdb1 mounted on /tmp/mnt/ASUS.
Jun 14 07:45:40 dMP17 kernel: nf_conntrack_rtsp v0.6.21 loading
Jun 14 07:45:40 dMP17 kernel: nf_nat_rtsp v0.6.21 loading
Jun 14 07:45:40 dMP17 script: Running /jffs/scripts/post-mount (args: /tmp/mnt/ASUS)
Jun 14 07:45:40 dMP17 kernel: Adding 524284k swap on /dev/sda1.  Priority:-1 extents:1 across:524284k
Jun 14 07:45:40 dMP17 script: Running /jffs/scripts/firewall-start (args: eth0)

I assume the ext2 warning shows up just because it's a non-journaling fs. For some reason, I don't know why, as both the USB-drives are permanently connected at the back of the router, on the highest point in the living room, where even my cats can't get to, /dev/sda2 appears to have an issue, or at least diskmon believes so, because shortly after the Skynet error, the log shows:

Jun 14 07:45:57 dMP17 kernel:  EXT4-fs (sda2): recovery complete
Jun 14 07:45:57 dMP17 hotplug:  USB ext4 fs at /dev/sda2 mounted on /tmp/mnt/data
Jun 14 07:45:57 dMP17 usb:  USB ext4 fs at /dev/sda2 mounted on /tmp/mnt/data.
Jun 14 07:45:57 dMP17 kernel:  EXT4-fs (sda2): mounted filesystem with ordered data mode. Opts: user_xattr
Jun 14 07:45:57 dMP17 rc_service:  ntp 1129:notify_rc restart_diskmon
Jun 14 07:45:57 dMP17 disk_monitor:  Finish
Jun 14 07:45:58 dMP17 client:  bound xxx.xxx.xxx.xxx via xxx.xxx.xxx.xxx during 3600 seconds.
Jun 14 07:45:58 dMP17 script:  Running /jffs/scripts/post-mount (args: /tmp/mnt/data)
Jun 14 07:45:59 dMP17 start_nat_rules:  apply the nat_rules(/tmp/nat_rules_eth0_eth0)!
Jun 14 07:46:00 dMP17 rc_service:  service 1303:notify_rc restart_dnsmasq
Jun 14 07:46:00 dMP17 rc_service:  waitting "start_firewall" via  ...
Jun 14 07:46:00 dMP17 monitor:  be idle
Jun 14 07:46:01 dMP17 script:  Running /jffs/scripts/firewall-start (args: eth0)
Jun 14 07:46:01 dMP17 dnsmasq:  exiting on receipt of SIGTERM
Jun 14 07:46:01 dMP17 ntpd:  ntpd 4.2.8p9-win@1.3728 Sat Mar 18 09:20:25 UTC 2017 (2): Starting
Jun 14 07:46:01 dMP17 ntpd:  Command line: ntpd -c /jffs/etc/ntp.conf
Jun 14 07:46:01 dMP17 ntpd:  proto: precision = 1.682 usec (-19)
Jun 14 07:46:01 dMP17 ntpd:  Listen normally on 0 lo 127.0.0.1:123
Jun 14 07:46:01 dMP17 ntpd:  Listen normally on 1 br0 192.168.1.1:123
Jun 14 07:46:01 dMP17 ntpd:  Listening on routing socket on fd #18 for interface updates
Jun 14 07:46:01 dMP17 config:  Appending content of /jffs/configs/dnsmasq.conf.add.
Jun 14 07:46:01 dMP17 script:  Running /jffs/scripts/dnsmasq.postconf (args: /etc/dnsmasq.conf)
Jun 14 07:46:02 dMP17 marco:  Started ntpd from /jffs/scripts/services-start.
Jun 14 07:46:03 dMP17 marco:  AB-Solution added entries via /jffs/scripts/post-mount
Jun 14 07:46:03 dMP17 marco:  Started pixelserv-tls (AB-Solution) from /jffs/scripts/services-start.
Jun 14 07:46:03 dMP17 pixelserv:  pixelserv-tls version: v35.HZ12.Kj compiled: May 30 2017 22:33:13 options: 192.168.1.2
Jun 14 07:46:03 dMP17 rc_service:  service 1361:notify_rc restart_httpd
Jun 14 07:46:03 dMP17 rc_service:  waitting "restart_dnsmasq" via  ...
Jun 14 07:46:03 dMP17 pixelserv:  Listening on :192.168.1.2:80
Jun 14 07:46:03 dMP17 pixelserv:  Listening on :192.168.1.2:443
Jun 14 07:46:04 dMP17 marco:  AB-Solution added entries via ab_dnsmasq_postconf.sh
Jun 14 07:46:04 dMP17 marco:  AB-Solution linked ab_dnsmasq_postconf.sh via /jffs/scripts/dnsmasq.postconf
Jun 14 07:46:04 dMP17 kernel:  DROP IN=eth0 OUT= MAC=2c:4d:54:49:71:30:00:01:5c:79:5c:46:08:00 SRC=93.159.194.117 DST=xxx.xxx.xxx.xxx LEN=40 TOS=0x00 PREC=0x00 TTL=245 ID=51139 PROTO=TCP SPT=11256 DPT=2323 SEQ=0 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0
Jun 14 07:46:04 dMP17 RT-AC68U_WHITE:  start httpd
Jun 14 07:46:06 dMP17 rc_service:  hotplug 996:notify_rc restart_nasapps
Jun 14 07:46:06 dMP17 iTunes:  daemon is stopped
Jun 14 07:46:06 dMP17 Server:  daemon is stopped
Jun 14 07:46:08 dMP17 Server:  smb daemon is stopped
Jun 14 07:46:08 dMP17 kernel:  gro disabled
Jun 14 07:46:08 dMP17 Timemachine:  daemon is stopped
Jun 14 07:46:08 dMP17 kernel:  gro enabled with interval 2
Jun 14 07:46:13 dMP17 Skynet:  [INFO] Startup Initiated ... ... ...
Jun 14 07:46:13 dMP17 kernel:  ip_set: protocol 6
Jun 14 07:46:15 dMP17 Skynet:  [Complete] 0 IPs / 0 Ranges Banned. 0 New IPs / 0 New Ranges Banned. 0 Inbound / 0 Outbound Connections Blocked! [3s]

I assume the first line in the part above shows how diskmon uses the journaling fs to check and/or fixes any errors it has detected. Please do correct me if I'm wrong. Skynet still appears to be running in the background, as it apparently found a way to restart or becomes active when /mnt/data gets mounted
@ 07:46:13. Not sure whether I need to restart in debug mode? It appears to run just fine, even as we speak:

Jun 14 09:00:01 dMP17 Skynet: [Complete] 0 IPs / 0 Ranges Banned. 0 New IPs / 0 New Ranges Banned. 0 Inbound / 0 Outbound Connections Blocked! [1s]

Maybe it's nothing, but as it differs from regular behaviour, I thought it might be worth mentioning.
And yes, I'm an autist and I'm proud of it

 

[Adamm]

Interesting that (/dev/sda2) took so long to mount, I assumed 60 seconds would be long enough even for the slowest devices (my usb here mounts 13 seconds before restart_firewall is even called). It seems part of the reason it was so slow is because it preformed a fsck on boot. But it looks that the second instance of Skynet eventually caught the USB when it was properly mounted so no harm done in this instance and everything eventually ran as per usual.

That being said, I have increased the attempts to 10 (100 seconds) which would have caught the USB in this case. Beyond that there's no much else to be done after updating, possibly try run "e2fsck" on all your devices to get rid of the errors in future (I think this can be done via the GUI).

 

[MarCoMLXXV]

I don't now what happened either, with /dev/sda2, besides the fact that it apparently needed checking. It's a fast USB 3.0 drive, so I'm just as surprised as you are.

Just to be clear: do I still have to run

sh /jffs/scripts/firewall debug restart

?

 

[elorimer]

A couple of things:

1. Is there a difference between whitelist and unban?
2. In the help section of the readme, whitelist is explained as banning things (I think this was a copy/paste error? But maybe a place to explain the difference between whitelist and unban, if any)
3. Wow, these lists ban a whole lot of things! clonezilla.org, for example, the 3M cloudbook site, several wordpress blogs I go to, and the list is growing by thousands an hour.

 

[MarCoMLXXV]

Oh, in addition to my previous message: I'll check my drives to make sure, but I remember having read somewhere it's best not to do that when having Entware-ng installed. I tried it in the beginning, router had lots of trouble to properly unmount the drives, and now that I have several scripts running from /dev/sda2, I'll just do it manually on my Ubuntu powered laptop to make sure I don't break things

 

[Adamm]

Is there a difference between whitelist and unban?

Yes, whitelist will prevent an IP from ever being blocked, where as unban will only remove it when run from the current blacklist. Which means if lets say your favourite site google.com appears in a list you refresh frequently (aka using banmalware), once the list is refreshed the IP will be banned again. But if you whitelisted it, the IP will stay unbanned as the whitelist takes priority.

In the help section of the readme, whitelist is explained as banning things

Thanks for pointing it out, it was a bad copy paste, consider it fixed.

3. Wow, these lists ban a whole lot of things! clonezilla.org, for example, the 3M cloudbook site, several wordpress blogs I go to, and the list is growing by thousands an hour.

That's unfortunately the nature of shared hosting, some webservers host hundreds if not thousands of websites on the same IP to maximise profit. That means if one bad site is hosted on the same IP, it can potentially get every other site blacklisted in the process. Using the example you provided of clonezilla.org;

This IP is featured on the abuse.ch ransomware tracker list, the server hosts 266 different websites and multiple are blacklisted as ransomware distributors (a website buyviagrasofttabs is even hosted on the same server ).

Because this is IP based blacklisting and not DNS (domain), the blacklist considers clonezilla.org the same as buyviagrasofttabs. So you can probably get a good picture now of why it was blocked in the first place. That being said this isn't the end of the world, it just requires a little user interaction upon setup whitelisting any websites that you visit that may have been blacklisted that you know are safe.

So in this situation, its best to whitelist clonezilla.org, then you will not have to worry about it being blocked again in future. Now this shouldn't be an issue for 99% of websites you visit, but only the ones on cheaper/unmanaged webhosts that were incorrectly listed.

If you are ever wondering which list a said IP is featured on, you can use the "sh /jffs/scripts/firewall stats search malware IPHERE" command and investigate from there.

Hope this clears things up

 

[Adamm]

Just to be clear: do I still have to run

No skynet ended up running on the second restart_firewall event, so its running as per usual.

Oh, in addition to my previous message: I'll check my drives to make sure

Probably a good idea, I'm sure its fine, linux tends to just be picky about these things sometimes.