Skynet Skynet - Router Firewall & Security Enhancements

[MarCoMLXXV]

I just tried to import the Microsoft Telemetry list from this post with the import function you added in this post and got the same error shown in this post.

The result:

marco@RT-AC68U:/tmp/mnt/ASUS# sh /jffs/scripts/firewall import https://raw.githubusercontent.com/shounak-de/misc-scripts/master/telemetry_and_scanners.txt

<snip>

This Function Extracts All IPs And Adds Them ALL To Blacklist
Custom List Detected: https://raw.githubusercontent.com/shounak-de/misc-scripts/master/telemetry_and_scanners.txt
Filtering IPv4 Addresses
Filtering IPv4 Ranges
Adding IPs To Blacklist
Saving Changes
/jffs/scripts/firewall: line 926: arithmetic syntax error

What to do? Update and run again? How does the script and/or ipset deal with duplicates?

 

[Adamm]

I just tried to import the Microsoft Telemetry list from this post with the import function you added in this post and got the same error shown in this post.

Run "debug restart" and this should be resolved. I recently changed startup behaviour to not include a flush of the raw IPTable table (to play nicer with other scripts) as AsusWRT doesn't flush this table upon restart_firewall. I assume you had old rules from a revision I made earlier, I now flush this table during the update function for the event the rules change again and the script "forgets" to remove them.

What to do? Update and run again? How does the script and/or ipset deal with duplicates?

Duplicates aren't an issue, an IP can only be added to the IPSet once so the script will automatically handle these situations. If you try process the list again for example it would simply just add nothing as all IP's already exist.

 

[MarCoMLXXV]

Should I update first? I'm on the last but one version, 4.8.2 I believe (I can't keep up with your pace, sorry...)

 

[Adamm]

(I can't keep up with your pace, sorry...)

I'm almost done for the night ...

Should I update first? I'm on the last but one version, 4.8.2 I believe

Sure is always best to update first.

 

[MarCoMLXXV]

I'm almost done for the night ...

Glad to hear that

I'll update first. The ''debug restart' are command line parameters I assume? You wrote 'it should be fixed', does that mean there's no need to import the list again to make sure all is imported?

 

[MarCoMLXXV]

Okay

 sh /jffs/scripts/firewall update -f

caused a flood of lines filling up the Putty buffer, so I guess that was an import into ipset. It ended with the same error.

sh /jffs/scripts/firewall debug restart

Seems to have fixed the error. Anything else to do? Regular restart?

 

[Adamm]

caused a flood of lines filling up the Putty buffer, so I guess that was an import into ipset. It ended with the same error.

I was printing a log conversion rather then appyling the edit, you can tell its late. This can safely be ignored though.

Seems to have fixed the error. Anything else to do? Regular restart?

You should be fine as is, the "debug restart" command is basically a fresh slate networking wise. And no, you will not need to import the list again.

Thanks for testing everything out and pointing out my stupid mistakes

 

[MarCoMLXXV]

You should be fine as is, the "debug restart" command is basically a fresh slate networking wise. And no, you will not need to import the list again."

Looking at syslog the count of banned IP has increased with around 60 IP-adresses. Does that sound about right? Or, even beter, how can I verify myself?

Thanks for testing everything out and pointing out my stupid mistakes

You're welcome, my pleasure entirely. Times two

 

[Adamm]

Looking at syslog the count of banned IP has increased with around 60 IP-adresses. Does that sound about right? Or, even beter, how can I verify myself?

That list is 69 entries long, I believe 7 or so are already included from the banmalware lists so that number is very accurate. Usually the log would have printed the exact amount of entries added, but due to the error you didn't see it. You could verify by using the "deport" command (aka remove them all) then "import" them again, but its safe to say they were added.

 

[steelskinz]

Maybe a stupid question but can we use local filter.list in jffs or usb device to prevent uploading file to internet ? Thanks.

 

[Adamm]

Maybe a stupid question but can we use local filter.list in jffs or usb device to prevent uploading file to internet ? Thanks.

At the moment it only supports url's as custom input, I recommend using something like pastebin (you will need to link to the raw file) then you can just private the paste or set an expiry.

 

[MarCoMLXXV]

Another day, another challenge...

When browsing through my logfiles, I noticed that Skynet behaves differently, it looks like it's called twice, detects a lockfile and exits one of the initiated processes. I've pasted the (filtered) logs from Papertrailapp.com, the behaviour starts after upgrading to 4.8.2, but I'm not sure whether it has to do with 4.8.2 (as 4.8.6 is the current version, if I'm not mistaken) or with the issues you helped me with yesterday (see this post and the following).

The filtered logfiles (going back to yesterday, before I imported the Microsoft Telemetry IP-list) up until now:

https://pastebin.com/EngCF4HK

The double call seems to start at line 111 in the pastebin logfile.

I've checked

crontab -l

which outputs

marco@RT-AC68U:/tmp/home/root# crontab -l
00 1 * * Mon /tmp/mnt/data/adblocking/addon/update-hosts.add cronjob #UpdateHosts#
20 5 * * * /tmp/mnt/data/adblocking/addon/rotate-logs.add #RotateLogs#
30 0 * * Mon /tmp/mnt/data/adblocking/scripts/weekly-log-stats.sh #WeeklyStats#
25 1 * * 1 sh /jffs/scripts/firewall banmalware #Firewall_banmalware#
25 1 * * * sh /jffs/scripts/firewall update #Firewall_autoupdate#
0 * * * * /jffs/scripts/firewall save #Firewall_save#
25 1 * * 1 sh /jffs/scripts/firewall banmalware #Skynet_banmalware#
25 1 * * * sh /jffs/scripts/firewall update #Skynet_autoupdate#
0 * * * * /jffs/scripts/firewall save #Skynet_save#

and the start of Skynet with

cat /jffs/scripts/firewall-start

which outputs

#!/bin/sh

[ -x /jffs/dnscrypt/manager ] && /jffs/dnscrypt/manager fw-rules
sleep 10; sh /jffs/scripts/firewall start debug banmalware autoupdate usb=/tmp/mnt/data # Skynet Firewall Addition

I noticed in crontab -l two nearly similar lines:

0 * * * * /jffs/scripts/firewall save #Firewall_save#
0 * * * * /jffs/scripts/firewall save #Skynet_save#

Is this wat causes it? How can I remove one of them?

 

[Adamm]

Is this wat causes it? How can I remove one of them?

Updating to v4.8.8 will fix this. It was an oversight on my end when reorganising the code, I changed the cron names yesterday but didn't have great cron management in the event they were renamed and the router hadn't rebooted (thus force clearing the old ones). I noticed my mistake a few hours ago and pushed appropriate fixes and better management so in future versions these can be renamed without consequences.

 

[MarCoMLXXV]

Thanks for the fix. Just installed v4.8.8 and the duplicate cronjob has been removed, so it should be solved.

 

[TheUntouchable]

Could it be that with the latest version stats for outbound blocked connections is not working anymore..? Got blocks @ the syslog but nothing about them @ the stats

 

[Adamm]

Could it be that with the latest version stats for outbound blocked connections is not working anymore..? Got blocks @ the syslog but nothing about them @ the stats

It's definitely all working on my end, make sure you are on the latest version (4.8.8). If you still don't see them being transferred to your stats after appearing in syslog, possibly try a "debug restart". But beyond that everything should be running smooth.

Debug Data Detected in /tmp/mnt/Main/skynet/skynet.log - 124.0K
Monitoring From Jun 09 22:32:21 To Jun 10 05:32:32
489 Total Events Detected
155 Unique IPs
3 Autobans Issued
4 Manual Bans Issued

Top 10 Targeted Ports (Inbound); (Torrent Clients May Cause Excess Hits In Debug Mode)
130x https://www.speedguide.net/port.php?port=18158
34x https://www.speedguide.net/port.php?port=23
27x https://www.speedguide.net/port.php?port=1433
18x https://www.speedguide.net/port.php?port=22
17x https://www.speedguide.net/port.php?port=56704
5x https://www.speedguide.net/port.php?port=5060
4x https://www.speedguide.net/port.php?port=1900
2x https://www.speedguide.net/port.php?port=8080
2x https://www.speedguide.net/port.php?port=80
2x https://www.speedguide.net/port.php?port=3392

Top 10 Source Ports (Inbound);
17x https://www.speedguide.net/port.php?port=5051
15x https://www.speedguide.net/port.php?port=53969
11x https://www.speedguide.net/port.php?port=51523
10x https://www.speedguide.net/port.php?port=64575
10x https://www.speedguide.net/port.php?port=11406
6x https://www.speedguide.net/port.php?port=11706
5x https://www.speedguide.net/port.php?port=60884
5x https://www.speedguide.net/port.php?port=50246
4x https://www.speedguide.net/port.php?port=9090
4x https://www.speedguide.net/port.php?port=6881

Last 10 Unique Connections Blocked (Inbound);
https://otx.alienvault.com/indicator/ip/121.168.226.200
https://otx.alienvault.com/indicator/ip/46.99.148.130
https://otx.alienvault.com/indicator/ip/46.173.73.53
https://otx.alienvault.com/indicator/ip/162.144.70.81
https://otx.alienvault.com/indicator/ip/89.29.205.130
https://otx.alienvault.com/indicator/ip/150.242.68.87
https://otx.alienvault.com/indicator/ip/123.207.171.48
https://otx.alienvault.com/indicator/ip/121.163.160.90
https://otx.alienvault.com/indicator/ip/81.171.2.145
https://otx.alienvault.com/indicator/ip/104.236.175.230

Last 10 Unique Connections Blocked (Outbound);
https://otx.alienvault.com/indicator/ip/175.197.147.157
https://otx.alienvault.com/indicator/ip/94.254.130.146
https://otx.alienvault.com/indicator/ip/59.18.148.226
https://otx.alienvault.com/indicator/ip/5.157.7.11
https://otx.alienvault.com/indicator/ip/175.199.244.11
https://otx.alienvault.com/indicator/ip/24.43.69.30
https://otx.alienvault.com/indicator/ip/175.205.38.155
https://otx.alienvault.com/indicator/ip/222.100.116.194
https://otx.alienvault.com/indicator/ip/46.116.125.176
https://otx.alienvault.com/indicator/ip/121.143.107.219

Last 10 Autobans;
https://otx.alienvault.com/indicator/ip/75.138.185.34
https://otx.alienvault.com/indicator/ip/46.116.125.176
https://otx.alienvault.com/indicator/ip/54.173.193.248

Last 10 Manual Bans;
https://otx.alienvault.com/indicator/ip/199.101.135.115
https://otx.alienvault.com/indicator/ip/212.224.124.112
https://otx.alienvault.com/indicator/ip/37.60.243.149
https://otx.alienvault.com/indicator/ip/34.225.36.129

Last 10 Unique HTTP(s) Blocks;

Top 10 HTTP(s) Blocks;

Top 10 Blocks (Inbound);
34x https://otx.alienvault.com/indicator/ip/46.116.125.176
19x https://otx.alienvault.com/indicator/ip/220.72.219.23
19x https://otx.alienvault.com/indicator/ip/175.199.244.11
15x https://otx.alienvault.com/indicator/ip/118.34.151.56
10x https://otx.alienvault.com/indicator/ip/193.138.219.233
7x https://otx.alienvault.com/indicator/ip/213.32.7.73
6x https://otx.alienvault.com/indicator/ip/49.204.132.224
5x https://otx.alienvault.com/indicator/ip/5.157.7.155
4x https://otx.alienvault.com/indicator/ip/222.186.56.40
3x https://otx.alienvault.com/indicator/ip/61.164.149.128

Top 10 Blocks (Outbound);
61x https://otx.alienvault.com/indicator/ip/168.167.35.78
31x https://otx.alienvault.com/indicator/ip/5.157.7.155
30x https://otx.alienvault.com/indicator/ip/46.116.125.176
19x https://otx.alienvault.com/indicator/ip/5.157.7.11
19x https://otx.alienvault.com/indicator/ip/220.123.214.148
15x https://otx.alienvault.com/indicator/ip/24.43.69.30
7x https://otx.alienvault.com/indicator/ip/94.254.130.146
5x https://otx.alienvault.com/indicator/ip/222.102.143.27
3x https://otx.alienvault.com/indicator/ip/72.52.4.120
2x https://otx.alienvault.com/indicator/ip/218.153.218.178

Skynet: [Complete] 138542 IPs / 5325 Ranges banned. 0 New IPs / 0 New Ranges Banned. 236 IP / 221 Range Connections Blocked! [2s]

 

[TheUntouchable]

Just restarted the firewall, will have a look the next hours on it

 

[faria]

Jun 9 20:45:17 Skynet: [INFO] Lock File Detected - Exiting

Hi guys ,the above message has been appearing for a few hours, in my logs, any idea what it means?

 

[Adamm]

Jun 9 20:45:17 Skynet: [INFO] Lock File Detected - Exiting

Hi guys ,the above message has been appearing for a few hours, in my logs, any idea what it means?

Same bug @MarCoMLXXV had here. Updating to v4.8.8 will fix it

 

[Adamm]

I suggest users update to v4.9.3 and re-generate their whitelists via;

sh /jffs/scripts/firewall whitelist remove

There was a bug where the script was detecting some outgoing connections as Private IP's and incorrectly whitelisting them. While there is no harm in this it will prevent them from being blocked in future, so its best to purge the whitelist.