Skynet Skynet - Router Firewall & Security Enhancements

[DonnyJohnny]

It was just a normal scheduled corn job ... I set it at every 6 hr 5 min...
This is not first time I saw this.. it happened in the last 24 hr. It seems to have this error at 0005 hr only throughout the day. I will change the update to every 6 hr at 10min and monitor.

Could it be memory issue? Due to recent update of pixelserv-tls and I am using dnscrypt-proxy v2 which used lots of memory resources.
Prior to the pixelserv-tls, the update is fine thou..

Could be data drive issue too.. I will find time to do a scan for error when I am free during weekend..

Very strange
This time no error message but the update is still not correct... seems half done.

Mar 16 06:00:16 Skynet: [Complete] 234155 IPs / 39700 Ranges Banned. 0 New IPs / 0 New Ranges Banned. 1936 Inbound / 1834 Outbound Connections Blocked! [16s]
Mar 16 06:13:11 Skynet: [Complete] 58418 IPs / 33110 Ranges Banned. -175737 New IPs / -6590 New Ranges Banned. 1945 Inbound / 1834 Outbound Connections Blocked! [191s]

Wonder why.. when I manually update, it is fine..
sh /jffs/scripts/firewall banmalware

Mar 16 10:47:33 Skynet: [Complete] 234899 IPs / 39811 Ranges Banned. 176481 New IPs / 6701 New Ranges Banned. 2045 Inbound / 1834 Outbound Connections Blocked! [83s]

Realised that it took 191s for the failed update and when during manual update it took only 83s.

Update at 1210hrs, no errors, update 124s. Strange.. seems random..

Mar 16 12:12:04 Skynet: [Complete] 235001 IPs / 39807 Ranges Banned. 102 New IPs / -4 New Ranges Banned. 2148 Inbound / 1834 Outbound Connections Blocked! [124s]

 

[Adamm]

Very strange
This time no error message but the update is still not correct... seems half done.

Mar 16 06:00:16 Skynet: [Complete] 234155 IPs / 39700 Ranges Banned. 0 New IPs / 0 New Ranges Banned. 1936 Inbound / 1834 Outbound Connections Blocked! [16s]
Mar 16 06:13:11 Skynet: [Complete] 58418 IPs / 33110 Ranges Banned. -175737 New IPs / -6590 New Ranges Banned. 1945 Inbound / 1834 Outbound Connections Blocked! [191s]

Wonder why.. when I manually update, it is fine..
sh /jffs/scripts/firewall banmalware

Mar 16 10:47:33 Skynet: [Complete] 234899 IPs / 39811 Ranges Banned. 176481 New IPs / 6701 New Ranges Banned. 2045 Inbound / 1834 Outbound Connections Blocked! [83s]

Realised that it took 191s for the failed update and when during manual update it took only 83s.

Update at 1210hrs, no errors, update 124s. Strange.. seems random..

Mar 16 12:12:04 Skynet: [Complete] 235001 IPs / 39807 Ranges Banned. 102 New IPs / -4 New Ranges Banned. 2148 Inbound / 1834 Outbound Connections Blocked! [124s]

The fact the process is taking from 80s - 190s is a cause of concern, this device should complete the process within 30s-45s from memory, and devices like the AC86U complete it in >15s. Sounds like to me something is hogging resources on your device which could cause behavior like you are seeing.

I suggest try installing a swap file and investigating what is using up all your CPU.

 

[Mutzli]

It was just a normal scheduled corn job ... I set it at every 6 hr 5 min...
This is not first time I saw this.. it happened in the last 24 hr. It seems to have this error at 0005 hr only throughout the day. I will change the update to every 6 hr at 10min and monitor.

Could it be memory issue? Due to recent update of pixelserv-tls and I am using dnscrypt-proxy v2 which used lots of memory resources.
Prior to the pixelserv-tls, the update is fine thou..

Could be data drive issue too.. I will find time to do a scan for error when I am free during weekend..

I had a similar issue and it was my USB drive that was failing. Replaced it with a new one and everything was back to normal again.

 

[2992]

I had a similar issue and it was my USB drive that was failing. Replaced it with a new one and everything was back to normal again.

Had the same issue, and it was not the USBDrive. (I have learnt that after trying about 10 different USBDrives, formated in different formats and by different OSs.)
So, I had to re-format the USBDrive properly, and then I've reinstalled the whole router and the ABS/Skynet/DNSCrypt scripts again from scratch (thanks AMTM for that), then from there onwards everything was and still is fine (so far).

 

[DonnyJohnny]

The fact the process is taking from 80s - 190s is a cause of concern, this device should complete the process within 30s-45s from memory, and devices like the AC86U complete it in >15s. Sounds like to me something is hogging resources on your device which could cause behavior like you are seeing.

I suggest try installing a swap file and investigating what is using up all your CPU.

I already have a 500mb swap. And previously it is working good.
Regards the 80+ Sec, it seems normal as I have country block and custom filter list. If I remember correctly, the default list will still takes me around 60+ Sec.

I did a restart of router which is up for 22days... could be some swap corruption. The first round of schedule update is working good at 0000hrs. Will see the next update at 0600hr.

Memory available seems to be more at 58mb compared to 28mb before restart. I doubt is the cpu as the router is not heavily in use. Avg at 0.3-0.7.

 

[Adamm]

I already have a 500mb swap. And previously it is working good.
Regards the 80+ Sec, it seems normal as I have country block and custom filter list. If I remember correctly, the default list will still takes me around 60+ Sec.

I did a restart of router which is up for 22days... could be some swap corruption. The first round of schedule update is working good at 0000hrs. Will see the next update at 0600hr.

Memory available seems to be more at 58mb compared to 28mb before restart. I doubt is the cpu as the router is not heavily in use. Avg at 0.3-0.7.

80s+ does still seem excessively high, maybe other users running a AC68U can post their times for comparison so I don't have to fire up my dusty unit

Like @Mutzli suggested, it could also very well be a USB related issue. The behavior you are experience is definitely not normal and in my opinion not caused directly by Skynet.

 

[Adamm]

If anyone would like to do some additional testing, I've updated the telemetry list locally to include some more Microsoft IP's whose only intention are to collect data.

Sometimes these updates accidentally break Microsoft services (Office 360, Xbox, onedrive, azure, mail, windows updates). So for anyone willing to test, please use the following custom banmalware filter and focus on testing those services if you use any of them;

sh /jffs/scripts/firewall banmalware https://pastebin.com/raw/5CjbP2Bf

If you do run into any issues, use the usual IP debugging procedure and report back here with any conflicting IP's causing issues. Thanks!

 

[ThanksForThis]

Thank you for your commitment to this project @Adamm.

 

[DonnyJohnny]

@Adamm
I have since reboot the router and the update is now normal for the past 24hr.

I think the 83s-130+ is normal due to the extensive list I have...
I have a huge list of country ban and a bigger custom filter list.
I did 3 test of update. 1st with country ban and custom list (135s), 2nd with country ban and default list (69s), 3rd with default list without any country ban (50s).

With country ban and custom list

Downloading filter.list         [1s]
Refreshing Whitelists           [8s]
Consolidating Blacklist         [23s]
Saving Changes                  [14s]
Removing Previous Malware Bans  [5s]
Filtering IPv4 Addresses        [11s]
Filtering IPv4 Ranges           [1s]
Applying Blacklists             [22s]

Skynet: [Complete] 231917 IPs / 39753 Ranges Banned. -744 New IPs / -40 New Ranges Banned. 1172 Inbound / 265 Outbound Connections Blocked! [135s]

With country ban and default list

Filter URL Reset
Downloading filter.list         [1s]
Refreshing Whitelists           [7s]
Consolidating Blacklist         [12s]
Saving Changes                  [13s]
Removing Previous Malware Bans  [5s]
Filtering IPv4 Addresses        [5s]
Filtering IPv4 Ranges           [0s]
Applying Blacklists             [10s]

For False Positive Website Bans Use; ( sh /jffs/scripts/firewall whitelist domain URL )

Skynet: [Complete] 102073 IPs / 33845 Ranges Banned. -129844 New IPs / -5908 New Ranges Banned. 1172 Inbound / 265 Outbound Connections Blocked! [69s]

With just the default list without country ban

Filter URL Reset
Downloading filter.list         [1s]
Refreshing Whitelists           [4s]
Consolidating Blacklist         [12s]
Saving Changes                  [5s]
Removing Previous Malware Bans  [2s]
Filtering IPv4 Addresses        [5s]
Filtering IPv4 Ranges           [0s]
Applying Blacklists             [8s]

For False Positive Website Bans Use; ( sh /jffs/scripts/firewall whitelist domain URL )

Skynet: [Complete] 102073 IPs / 1543 Ranges Banned. 0 New IPs / 0 New Ranges Banned. 1173 Inbound / 265 Outbound Connections Blocked! [50s]

 

[Adamm]

@Adamm
I have since reboot the router and the update is now normal for the past 24hr.

I think the 83s-130+ is normal due to the extensive list I have...
I have a huge list of country ban and a bigger custom filter list.
I did 3 test of update. 1st with country ban and custom list (135s), 2nd with country ban and default list (69s), 3rd with default list without any country ban (50s).

The timers make more sense now, I assumed when you got these huge timers it was from cmdline initiation, not using the menu. There was actually a logic bug that was starting the "overall" timer from the moment you open the menu, so it was increasing the time significantly.

The examples you posted would be as follows once corrected;

1) 135s --> 85s
2) 69s --> 53s
3) 50s --> 37s

This has been corrected in the v6.0.0 update when it is eventually released. You may want to check you are not running unnecessary duplicates in your custom filter though. Some of the firehol lists for example are a combination of 10+ lists in an optimized format to reduce overall IPSet entries. The firehol website gives good analysis on this type of thing so you aren't adding unnessesary lists.


As for the errors you were experiencing, I'm glad a reboot fixed it, it did seem like an issue outside of Skynets hands.

 

[DonnyJohnny]

The timers make more sense now, I assumed when you got these huge timers it was from cmdline initiation, not using the menu. There was actually a logic bug that was starting the "overall" timer from the moment you open the menu, so it was increasing the time significantly.

The examples you posted would be as follows once corrected;

1) 135s --> 85s
2) 69s --> 53s
3) 50s --> 37s

This has been corrected in the v6.0.0 update when it is eventually released. You may want to check you are not running unnecessary duplicates in your custom filter though. Some of the firehol lists for example are a combination of 10+ lists in an optimized format to reduce overall IPSet entries. The firehol website gives good analysis on this type of thing so you aren't adding unnessesary lists.


As for the errors you were experiencing, I'm glad a reboot fixed it, it did seem like an issue outside of Skynets hands.

Nice,
I using level 1-3 firehol and some malware/reputation list like proxy, bitcoin, randomware, etc. I did look thru those list to minimise duplicate.
The bulk mainly from level 1 list I think.
List I used:
https://pastebin.com/raw/QCAybceU

 

[JaimeZX]

Howdy!

So I've just installed Skynet via the amtm 1.1 script. Super easy.
Per recommendation, I installed it on my USB stick where I have ABS. Unfortunately it's only a 256MB stick (seemed like plenty when ABS advertised needing ~7MB). Anyway, that means I opted for the "no swap file" install since I had no option less than 256MB.

The problem is that it returns 0 IPs / 0 Ranges Banned. 0 New IPs / 0 New Ranges Banned. 0 Inbound / 0 Outbound Connections Blocked! upon loading.

If I try (10) Update Skynet -> option 3/force -> it just crashes and reloads.

Do I need to rustle up a bigger thumbdrive and reinstall all of these scripts on that to make Skynet work? Else I can install in /jffs, but that wasn't the "recommended" option.

Side note: is it normal for both CPU cores to go crazy for about 90 seconds on Skynet load? (Ref: attachment)

 

[skeal]

How
So I've just installed Skynet via the amtm 1.1 script. Super easy.
Per recommendation, I installed it on my USB stick where I have ABS. Unfortunately it's only a 256MB stick (seemed like plenty when ABS advertised needing ~7MB). Anyway, that means I opted for the "no swap file" install since I had no option less than 256MB.

The problem is that it returns 0 IPs / 0 Ranges Banned. 0 New IPs / 0 New Ranges Banned. 0 Inbound / 0 Outbound Connections Blocked! upon loading.

If I try (10) Update Skynet -> option 3/force -> it just crashes and reloads.

Do I need to rustle up a bigger thumbdrive and reinstall all of these scripts on that to make Skynet work? Else I can install in /jffs, but that wasn't the "recommended" option.

Side note: is it normal for both CPU cores to go crazy for about 90 seconds on Skynet load? (Ref: attachment)

To make skynet block from list you need to run banmalware command. Yes a bigger thumb drive is very much advised. Both of the scripts you have installed are use a bit of memory. Yes the load at startup makes the graph go crazy.

 

[Adamm]

The problem is that it returns 0 IPs / 0 Ranges Banned. 0 New IPs / 0 New Ranges Banned. 0 Inbound / 0 Outbound Connections Blocked! upon loading.

This is normal until you ban something or use the banmalware command. (In v6 if banmalware was selected during the install process this will be done automatically/immediately rather then waiting for the first cronjob)

If I try (10) Update Skynet -> option 3/force -> it just crashes and reloads.

Can you post any output from when it crashes (do note this feature is for updating the script itself)

Do I need to rustle up a bigger thumbdrive and reinstall all of these scripts on that to make Skynet work? Else I can install in /jffs, but that wasn't the "recommended" option.

At the moment that size will be sufficient, but as of v6.0.0 which will be released in the near future a 512MB+ USB will be required due to swap files becoming mandatory (and jffs installs will be removed also). So to save yourself the trouble in the future I’d look at getting a bigger USB. Luckily you can find them for around the $1 mark so hopefully won’t be an issue for most users.

Side note: is it normal for both CPU cores to go crazy for about 90 seconds on Skynet load? (Ref: attachment)

Completely normal, much like a computer Skynet has to configure and setup various services on boot (plus any custom scripts). After doing so the usage will drop back to idle levels.

 

[JaimeZX]

To make skynet block from list you need to run banmalware command.

:facepalm: Done! Haha.

Yes a bigger thumb drive is very much advised. Both of the scripts you have installed are use a bit of memory. Yes the load at startup makes the graph go crazy.

Noted. I figured this 256MB one isn't good for much else, would be perfect here. Mebbe I grab a 512 for $2 off Amazon, eh?

 

[ThanksForThis]

Just a heads up:
Merlin 384.3 > .4 incurred an issue.

With aiProtection Two-Way IPS enabled, it states an infected device at 127.0.0.1 was blocked due to attempted Blacknurse ICMP attack when trying to upgrade via WebUI.
I easily circumvented this by disabling Two-Way IPS and rebooting to proceed with updating the firmware, and then reverted back to enabling Two-Way IPS.

I didn't look into it, but I assumed since it was an attack reported on ics.sans.edu, I assumed that firehol level 2 was reporting it to Skynet and the block occurred.
Additionally, I noticed DNS resolution was completely inoperable after updating and re-enabling aiProtection Two-Way IPS. Attempting to execute Skynet while incurring DNS failure caused a hang on the ASCII presentation and the script does not proceed.
Turning off Two-Way IPS resolved this entirely, again.

Just wanted to provide that info before the FW update gets a lot of users migrating.

I did not attempt to whitelist localhost address and then upgrade as I was simply trying to upgrade and move on, and in hindsight it would give some useful debug info on what is specifically to blame. Potentially if others going through the firmware upgrade process running Skynet become aware, they could provide information regarding the ability to replicate.

Your effort is appreciated greatly.

Update --- an hour later
Re-enabling Two-Way IPS after updating Skynet via script update, and Banmalware iplists update seems to work fine. Will report back if anything changes.

 

[JaimeZX]

Can you post any output from when it crashes (do note this feature is for updating the script itself)

There isn't one; it just disappears and I find myself back at the amtm menu. :dunno:
Edit: if it's logging something somewhere I'm happy to post the output of the log; just lemme know.

At the moment that size will be sufficient, but as of v6.0.0 which will be released in the near future a 512MB+ USB will be required due to swap files becoming mandatory (and jffs installs will be removed also). So to save yourself the trouble in the future I’d look at getting a bigger USB. Luckily you can find them for around the $1 mark so hopefully won’t be an issue for most users.

Hm. Yes. Yet strangely the cheapest one I could find on Amazon that didn't look like it was made in some Chinese guy's back shed in his spare time was $9. I'll keep my eye out.

 

[Adamm]

Just a heads up:
Merlin 384.3 > .4 incurred an issue.

With aiProtection Two-Way IPS enabled, it states an infected device at 127.0.0.1 was blocked due to attempted Blacknurse ICMP attack when trying to upgrade via WebUI.
I easily circumvented this by disabling Two-Way IPS and rebooting to proceed with updating the firmware, and then reverted back to enabling Two-Way IPS.

I didn't look into it, but I assumed since it was an attack reported on ics.sans.edu, I assumed that firehol level 2 was reporting it to Skynet and the block occurred.
Additionally, I noticed DNS resolution was completely inoperable after updating and re-enabling aiProtection Two-Way IPS. Attempting to execute Skynet while incurring DNS failure caused a hang on the ASCII presentation and the script does not proceed.
Turning off Two-Way IPS resolved this entirely, again.

Just wanted to provide that info before the FW update gets a lot of users migrating.

I did not attempt to whitelist localhost address and then upgrade as I was simply trying to upgrade and move on, and in hindsight it would give some useful debug info on what is specifically to blame. Potentially if others going through the firmware upgrade process running Skynet become aware, they could provide information regarding the ability to replicate.

Your effort is appreciated greatly.

Update --- an hour later
Re-enabling Two-Way IPS after updating Skynet via script update, and Banmalware iplists update seems to work fine. Will report back if anything changes.

Strange behavior indeed but it seems more related to the Trend Micro engine, possibly due to the "signature" version you were using at the time. I'd say just keep an eye on it and try force updating the signature version if you haven't already.

There isn't one; it just disappears and I find myself back at the amtm menu. :dunno:
Edit: if it's logging something somewhere I'm happy to post the output of the log; just lemme know.

Scroll up and it should be there, amtm uses the "clear" command upon exiting of another script which essentially clears old text off the screen.

 

[JaimeZX]

Scroll up and it should be there, amtm uses the "clear" command upon exiting of another script which essentially clears old text off the screen.

OK - my mistake... I scrolled up and it does say: Skynet: [INFO] Skynet Sucessfully Updated - Restarting Firewall.

It's just that since you then clear the screen and go back to the AMTM menu immediately thereafter, the "success" message isn't actually displayed for a useful period of time. Maybe put a 0.25s delay before going back to the menu? That'd be long enough to register but not long enough to slow down any particular work someone was doing... On the other hand now that I know what's going on it's irrelevant to me. For us n00bs though, it'd be useful.

 

[Wisiwyg]

Re using the geoip ban function, is it possible to ban all except an allowed list?

Something like:
ban country all
allow country "us uk"

Or is it necessary to name each as in:
ban country "pk cn sa ru ..."