Skynet Skynet - Router Firewall & Security Enhancements

[Adamm]

Thanks, updated my previous note. Discovered it was enumerating millions of files on the second USB drive. Removed that one and install no longer took forever.

Perhaps you can help with secondary issue? At startup, SkyNet says it's not blocking anything. I read that as the default domains not loading as expected, so tried to update the Malware Blacklist with visible logging.

sh -x /jffs/scripts/firewall banmalware​

That failed with "List Content Error Detected" and this last section of output.

+ dos2unix /tmp/mnt/UTILS/skynet/lists/*
+ printf \033[1;32m%s\033[0m\b\b\b --*
+ grep -qF * /tmp/skynet/skynet.manifest
+ usleep 250000
+ rm -rf *
+ grep -qE ^([0-9]{1,3}\.){3}[0-9]{1,3}(/[0-9]{1,2})?$ /tmp/mnt/UTILS/skynet/lists/*
+ date +%s
+ Red [27s]
+ printf -- \033[1;31m%s\033[0m\n [27s]
+ result=[27s]
+ printf %-8s\n [27s]
+ printf \b\b\b
[27s]
+ printf %-35s\n [*] List Content Error Detected - Stopping Banmalware
[*] List Content Error Detected - Stopping Banmalware
+ nocfg=1
+ result=1
+ [ 1 != 1 ]
+ [ -f /tmp/skynet/spinstart ]
+ Clean_Temp
+ rm -rf /tmp/skynet/lists /tmp/skynet/skynet.manifest /tmp/skynet/spinstart
+ mkdir -p /tmp/skynet/lists
+ Spinner_End​

Here's shell output without logging.

Downloading filter.list | [3s]
Refreshing Whitelists | [100s]
Consolidating Blacklist | [37s]
[*] List Content Error Detected - Stopping Banmalware​

[#] 0 IPs (+0) -- 0 Ranges Banned (+0) || 0 Inbound -- 0 Outbound Connections Blocked! [banmalware] [156s]​

Is that bolded grep line what triggers the content error? I checked /tmp/mnt/UTILS/skynet/lists/ and the folder's empty on my system, so if it's looking for contents there, I can see why it failed.

Your whitelist refresh time is excessively long, do you happen to have some crazy diversion whitelist in place? Skynet has to convert these all from domains to IP's which can cause issues if you have thousands of unnecessary domains listed.

 

[MWGlidden]

Your whitelist refresh time is excessively long, do you happen to have some crazy diversion whitelist in place? Skynet has to convert these all from domains to IP's which can cause issues if you have thousands of unnecessary domains listed.

This SkyNet install followed a default Diversion install and I didn't change anything in-between. I next tried disabling Diversion, followed by SkyNet list reset. Appear to have same issue.

[$] /jffs/scripts/firewall banmalware reset

Filter URL Reset
Downloading filter.list | [2s]
Refreshing Whitelists | [89s]
Consolidating Blacklist | [29s]
[*] List Content Error Detected - Stopping Banmalware
-*-​

Other info I can post that would suss out why the whitelists take so long to fetch? Worth uninstalling Diversion altogether beforehand?

 

[AntonK]

Does Skynet require the Entware packages installed?

More generally, what does Skynet require other than itself?

Thanks,
Anton

 

[Adamm]

Other info I can post that would suss out why the whitelists take so long to fetch?

sh -x /jffs/scripts/firewall whitelist refresh

and

sh -x /jffs/scripts/firewall banmalware

Does Skynet require the Entware packages installed?

More generally, what does Skynet require other than itself?

Skynet has no dependence besides running Merlins firmware.

 

[L&LD]

@Adamm Skynet does require a USB drive inserted into the router.

 

[dave14305]

@Adamm Skynet does require a USB drive inserted into the router.

And it will want to conditionally install curl, sqlite3-cli or coreutils-base64 from Entware depending on your firmware version and options chosen. base64 could be eliminated by using openssl enc -a -d to decode base64 text.

 

[Adamm]

And it will want to conditionally install curl, sqlite3-cli or coreutils-base64 from Entware depending on your firmware version and options chosen. base64 could be eliminated by using openssl enc -a -d to decode base64 text.

curl is only installed if the user is on an older firmware version that doesn't support the newer curl features. sqlite3-cli functionality can probably be modified to check for the native version first that was included sometime last year. As for using openssl to decode base64, it only accepts file input so its much simpler to use coreutils-base64 as this functionality is only used when dnscrypt is detected (thus entware would also be installed)

 

[dave14305]

As for using openssl to decode base64, it only accepts file input

I pipe to openssl enc -a -d in one of my scripts.

# echo Adamm | openssl enc -a | openssl enc -a -d
Adamm

 

[AntonK]

And it will want to conditionally install curl, sqlite3-cli or coreutils-base64 from Entware depending on your firmware version and options chosen. base64 could be eliminated by using openssl enc -a -d to decode base64 text.

And not Scribe or uiScribe to create the Skynet tab under Firewall?

 

[Adamm]

I pipe to openssl enc -a -d in one of my scripts.

# echo Adamm | openssl enc -a | openssl enc -a -d
Adamm

I stand corrected, my memory was a little off. The real issue was the inconsistent results we got when trying to decode dnscrypt stamps;

skynet@RT-AX88U-DC28:/tmp/home/root# echo "AgMAAAAAAAAADzE0OS4xNTQuMTUzLjE1MyA-GhoPbFPz6XpJLVcIS1uYBwWe4FerFQWHb9g_2j24OBhhZGZyZWUudXNhYmxlcHJpdmFjeS5uZXQKL2Rucy1xdWVyeQ" | base64 -d 2>/dev/null
149.154.153.153

skynet@RT-AX88U-DC28:/tmp/home/root# echo "AgMAAAAAAAAADzE0OS4xNTQuMTUzLjE1MyA-GhoPbFPz6XpJLVcIS1uYBwWe4FerFQWHb9g_2j24OBhhZGZyZWUudXNhYmxlcHJpdmFjeS5uZXQKL2Rucy1xdWVyeQ" | openssl enc -a -d
skynet@RT-AX88U-DC28:/tmp/home/root#

iirc this is due to the use of underscores and dashes that are not "valid" base64 characters.

 

[Adamm]

And not Scribe or uiScribe to create the Skynet tab under Firewall?

The Skynet tab is completely independent of other scripts and generated by Skynet its-self.

 

[nev39]

Tried to install Skynet on John's latest fork (E5) via AMTM. It displays following:

Skynet install failed,
 IPSet version on router not supported:

 ipset v4.5, protocol version 4.
 Kernel module protocol version 4.

How can i install it?

 

[ColinTaylor]

Tried to install Skynet on John's latest fork (E5) via AMTM. It displays following:

Skynet install failed,
 IPSet version on router not supported:

 ipset v4.5, protocol version 4.
 Kernel module protocol version 4.

How can i install it?

I've just answered that question in your other post. Please don't double post.

Unfortunately it does not support MIPS based routers due to their ancient kernel.

 

[TonyK132]

I just tried to do Skynet from amtm, and it was giving me a script error message. When I looked at the firewall script, it was only 1500 lines or so, much less than the 5800 or so lines from the saved firewall script from yesterday. So I loaded the previous day's version, restarted amtm, uninstalled Skynet, then reinstalled, and now all is OK. But the question is, how can the script get screwed up like that? It may have happened when Skynet auto-updated itself at 1:13am this morning (Mon)? I looked in the GUI log, and there is no record of Skynet doing a check for auto-update, or if it attempted to do an auto-update. Is that the right place to look for those messages?

 

[iJorgen]

I have noticed an issue each morning the past week that one (or more) lists seems to fail during the nightly updates. When I ran the manual update procedure today it added 191.104 IP's.

Any way to detect this deviation in SkyNet's update-procedure and maybe retry a few times later or keep the current list?!

[#] 301346 IPs (+191104) -- 1728 Ranges Banned (+98) || 4002 Inbound -- 0 Outbound Connections Blocked!

 

[Adamm]

Any way to detect this deviation in SkyNet's update-procedure and maybe retry a few times later or keep the current list?!

Skynet makes 3 attempts that will time out after 3 seconds trying to download each list. You could try restarting skynet as that will change the cronjob update time as the issue may be related to connectivity issue on yours or the hosts end.

 

[Ubimo]

How can I stop skynet from ssh terminal command line?
I cannot load the menu, due to no internet connection and no ntp.
The menu says " waiting for ntp sync" and then exits.

 

[Adamm]

How can I stop skynet from ssh terminal command line?

firewall disable

 

[Ubimo]

This does not work.
The menu starts to load, "waiting for ntp sync" and then exits.

 

[wbartels]

skynet@RT-AX88U-DC28:/tmp/home/root# echo "AgMAAAAAAAAADzE0OS4xNTQuMTUzLjE1MyA-GhoPbFPz6XpJLVcIS1uYBwWe4FerFQWHb9g_2j24OBhhZGZyZWUudXNhYmxlcHJpdmFjeS5uZXQKL2Rucy1xdWVyeQ" | openssl enc -a -d
skynet@RT-AX88U-DC28:/tmp/home/root#

This is not standard Base64, but URL safe Base64, encoding is simple:
replace + with -
replace / with _
and delete trailing =
Decoding is more difficult, because the = need to be reconstructed.
Below I will post my PHP solution, witch could be rewritten for Shell script:

//  +------------------------------------------------------------------------+
//  | base64url encode                                                       |
//  +------------------------------------------------------------------------+
function base64url_encode($string) {
    // http://www.ietf.org/rfc/rfc4648.txt
    return rtrim(strtr(base64_encode($string), '+/', '-_'), '=');
}


//  +------------------------------------------------------------------------+
//  | base64url decode                                                       |
//  +------------------------------------------------------------------------+
function base64url_decode($string) {
    // Thanks gutzmer at usa dot net
    // http://php.net/manual/en/function.base64-encode.php#103849
    return base64_decode(str_pad(strtr($string, '-_', '+/'), strlen($string) % 4, '=', STR_PAD_RIGHT));
}