Skynet Is default firewall good enough?

[JTnola]

That means there is a list some where in there breaking things. A list that has a nonreal ipv4 address most likely. As I mentioned before, skynet has no actual protection against this. So, it will most likely fail the moment a list has a none real ip address. What I mean by a non-real ipv4 address is an address the is out side the range of real addresses. Skynet could fix this issue by strengthening its ipv4 grep regexp.

Right. I just thought you’d want to know. But if it’s not helpful to post this, I’m happy to follow your lead. Just say the word.

(My only real interest here has to do with country bans. You had mentioned that they won’t load in scenarios such as this …)

 

[Viktor Jaep]

Looks like it's happened to the @Viktor Jaep list as well, showing zero ranges also.

Back to normal after eliminating that list...

 

[SomeWhereOverTheRainBow]

Right. I just thought you’d want to know. But if it’s not helpful to post this, I’m happy to follow your lead. Just say the word.

(My only real interest here has to do with country bans. You had mentioned that they won’t load in scenarios such as this …)

Yea so I am glad you mentioned it. I don't really have control over when a list decides to use a fake ip. I can remove the list from my filter list, but the real change needs to come from the developer of skynet because this can happen to anybody using any list of filters. Skynet should use a better regexp such that non real ip addresses is filtered out. This is completely doable.

 

[Viktor Jaep]

OK... so I put together a little script that validates your filter entries, and determines if there's a bad IP on one of these lists...

Filter Validator tests the IPv4 addresses (and IPv6 if present) on a given filter list that are to be used with the Skynet Firewall on Asus-Merlin Firmware in order to block incoming/outgoing IPs. This script arose out of the need to determine exactly which blacklist URL contained an invalid IP that was causing our Skynet firewalls to fail importing the correct IP sets due to an invalid IP somewhere on these lists.

If there's an issue with an IP on any of these lists, you'll see something like this:

---

Usage Guide​

Execute the script as such: sh filtervalidator.sh

Upon execution, it will ask for a valid URL to the specified filter list to be tested. For example, here is a valid filter list URL that will be used if you press enter: https://raw.githubusercontent.com/ViktorJp/Skynet/main/filter.list

NOTE: Should any list come back with any invalid IP entries (marked in Red), it would be advisable to #COMMENT out the offending entry in your filter list in order to get Skynet back in working condition, or get in touch with the entity that takes care of the list in order to correct their mistake.

---

Download​

curl --retry 3 "https://raw.githubusercontent.com/ViktorJp/FilterValidator/master/filtervalidator.sh" -o "/jffs/scripts/filtervalidator.sh" && chmod a+rx "/jffs/scripts/filtervalidator.sh"

 

[Viktor Jaep]

OK... so I put together a little script that validates your filter entries, and determines if there's a bad IP on one of these lists...

View attachment 48318

Filter Validator tests the IPv4 addresses (and IPv6 if present) on a given filter list that are to be used with the Skynet Firewall on Asus-Merlin Firmware in order to block incoming/outgoing IPs. This script arose out of the need to determine exactly which blacklist URL contained an invalid IP that was causing our Skynet firewalls fail importing the correct IP sets due to an invalid IP somewhere on these lists.

If there's an issue with an IP on any of these lists, you'll see something like this:

View attachment 48319

---

Usage Guide​

Execute the script as such: sh filtervalidator.sh

Upon execution, it will ask for a valid URL to the specified filter list to be tested. For example, here is a valid filter list URL that will be used if you press enter: https://raw.githubusercontent.com/ViktorJp/Skynet/main/filter.list

NOTE: Should any list come back with any invalid IP entries (marked in Red), it would be advisable to #COMMENT out the offending entry in your filter list in order to get Skynet back in working condition, or get in touch with the entity that takes care of the list in order to correct their mistake.

---

Download​

curl --retry 3 "https://raw.githubusercontent.com/ViktorJp/FilterValidator/master/filtervalidator.sh" -o "/jffs/scripts/filtervalidator.sh" && chmod a+rx "/jffs/scripts/filtervalidator.sh"

Oh, and just as a caveat... regex is my kryptonite, and I can't be quite sure if this will work to really nip everything in the bud... so @SomeWhereOverTheRainBow, knowing you're a wizard at this stuff -- if you want to collaborate on this to make it more effective, I would be happy to pitch in to help.

 

[JTnola]

Y’all are the best !!!

Yea so I am glad you mentioned it. I don't really have control over when a list decides to use a fake ip. I can remove the list from my filter list, but the real change needs to come from the developer of skynet because this can happen to anybody using any list of filters. Skynet should use a better regexp such that non real ip addresses is filtered out. This is completely doable.

OK... so I put together a little script that validates your filter entries, and determines if there's a bad IP on one of these lists...

View attachment 48318

Filter Validator tests the IPv4 addresses (and IPv6 if present) on a given filter list that are to be used with the Skynet Firewall on Asus-Merlin Firmware in order to block incoming/outgoing IPs. This script arose out of the need to determine exactly which blacklist URL contained an invalid IP that was causing our Skynet firewalls to fail importing the correct IP sets due to an invalid IP somewhere on these lists.

If there's an issue with an IP on any of these lists, you'll see something like this:

View attachment 48319

---

Usage Guide​

Execute the script as such: sh filtervalidator.sh

Upon execution, it will ask for a valid URL to the specified filter list to be tested. For example, here is a valid filter list URL that will be used if you press enter: https://raw.githubusercontent.com/ViktorJp/Skynet/main/filter.list

NOTE: Should any list come back with any invalid IP entries (marked in Red), it would be advisable to #COMMENT out the offending entry in your filter list in order to get Skynet back in working condition, or get in touch with the entity that takes care of the list in order to correct their mistake.

---

Download​

curl --retry 3 "https://raw.githubusercontent.com/ViktorJp/FilterValidator/master/filtervalidator.sh" -o "/jffs/scripts/filtervalidator.sh" && chmod a+rx "/jffs/scripts/filtervalidator.sh"

 

[SomeWhereOverTheRainBow]

Oh, and just as a caveat... regex is my kryptonite, and I can't be quite sure if this will work to really nip everything in the bud... so @SomeWhereOverTheRainBow, knowing you're a wizard at this stuff -- if you want to collaborate on this to make it more effective, I would be happy to pitch in to help.

It looks good, but ipv6 can be abit trickier because you have valid expanded addresses and valid unexpanded addresses as well. So what I do is I have a function that expands all ipv6 before attempting to validate. But your validation script should particularly only be used for ipv4 since skynet doesn't work with ipv6. I will not really be able to look at this till I am back from my work trip, but this will only pull all real ipv4 addresses from a list @Viktor Jaep

grep -E '^(((25[0-5]|(2[0-4]|1[0-9]|[1-9]|)[0-9])\.){3}(25[0-5]|(2[0-4]|1[0-9]|[1-9]|)[0-9])(\/(1?[0-9]|2?[0-9]|3?[0-2]))?)$'

This will give all invalid ipv4 if you modify the pipe for ipv4 in your script @Viktor Jaep

grep -vE '^(((25[0-5]|(2[0-4]|1[0-9]|[1-9]|)[0-9])\.){3}(25[0-5]|(2[0-4]|1[0-9]|[1-9]|)[0-9])(\/(1?[0-9]|2?[0-9]|3?[0-2]))?)$'

E.g.

ipresults=$(curl --silent --retry 3 --request GET --url $blacklisturl | grep "^[^#;]" | grep "\s*$" | grep -vE '^(((((25[0-5]|(2[0-4]|1[0-9]|[1-9]|)[0-9])\.){3}(25[0-5]|(2[0-4]|1[0-9]|[1-9]|)[0-9])(\/(1?[0-9]|2?[0-9]|3?[0-2]))?)|(([0-9A-f]{0,4}:){1,7}[0-9A-f]{0,4}:?(\/(1?[0-2][0-8]|[0-9][0-9]))?))$')

 

[Viktor Jaep]

It looks good, but ipv6 can be abit trickier because you have valid expanded addresses and valid unexpanded addresses as well. So what I do is I have a function that expands all ipv6 before attempting to validate. But your validation script should particularly only be used for ipv4 since skynet doesn't work with ipv6. I will not really be able to look at this till I am back from my work trip, but this will only pull all real ipv4 addresses from a list @Viktor Jaep

grep -E '^(((25[0-5]|(2[0-4]|1[0-9]|[1-9]|)[0-9])\.){3}(25[0-5]|(2[0-4]|1[0-9]|[1-9]|)[0-9])(\/(1?[0-9]|2?[0-9]|3?[0-2]))?)$'

This will give all invalid ipv4 if you modify the pipe for ipv4 in your script @Viktor Jaep

grep -vE '^(((25[0-5]|(2[0-4]|1[0-9]|[1-9]|)[0-9])\.){3}(25[0-5]|(2[0-4]|1[0-9]|[1-9]|)[0-9])(\/(1?[0-9]|2?[0-9]|3?[0-2]))?)$'

Thanks @SomeWhereOverTheRainBow! I think the IPv4 stuff is working OK, but I will check this against what I've got... The whole reason I threw the IPv6 regex in there was to validate valid IPs because one of the lists (https://raw.githubusercontent.com/SecOps-Institute/Tor-IP-Addresses/master/tor-exit-nodes.lst) contains both IPv4 and IPv6... and having only the IPv4 regex in there was causing it to error. Since I didn't have a way to filter this out, at least I wanted to check to make sure these addresses were valid so they wouldn't error out... and the other crazy thing I was trying to do was validate both IPv4 and v6 in the same statement. LOL.

 

[SomeWhereOverTheRainBow]

@Viktor Jaep

I edited it one more time to show you my complete modification of the ipresult of your script. You have to do something different with ipv6. When I get back from vacation I will submit a good validation for that for you.

 

[SomeWhereOverTheRainBow]

For your case @Viktor Jaep

ipresults=$(curl --silent --retry 3 --request GET --url $blacklisturl | grep "^[^#;]" | grep "\s*$" | grep -vE '^((((25[0-5]|(2[0-4]|1[0-9]|[1-9]|)[0-9])\.){3}(25[0-5]|(2[0-4]|1[0-9]|[1-9]|)[0-9])(\/(1?[0-9]|2?[0-9]|3?[0-2]))?)|(([0-9A-f]{0,4}:){1,7}[0-9A-f]{0,4}:?(\/(1?[0-2][0-8]|[0-9][0-9]))?))$')

This should handle both ipv4 and ipv6.

I had to edit because there were too many of these ()

 

[Viktor Jaep]

For your case @Viktor Jaep

ipresults=$(curl --silent --retry 3 --request GET --url $blacklisturl | grep "^[^#;]" | grep "\s*$" | grep -vE '^(((((25[0-5]|(2[0-4]|1[0-9]|[1-9]|)[0-9])\.){3}(25[0-5]|(2[0-4]|1[0-9]|[1-9]|)[0-9])(\/(1?[0-9]|2?[0-9]|3?[0-2]))?)|(([0-9A-f]{0,4}:){1,7}[0-9A-f]{0,4}:?(\/(1?[0-2][0-8]|[0-9][0-9]))?))$')

This should handle both ipv4 and ipv6.

Wut!! I'll check it out and let you know!

 

[Viktor Jaep]

Moved this discussion on the Skynet Filter Validator over to:

Skynet - Filter Validator v0.7 - Skynet Firewall Filter List IPv4 Integrity Validator

Filter Validator v0.7 A collaborative project between @Viktor Jaep and @SomeWhereOverTheRainBow! Filter Validator tests the IPv4 addresses (and IPv6 if present) on a given filter list that are to be used with the Skynet Firewall running on Asus-Merlin Firmware in order to block...

www.snbforums.com

 

[Martinski]

grep -E '^(((25[0-5]|(2[0-4]|1[0-9]|[1-9]|)[0-9])\.){3}(25[0-5]|(2[0-4]|1[0-9]|[1-9]|)[0-9])(\/(1?[0-9]|2?[0-9]|3?[0-2]))?)$'

This will give all invalid ipv4 if you modify the pipe for ipv4 in your script @Viktor Jaep

grep -vE '^(((25[0-5]|(2[0-4]|1[0-9]|[1-9]|)[0-9])\.){3}(25[0-5]|(2[0-4]|1[0-9]|[1-9]|)[0-9])(\/(1?[0-9]|2?[0-9]|3?[0-2]))?)$'

If you're also seeking to minimize the workload on the Regular Expression Engine, you could streamline the regex for the last part of the CIDR notation.

FROM:

(\/(1?[0-9]|2?[0-9]|3?[0-2]))

TO:

(\/(1?[0-9]|2[0-9]|3[0-2]))

The question marks in "|2?[0-9]" and "|3?[0-9]" are not needed because the 1st part of the regex ("1?[0-9]I") already includes the possibility of single-digit entries. The simpler regex is a little more efficient because it results in fewer pattern-matching parsing rules. Yes, under "normal conditions" the improvement would be insignificant, but when the script starts parsing many hundreds of thousands or millions of IP addresses every bit helps, IMO.

Just my 2 cents.

 

[Ubimo]

Why is the IP adress red? The AC86U is behind a Starlink router. It does not block anything.

 

[JTnola]

Why is the IP adress red? The AC86U is behind a Starlink router. It does not block anything.

View attachment 48327

Isn’t that a private IP address?

 

[John Fitzgerald]

Why is the IP address red? The AC86U is behind a Starlink router. It does not block anything.

View attachment 48327

The Starlink is not in "Bridge Mode".

Starlink Bypass Mode (Bridge Mode) – Protectli Knowledge Base

protectli.com

 

[Ubimo]

Thanks, but Starlink router is in bypass mode.

 

[John Fitzgerald]

Thanks, but Starlink router is in bypass mode.

The IP Address in red should be the same as the one that the ISP gives or assigns for your Modem.
If you changed something on your end then look at that. (post #375: Isn’t that a private IP address?)
How does that IP Address compare to the one assigned by Starlink?

Also, Skynet has been updated to 7.3.6 : https://github.com/Adamm00/IPSet_ASUS

 

[commodoro]

The function that identifies whether an IP address is private or not is as follows

https://github.com/Adamm00/IPSet_ASUS/blob/2b47086755848735db585d2b2edc11b6e9bac015/firewall.sh#L751

which includes as private IPs even those that start with 100
Now I'm no expert on IP addressing but I know the following ranges as private IPs

10.0.0.0 - 10.255.255.255 (10/8 prefix)
172.16.0.0 - 172.31.255.255 (172.16/12 prefix)
192.168.0.0 - 192.168.255.255 (192.168/16 prefix)
169.254.0.0 - 169.254.255.255 (169.254/16 prefix linklocal)

and the local host 127.0.0.1

 

[John Fitzgerald]

The function that identifies whether an IP address is private or not is as follows

https://github.com/Adamm00/IPSet_ASUS/blob/2b47086755848735db585d2b2edc11b6e9bac015/firewall.sh#L751

which includes as private IPs even those that start with 100
Now I'm no expert on IP addressing but I know the following ranges as private IPs

10.0.0.0 - 10.255.255.255 (10/8 prefix)
172.16.0.0 - 172.31.255.255 (172.16/12 prefix)
192.168.0.0 - 192.168.255.255 (192.168/16 prefix)

Starlink does things a bit differently:

Can You Get A Static IP From Starlink? | Starlink Hardware

Starlink doesn't offer static IP addresses. You can get a public IP on the Priority plan. This article explains a workaround using a VPN.

www.starlinkhardware.com

Starlink Router Configuration Guide | Starlink Hardware

Learn how to easily set up and configure your Starlink WiFi router using the Starlink app.

www.starlinkhardware.com

See Hacks section: https://hackaday.com/2021/05/24/starlink-a-review-and-some-hacks/

Musings from a Starlink user

https://www.reddit.com/r/Starlink_Support/comments/vjtqb3

https://www.reddit.com/r/Starlink/comments/tbuz0a

How To Use Your Own Wi-Fi Router With Starlink

Want to use your own Wi-Fi router or mesh system with Starlink? Chances are it's compatible, if you buy a simple adapter. Here's how to find the best router for Starlink to take your coverage and speeds to the next level.

www.pcmag.com

Starlink Update: February 2021

I wanted to share some additional insights I’ve garnered on Starlink service. Some of it from a YouTube video / its description (Thanks to Phil Armishaw for posting it in a comment on my Dece…

turtleherding.com

Do I get a public IP address with Starlink? Not at this time. Starlink uses Carrier Grade Network Address Translation ( CGNAT ). This is the same technology used by cellular carriers and commonly uses an IP in the 100.64.0.0/10 range.