iblocklist.com generic ipset loader for ipset v6 and v4

[dandle]

Hi redhat. Thanks for the script! Is there a way to change the location of where the .gz zips are located? They're currently sitting in /jffs/ipset_lists and I'd like to move it to a USB location if possible. Thanks.

 

[redhat27]

Sure. You'd need to change this line to reference your usb location

 

[unsynaps]

I do not know if I am a derp and missed it going through this thread or what. But I have a questions.

1) Does this only block NEW (unsolicited) connections? Allowing ones that are RELATED,ESTABLISHED?
2) Is there an easy way to only allow 1-2 countries without having to make a huge line of all the ones you want to block?
EX: Leave BLOCKLIST_INDEXES empty and out the 1-2 country numbers in ALLOWLIST_INDEXES.
3) It looks like the country lists block to and from. Is there a way to make it only from?

Basically I want to:
- DROP all NEW (unsolicited) traffic from anywhere but the US.
- ACCEPT all RELATED,ESTABLISHED traffic. So if I actually go to a server in another country everything works fine.

 

[unsynaps]

I am going to have to hold off on using this now that I have done some testing. Nothing wrong with the script, its the blacklists.

Only enabled lists 2 and 11. Found out that Twitch video was blocked (chat loaded ok), steam would not connect, and FFXIV would not update. While making a white list for Twitch probably would not be that big of a deal and there is a list for Steam the list for Square Enix (FFXIV) is missing an entire range of IP's needed to whitelist. Because of this i have serious doubts about ALL of their lists. Making it seem like I will have to take hords of time making a custom white list to make sure everything works.

 

[redhat27]

@unsynaps Regarding your last two posts, I'm going to implement a whitelist CIDR (for local/private IPs that are sometimes included in the ipset data) and a blacklist CIDR, as that was requested (see here and the prior post)

I'll try to have the changes completed by today

 

[unsynaps]

Ohh cool. Well that solves that extra block of IP's for Square Enix that iBlocklist didn't pick up.

Found the block googleing for like 2 seconds. lol

Gives me time to read more about iptables and ipset. Never messed with iptables outside FILTER and NAT.
Diving into the other tables also led me to why WoL from the internet was not working even though I forwarded UDP 9 to the broadcast address.

 

[redhat27]

Okay people, a new version of iblocklist-loader is up (1.2)
The changes are in the extended version only.

It now allows you to specify a whitelist and a blacklist CIDR file where you can add manual entries. This feature has been requested a few times. In addition, if you do not have a whilelist CIDR file, one would be created that whitelists internal IPs that are unroutable over internet. This will alleviate the pain a lot of folks have experienced with bogon and other lists.

I am planning to get rid of the original version and only keep the extended version. Does anyone object to that? As @Xentrk pointed out, keeping both iblocklist-loader.sh (original script) and iblocklist-loader-v2.sh (extended version) may be confusing to many.

 

[Xentrk]

I am planning to get rid of the original version and only keep the extended version. Does anyone object to that? As @Xentrk pointed out, keeping both iblocklist-loader.sh (original script) and iblocklist-loader-v2.sh (extended version) may be confusing to many.

No objections with me. I vote for only keeping the extended version.

 

[shooter40sw]

Okay people, a new version of iblocklist-loader is up (1.2)
The changes are in the extended version only.

It now allows you to specify a whitelist and a blacklist CIDR file where you can add manual entries. This feature has been requested a few times. In addition, if you do not have a whilelist CIDR file, one would be created that whitelists internal IPs that are unroutable over internet. This will alleviate the pain a lot of folks have experienced with bogon and other lists.

I am planning to get rid of the original version and only keep the extended version. Does anyone object to that? As @Xentrk pointed out, keeping both iblocklist-loader.sh (original script) and iblocklist-loader-v2.sh (extended version) may be confusing to many.

Is the version 2 of the script the extended one?, Iv been using the 1.1 until now runs fast...

 

[Xentrk]

Had a new issue starting last night. In the MS Edge Browser, when you open up a tab, icons from recent sites display on top. Below this are news stories. I sometimes click on one of these stories. They go to msn.com English usa site. But the pages were not loading. If I copied the url into Firefox, I had the same result. Yet, I could go directly to msn.com and click on stories and they would load with no issue. I rebooted without running my usual scripts. I ran them manually until I confirmed it was iblocklist-loader.sh.

I do use the blacklist-domains.txt to block MS Telemetry, etc.

I used AB-Solution, follows the log file option, to capture domain names that were spun up when browsing the site when I had started the router fresh without running any scripts. I then got all names with msn in them. The second domain name I whitelisted made it work again.

The fix was to whitelist: img-s-msn-com.akamaized.net

I am not sure why this happened. I have another using the same script that did not give me this issue.

 

[redhat27]

Also, here is another url in the blacklist-domains.txt that reports as invalid:

nslookup: can't resolve 'public-family.api.account.microsoft.com'

Don't know if I'd mentioned it, but I removed that entry a while ago.

 

[redhat27]

Is the version 2 of the script the extended one?, Iv been using the 1.1 until now runs fast..

Yes this is the extended script, and this is the original one. A bit confusing to keep both around. I'll get rid of the original one soon and just keep the extended one. The one thing that is stopping me is the thought that all the places I'd need to update the references (wiki, posts, etc)

 

[VZ3]

@redhat27
Thanks, you have wrote nice utility MatchIP.

But requirements to install entware (to have USB drive with ext3 available) and then opkg install coreutils-paste kinda too much hassle.

You can just replace your "| paste -s" with "| tr '\n' ' ' " and that's it. MatchIP will work without that extra thingy.

 

[redhat27]

Good change! The joys of community contribution
Please feel free to update the wiki.

If you want to pass on that, I'll update it when I get a chance.

 

[VZ3]

@redhat27

One more thing. I'm playing with blocklist-loader-v2.sh. And country list [src.dst] traffic modifiers kinda reversed or I did not get it.

I tested on United-Kingdom. I put UK in BLOCKLIST_INDEXES. If I set traffic field to [src] only then I cannot update my ubuntu (address in UK). Also I cannot access UK websites. BUT when I change it to [dst] in that index line then I can access UK no problem.

Maybe it's too late and I did not think clearly... Idk.

Edit: I just tested blocking US as a country. Well, no matter what traffic flag I choose [src] or [dst] I cannot connect to my VPS's (multiple US addresses), cannot hit sites like CNN.com, cannot hit this SNB forum... I just don't understand why... I see at US match-set record in iptables raw with etither [src] or [dst] but cannot connect neither way.

PS: Well, I just tried ya-malware-block script instead and I think I'm going to stick with it. Though, just out of curiosity, why [src|dst] kinda flaky in match-sets...

 

[redhat27]

src is where the traffic originates. dst is the the target for the traffic. Blocking the [src] is mostly all that is needed, specifically if the idea is to block incoming traffic from outside (potentially dangerous/mal-intentioned sites on the internet)
You'd typically use [dst] to prevent your lan machines to "dial out" to unwanted destinations. Example is when you use microsoft windows and want stop your machines from sending identifying or other data to M$ telemetry servers.

 

[redhat27]

Also, ya-malware-block and iblocklist-loader do not conflict with each other. I use both. They have different purposes, although there may be some overlap with some of the lists. If an IP is blocked on both lists from the two scripts, there is no harm: The iptables filter that blocks it first wins, the other rule will not be enacted.

 

[amplatfus]

src is where the traffic originates. dst is the the target for the traffic. Blocking the [src] is mostly all that is needed, specifically if the idea is to block incoming traffic from outside (potentially dangerous/mal-intentioned sites on the internet)
You'd typically use [dst] to prevent your lan machines to "dial out" to unwanted destinations. Example is when you use microsoft windows and want stop your machines from sending identifying or other data to M$ telemetry servers.

Hi,

Even I have only src activated it seems that some sites could not be accessed until are added to the white-list.
Please am I doing something wrong?
Thanks!

 

[redhat27]

Hi,

Even I have only src activated it seems that some sites could not be accessed until are added to the white-list.
Please am I doing something wrong?
Thanks!

Enable both [src,dst] on sites you'd want to access in your whitelist.
Can you give me an example of the site you want to access that is blocked? Perhaps an avtivated list is legitimately blocking it?

 

[amplatfus]

Enable both [src,dst] on sites you'd want to access in your whitelist.
Can you give me an example of the site you want to access that is blocked? Perhaps an avtivated list is legitimately blocking it?

Thanks. For instance dropbox.com
I have activated below lists

 BLOCKLIST_INDEXES="6 9 10 11 14 15 16 17 25 27 28 31 34"

The lists are activated as this:

List006="rangetest        Bluetack     http://list.iblocklist.com/?list=plkehquoahljmyxjixpu  src"

So the question is why browsing to Dropbox is not working. It works only after adding it in white-lists.

Thank you,!

Sent from my ONE A2003 using Tapatalk