iblocklist.com generic ipset loader for ipset v6 and v4

[redhat27]

I am wondering if somewhere, your script erases all previous IPTABLES

It doesn't

do I have to write my own IPTABLES rules after your script, and not before, or if it doesn't matter in which order the scripts are run?

Depends if you are modifying the same chain. Also -I (inserts) are added to the beginning of the chain, and -A (appends) are added at the end

My question is, if the list is not passworded at all, then why do I still get the error?
gunzip: invalid magic

Try this:

wget -q -O test.data.gz "http://list.iblocklist.com/?list=LISTID&fileformat=p2p&archiveformat=gz&username=MYUSERNAME&pin=MYPIN"
gunzip test.data.gz

Where you fill in your id, user and pin in the url

If you do not get any error, then you should be good. I suspect you'll get the same error. Let me know.

 

[p1r473]

It doesn't
Depends if you are modifying the same chain. Also -I (inserts) are added to the beginning of the chain, and -A (appends) are added at the end.

I am trying like these with either -I or -A (which should I use? It seems only -I works, and -A doesnt???):
iptables -I INPUT -p tcp --dport 22 -j DROP
iptables -I INPUT -p tcp -s 192.168.1.48 --dport 22 -j ACCEPT

Can I safely call this script before/after your script, and they will both work well together?

It doesn't
Try this:

wget -q -O test.data.gz "http://list.iblocklist.com/?list=LISTID&fileformat=p2p&archiveformat=gz&username=MYUSERNAME&pin=MYPIN"
gunzip test.data.gz

Where you fill in your id, user and pin in the url

If you do not get any error, then you should be good. I suspect you'll get the same error. Let me know.

This worked fine when I putty'ed into my router to try it! So I don't know why the script hates it! Extracted the file out of the gzip perfectly

I am noticing a sample line from your list example is:
http://list.iblocklist.com/?list=zw
However, the premium one has additional parameters (username, pin, fileformat, archiveformat). So I tried both of these lines:
List296="Premium I-Blocklist http://list.iblocklist.com/?list=LIST&username=USERNAME&pin=PIN src,dst"
and I tried
List296="Premium I-Blocklist http://list.iblocklist.com/?list=LIST&fileformat=p2p&archiveformat=gz&username=USERNAME&pin=PIN src,dst"

And neither worked. Its something to do with the parameters I think, because thats the only difference I can see. The actual file itself isn't password protected

 

[bayern1975]

can't connect over openvpn anymore with iblocklist, iblock-ipsets installed? Where should be a problem?

Poslano z mojega EVA-L09 z uporabo Tapatalk

 

[redhat27]

@p1r473 Can you try the latest script from github now and see if that works for you? I've made a change that should fix the issue you were facing.

 

[redhat27]

@bayern1975 Are you positive that the one of the ipsets loaded by the script is at fault? You can try to see if the IP you are connecting from (via VPN) is in the blocklist.
To test if the IP is in the blocklist, test with

ipset test [set] [ip]

example:
ipset test BluetackMicrosoftCIDR 123.123.123.123

You might need to use that IP in each of the ipsets that you are using.

But, before trying all that you can just whitelist the IP for your VPN first:

create a /jffs/configs/host.add (or anywhere that will add to your routers hosts file) entry with:

123.123.123.123 my_vpn_ip

where you will substitute your vpn ip for 123.123.123.123
and then include my_vpn_ip in the file defined on your WHITELIST_DOMAINS_FILE= line. You may need to reboot your router for the hosts entry to resolve

 

[bayern1975]

do you mean I put in putty like this?
ipset test 10.8.0.0/24

I think it is to hard for understanding for me....

Poslano z mojega EVA-L09 z uporabo Tapatalk

 

[redhat27]

No, like ipset test BluetackMicrosoftCIDR <your-external-vpn-ip-where-you-vpn-from>. Its easier to whitelist your external vpn ip first. Is the IP from where you VPN a static IP or a dynamic IP?

You can also try taking out Microsoft from being blocked, and see if that helps (list index 12)

 

[bayern1975]

it working again...i delete openvpn profile and create new one.....i think something lost somewhere....thank you for helping me....

 

[p1r473]

@p1r473 Can you try the latest script from github now and see if that works for you? I've made a change that should fix the issue you were facing.

Works great with premium now thanks!

Can anyone please try this, and tell me if this causes them to lose all internet access? I wondering if one of the lists is bad, or if I just have too many, but it seems when I have these all enabled I lose internet access. I can still do DNS lookups, but I cant even ping a site. It seems I don't lose access to EVERY site (can still access snbforums) but I lose access to many sites (google, pokemon.com, github, all unreachable)
BLOCKLIST_INDEXES="2 6 3 7 8 9 11 15 17 23 24 25 26 27 28 29 30 31 33 34 35"
If anyone can confirm these indexes break the internet for them too, I will go through them one at a time and try to find which one is breaking internet access

Regarding getting this working with an additional firewall script:
I am trying like these with -I. It seems -A causes me to not be able to SSH no matter the order of the lines.
iptables -I INPUT -p tcp --dport 22 -j DROP
iptables -I INPUT -p tcp -s 192.168.1.48 --dport 22 -j ACCEPT
Can I safely call this script before/after your script, and they will both work well together?

 

[redhat27]

Can anyone please try this, and tell me if this causes them to lose all internet access? I wondering if one of the lists is bad, or if I just have too many, but it seems when I have these all enabled I lose internet access. I can still do DNS lookups, but I cant even ping a site. It seems I don't lose access to EVERY site (can still access snbforums) but I lose access to many sites (google, pokemon.com, github, all unreachable)
BLOCKLIST_INDEXES="2 6 3 7 8 9 11 15 17 23 24 25 26 27 28 29 30 31 33 34 35"

I would recommend starting small (maybe with the default list that is in github) and then work your way up. If you see your favourite site blocked after you add the list that you want, then you can try whitelisting the TLD first

iptables -I INPUT -p tcp --dport 22 -j DROP
iptables -I INPUT -p tcp -s 192.168.1.48 --dport 22 -j ACCEPT
Can I safely call this script before/after your script, and they will both work well together?

They should work together without any issue,
BUT
It looks like you are using ssh from within your lan. You can just disable ssh on the wan from the ui (unless you specifically want to allow ssh from only one lan client) That way you would not need these rules.

 

[amplatfus]

Works great with premium now thanks!

Can anyone please try this, and tell me if this causes them to lose all internet access? I wondering if one of the lists is bad, or if I just have too many, but it seems when I have these all enabled I lose internet access. I can still do DNS lookups, but I cant even ping a site. It seems I don't lose access to EVERY site (can still access snbforums) but I lose access to many sites (google, pokemon.com, github, all unreachable)
BLOCKLIST_INDEXES="2 6 3 7 8 9 11 15 17 23 24 25 26 27 28 29 30 31 33 34 35"
If anyone can confirm these indexes break the internet for them too, I will go through them one at a time and try to find which one is breaking internet access

Regarding getting this working with an additional firewall script:
I am trying like these with -I. It seems -A causes me to not be able to SSH no matter the order of the lines.
iptables -I INPUT -p tcp --dport 22 -j DROP
iptables -I INPUT -p tcp -s 192.168.1.48 --dport 22 -j ACCEPT
Can I safely call this script before/after your script, and they will both work well together?

I had similar issues with List007="bogon Bluetack" as I remember.
Might be other list (I do not remember which one) but I suggest to delete this one first.

 

[bayern1975]

i tested today most of lists from 1 to 37.....if i choose all then internet stop working.....i do not know but will be fine to share here what lists are ok to get active.....here is my at the moment but do not know it is good or no.....share yours here on board......

# Block traffic from any of the above lists
BLOCKLIST_INDEXES="8 9 10 12 13 14 17 27 28 29"

 

[p1r473]

I did manage to get this error once:
./iblocklist-loader.sh: line 515: syntax error: unexpected "(" (expecting "fi")
Seems to be a syntax error... I didn't get it often, just once during my testing.

I tested all the lists that interested me by trying about 50 different domains while web surfing with every list.

 

[bayern1975]

yes, if bbc.co.uk is blocked is not good and speedtest.com also blocked.....we need, find best solution to use these lists....if they block regulary website we do not need it to have active...i am not happy at all with most of lists from 1 to 37......

 

[bayern1975]

speedtest.net not working if you have list number 29 active....I think list 29 blocks most website....

Poslano z mojega EVA-L09 z uporabo Tapatalk

 

[p1r473]

Oh yes I can confirm that bayern1975 thanks for your contribution!
I have taken 29 out of the list

Please everyone test:
8 9 11 15 14 16 17 24 25 26 27 28 31 34

Up to date bad list:
6 7 30 35 (blocks A LOT)
2 (blocked teamviewer.com)
33 (blocked bbc.co.uk)
29 (blocked speedtest.net)
Premium Malicious list maintained by Squidblacklist (blocked eotugame.com)

 

[redhat27]

I know some lists are not very useful (I seen bogon and rangetests, especially) but I would like to reiterate that if a list appears useful, but only a few of your favourite sites are blocked, please add the domains to the whitelist file, and you should be good to go. I would request that you share the domains that you had to add to make the process easy for others.

Just a suggestion.

EDIT: I would also add your domains to the github, so that others can use the whitelist file and will only need to add theirs. Wh can keep growing that file.

 

[p1r473]

The problem is my testing methodology was to use a "Refresh all tabs" chrome addon onto the 50 websites I had open. While I only noticed one domain out of the 50 I had open was blocked, there could potentially be thousands blocked, so more testing must be done onto these possibly bad lists. I mean what if 1/50th of the whole internet is blocked??

 

[p1r473]

What I will try to do is add 2, 33, 29 back in and whitelist those domains, and I can report back if any additional sites were blocked

 

[redhat27]

I agree, that that is not a foolproof way to determine if a lot of sites are blocked, but you only need to unblock your favourite sites. Also testing this addition to whitelistdomains should not involve rebooting your router or anything that takes too long:

After adding the domain to the whitelist domains file, you just manually run the script. It will NOT process the already loaded lists if USE_LOCAL_CACHE=Y, it will just re-process the whitelistdomains (and blacklistdomains if you are using v2). The turnaround time should be fairly quick.