:: ipBLOCKer :: Category blocking using iptables and ipsets

[spalife]

Blocks IPs & CIDR's tagged as

Adware Country Custom ETF
Malware Shalla(exp) Spam
Tor-Exits

Current Version: 1.1
See ChangeLog1_1.txt on github
or on post2 if upgrading
from Version 1.0

:: Features ::
- Command Line Driven with Menu Assist
- Category based Blocking
- Select Categories to Block
- Incremental Updates
- Turn on/off Blocking temporarily
- Blocking Status
- Configurable System
- Control how much data gets into
Categories with Buckets and
Maximum Entries.
System seeds with Max Entries of
65500 and 10 Buckets.
- Scheduled and Manually triggered
Refreshes
- White-listing of ips,cidrs,urls
- Custom category to add any
requried ips,cidrs,urls
- Supports additions to existing
Filters & Categories
- Check system for Blocking Status
of a ips,cidrs,urls
- Backup and Restore
- Uninstall a Category or System

:: System Requirements ::
bash(4.3+) diff(3.3+) grep(2.26+)
sort(8.2+) split(8.2+) xargs(5.2+)
kernel (2.4+) ipset (4.5+)
iptables (1.3+)

:: Installation ::
1. On Asuswrt-Merlin install Entware-ng

2. After Entware Installation
install below packages

  opkg install bash diffutils grep coreutils-sort coreutils-split findutils

3. Download ipBLOCKer
( create a seperate directory for
ipBLOCKer and download package )

mkdir -P ipBLOCKer; cd ipBLOCKer;

curl -S -# -O https://raw.githubusercontent.com/spalife/ipBLOCKer/master/ipBLOCKer.sh;
curl -S -# -O https://raw.githubusercontent.com/spalife/ipBLOCKer/master/includes_ipBLOCKer.sh;
curl -S -# -O https://raw.githubusercontent.com/spalife/ipBLOCKer/master/flib_ipBLOCKer.sh;

chmod +x *.sh;

 ./ipBLOCKer.sh setup

#( Select Categories to Block from Menu )

4. exit out of ssh/telnet
ssh/telnet again for alias to work

   cd ipBLOCKer

If you want to add to an existing
category filters or delete from
them, you can do them now.

We can wait for refresh schedule
to update selected categories
or can start a refresh manually.

Start off with some seeded
blocks by running command

  block refresh custom

:: Files Changed ::
( uninstall will remove this changes )

# /jffs/scripts/firewall-start
# ~/.profile
# ~/.bash_profile
# crontab

:: Thanks to ::
@RMerlin (Asuswrt-Merlin)
@thiggins (snbforums.com)

Inspiring contributions
Knowledge and Sharing
@john9527 @ryzhov_al @kvic
@Adamm @Martineau @thelonelycoder
@TeHashX @bigeyes0x0 @swetoast
@redhat27 @joegreat @sfx2000
@tomsk @Zirescu @ColinTaylor


Appreciate
Feedback and Suggestions


Readme
Usage
Limitations
Upcoming Features
Category Blocking Info
Refresh Schedule
FAQ
Screenshots
Found Here

 

[spalife]

[ RESERVERD ]
Could upload only 5 snapsshots in above post.

ipBLOCKer Version 1.1 ChangeLog

Upgrade instructins:
If you are upgrading ipBLOCKer from Version 1.0 to 1.1.
Run the below commands on Asuswrt-Merlin.

block backup
or
ipBLOCKer.sh backup

cp /jffs/scripts/firewall-start /jffs/scripts/firewall-start.backup;
cat /jffs/scripts/firewall-start | grep -Ev ipBLOCKEer > /jffs/scripts/firewall-start.temp
mv /jffs/scripts/firewall-start.temp /jffs/scripts/firewall-start

Installation Instructions:
Refer to Download ipBLOCKer Instructions in
Readme on GitHub.

:: CHANGE LOG ::

1.Reduced the dependency on firewalll-start.
Now when ipBLOCKer is started manually or
through a scheduled refresh, does a self check
for missing buckets and firewall rules and restores
them from available saved states which are autosaved
by the system in refresh folder:
refresh/iptables.save
refresh/ipset.save
2.User can also manually initiate the saved state restore
even though system does it automatically.
This is ONLY needed when the router has restarted
or users have scheduled off/on through router admin.
Now they can include a cron job to restore
from saved state so that it times with their restarts.
block synch_all
or
ipBLOCKer.sh synch_all
3.Cleaning up of remnants improved
4.Simulteanous run check implemented
Now the system checks if another instance
of ipBLOCKer is running either initiated by
user or as a scheduled refresh and
alerts the user.
The user can wait for ipBLOCKer to free up
or cancel his activity.
If the user has choosen to wait he is automatically taken
into ipBLOCKer when it becomes available.
Wait time defaults to 5 mins.
Menu input and confirm activities too auto timeout
now after wait_time.
Scenario:
ie user starts an activity with ipBLOCKer
and does not complete his activity.
The system times out the activity per
user initiated intent
i.e., user was in
block add white-list
Enters a List of ips / cidrs / urls to white-list
and does not confirm the activity
If it is more than the wait_time,
the system now assumes the original intent and
times out with a yes confirm and completes the activity.
5.Menu titles are now consistent across options

 

[thelonelycoder]

Wow, looks very interesting and versatile.
Since you also block ads, be prepared to answer questions about compatibility with AB-Solution.

 

[spalife]

Thanks.
The contributions from the likes of you in this forum is an inspiration.

AB-Solution is the goto choice for adblocking.
This is more of a AIO

 

[pattiri]

I've installed this and started to refresh all categories, it's taking some time but it seems it's working great. I can't even access web sites that are located in my country since Turkey is flagged as blocked country. lol

I think I need to disable county blocking

Anyway, this is a great work! thanks!

 

[spalife]

I've installed this and started to refresh all categories, it's taking some time but it seems it's working great. I can't even access web sites that are located in my country since Turkey is flagged as blocked country. lol

I think I need to disable county blocking

Anyway, this is a great work! thanks!

block refresh all
will take time the FIRST time.
Subsequent refreshes will be faster as the system knows
what blocking data you already have and will update
with the FRESHLY tagged blocks.

Country blocking can be quite effective,
though politically incorrect.
When we have guests over or
children roaming on the net,
we loose control over
where they go
what they do and
what they download and
from where

After your refresh is completed.
Do the below so that you can have country blocking
as well as remove Turkey from it.

# Since the refresh is already done
# ssh/telnet into the router

block uninstall country

cd ipBLOCKer
nano filters/country.urls

# Delete the Turkey specific filter
# i.e.,
#http://www.ipdeny.com/ipblocks/data/countries/tr.zone
# Save the file in nano by pressing ctrl+x and y

# in the command prompt
# run the below to get your country blocking again

block refresh country

Quick Q:
Are you on a MIPS or a ARM based router ?

Never mind saw your signature...

 

[pattiri]

Thanks for the tip @spalife I'll do this. meaanwhile I've added www.speedtest.net to whitelist but session stopped, I think its being processed but it has been about 10 minutes since I entered "www.speedtest.net" is it normal that adding a domain to while-list takes too much time?

admin@Fatiii:/tmp/home/root/ipBLOCKer# ./ipBLOCKer.sh add white-list
Option: add white-list                                                                                                               Enter white-list Website, IP or CIDR values below. Press ENTER when Done.
Example: www.somesite.com or 123.123.123.123 or 123.123.123.123/24

www.speedtest.net

both CPU cores are below 5% btw.

 

[spalife]

Nope it hardly takes a minute to add a single url to white-list
I just did it.

block add white-list
Option: add white-list
 Enter white-list Website, IP or CIDR values below. Press ENTER when Done.
Example: www.somesite.com or 123.123.123.123 or 123.123.123.123/24

www.speedtest.net


Entered Websites's:
www.speedtest.net
Please wait retrieving IP Address of Websites....

Retrieved IP's for Websites:
93.184.219.82


Option: add white-list

Are you Sure? [y/n]: y
ipBLOCKer: Total Downloaded & Deduplicated IP's:    1
ipBLOCKer: Total Existing IP's:                     35
ipBLOCKer: Incremental refresh IP's:                1
ipBLOCKer: Total Downloaded & Deduplicated CIDR's:  0
ipBLOCKer: Total Existing CIDR's:                   0
Please wait refresh CIDR categories will begin ....

Option: add white-list .... Done

between you need to press ENTER (twice)
Once after entering the IP
Second time on a blank line to confirm you have no more entries

 

[pattiri]

Nope it hardly takes a minute to add a single url to white-list
I just did it.

block add white-list
Option: add white-list
 Enter white-list Website, IP or CIDR values below. Press ENTER when Done.
Example: www.somesite.com or 123.123.123.123 or 123.123.123.123/24

www.speedtest.net


Entered Websites's:
www.speedtest.net
Please wait retrieving IP Address of Websites....

Retrieved IP's for Websites:
93.184.219.82


Option: add white-list

Are you Sure? [y/n]: y
ipBLOCKer: Total Downloaded & Deduplicated IP's:    1
ipBLOCKer: Total Existing IP's:                     35
ipBLOCKer: Incremental refresh IP's:                1
ipBLOCKer: Total Downloaded & Deduplicated CIDR's:  0
ipBLOCKer: Total Existing CIDR's:                   0
Please wait refresh CIDR categories will begin ....

Option: add white-list .... Done

I think there is another problem for me because I can't access speedtest.net even though AB-solution and ipBLOCKer both disabled. I will check it. Thanks again.

 

[Xentrk]

block refresh all
will take time the FIRST time.
Subsequent refreshes will be faster as the system knows
what blocking data you already have and will update
with the FRESHLY tagged blocks.

Country blocking can be quite effective,
though politically incorrect.
When we have guests over or
children roaming on the net,
we loose control over
where they go
what they do and
what they download and
from where

After your refresh is completed.
Do the below so that you can have country blocking
as well as remove Turkey from it.

# Since the refresh is already done
# ssh/telnet into the router

block uninstall country

cd ipBLOCKer
nano filters/country.urls

# Delete the Turkey specific filter
# i.e.,
#http://www.ipdeny.com/ipblocks/data/countries/tr.zone
# Save the file in nano by pressing ctrl+x and y

# in the command prompt
# run the below to get your country blocking again

block refresh country

Quick Q:
Are you on a MIPS or a ARM based router ?

Never mind saw your signature...

What is the process for adding a country, say China for example? I see ip addresses from China in my Snort logs on pfSense always probing MySQL ports.

 

[spalife]

What is the process for adding a country, say China for example? I see ip addresses from China in my Snort logs on pfSense always probing MySQL ports.

The opposite of delete.

# Adding China to country blocking

cd ipBLOCKer
nano filters/country.urls

# Go to the end of the file
# Copy paste the below
http://www.ipdeny.com/ipblocks/data/countries/cn.zone

# Save the file by ctrl+x and y
# in the command prompt

block refresh country

# if the country refresh has run before
# it will ONLY update with china otherwise
# with all the countries currently in the country filter

 

[spalife]

I think there is another problem for me because I can't access speedtest.net even though AB-solution and ipBLOCKer both disabled. I will check it. Thanks again.

A thought came by
if ab-solution is not blocking speedtest.net
do the below go to
https://www.digwebinterface.com/?

enter speedtest.net and
Nameservers ALL.
Copy all the ip's from the results, come back to ipBLOCKer
run block check and paste all the ip's.

If any of them are blocked run block add white-list
paste all the ip's into white-list.

The reason i mention this steps is because,
speedtest could have multiple changing ip's
depending on who is accesing their site
specifically if it uses cloudfare services.

I use 3 dnscrypt-proxy resolvers,
everytime i query speedtest i endup with a different ip,
till i get the entire list.

 

[pattiri]

A thought came by
if ab-solution is not blocking speedtest.net
do the below go to
https://www.digwebinterface.com/?

enter speedtest.net and
Nameservers ALL.
Copy all the ip's from the results, come back to ipBLOCKer
run block check and paste all the ip's.

If any of them are blocked run block add white-list
paste all the ip's into white-list.

The reason i mention this steps is because,
speedtest could have multiple changing ip's
depending on who is accesing their site
specifically if it uses cloudfare services.

I use 3 dnscrypt-proxy resolvers,
everytime i query speedtest i endup with a different ip,
till i get the entire list.

Thank you mate. in fact the real problem was; I missing that part;

"between you need to press ENTER (twice)
Once after entering the IP
Second time on a blank line to confirm you have no more entries"

All working fine right now

 

[bayern1975]

amazing script....will tested if got it working.....

how to check if it working? i got in syslog just this....

admin: ipBLOCKer Restoring ipsets...
admin: ipBLOCKer Restoring iptables...

 

[spalife]

amazing script....will tested if got it working.....

how to check if it working? i got in syslog just this....

admin: ipBLOCKer Restoring ipsets...
admin: ipBLOCKer Restoring iptables...

Looks like your firewall restarted.
You should see a lot more chatter in the syslog.

Run the below command which will show you
the categories you have selected, blocking data
loaded for them and hit count

block status

 

[kvic]

Woohoo! Another contestant to the ipset blocker biz.

I have the impression you're quite a laid back member for the past few years. So this sounds like a little big bang. I feel excited to see your work.

Also glad to see you stay with iptables "filter". Good choice.

 

[spalife]

Woohoo! Another contestant to the ipset blocker biz.

I have the impression you're quite a laid back member for the past few years. So this sounds like a little big bang. I feel excited to see your work.

Also glad to see you stay with iptables "filter". Good choice.

Could not spare time before.
The current effort took about 4 months few hours a day
and a month+ of testing to see how effective and useful
it could be.

Quite a bit of activity on this topic currently,
its good as security awareness is always helpful.

iptables "filter" ...
wonder how many connotations it leads to ....

 

[kvic]

iptables "filter" ...
wonder how many connotations it leads to ....

Not much really Just want to break the news that it's more efficient to use "filter" rather than "raw" tables.

 

[steelskinz]

Another blocker to test ! Thanks !

 

[bayern1975]

Looks like your firewall restarted.
You should see a lot more chatter in the syslog.

Run the below command which will show you
the categories you have selected, blocking data
loaded for them and hit count

block status

hmm, thank you for reply but can`t access overy putty....something is wrong in my case?

admin@RT-AC3200-7180:/jffs/ipBLOCKer# sh ipBLOCKer.sh status
ipBLOCKer.sh: line 13: shopt: not found
ipBLOCKer.sh: line 14: shopt: not found
ipBLOCKer.sh: /jffs/ipBLOCKer/includes_ipBLOCKer.sh: line 29: declare: not found
ipBLOCKer.sh: /jffs/ipBLOCKer/includes_ipBLOCKer.sh: line 65: declare: not found
ipBLOCKer.sh: /jffs/ipBLOCKer/includes_ipBLOCKer.sh: line 77: declare: not found
ipBLOCKer.sh: /jffs/ipBLOCKer/includes_ipBLOCKer.sh: line 81: declare: not found
ipBLOCKer.sh: /jffs/ipBLOCKer/includes_ipBLOCKer.sh: line 87: declare: not found
ipBLOCKer.sh: /jffs/ipBLOCKer/includes_ipBLOCKer.sh: line 90: MAX_ENTRIES: parameter not set
admin@RT-AC3200-7180:/jffs/ipBLOCKer#