How to Dynamically Ban Malicious IP's using IPSet (Martineau version)

Martineau · May 5, 2017

amplatfus said:
I have the FTP 21 port open.
I have tried to enter wrong password from my mobile LTE connection to FTP many times per minute (in order to block that external IP) and I could still login immediately.

This solution will protect somehow this port even is open?

The Blacklist rules are inserted at the appropriate point in the '-t filter INPUT/FORWARD' chains just before the final 'DROP/logdrop' rules.

So if you have any port services such as FTP etc. 'open', then you will need to review the '-t filter / -t nat' rules to identify where the preceding 'ACCEPT' rules are defined that will be processed before hitting the Blacklist rules.

NOTE: There are several posts/examples/ showing techniques for 'throttling' / restricting the number of access attempts per minute etc. to even using IPSETs to implement 'port knocking security' which may better serve your needs.

Martineau · May 5, 2017

HRearden said:
I think this addition and some of the Trend Micro Network AIprotection may get in each other's way, specifically Malicious Sites Blocking and Vulnerability Protection. At least I get some different error messages when I run the script and have those on. I have had a hard time finding out exactly what and how those work. Does this script attempt to do the same thing? Can they both run?
rearden

I have never bothered with the Trend Micro AI 'protection' so I wouldn't know what it does when the EULA prompt is accepted.

As explained in a previous post, the Blacklist blocking is pretty much applied as the last rules that are processed rather than the first, so I can't see how the script would affect the Trend Micro stuff?

You will need to manually list the Firewall rules to confirm the precise order in which they are triggered.

skeal · May 5, 2017

Hey there. My reports from this script are not being saved to usb. I wonder did I mess something up? My usb drive is labelled EXT4.

Code:

################################################Customise for local use #############################################
if [ -d  "/tmp/mnt/"EXT4 ];then
    DIR="/tmp/mnt/"EXT4                # <== USB Location of IPSET save/restore configuration
else
    DIR="/tmp"                                #         NOTE: /TMP isn't permanent! ;-) but allows testing of save/restore
fi

HHMMSS="168:00:00"                            # <== Specify retention period to keep Blacklist entries or passed via 'init reset' hh:mm:ss' invocation
                                            #            e.g. 168 hrs = 7 days
#####################################################################################################################

Xentrk · May 5, 2017

skeal said:

Hey there. My reports from this script are not being saved to usb. I wonder did I mess something up? My usb drive is labelled EXT4.

Code:

################################################Customise for local use #############################################
if [ -d  "/tmp/mnt/"EXT4 ];then
    DIR="/tmp/mnt/"EXT4                # <== USB Location of IPSET save/restore configuration
else
    DIR="/tmp"                                #         NOTE: /TMP isn't permanent! ;-) but allows testing of save/restore
fi

HHMMSS="168:00:00"                            # <== Specify retention period to keep Blacklist entries or passed via 'init reset' hh:mm:ss' invocation
                                            #            e.g. 168 hrs = 7 days
#####################################################################################################################

You can change
MYROUTER=$(nvram get computer_name)
to
MYROUTER=EXT4

And not require any further changes to the code. Or you can try making this change with the double quotes, but I did not test it:

DIR="/tmp/mnt/"EXT4 to DIR="/tmp/mnt/EXT4"

Try this little script to test the first suggestion. It will return: /tmp/mnt/EXT4 to the terminal

Code:

#!/bin/sh

#MYROUTER=$(nvram get computer_name)
MYROUTER=EXT4
DIR="/tmp/mnt/"$MYROUTER                # <== USB Location of IPSET save/restore configuration
echo $DIR

Jack Yaz · May 5, 2017

I think this script is blocking dns crypt, I've disabled country filter, malware amd this one by one and when this enables i lose ability to ping when using dnscrypt

Xentrk · May 5, 2017

The help and status function is broken with a fresh download of v3.0.5 (v3.0.4 is okay):

Code:

sh IPSET_Block.sh -h
: not found.sh: line 46:
: No such file or directory
: not found.sh: line 50: }
: not found.sh: line 51:
: not found.sh: line 54:
: not found.sh: line 57:
: not found.sh: line 61:
iptables: No chain/target/match by that name.
: not found.sh: line 63:
: not found.sh: line 66:
: not found.sh: line 69:
: not found.sh: line 70:
: not found.sh: line 72:
: not found.sh: line 79: }
0
: not found.sh: line 82: }
'rintf: invalid number '0
'rintf: invalid number '0
'rintf: invalid number '0
00:00:00
: not found.sh: line 88: }
IPSET_Block.sh: local: line 91: not in a function
: not found.sh: line 91: shift
IPSET_Block.sh: local: line 92: not in a function
IPSET_Block.sh: line 93: redir error

Adding this to the end of 3.0.4 works.

Code:

if [ -f /jffs/scripts/HackerPorts.sh ]; then
    /jffs/scripts/HackerPorts.sh num=3      # Requires HackerPorts v2.xx
fi
exit 0

Martineau · May 6, 2017

skeal said:

Hey there. My reports from this script are not being saved to usb. I wonder did I mess something up? My usb drive is labelled EXT4.

Code:

################################################Customise for local use #############################################
if [ -d  "/tmp/mnt/"EXT4 ];then
    DIR="/tmp/mnt/"EXT4                # <== USB Location of IPSET save/restore configuration
else
    DIR="/tmp"                                #         NOTE: /TMP isn't permanent! ;-) but allows testing of save/restore
fi

HHMMSS="168:00:00"                            # <== Specify retention period to keep Blacklist entries or passed via 'init reset' hh:mm:ss' invocation
                                            #            e.g. 168 hrs = 7 days
#####################################################################################################################

https://www.snbforums.com/threads/y...sing-ipset-v4-and-v6.38935/page-3#post-322591

Martineau · May 6, 2017

Xentrk said:
The help and status function is broken with a fresh download of v3.0.5 (v3.0.4 is okay):

So rather than copy'n'paste directly into the WinSCP editor on the router, I followed the simple advice posted by @octopus and I clicked on the 'Download' button.

This created the 'ipset_block.sh' file in the Win10 download folder.

I then dragged the file to the '/jffs/scripts/' folder in the WinSCP window pane and (PF 9) to set the execution permission.
NOTE: This fortunately? did not overwrite the 'IPSET_Block.sh' file thanks to Unix case-sensitive names!

.....and I can confirm that the script then apparently appears to be broken:

Code:

./ipset_block.sh -h

-sh: ./ipset_block.sh: not found

sh -v ipset_block.sh -h

sh: can't open 'ipset_block.sh'

sh -x ipset_block.sh -h

+ VER=v3.05
+
: not found.sh: line 46:
+ awk /^#==/{f=1} f{print; if (!NF) exit} ipset_block.sh
: No such file or directory
+ }
: not found.sh: line 50: }
<snip>

However this fixes it

Code:

dos2unix /jffs/scripts/ipset_block.sh

I guess the old-skool method of manually copy'n'paste still has its merits..despite also being error prone.

Not sure if I would be better served by only hosting the scripts on GitHub..Hopefully your curl-based download script would eliminate this tedious EOF issue.

EDIT: To use the call to conveniently run HackerReports from IPSET_Block.sh v3.04 you should manually fix the script until I get chance to post the updated files on pastebin/GitHub.

Basically LINE:187 in the HackerPorts.sh script: should be reversed

Code:

head -n $TOPX $LOGFILE".tmp" | sort -nr     >> $LOGFILE.new

change to

sort -nr  $LOGFILE".tmp"  | head -n $TOPX >> $LOGFILE.new

Jack Yaz · May 6, 2017

I think I need to add the OpenDNS proxy to whitelist, since as soon as IPSet kicks in, I lose internet connectivity from clients, though the router can ping everything just fine? Or is that something completely separate to this script?

dnscrypt-proxy[880]: Proxying from 127.0.0.1:65053 to 208.67.220.220:443

Martineau · May 6, 2017

Jack Yaz said:
I think I need to add the OpenDNS proxy to whitelist, since as soon as IPSet kicks in, I lose internet connectivity from clients, though the router can ping everything just fine? Or is that something completely separate to this script?

dnscrypt-proxy[880]: Proxying from 127.0.0.1:65053 to 208.67.220.220:443

I don't use dnscrypt but you should check to see if the IP address actually exists in the Blacklist IPSET or simply issue

Code:

./IPSET_Block.sh   unban   208.67.220.220

I do extract IPs to ban from all interfaces, so I'm not sure if modifying the rule to explicitly only monitor the WAN interface would eradicate the issue:

Code:

iptables -D Blacklist 1

iptables -I Blacklist -i $(nvram get wan0_ifname) -m state --state NEW -j SET --add-set Blacklist src

iptables --line -nvL Blacklist

Jack Yaz · May 6, 2017

When I next use your script I'll take a look (wife would like to use the internet today rather than me messing about with it haha)

Martineau · May 6, 2017

Jack Yaz said:
.... (wife would like to use the internet today rather than me messing about with it haha)

Prudent decision although you could always let her have your credit card?

...although I suspect she already has the details and would prefer to purchase online rather than hit the 'mall'

P.S. I warned you about my shonky scripts not being it fit-for-other-peoples-purpose!

Jack Yaz · May 6, 2017

Got it in one! Haha

Nah it's a good script and worked fine until I had the wise idea of looking into dnscrypt! I suspect whitelisting the IP would do it, and once I bring the script back up I'll see if it is being banned. Though I think I tried doing IPSet reset, which should have allowed everything?

Martineau · May 6, 2017

Jack Yaz said:
I think I tried doing IPSet reset, which should have allowed everything?

Correct, although I can't recall if this sequence is more reliable:

Code:

./IPSET_Block.sh save

./IPSET_Block.sh init reset

Then do your dnscrypt testing.....then restore the saved Blacklist IPSET

Code:

./IPSET_Block.sh init

Csection · May 6, 2017

I am running 3.04 with the HackerPorts addin.
I think it is not working correctly either.
(IPSET_Block.sh): 12368 v3.04 © 2016-2017 Martineau, Dynamic IPSET Blacklist banning request.....
May 6 10:00:00 (IPSET_Block.sh): 12368 Saving IPSET Block rules to /mnt/ASUS/IPSET_Logs/IPSET_Block.config.....
May 6 10:00:01 (IPSET_Block.sh): 12368 Summary Blacklist: 0 Successful blocks! ( 6146 IPs currently banned - 6146 expired )
May 6 10:00:01 (HackerPorts.sh): 12436 v2.02 Hacker Port attacks Report starting.....
May 6 10:00:01 (HackerPorts.sh): 12436 Hacker report created '/tmp/HackerReport.txt' - Statistics: Total Unique Ports attacked: 12 (out of 39 attempts) tracked using SYSLOG between May 6 09:30:26 - May 6 10:00:01.

Martineau · May 6, 2017

Csection said:
I am running 3.04 with the HackerPorts addin.
I think it is not working correctly either.
(IPSET_Block.sh): 12368 v3.04 © 2016-2017 Martineau, Dynamic IPSET Blacklist banning request.....
May 6 10:00:00 (IPSET_Block.sh): 12368 Saving IPSET Block rules to /mnt/ASUS/IPSET_Logs/IPSET_Block.config.....
May 6 10:00:01 (IPSET_Block.sh): 12368 Summary Blacklist: 0 Successful blocks! ( 6146 IPs currently banned - 6146 expired )
May 6 10:00:01 (HackerPorts.sh): 12436 v2.02 Hacker Port attacks Report starting.....
May 6 10:00:01 (HackerPorts.sh): 12436 Hacker report created '/tmp/HackerReport.txt' - Statistics: Total Unique Ports attacked: 12 (out of 39 attempts) tracked using SYSLOG between May 6 09:30:26 - May 6 10:00:01.

Please provide the output of the diagnostic commands:

Code:

grep -E "Block.*" /tmp/syslog.log | wc -l

grep -E "Block.*" /tmp/syslog.log | tail

grep    "Block IN" | tail | sed -e 's/.*SRC=\(.*\)SEQ.*/\1/'

Xentrk · May 6, 2017

Martineau said:
So rather than copy'n'paste directly into the WinSCP editor on the router, I followed the simple advice posted by @octopus and I clicked on the 'Download' button.

This created the 'ipset_block.sh' file in the Win10 download folder.

I then dragged the file to the '/jffs/scripts/' folder in the WinSCP window pane and (PF 9) to set the execution permission.
NOTE: This fortunately? did not overwrite the 'IPSET_Block.sh' file thanks to Unix case-sensitive names!

.....and I can confirm that the script then apparently appears to be broken:

Code:

./ipset_block.sh -h -sh: ./ipset_block.sh: not found sh -v ipset_block.sh -h sh: can't open 'ipset_block.sh' sh -x ipset_block.sh -h + VER=v3.05 + : not found.sh: line 46: + awk /^#==/{f=1} f{print; if (!NF) exit} ipset_block.sh : No such file or directory + } : not found.sh: line 50: } <snip>

However this fixes it

Code:

dos2unix /jffs/scripts/ipset_block.sh

I guess the old-skool method of manually copy'n'paste still has its merits..despite also being error prone.

Not sure if I would be better served by only hosting the scripts on GitHub..Hopefully your curl-based download script would eliminate this tedious EOF issue.

EDIT: To use the call to conveniently run HackerReports from IPSET_Block.sh v3.04 you should manually fix the script until I get chance to post the updated files on pastebin/GitHub.

Basically LINE:187 in the HackerPorts.sh script: should be reversed

Code:

head -n $TOPX $LOGFILE".tmp" | sort -nr >> $LOGFILE.new change to sort -nr $LOGFILE".tmp" | head -n $TOPX >> $LOGFILE.new

Normally, maybe 98% of the time, I do copy and paste from raw to MobaXterm editor in sftp session with ASCII turned on. I never had to deal with those pesky DOS characters. But this time, I did the wget command to download the file.

Code:

wget https://pastebin.com/raw/zQ3KEe8P -O /jffs/scripts/IPSET_Block.sh

I'll try again.
Thank you!

UPDATE: I confirmed that the dos2unix command fixed the issue with the file I downloaded using the wget command posted above. Now, I just need some hacker attempts so I can test the report out.

Martineau · May 6, 2017

Xentrk said:
UPDATE: I confirmed that the dos2unix command fixed the issue with the file I downloaded using the wget command posted above. Now, I just need some hacker attempts so I can test the report out.

Thanks for the feedback, so basically if using 'wget' then you must include the dos2unix utility

Code:

wget https://pastebin.com/raw/zQ3KEe8P -O /jffs/scripts/IPSET_Block.sh; dos2unix /jffs/scripts/IPSET_Block.sh

Xentrk · May 6, 2017

Martineau said:
Thanks for the feedback, so basically if using 'wget' then you must include the dos2unix utility

Code:

wget https://pastebin.com/raw/zQ3KEe8P -O /jffs/scripts/IPSET_Block.sh; dos2unix /jffs/scripts/IPSET_Block.sh

It appears to be this way, at least for pastebin files. Never had to do this with GitHub files. Maybe it is a feature of pastebin?

I never had to issue the dos2unix command in the past for files I've downloaded this way. I like the suggestion above and I'm sure this will help others as well.

Csection · May 6, 2017

Martineau said:
Please provide the output of the diagnostic commands:

Code:

grep -E "Block.*" /tmp/syslog.log | wc -l grep -E "Block.*" /tmp/syslog.log | tail grep "Block IN" | tail | sed -e 's/.*SRC=$.*$SEQ.*/\1/'

How do I provide this?
I ran it but it seemed to hang on the last one the "Block IN" one.

How to Dynamically Ban Malicious IP's using IPSet (Martineau version)

Part of the Furniture

Part of the Furniture

Part of the Furniture

Part of the Furniture

Part of the Furniture

Part of the Furniture

Part of the Furniture

Part of the Furniture

Part of the Furniture

Part of the Furniture

Part of the Furniture

Part of the Furniture

Part of the Furniture

Part of the Furniture

Senior Member

Part of the Furniture

Part of the Furniture

Part of the Furniture

Part of the Furniture

Senior Member

Similar threads

Similar threads

Support SNBForums w/ Amazon

Sign Up For SNBForums Daily Digest