How to Dynamically Ban Malicious IP's using IPSet (Martineau version)

[Xentrk]

Did you follow post

https://www.snbforums.com/threads/h...t-martineau-version.38748/page-16#post-327312

and the apply the code fix to HackerPorts.sh ?

If you have applied the fix, and ensured that 'init nolog' is present in services-start then if it is inconvenient to reboot the router simply issue:

./IPSET_Block.sh   nolog

and if IPSET tracking is already correctly enabled, the Syslog 'spam' messages should immediately stop, and HackerPorts.sh should now continue to create its reports from the tracking IPSET.

I did try those changes. I still have the message

***ERROR Syslog Tracking DISABLED? - check '/jffs/scripts/services-start' to run 'IPSET_Block.sh init'

I wiped my hack attempts in error though. So nothing to report on. I'll need to do a better job of making a backup before making changes.

I will switch over to another router, make a backup of the hack attempts, and try the changes there to see if I have better luck.

 

[Martineau]

I did try those changes. I still have the message

***ERROR Syslog Tracking DISABLED? - check '/jffs/scripts/services-start' to run 'IPSET_Block.sh init'

I'm sorry but with all due respect I don't think that can be true?

i.e. If you haven't patched HackerPorts v2.06 then if the string '/IPSET_Block.sh init nolog' exists in services-start then HackerPorts.sh will not run even if the BlacklistTRK IPSET exists.

If IPSET BlacklistTRK doesn't exist then even with the patch, HackerPorts.sh will correctly refuse to run if '/IPSET_Block.sh init nolog' exists in services-start i.e. where would Hackerports.sh get its stats from?

I wiped my hack attempts in error though. So nothing to report on. I'll need to do a better job of making a backup before making changes.

I will switch over to another router, make a backup of the hack attempts, and try the changes there to see if I have better luck.

NOTE: You can obviously use any copy of IPSET_Block.config from any other router to use on a different router rather than wait until the local Blacklist IPSET is populated with entries that are unique to the target router.

 

[Xentrk]

This is the change I made.

#    if [ "$(ipset $LIST BlacklistTRK 2> /dev/null | wc -l)" -gt 0 ]; then
ipset list BlacklistTRK >/dev/null 2>&1;if [ $? -eq 0 ]; then
        STATUS=$((STATUS+2))                        # Yes - IPSET
    fi

    echo $STATUS","$FN

}

Note how I commented the original line of code and pasted the new line below. Another pair of eyes would be helpful!

Thanks for the suggestion on the IPSET_Block.config file. Great way to recover for testing purposes. I like it!

 

[Martineau]

This is the change I made.

#    if [ "$(ipset $LIST BlacklistTRK 2> /dev/null | wc -l)" -gt 0 ]; then
ipset list BlacklistTRK >/dev/null 2>&1;if [ $? -eq 0 ]; then
        STATUS=$((STATUS+2))                        # Yes - IPSET
    fi

Note how I commented the original line of code and pasted the new line below. Another pair of eyes would be helpful!

Thanks for the suggestion on the IPSET_Block.config file. Great way to recover for testing purposes. I like it!

So either 'list' isn't valid syntax for your version of ipset ? perhaps try '-L' in place of 'list', or the BlacklistTRK IPSET doesn't exist, in which case issue

./IPSET_Block.sh    init   reset   ipset

again.

If this doesn't resolve your issue, then you will need to provide the output with debugging turned on for that line that you changed.

 

[Xentrk]

So either 'list' isn't valid syntax for your version of ipset ? perhaps try '-L' in place of 'list', or the BlacklistTRK IPSET doesn't exist, in which case issue

./IPSET_Block.sh    init   reset   ipset

again.

If this doesn't resolve your issue, then you will need to provide the output with debugging turned on for that line that you changed.

Thanks for taking the time for looking into this. Sorry for my late reply. Here is the output from verbose execution of the script.

https://pastebin.com/hNVYeJfZ

I'll head on over to one of the other routers for some testing there.

 

[Martineau]

Here is the output from verbose execution of the script.

https://pastebin.com/??????????

WTF!!!!?

 

[Xentrk]

Here is the output from the second router. A little different:
https://pastebin.com/XpmVEE7S

Update: The HackerPorts.sh all report is working on the second router now that I applied this command:

./IPSET_Block.sh init reset ipset

   v2.06 © 2016-2017 Martineau, Hacker Port attacks Report.....

Retrieving IPSET BlacklistTRK data for 'eth0' violations, please wait.....

2 members retrieved from IPSET (BlacklistTRK - Entries auto-expire after 7 days 00:00:00 hrs)


29 May 19:00:38: # Unique Ports attacked via 'eth0': 2 (out of 2 attempts) tracked via IPSET


        Top 10 Ports attacked:
    1 http://www.speedguide.net/port.php?port=42495 (tcp,udp)  e.g. https://www.speedguide.net/ip/184.26.143.115
    1 http://www.speedguide.net/port.php?port=1080  (tcp,udp)  e.g. https://www.speedguide.net/ip/50.117.86.3

        Top 10 attackers:
    1 https://www.speedguide.net/ip/184.26.143.115
    1 https://www.speedguide.net/ip/50.117.86.3

I did that command on the first router following the posts with you and @skeal. But it still gave me the error I pasted above.

 

[Csection]

This is the change I made.

#    if [ "$(ipset $LIST BlacklistTRK 2> /dev/null | wc -l)" -gt 0 ]; then
ipset list BlacklistTRK >/dev/null 2>&1;if [ $? -eq 0 ]; then
        STATUS=$((STATUS+2))                        # Yes - IPSET
    fi

    echo $STATUS","$FN

}

Note how I commented the original line of code and pasted the new line below. Another pair of eyes would be helpful!

Thanks for the suggestion on the IPSET_Block.config file. Great way to recover for testing purposes. I like it!

Hey, Xen!
I'm nobody to comment but I have a suggestion.
Try putting a blank line after the line you commented out. Just a thought!

 

[Xentrk]

Hey, Xen!
I'm nobody to comment but I have a suggestion.
Try putting a blank line after the line you commented out. Just a thought!

I did what you suggested and no change in behavior. I had the thought of removing all traces of the script, rebooting, then reinstalling fresh followed by a reboot. But I will wait to see what @Martineau thinks after he had some time to look over things. This router is VPN all traffic. So that may be having some impact.

 

[Csection]

I did what you suggested and no change in behavior. I had the thought of removing all traces of the script, rebooting, then reinstalling fresh followed by a reboot. But I will wait to see what @Martineau thinks after he had some time to look over things. This router is VPN all traffic. So that may be having some impact.

Yeah!
I just thought maybe on a slight chance the line you entered may have appended to the comment line and wasn't executing.

 

[skeal]

HI there sorry if you are busy but I think I messed things up by upgrading another script. I updated @redhat27 ya-malware-block and since doing so my IPSET_Block is not successfully blocking anything and states when I run IPSET_Block.sh it tells me "warning ipset blocking is in tracking mode only". I am sorry to burden you with this question but other than reverting back to old settings and scripts as a cure I have no ideas. I apologize in advance of my ignorance of how these scripts work I've been reading as much as I can.
EDIT: I am so sorry for asking this question as a reboot fixed everything. Sorry for my noobness!

 

[Martineau]

My IPSET_Block is not successfully blocking anything and states when I run IPSET_Block.sh it tells me "warning ipset blocking is in tracking mode only".

There is a command line switch in IPSET_Block.sh, that is available:

./IPSET_Block.sh   init   trackonly

   v4.04 © 2016-2017 Martineau, Dynamic IPSET Blocking.....

 IPSET(s) restored from '/tmp/mnt/RT-AC68U/IPSET_Block.config'

 Restoring permanently banned I/P addresses to Blacklist from '/tmp/mnt/RT-AC68U/IPSET_Block.config.add'.....

 Tracking ONLY mode enabled...attackers will not be physically blocked!

 Summary Blacklist: 0+0 Successful blocks! ( 0 IPs currently banned - 2268 added ), Entries auto-expire after 3 days 00:00:00hrs, 91 permanently banned

   v2.07 © 2016-2017 Martineau, Hacker Port attacks Report.....

 ***Warning IPSET Blocking is in Tracking ONLY mode!

and whilst it is shown when you request the IPSET_Block '-h/help' function, its use isn't explicitly described - but its intended usage should be self explanatory?

To remove the tracking ONLY state, simply issue

./IPSET_Block.sh   init

However, if you didn't invoke the 'trackonly' directive, then all I can assume is that the INPUT Blacklist DROP rule was somehow purged?

29     153 10859 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set Blacklist src

However, as you have found, it would appear that you correctly only have 'init' specified in the ./IPSET_Block.sh start request @boot, and that fixed your 'issue'.

 

[Csection]

I just ran into a problem.
When IPSET_Block.sh ran on the hourly scheduled time. I got the following output:
Jun 6 21:34:44 (IPSET_Block.sh): 9500 v4.03 © 2016-2017 Martineau, Dynamic IPSET Blacklist banning request.....
Jun 6 21:34:44 (IPSET_Block.sh): 9500 ***ERROR IPSET Blacklist does not exist! - Please run 'IPSET_Block.sh init'
Jun 6 21:34:51

I tried to run the init function, but it did not fix the problem.

 

[Martineau]

I just ran into a problem.
When IPSET_Block.sh ran on the hourly scheduled time. I got the following output:
Jun 6 21:34:44 (IPSET_Block.sh): 9500 v4.03 © 2016-2017 Martineau, Dynamic IPSET Blacklist banning request.....
Jun 6 21:34:44 (IPSET_Block.sh): 9500 ***ERROR IPSET Blacklist does not exist! - Please run 'IPSET_Block.sh init'

I tried to run the init function, but it did not fix the problem.

21:34:44 does seem a strange 'hourly' cron invocation, but you can obviously confirm if it has been physically deleted (rather than the script incorrectly failing) by using:

ipset list | grep Blacklist | uniq

So if the IPSET has indeed been purged (how/why?? ), then if there is no IPSET_Block.config on a persistent USB drive for the 'init' directive to automatically restore from, then you will need to issue:

./IPSET_Block.sh   init   reset

Unless there is anything significant logged in Syslog, then it is going to be difficult to determine the cause.

 

[Csection]

I rebooted the router and then I ran the function ./IPSET_Block.sh init reset ipset, but that didn't help either.
command line output from IS_B init:
admin@RT-AC3100-0000:/jffs/scripts# ./IPSET_Block.sh init

v4.03 © 2016-2017 Martineau, Dynamic IPSET Blocking.....

IPSET(s) restored from '/mnt/ASUS/IPSET_Logs/IPSET_Block.config'

Restoring permanently banned I/P addresses to Blacklist from '/mnt/ASUS/IPSET_Logs/IPSET_Block.config.add'.....
ipset v6.29: The set with the given name does not exist
ipset v6.29: The set with the given name does not exist
ipset v6.29: The set with the given name does not exist
iptables v1.4.14: Set Blacklist doesn't exist.

Try `iptables -h' or 'iptables --help' for more information.
iptables v1.4.14: Set Whitelist doesn't exist.

Try `iptables -h' or 'iptables --help' for more information.
(IPSET_Block.sh): 12103 **ERROR** Unable to add - INPUT --match-set Whitelist RC=0

iptables v1.4.14: Set Blacklist doesn't exist.

Try `iptables -h' or 'iptables --help' for more information.
iptables v1.4.14: Set Blacklist doesn't exist.

Try `iptables -h' or 'iptables --help' for more information.
Bad argument `58'
Try `iptables -h' or 'iptables --help' for more information.
iptables v1.4.14: Set Whitelist doesn't exist.

Try `iptables -h' or 'iptables --help' for more information.
ipset v6.29: The set with the given name does not exist

***ERROR IPSET Blacklist does not exist! - Please run 'IPSET_Block.sh init'

admin@RT-AC3100-0000:/jffs/scripts#

21:34:44 does seem a strange 'hourly' cron invocation, but you can obviously confirm if it has been physically deleted (rather than the script incorrectly failing) by using:

ipset list | grep Blacklist | uniq

So if the IPSET has indeed been purged (how/why?? ), then if there is no IPSET_Block.config on a persistent USB drive for the 'init' directive to automatically restore from, then you will need to issue:

./IPSET_Block.sh   init   reset

Unless there is anything significant logged in Syslog, then it is going to be difficult to determine the cause.

I'm sorry if I gave confusing info. I ran the script from the command line to give you something to let you know what was going on.
I just ran the "init reset" and it appears to have fixed the script.
Here is the output from that run:
admin@RT-AC3100-0000:/jffs/scripts# ./IPSET_Block.sh init reset

v4.03 © 2016-2017 Martineau, Dynamic IPSET Blocking.....

IPSETs: 'Blacklist*/Whitelist*' created empty - reset!

Restoring permanently banned I/P addresses to Blacklist from '/mnt/ASUS/IPSET_Logs/IPSET_Block.config.add'.....
iptables: No chain/target/match by that name.
iptables: No chain/target/match by that name.
Bad argument `59'
Try `iptables -h' or 'iptables --help' for more information.

Summary Blacklist: 0+0 Successful blocks! ( 0 IPs currently banned - 2 added ), Entries auto-expire after 7 days 00:00:00hrs, 2 permanently banned


v2.06 © 2016-2017 Martineau, Hacker Port attacks Report.....

Scanning /tmp/syslog.log for ANY interface (IN=eth0) violations, please wait.....

53 records scanned from Syslog ('/tmp/syslog.log')


07 Jun 07:20:13: # Unique Ports attacked via ANY interface: 2 (out of 3 attempts) tracked via SYSLOG, Jun 7 07:18:36 - Jun 7 07:20:13


Top 3 Ports attacked:
2 http://www.speedguide.net/port.php?port=23 e.g. https://www.speedguide.net/ip/190.0.47.138
1 http://www.speedguide.net/port.php?port=3389 e.g. https://www.speedguide.net/ip/109.237.108.143

Top 3 attackers:
1 https://www.speedguide.net/ip/190.0.47.138
1 https://www.speedguide.net/ip/109.237.108.143

Last 3 most recent attackers:
https://www.speedguide.net/ip/190.0.47.138
https://www.speedguide.net/ip/109.237.108.143

 

[Csection]

Yep!
She's working again!
I did a manual command line "Save" and it worked.
Damn!
I tried everything except a "Reset".
Thanks for the help!
I'm not sure what caused the problem. All I can say is it seemed to start after I did an "Unban" from the command line and found out the ip address was not in the "Blacklist".
That is when "All Hell" broke loose!

 

[Martineau]

.....it seemed to start after I did an "Unban" from the command line and found out the ip address was not in the "Blacklist".

That is when "All Hell" broke loose!

Ahh, debugging my shonky code , is much easier when you provide a detailed description of your actions.

So in the 'unban' clause, the code should only attempt to remove the I/P from 'IPSET_Block.config' if it exists, but (because of the brain-dead syntax) it STUPIDLY always destroys 'IPSET_Block.config' - hence the next time you reboot, the Blacklist IPSET no longer exists.

Here is the patch:

if [ $? -eq 0 ];then
   echo -e "\n"$unbannedip "Is Now Unbanned"$cRESET
fi
echo "`sed /$unbannedip/d $DIR/IPSET_Block.config`" > $DIR/IPSET_Block.config

change to

if [ $? -eq 0 ];then
   echo -e "\n"$unbannedip "Is Now Unbanned"$cRESET
   sed -i '/$unbannedip/d' $DIR/IPSET_Block.config
fi

Abject apologies for the inconvenience.

 

[Alfsu]

@Martineau
Thank you so much for this excellent script, I've been running rev 3.05 for sometime now without any problems.
Where or how can I do get hold of rev 4.03, or is it not available at all? I see it being reported on in the threat but the OP link still points to the old revision.
Cheers!

 

[Martineau]

@Martineau
Thank you so much for this excellent script, I've been running rev 3.05 for sometime now without any problems.
Where or how can I do get hold of rev 4.03, or is it not available at all? I see it being reported on in the threat but the OP link still points to the old revision.
Cheers!

It is nominally still in beta, but I will send you the temporary pastebin link....

 

[Csection]

Ahh, debugging my shonky code , is much easier when you provide a detailed description of your actions.

So in the 'unban' clause, the code should only attempt to remove the I/P from 'IPSET_Block.config' if it exists, but (because of the brain-dead syntax) it STUPIDLY always destroys 'IPSET_Block.config' - hence the next time you reboot, the Blacklist IPSET no longer exists.

Here is the patch:

if [ $? -eq 0 ];then
   echo -e "\n"$unbannedip "Is Now Unbanned"$cRESET
fi
echo "`sed /$unbannedip/d $DIR/IPSET_Block.config`" > $DIR/IPSET_Block.config

change to

if [ $? -eq 0 ];then
   echo -e "\n"$unbannedip "Is Now Unbanned"$cRESET
   sed -i '/$unbannedip/d' $DIR/IPSET_Block.config
fi

Abject apologies for the inconvenience.

Should I go ahead and run a test with a valid ip such as 8.8.8.8 to check it out?