Malware Filter / bad host IPSET

[spalife]

Another alternative

#WAN Down Values...
noLnk="0.0.0.0";

#Get values from NVRAM
ipAddr="$(nvram get wan0_ipaddr)";
lnkWan="$(nvram get link_internet)";

if ( ([ $ipAddr == $noLnk ]) || ([ $lnkWan == 0 ]) );
then
echo "Network Down"
fi

 

[swetoast]

im probobly not gonna rely on nvram for future revision since its so bound to asuswrt and i want it to be more generic so im gonna keep on figuring out a way for it to work out of the box for all types.

 

[swetoast]

So scrapped the nvram solution so that other distroes can use this script so far the script is only in my git as always its a test version for anyone brave enough to try

 

[visortgw]

So scrapped the nvram solution so that other distroes can use this script so far the script is only in my git as always its a test version for anyone brave enough to try

This works for me on two arm and one mips routers (all ASUS) running merlin firmware.

 

[elorimer]

No idea what you are tryinig to do, but is it not possible to use ping in a bash script? Why not do the same ping wanduck is doing to that microsoft ncsi address?

RESULT="1"
PING=$(ping www.msftncsi.com -c 1 | grep -E -o '[0-9]+ received' | cut -f1 -d' ')
if [ "$RESULT" != "$PING" ]
then
   DO SOMETHING
else
   DO SOMETHING
fi

 

[redhat27]

but is it not possible to use ping in a bash script

I believe he has already taken that approach in his git repo

 

[visortgw]

No idea what you are tryinig to do, but is it not possible to use ping in a bash script? Why not do the same ping wanduck is doing to that microsoft ncsi address?

RESULT="1"
PING=$(ping www.msftncsi.com -c 1 | grep -E -o '[0-9]+ received' | cut -f1 -d' ')
if [ "$RESULT" != "$PING" ]
then
   DO SOMETHING
else
   DO SOMETHING
fi

But definitely not to that host as many people block it...

 

[swetoast]

the only reason online check is in the script is due to if your wan connection is down then it doesnt run and remove the lists, had serious downtime awhile back and i noticed that malware-filter spammed the syslog with empty results and thats why i included online check since its on a cron job and not running from wan_start

in anycase "happy beerday since its friday "

 

[Xentrk]

So scrapped the nvram solution so that other distroes can use this script so far the script is only in my git as always its a test version for anyone brave enough to try

Updated malware-block to use revised check_online function and can confirm it works on DD-WRT and ASUS Merlin.

 

[elorimer]

I believe he has already taken that approach in his git repo

Ah, thanks. Didn't see your post.

 

[swetoast]

kewl will update tomorrow once is im sober haha dont let this deter anyone from trying my stuff

 

[swetoast]

Here is the update as promised

#!/bin/sh
# Author: Toast
# Contributers: Octopus, Tomsk, Neurophile, jimf, spalife, visortgw, Cedarhillguy, redhat27
# Testers: shooter40sw
# Supporters: lesandie
# Revision 19

blocklist=/jffs/malware-filter.list                     # Set your path here
retries=3                                               # Set number of tries here
regexp=`echo "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b"`         # Dont change this value

case $(ipset -v | grep -o "v[4,6]") in
  v6)
    MATCH_SET='--match-set'; CREATE='create'; ADD='add'; SWAP='swap'; IPHASH='hash:ip'; NETHASH='hash:net family inet'; FLUSH='flush'; DESTROY='destroy';
    lsmod | grep -q "xt_set" || \
    for module in ip_set ip_set_nethash ip_set_iphash xt_set; do
      insmod $module
    done;;
  v4)
    MATCH_SET='--set'; CREATE='--create'; ADD='--add'; SWAP='--swap'; IPHASH='iphash'; NETHASH='nethash'; FLUSH='--flush'; DESTROY='--destroy'
    lsmod | grep -q "ipt_set" || \
    for module in ip_set ip_set_nethash ip_set_iphash ipt_set; do
      insmod $module
    done;;
  *) echo "unsupported version"; exit 1 ;;
esac

check_online () {
  ping -q -c 1 google.com >/dev/null 2>&1 && get_list || exit 1
}

get_list () {
url=https://gitlab.com/swe_toast/malware-filter/raw/master/malware-filter.list
if [ ! -f $blocklist ]
then wget $url -O $blocklist; get_source; else get_source; fi
}

get_source () {
wget -q --tries=$retries --show-progress -i $blocklist -O /tmp/malware-filter-raw.part
    awk '!/(^127\.)|(^10\.)|(^172\.1[6-9]\.)|(^172\.2[0-9]\.)|(^172\.3[0-1]\.)|(^192\.168\.)/' /tmp/malware-filter-raw.part > /tmp/malware-filter-presort.part
    cat /tmp/malware-filter-presort.part | grep -oE "$regexp" | sort -u > /tmp/malware-filter-sorted.part
}

run_ipset () {
echo "adding ipset rule to firewall this will take time."
ipset -L malware-filter >/dev/null 2>&1
if [ $? -ne 0 ]; then
    if [ "$(ipset --swap malware-filter malware-filter 2>&1 | grep -E 'Unknown set|The set with the given name does not exist')" != "" ]; then
    nice -n 2 ipset $CREATE malware-filter $IPHASH
    if [ -f /opt/bin/xargs ]; then
    /opt/bin/xargs -P10 -I "PARAM" -n1 -a /tmp/malware-filter-sorted.part nice -n 2 ipset  $ADD malware-filter PARAM
    else cat /tmp/malware-filter-sorted.part | xargs -I {} ipset $ADD malware-filter {}; fi
fi
else
    nice -n 2 ipset $CREATE malware-update $IPHASH
    if [ -f /opt/bin/xargs ]; then
    /opt/bin/xargs -P10 -I "PARAM" -n1 -a /tmp/malware-filter-sorted.part nice -n 2 ipset  $ADD malware-update PARAM
    else cat /tmp/malware-filter-sorted.part | xargs -I {} ipset $ADD malware-update {}; fi
    nice -n 2 ipset $SWAP malware-update malware-filter
    nice -n 2 ipset $DESTROY malware-update
fi
iptables -L | grep malware-filter > /dev/null 2>&1
if [ $? -ne 0 ]; then
    nice -n 2 iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
else
    nice -n 2 iptables -D FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
    nice -n 2 iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
fi }

cleanup () {
logger -s -t system "Malware Filter loaded $(ipset -L malware-filter | wc -l | awk '{print $1-7}') unique ip addresses."
find /tmp -name 'malware-filter-*.part' -exec rm {} +
}

check_online
run_ipset
cleanup

exit $?

 

[shooter40sw]

Hi, I tested this one, and the difference between this and the 17 version is that now I get printed on the ssh screen this:

pset v4.5: 181.214.63.133 is already in set malware-update.
ipset v4.5: 181.215.113.46 is already in set malware-update.
ipset v4.5: 181.215.244.14 is already in set malware-update.
system: Malware Filter loaded 37930 unique ip addresses.

and I suppose that it does it the 37930 times. The script works

Here is the update as promised

#!/bin/sh
# Author: Toast
# Contributers: Octopus, Tomsk, Neurophile, jimf, spalife, visortgw, Cedarhillguy, redhat27
# Testers: shooter40sw
# Supporters: lesandie
# Revision 19

blocklist=/jffs/malware-filter.list                     # Set your path here
retries=3                                               # Set number of tries here
regexp=`echo "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b"`         # Dont change this value

case $(ipset -v | grep -o "v[4,6]") in
  v6)
    MATCH_SET='--match-set'; CREATE='create'; ADD='add'; SWAP='swap'; IPHASH='hash:ip'; NETHASH='hash:net family inet'; FLUSH='flush'; DESTROY='destroy';
    lsmod | grep -q "xt_set" || \
    for module in ip_set ip_set_nethash ip_set_iphash xt_set; do
      insmod $module
    done;;
  v4)
    MATCH_SET='--set'; CREATE='--create'; ADD='--add'; SWAP='--swap'; IPHASH='iphash'; NETHASH='nethash'; FLUSH='--flush'; DESTROY='--destroy'
    lsmod | grep -q "ipt_set" || \
    for module in ip_set ip_set_nethash ip_set_iphash ipt_set; do
      insmod $module
    done;;
  *) echo "unsupported version"; exit 1 ;;
esac

check_online () {
  ping -q -c 1 google.com >/dev/null 2>&1 && get_list || exit 1
}

get_list () {
url=https://gitlab.com/swe_toast/malware-filter/raw/master/malware-filter.list
if [ ! -f $blocklist ]
then wget $url -O $blocklist; get_source; else get_source; fi
}

get_source () {
wget -q --tries=$retries --show-progress -i $blocklist -O /tmp/malware-filter-raw.part
    awk '!/(^127\.)|(^10\.)|(^172\.1[6-9]\.)|(^172\.2[0-9]\.)|(^172\.3[0-1]\.)|(^192\.168\.)/' /tmp/malware-filter-raw.part > /tmp/malware-filter-presort.part
    cat /tmp/malware-filter-presort.part | grep -oE "$regexp" | sort -u > /tmp/malware-filter-sorted.part
}

run_ipset () {
echo "adding ipset rule to firewall this will take time."
ipset -L malware-filter >/dev/null 2>&1
if [ $? -ne 0 ]; then
    if [ "$(ipset --swap malware-filter malware-filter 2>&1 | grep -E 'Unknown set|The set with the given name does not exist')" != "" ]; then
    nice -n 2 ipset $CREATE malware-filter $IPHASH
    if [ -f /opt/bin/xargs ]; then
    /opt/bin/xargs -P10 -I "PARAM" -n1 -a /tmp/malware-filter-sorted.part nice -n 2 ipset  $ADD malware-filter PARAM
    else cat /tmp/malware-filter-sorted.part | xargs -I {} ipset $ADD malware-filter {}; fi
fi
else
    nice -n 2 ipset $CREATE malware-update $IPHASH
    if [ -f /opt/bin/xargs ]; then
    /opt/bin/xargs -P10 -I "PARAM" -n1 -a /tmp/malware-filter-sorted.part nice -n 2 ipset  $ADD malware-update PARAM
    else cat /tmp/malware-filter-sorted.part | xargs -I {} ipset $ADD malware-update {}; fi
    nice -n 2 ipset $SWAP malware-update malware-filter
    nice -n 2 ipset $DESTROY malware-update
fi
iptables -L | grep malware-filter > /dev/null 2>&1
if [ $? -ne 0 ]; then
    nice -n 2 iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
else
    nice -n 2 iptables -D FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
    nice -n 2 iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
fi }

cleanup () {
logger -s -t system "Malware Filter loaded $(ipset -L malware-filter | wc -l | awk '{print $1-7}') unique ip addresses."
find /tmp -name 'malware-filter-*.part' -exec rm {} +
}

check_online
run_ipset
cleanup

exit $?

 

[swetoast]

must have been interrupted when it ran before but its not a problem since it tries again then swaps the rules.

 

[florid]

Just a suggestion: in FORWARD chain use REJECT that prevent DROP wait out timing and drop after n-seconds.

 iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT

I do not agree with this suggestion. As this is for blocking known bad, why we need to use REJECT to actively respond to all malicious requests? REJECT only increases the router load and potentially allows more malicious requests.
In company firewall configuration, it is common sense to use the DROP action instead of REJECT.
This is not like M$ privacy filter that timeout will be a concern for web page loading. As malware filter, it is more IP based, not URL based. Timeout is not usually observed by normal users. It's the time to check your system security if you see long waiting time loading a suspicious website.

 

[swetoast]

well thats easily changed i can make a simple value for it in the settings does that sound better ?

 

[florid]

well thats easily changed i can make a simple value for it in the settings does that sound better ?

If there's option to choose REJECT or DROP, it will suit everyone.

BTW, are these the three lines to change REJECT to DROP if I want to modify it myself? I am not a script person, only working in infosec.

nice -n 2 iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
else
   nice -n 2 iptables -D FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
   nice -n 2 iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT

 

[shooter40sw]

Hi @swetoast a question, apart of the discussion of whether reject or drop is the correct option, maybe drop is the correct one... but Im looking at the command iptables -L -v -n and Im looking and rejected inputs...

 pkts bytes target     prot opt in     out     source               destination
    5   915 REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           set malware-filter src,dst reject-with icmp-port-unreachable
7865K 2324M ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           state RELATED,ESTABLISHED

So great its working, ... is there a command that I cant execute that can tell me if the rejected package is from what source IP address? I want to know if this is coming from an internal IP or external

regards, great scripts!

 

[swetoast]

@florid next revision will have it, and yes those are the 3 lines or you could grab a new version from my git.
@shooter40sw yes you have to create an additional rule that logs rejected packages.

 

[octopus]

I do not agree with this suggestion. As this is for blocking known bad, why we need to use REJECT to actively respond to all malicious requests? REJECT only increases the router load and potentially allows more malicious requests.
In company firewall configuration, it is common sense to use the DROP action instead of REJECT.
This is not like M$ privacy filter that timeout will be a concern for web page loading. As malware filter, it is more IP based, not URL based. Timeout is not usually observed by normal users. It's the time to check your system security if you see long waiting time loading a suspicious website.

Everyone can do what they want with their skills and experience. Feel free to do what suits the individual.