Malware Filter / bad host IPSET

[swetoast]

nice is just so that other applications have priority if there is a load cycle

 

[Nutz2U2]

Hi to Everyone,

Wish to thank ALL the boys and girls who made this script workable.

Jan 12 15:02:55 system: Malware Filter loaded 35632 unique ip addresses.

Trying the "Privacy Filter" next .

 

[radionerd]

I am trying to install the latest v7 malware script. I am not great with unix commands, but am very persistent. My head hurts, Little help?
I'm Just trying to get into the cd /opt
Please don't tear my head off, I know there has to be a simple command to allow

admin@HOMES:/# cd /opt
-sh: cd: can't cd to /opt

I'm blocked by forum filters again

 

[Cedarhillguy]

I am trying to install the latest v7 malware script. I am not great with unix commands, but am very persistent. My head hurts, Little help?
I'm Just trying to get into the cd /opt
Please don't tear my head off, I know there has to be a simple command to allow

admin@HOMES:/# cd /opt
-sh: cd: can't cd to /opt

The /opt directory probably doesn't exist. Try cd / to get to the root directory and then ls -trl which will list the directories and files. You can place the script in another directory such as /jffs/scripts, but if so.. would need to modify the path in the script. If you want to make a directory the command is mkdir directoryname

 

[radionerd]

The /opt directory probably doesn't exist. Try cd / to get to the root directory and then ls -trl which will list the directories and files. You can place the script in another directory such as /jffs/scripts, but if so.. would need to modify the path in the script. If you want to make a directory the command is mkdir directoryname

Thanks for the response, I tried to attach the code but got blocked by the forum's filters. opt shows up. Trying to post a jpg of it since forum filters are blocking me.

 

[radionerd]

My problem was /opt was simlinked to a deleted ->tmp/opt.

I'm rocking the script, Thanks guys!

Jan 17 21:58:13 kernel: net/ipv4/netfilter/ip_set_iphash.c: iphash_retry: rehashing of set malware-update triggered: hashsize grows from 88573 to 132859
Jan 17 21:59:24 system: Malware Filter loaded 33069 unique ip addresses.

 

[swetoast]

the path /opt can be replaced with whatever your heart desires so your not limited to /opt

 

[radionerd]

Thanks for working on the script!

I have been looking at blocked IP's from my logs, then in ipset -Test to see if they are in the malware-filter block list, nice!

ipset v4.5: -T requires setname and IP
Try `ipset -H' or 'ipset --help' for more information.
admin@HOMES:/jffs/scripts# ipset -T malware-filter 74.82.47.14
74.82.47.14 is in set malware-filter.
admin@HOMES:/jffs/scripts# ipset -T malware-filter 2.178.244.64
2.178.244.64 is NOT in set malware-filter.
admin@HOMES:/jffs/scripts#

/usr/sbin/iptables -L FORWARD -v -n
Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    1    40 REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           set malware-filter src,dst reject-with icmp-port-unreachable
1527K 2017M ipttolan   all  --  *      br0     0.0.0.0/0            0.0.0.0/0   
 777K   70M iptfromlan  all  --  br0    *       0.0.0.0/0            0.0.0.0/0 
    0     0 ACCEPT     udp  --  *      *       0.0.0.0/0            224.0.0.0/4
2287K 2085M ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           state RELATED,ESTABLISHED
    0     0 logdrop    all  --  !br0   eth0    0.0.0.0/0            0.0.0.0/0   
    0     0 logdrop    all  --  eth0   *       0.0.0.0/0            0.0.0.0/0           state INVALID
    0     0 ACCEPT     all  --  br0    br0     0.0.0.0/0            0.0.0.0/0   
  630 51065 SECURITY   all  --  eth0   *       0.0.0.0/0            0.0.0.0/0   
16782 1512K NSFW       all  --  *      *       0.0.0.0/0            0.0.0.0/0   
  614 50105 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           ctstate DNAT
16168 1462K ACCEPT     all  --  br0    *       0.0.0.0/0            0.0.0.0/0   
admin@HOMES:/jffs/scripts#

 

[Goobi]

Hello everyone,

When I try and run the latest script off the wiki (v7), it appears to just hang after the wgets. This is what I see:

superuser@Kepler:/jffs/scripts# /opt/bin/malware-block
/opt/var/cache/malware-filter/malware-list.pre 100%[============================================================================================================>] 160.20K --.-KB/s in 0.1s
/opt/var/cache/malware-filter/malware-list.pre [ <=> ] 2.12K --.-KB/s in 0s
/opt/var/cache/malware-filter/malware-list.pre [ <=> ] 10.55K --.-KB/s in 0.1s
/opt/var/cache/malware-filter/malware-list.pre 100%[============================================================================================================>] 16.99K --.-KB/s in 0.05s
/opt/var/cache/malware-filter/malware-list.pre 100%[============================================================================================================>] 253.30K --.-KB/s in 0.1s

My /opt/var/cache/malware-filter directory has the following files:

superuser@Kepler:/tmp/home/root# ls /opt/var/cache/malware-filter
malware-filter.list malware-list.pre

Looking at the script, this is what it appears to be "hung" at:

cat $path/malware-list.pre | grep -oE "$regexp" | sort -u >$path/malware-filter.txt

I am running 380.64 on a 68U. I got two additional IPSETs running (BlockedCountries & MicrosoftSpyServers) with no problems so I think ipset itself is fine, but for some reason, the malware script seems to simply hang after wget. I did not see any errors in the log. Did I maybe typo something in the script? Here is the unaltered lines from the script:

get_list () {
mkdir -p $path
wget -q --show-progress -i $path/malware-filter.list -O $path/malware-list.pre
cat $path/malware-list.pre | grep -oE "$regexp" | sort -u >$path/malware-filter.txt
}

As of now, it has been sitting there for about 20 minutes and top does not show any relevant activity. Any suggestions to point me in the right direction would be greatly appreciated.

 

[Nutz2U2]

Hi,
I believe it appears to be hung. I run similar scripts in my services-start like:-
sh /opt/bin/privacy-block
sh /opt/bin/malware-block
sh /jffs/scripts/p2p

followed by:

# Prepare IPset Lists and stop w10 snooping
cru a privacy-filter "5 0 * * * /opt/bin/privacy-block"
# Prepare IPset Lists and Malware Blacklists
cru a malware-filter "10 0 * * * /opt/bin/malware-block"
# Prepare IPset List and P2Pguardian
cru a P2P-filter "15 0 * * * /jffs/scripts/p2p"
# Prepare IPset List and Country Block
cru a country-filter "45 0 * * * /jffs/scripts/country"

Via shell i do iptables -L and none of the filters show - neither do any of the cronjobs in cru -l.
Wait a few minutes, privacy-block has completed, then malware-block and finally p2p script.
They then show up 1 by 1 in iptables -L.
After the scripts are completed, services-start completes and the cronjobs show in cru l.

It just takes some patience. At night-time they run without any interaction from me either.

But it did take a while to get it right .

 

[swetoast]

Hello everyone,

When I try and run the latest script off the wiki (v7), it appears to just hang after the wgets. This is what I see:

As of now, it has been sitting there for about 20 minutes and top does not show any relevant activity. Any suggestions to point me in the right direction would be greatly appreciated.

sometimes the host of these files have timeouts it happned to me too cant control it the only thing you can do is to try again but im gonna investigate if there is a possability to have timeout via wget if that happens so it skips lists.

 

[Goobi]

sometimes the host of these files have timeouts it happned to me too cant control it the only thing you can do is to try again but im gonna investigate if there is a possability to have timeout via wget if that happens so it skips lists.

Thanks for pointing me in the right direction. It appears that the following hosts on the list are unavailable at the moment based on ping:

http://cinsscore.com/list/ci-badguys.txt
http://sanyalnet-cloud-vps.freeddns.org/mirai-ips.txt

I commented those entries out and the script worked.

 

[swetoast]

So timeout value added so if the lists arent available then it skips.

#!/bin/sh
# Author: Toast
# Contributers: Octopus, Tomsk, Neurophile
# Testers: shooter40sw
# Revision 8

path=/opt/var/cache/malware-filter                      # Set your path here
timeout=60                                              # Set your timeout value here (seconds)
regexp=`echo "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b"`         # Dont change this value

case $(ipset -v | grep -oE "ipset v[0-9]") in
*v6) # Value for ARM Routers

    MATCH_SET='--match-set'
    HASH='hash:ip'
    SYNTAX='add'
    SWAPPED='swap'
    DESTROYED='destroy'
    OPTIONAL='family inet hashsize 2048 maxelem 65536'

     ipsetv=6
     lsmod | grep "xt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_hash_net ip_set_hash_ip xt_set
     do
          insmod $module
     done
;;

*v4) # Value for Mips Routers

    MATCH_SET='--set'
    HASH='iphash'
    SYNTAX='-q -A'
    SWAPPED='-W'
    DESTROYED='--destroy'
    OPTIONAL=''  

    ipsetv=4
     lsmod | grep "ipt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_nethash ip_set_iphash ipt_set
     do
          insmod $module
     done
;;
esac

get_list () {
        mkdir -p $path
        wget -q --timeout=$timeout --show-progress -i $path/malware-filter.list -O $path/malware-list.pre
        cat $path/malware-list.pre | grep -oE "$regexp" | sort -u >$path/malware-filter.txt
 }

run_ipset () {

get_list

echo "adding ipset rule to firewall this will take time."

ipset -L malware-filter >/dev/null 2>&1
if [ $? -ne 0 ]; then
    if [ "$(ipset --swap malware-filter malware-filter 2>&1 | grep -E 'Unknown set|The set with the given name does not exist')" != "" ]; then
    nice -n 2 ipset -N malware-filter $HASH $OPTIONAL
    for i in `cat $path/malware-filter.txt`; do nice -n 2 ipset $SYNTAX malware-filter $i ; done
fi
else
    nice -n 2 ipset -N malware-update $HASH $OPTIONAL
    for i in `cat $path/malware-filter.txt`; do nice ipset $SYNTAX malware-update $i ; done
    nice -n 2 ipset $SWAPPED malware-update malware-filter
    nice -n 2 ipset $DESTROYED malware-update
fi

iptables -L | grep malware-filter > /dev/null 2>&1
if [ $? -ne 0 ]; then
    nice -n 2 iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
else
    nice -n 2 iptables -D FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
    nice -n 2 iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
fi
}

run_ipset

logger -s -t system "Malware Filter loaded $(cat $path/malware-filter.txt | wc -l) unique ip addresses."
exit $?

 

[swetoast]

another good lists

https://rules.emergingthreats.net/fwrules/emerging-Block-IPs.txt
https://lists.blocklist.de/lists/ssh.txt
https://lists.blocklist.de/lists/bots.txt
http://www.abuseat.org/iotcc.txt

just remember the more you add to the list the longer it will take to make the rules in the firewall.

 

[jimf]

@swetoast, I tried adding the timeout independently. I also added --tries=3. Maybe I'm impatient, but the timeout by itself didn't seem to be working.

 

[swetoast]

cool can add that instead tnx

 

[swetoast]

#!/bin/sh
# Author: Toast
# Contributers: Octopus, Tomsk, Neurophile, jimf
# Testers: shooter40sw
# Revision 9

path=/opt/var/cache/malware-filter                      # Set your path here
retries=3                                               # Set number of tries here
timeout=60                                              # Set timeot value here
regexp=`echo "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b"`         # Dont change this value

case $(ipset -v | grep -oE "ipset v[0-9]") in
*v6) # Value for ARM Routers

    MATCH_SET='--match-set'
    HASH='hash:ip'
    SYNTAX='add'
    SWAPPED='swap'
    DESTROYED='destroy'
    OPTIONAL='family inet hashsize 2048 maxelem 65536'

     ipsetv=6
     lsmod | grep "xt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_hash_net ip_set_hash_ip xt_set
     do
          insmod $module
     done
;;

*v4) # Value for Mips Routers

    MATCH_SET='--set'
    HASH='iphash'
    SYNTAX='-q -A'
    SWAPPED='-W'
    DESTROYED='--destroy'
    OPTIONAL=''

    ipsetv=4
     lsmod | grep "ipt_set" > /dev/null 2>&1 || \
     for module in ip_set ip_set_nethash ip_set_iphash ipt_set
     do
          insmod $module
     done
;;
esac

get_list () {
        mkdir -p $path
        wget -q --tries=$retries  --timeout=$timeout --show-progress -i $path/malware-filter.list -O $path/malware-list.pre
        cat $path/malware-list.pre | grep -oE "$regexp" | sort -u >$path/malware-filter.txt
 }

run_ipset () {

get_list

echo "adding ipset rule to firewall this will take time."

ipset -L malware-filter >/dev/null 2>&1
if [ $? -ne 0 ]; then
    if [ "$(ipset --swap malware-filter malware-filter 2>&1 | grep -E 'Unknown set|The set with the given name does not exist')" != "" ]; then
    nice -n 2 ipset -N malware-filter $HASH $OPTIONAL
    for i in `cat $path/malware-filter.txt`; do nice -n 2 ipset $SYNTAX malware-filter $i ; done
fi
else
    nice -n 2 ipset -N malware-update $HASH $OPTIONAL
    for i in `cat $path/malware-filter.txt`; do nice ipset $SYNTAX malware-update $i ; done
    nice -n 2 ipset $SWAPPED malware-update malware-filter
    nice -n 2 ipset $DESTROYED malware-update
fi

iptables -L | grep malware-filter > /dev/null 2>&1
if [ $? -ne 0 ]; then
    nice -n 2 iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
else
    nice -n 2 iptables -D FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
    nice -n 2 iptables -I FORWARD -m set $MATCH_SET malware-filter src,dst -j REJECT
fi
}

run_ipset

logger -s -t system "Malware Filter loaded $(cat $path/malware-filter.txt | wc -l) unique ip addresses."
exit $?

 

[jimf]

I would do both. I'm not sure what the default timeout is, but remember it's setting 3 values (DNS, connect, and read). This is what I did:
wget -q --tries=3 --timeout=15 ...

Not sure what the default timeouts are, but this way I know what to expect. wget leaves it up to the system.

 

[swetoast]

changed the script above, tnx again

 

[icornish]

Hi,

I have implemented this script, and seems to work OK. Is it possible to include the Country & TorNode blocking that is covered in the first script on the wiki page by adding the appropriate URL's to the list file?