Yet another malware block script using ipset (v4 and v6)

[redhat27]

am I missing something or not set up right? this is what I get when I run ya-malware-block.sh -Loaded sets YAMalwareBlock1IP (3142), YAMalwareBlock2IP (1) and YAMalwareBlockCIDR (1) in 3 seconds

does it create files somewhere ? as I see none and how can I tell if it is actually blocking things?
thanks

Can you do:

rm /jffs/ipset_lists/ya-malware-block.url_list
wget --no-check-certificate -O /jffs/scripts/ya-malware-block.sh https://raw.githubusercontent.com/shounak-de/misc-scripts/master/ya-malware-block.sh
/jffs/scripts/ya-malware-block.sh

Also, can you do this as a test:

wget --no-check-certificate -q -O- "https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/firehol_level3.netset" | wc -l

and see if you get a number like 38433 or something big

 

[redhat27]

does it create files somewhere ? as I see none and how can I tell if it is actually blocking things?

It does create a /tmp file, but it gets deleted before the script exists. You'd be able to see the blocked packets with:
iptables -vL -t raw

 

[shooter40sw]

In this moment My internet is giving me issues but the new script took
YAMalwareBlock1IP (65535), YAMalwareBlock2IP (41787) and YAMalwareBlockCIDR (5154) in 1646 seconds
a very long time

 

[redhat27]

It should not take that long there is nothing in the new script that is different other than the iptables rules. Is your router busy at the time you ran it.. checking cpu load may help

 

[shooter40sw]

Well looks like something is wrong with the new script for me, dont know what, the previous script ran in 30 to 40 seconds this one has 4 minutes and counting

this is the top command a few seconds ago

Mem: 103616K used, 136056K free, 0K shrd, 8296K buff, 36096K cached
CPU:  0.5% usr  2.5% sys  0.7% nic 93.6% idle  0.0% io  0.0% irq  2.3% sirq

And in this time, already 5 minutes the router CPU does not go pass 5%


EDIT: Im testing deleting the quiet option on the wget so I can see how much its taking to download the files, I think there is my issue... Ill keep you posted

 

[redhat27]

If you have this script in the services-start, can you try a reboot? I think it is blocked by another process somewhere. The timing should show up in the syslog

 

[shooter40sw]

If you have this script in the services-start, can you try a reboot? I think it is blocked by another process somewhere. The timing should show up in the syslog

Its the download times its extremely slow , Im having issues with my ISP they are downloading at B/s it stuck downloading the raw.githubusercontent.com/firehol/blocklist-ipsets/master/cybercrime.ipset
It says the eta 3m 52s and its just stuck there without doing anything, I rebooted the router and tried again, because the other script was so fast...

 

[jack901]

I also run adblocker and I had to kill it to get :
wget --no-check-certificate -q -O- "https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/fi
rehol_level3.netset" | wc -l
38427

I did the rm and the wget and restarted adblocker and got this on when the adblocker was also running:
wget --no-check-certificate -q -O- "https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/f
i
> rehol_level3.netset" | wc -l
0
and on the test to check I get the following:
Chain PREROUTING (policy ACCEPT 4631K packets, 4431M bytes)
pkts bytes target prot opt in out source destination

Chain OUTPUT (policy ACCEPT 134K packets, 69M bytes)
pkts bytes target prot opt in out source destination

which I guess may take a while to get a hit.
Is adblocker not compatible as it sends it to pixelserv when that is running when I do the wget?

thanks

Can you do:

rm /jffs/ipset_lists/ya-malware-block.url_list
wget --no-check-certificate -O /jffs/scripts/ya-malware-block.sh https://raw.githubusercontent.com/shounak-de/misc-scripts/master/ya-malware-block.sh
/jffs/scripts/ya-malware-block.sh

Also, can you do this as a test:

wget --no-check-certificate -q -O- "https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/firehol_level3.netset" | wc -l

and see if you get a number like 38433 or something big

 

[skeal]

Updated by deleting .sh and .url_list,
ran the wget, left the cru as is, restarted the script, works like a charm. Excellent results in the log as well.

 

[redhat27]

Those who are stuck on wget, can you do these:

nslookup raw.githubusercontent.com
ping raw.githubusercontent.com

For me there are no issues with the updated script:

May 12 15:29:29 Firewall: /jffs/scripts/ya-malware-block.sh: Adding malware-block rules to firewall...
May 12 15:30:01 Firewall: /jffs/scripts/ya-malware-block.sh: Loaded sets YAMalwareBlock1IP (65535), YAMalwareBlock2IP (41547) and YAMalwareBlockCIDR (5147) in 32 seconds

 

[redhat27]

Well looks like something is wrong with the new script for me

You can still get the old version of the script from git history if you want to revert the changes and see if that makes any difference

 

[redhat27]

@shooter40sw and @jack901 can you do the
nslookup raw.githubusercontent.com
and send me the output?

 

[shooter40sw]

@shooter40sw and @jack901 can you do the
nslookup raw.githubusercontent.com
and send me the output?

It does respond, but it just stays stuck there "downloading" at 205 B/s with an eta 10m

 raw.githubusercontent.com
Server:    127.0.0.1
Address 1: 127.0.0.1 localhost.localdomain

Name:      raw.githubusercontent.com
Address 1: 151.101.0.133
Address 2: 151.101.64.133
Address 3: 151.101.128.133
Address 4: 151.101.192.133
:/tmp/home/root# ping raw.githubusercontent.com
PING raw.githubusercontent.com (151.101.0.133): 56 data bytes
64 bytes from 151.101.0.133: seq=2 ttl=57 time=111.679 ms
64 bytes from 151.101.0.133: seq=3 ttl=57 time=111.671 ms
64 bytes from 151.101.0.133: seq=5 ttl=57 time=113.192 ms
64 bytes from 151.101.0.133: seq=8 ttl=57 time=172.787 ms

 https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/firehol_level3.netset
Reusing existing connection to raw.githubusercontent.com:443.
HTTP request sent, awaiting response... 200 OK
Length: 565615 (552K) [text/plain]
Saving to: 'STDOUT'

-                                                75%[=============================================================================>                         ] 418.38K   205 B/s   eta 10m 19s

 

[redhat27]

My pings are a lot faster:

PING raw.githubusercontent.com (151.101.64.133): 56 data bytes
64 bytes from 151.101.64.133: seq=0 ttl=59 time=26.782 ms
64 bytes from 151.101.64.133: seq=1 ttl=59 time=22.697 ms
64 bytes from 151.101.64.133: seq=2 ttl=59 time=22.424 ms
64 bytes from 151.101.64.133: seq=3 ttl=59 time=21.956 ms

maybe a temporary network issue in your ISP?

 

[shooter40sw]

My pings are a lot faster:

PING raw.githubusercontent.com (151.101.64.133): 56 data bytes
64 bytes from 151.101.64.133: seq=0 ttl=59 time=26.782 ms
64 bytes from 151.101.64.133: seq=1 ttl=59 time=22.697 ms
64 bytes from 151.101.64.133: seq=2 ttl=59 time=22.424 ms
64 bytes from 151.101.64.133: seq=3 ttl=59 time=21.956 ms

maybe a temporary network issue in your ISP?

Yes my ISP at this moment is very slow.... I reported the issue a while ago but its kind of worse .. I hit enter on the offending github list,.... it started again and now its stuck in the github ssh list, so it has nothing to do with your script, it must be issues on My end, thanks for the hard work to make these scripts work fast and effective!

 

[jack901]

I just did it and for some reason it is going to pixelserv. so it is not going out. any idea how to fix with out killing adblocker?
thanks

@shooter40sw and @jack901 can you do the
nslookup raw.githubusercontent.com
and send me the output?

 

[jack901]

guess I am just going to remove it all and start over.

 

[redhat27]

Which adblocking solution are you using?
You should be able to whitelist raw.githubusercontent.com.

 

[jack901]

the ab-solution (sorry for the incorrect name) from john I believe down a few posts. I went to add it to the whitelist and it said it was not blocked.
so I uninstalled and deleted everything and in the process of reinstalling.
will see how this goes.
thanks for the help

Which adblocking solution are you using?
You should be able to whitelist raw.githubusercontent.com.

 

[jack901]

ok reinstalled first yours then ab-solution and it seems to work as I can now ping raw.githubusercontent.com and it goes thru.
so something got messed somewhere and now seems working.
thanks