Yet another malware block script using ipset (v4 and v6)

[drg]

Not seeing the blacklist file @ /jffs/ipset_lists/ya-malware-block.blacks

You mention this is optional. Do we add this file manually?

Nice script btw.

Yes, you have to create it manually.

 

[Csection]

Hi, everyone!
I ran the newest version of this script.
The first run seemed to go normal(65xxx blocks added to firewall etc.).
I then made some changes to the whites file to be sure I did not get locked out of my router and some pet urls I use.
Then I ran the script again and this is what I got:
Firewall: ./ya-malware-block.sh: Adding ya-malware-block rules to firewall...
Jun 5 08:10:20 Firewall: ./ya-malware-block.sh: Loaded sets YAMalwareBlock1IP (0) and YAMalwareBlockCIDR (0) in 4 seconds
Jun 5 08:10:27 kernel
This was of course run from the command line. The output is from syslog.
It seems to not be reloading firewall blocks.

 

[Jack Yaz]

Hi, everyone!
I ran the newest version of this script.
The first run seemed to go normal(65xxx blocks added to firewall etc.).
I then made some changes to the whites file to be sure I did not get locked out of my router and some pet urls I use.
Then I ran the script again and this is what I got:
Firewall: ./ya-malware-block.sh: Adding ya-malware-block rules to firewall...
Jun 5 08:10:20 Firewall: ./ya-malware-block.sh: Loaded sets YAMalwareBlock1IP (0) and YAMalwareBlockCIDR (0) in 4 seconds
Jun 5 08:10:27 kernel
It seems to not be reloading firewall blocks.

If you remove your changes to whites and re-run, what happens?

 

[Csection]

If you remove your changes to whites and re-run, what happens?

Yep!
It runs ok!
Jun 5 08:51:16 Firewall: ./ya-malware-block.sh: Adding ya-malware-block rules to firewall...
Jun 5 08:51:18 kernel: ACCEPT IN=br0 OUT=br0 SRC=192.168.1.8 DST=192.168.1.124 LEN=52 TOS=0x00 PREC=0x00 TTL=127 ID=31974 DF PROTO=TCP SPT=51149 DPT=9100 SEQ=235000509 ACK=0 WINDOW=8192 RES=0x00 SYN URGP=0 OPT (020405B40103030801010402) MARK=0x1
Jun 5 08:51:24 kernel: ACCEPT IN=br0 OUT=br0 SRC=192.168.1.8 DST=192.168.1.124 LEN=48 TOS=0x00 PREC=0x00 TTL=127 ID=31977 DF PROTO=TCP SPT=51149 DPT=9100 SEQ=235000509 ACK=0 WINDOW=8192 RES=0x00 SYN URGP=0 OPT (020405B401010402) MARK=0x1
Jun 5 08:51:25 Firewall: ./ya-malware-block.sh: Loaded sets YAMalwareBlock1IP (65536) YAMalwareBlock2IP (4916) and YAMalwareBlockCIDR (8856) in 9 seconds

What's up with that?
I only added about 10 ip addresses.

 

[Jack Yaz]

What tool did you use to add them?

It would be worth running the whites file through dos2unix to ensure any line endings are of the right format, which can cause scripts to not run correctly.

 

[Csection]

What tool did you use to add them?

It would be worth running the whites file through dos2unix to ensure any line endings are of the right format, which can cause scripts to not run correctly.

I used the editor in Mobaxterm command line, but running dos2unix did not fix it.

 

[Jack Yaz]

I used the editor in Mobaxterm command line, but running dos2unix did not fix it.

Are you able to show us the IPs you added?

 

[Csection]

Are you able to show us the IPs you added?

Here is a backup file(I will delete some of the personal ip's for privacy purposes):
^0\.
^10\.
^127\.
^169\.254\.
^172\.1[6-9]\.
^172\.2[0-9]\.
^172\.3[0-1]\.
^192\.168\.
213.230.210.230
192.124.249.10
192.168.1.4
192.168.1.8
192.168.1.14
192.168.1.17
192.168.1.19
192.168.1.11
192.168.1.13
192.168.1.12
192.168.1.20
192.168.1.22
98.xx.xxx.xxx
xxx.xx.xx.xx
xxx.xx.xx.xx
8.8.4.4
8.8.8.8
98.136.0.0
68.67.73.31
68.67.73.1

 

[Jack Yaz]

Here is a backup file(I will delete some of the personal ip's for privacy purposes):
^0\.
^10\.
^127\.
^169\.254\.
^172\.1[6-9]\.
^172\.2[0-9]\.
^172\.3[0-1]\.
^192\.168\.
213.230.210.230
192.124.249.10
192.168.1.4
192.168.1.8
192.168.1.14
192.168.1.17
192.168.1.19
192.168.1.11
192.168.1.13
192.168.1.12
192.168.1.20
192.168.1.22
98.xx.xxx.xxx
xxx.xx.xx.xx
xxx.xx.xx.xx
8.8.4.4
8.8.8.8
98.136.0.0
68.67.73.31
68.67.73.1

I think your 192.168. addresses can be removed, as the regex at the top will include all of them. I don't know if that would cause the script to fail however, so try removing those. I would also advise leaving a blank line at the end, as I've seen some scripts not play nice.

 

[Csection]

I think your 192.168. addresses can be removed, as the regex at the top will include all of them. I don't know if that would cause the script to fail however, so try removing those. I would also advise leaving a blank line at the end, as I've seen some scripts not play nice.

Ok!
I deleted the old whites file and let it d/l a new one.
The only reason I put those in there was because the old script locked me out of the the internet and I could not connect. I just wanted to make sure it didn't happen again.
Thanks for all the help!

 

[HRearden]

Just checking in: This thread's been quiet lately. Work kept me busy as well.

Does anyone have any issues with the new version 2.2?

It no longer uses wget, and uses curl. @HRearden does it work smoothly on tomato firmware?

Was the timing display (for terminal run) useful at all?

I am still having trouble on tomato by shibby. It is loading 0 sets. The problem is now with curl. The version does not do https.
I am running it both on merlin and tomato by shibby. On merlin it seems fine, although I have not updated to the latest version.

root@unknown:/jffs/scripts# curl -k https://raw.githubusercontent.com/shounak-de/misc-scripts/master/jffs/ipset_lists/ya-malware-block.urls -o /jffs/ipset_lists/ya-malware-block.urls
curl: (1) Protocol "https" not supported or disabled in libcurl
root@unknown:/jffs/scripts# which curl
/usr/sbin/curl
root@unknown:/jffs/scripts#

 

[drg]

Hi Guys,

Is there any way to limit the syslog to show only Inbound Accepted Traffic ? I know the original version was in this way

Thank you!

 

[redhat27]

You mention this is optional. Do we add this file manually?

Like @drg mentioned, yes, but only if you'd need to add to the default blacklist already provided by the default lists.

What's up with that?

I think what @Jack Yaz said about dos crlf characters is spot on. Try using dos2unix utility on that file. Just saw your mention on dos2unix not fixing that. All I can say is there may be some characters inadvertently messing things up. Try adding your IPs again with directly editing the file with vi or nano. Note that you do not need to add any internal IPs or your own external IP. The default should work in most cases. You only add if an IP you'd want to access in present in the YAMalwareBlock?IP (YAMalwareBlock1IP, YAMalwareBlock2IP, or YAMalwareBlock3IP). You can check if the IP you are trying to whitelist is in those lists by using the shell function.

I think your 192.168. addresses can be removed, as the regex at the top will include all of them.

Again, absolutely correct

I am still having trouble on tomato by shibby.

Okay. You mentioned wget without --no-check-certificate flag works good with https, right @HRearden ?

 

[redhat27]

Hi Guys,

Is there any way to limit the syslog to show only Inbound Accepted Traffic ? I know the original version was in this way

Thank you!

This script (current or prior versions) does not change any inbound logging.
iptables rule have the jump target "DROP" (dropped packets are no longer logged). ACCEPTed packet logging is enabled in the router web UI

But if you have DROPped packet logging as well, you'll only see the DROPs on the packets not blocked by the script.

However, you can see the blocked statistics using a simple command (see the blockstats alias here)

 

[redhat27]

I am still having trouble on tomato by shibby. It is loading 0 sets. The problem is now with curl. The version does not do https.
I am running it both on merlin and tomato by shibby. On merlin it seems fine, although I have not updated to the latest version.

root@unknown:/jffs/scripts# curl -k https://raw.githubusercontent.com/shounak-de/misc-scripts/master/jffs/ipset_lists/ya-malware-block.urls -o /jffs/ipset_lists/ya-malware-block.urls
curl: (1) Protocol "https" not supported or disabled in libcurl
root@unknown:/jffs/scripts# which curl
/usr/sbin/curl
root@unknown:/jffs/scripts#

Try this variant and let me know please (on tomato firmware):

wget -O /jffs/scripts/ya-malware-block.sh https://raw.githubusercontent.com/shounak-de/misc-scripts/master/ya-malware-block-tomato.sh

 

[Csection]

I think what @Jack Yaz said about dos crlf characters is spot on. Try using dos2unix utility on that file. Just saw your mention on dos2unix not fixing that. All I can say is there may be some characters inadvertently messing things up. Try adding your IPs again with directly editing the file with vi or nano. Note that you do not need to add any internal IPs or your own external IP. The default should work in most cases. You only add if an IP you'd want to access in present in the YAMalwareBlock?IP (YAMalwareBlock1IP, YAMalwareBlock2IP, or YAMalwareBlock3IP). You can check if the IP you are trying to whitelist is in those lists by using the shell function.

↑
I think your 192.168. addresses can be removed, as the regex at the top will include all of them.
Click to expand...
Again, absolutely correct


I tried to enter a few ip's manually using nano with Putty this time, but this did not work either.
I can't seem to enter anything into the "Whites" file.
When I do the output is zero's.
When I delete the old "Whites" file and d/l a new one. The script runs fine.
I am running this on an RT-3100 with Merlin ver. 380.66_4. Not sure if it is unique to that or not, but I'm at a loss.

 

[redhat27]

I can't seem to enter anything into the "Whites" file.

Try this:

echo "1.2.3.4" >> /jffs/ipset_lists/ya-malware-block.whites

Where you'd replace 1.2.3.4 with the IP you are trying to add to whitelist file

Also, is the IP you're trying to add found on the MatchIP shell function?

 

[Csection]

Try this:

echo "1.2.3.4" >> /jffs/ipset_lists/ya-malware-block.whites

Where you'd replace 1.2.3.4 with the IP you are trying to add to whitelist file
Does the "echo" go inside the "Whites" file?
Cause when I ran it from the command line. It did not produce any output.


Also, is the IP you're trying to add found on the MatchIP shell function?

I don't have Entware installed.

 

[redhat27]

The IP you are trying to add to the whites file should only be added if it is blocked by the YAMalwareBlock1IP or YAMalwareBloc2IP or YAMalwareBlock3IP

You can test the IP you are trying to add, first by checking if it is there:

ipset test YAMalwareBlock1IP 1.2.3.4
ipset test YAMalwareBlock2IP 1.2.3.4
ipset test YAMalwareBlock3IP 1.2.3.4 (you may not have a YAMalwareBlock3IP unless you are using Level4 FireHOL list)

replace 1.2.3.4 with the IP you are trying to add

 

[Csection]

The IP you are trying to add to the whites file should only be added if it is blocked by the YAMalwareBlock1IP or YAMalwareBloc2IP or YAMalwareBlock3IP

You can test the IP you are trying to add, first by checking if it is there:

ipset test YAMalwareBlock1IP 1.2.3.4
ipset test YAMalwareBlock2IP 1.2.3.4
ipset test YAMalwareBlock3IP 1.2.3.4 (you may not have a YAMalwareBlock3IP unless you are using Level4 FireHOL list)

replace 1.2.3.4 with the IP you are trying to add

Ok!
I misunderstood this feature. I thought it was for safe-guarding against future blockage.
Sorry for all the hassle. Course that is what makes this forum so much fun. Right?