Yet another malware block script using ipset (v4 and v6)

[redhat27]

Actually, I made a mistake too. Everyone who is using this latest version, can you delete your ya-malware-block.whites file so that the script re-downloads it? I corrected some entries there.

 

[shooter40sw]

Actually, I made a mistake too. Everyone who is using this latest version, can you delete your ya-malware-block.whites file so that the script re-downloads it? I corrected some entries there.

worked for me!

 

[skeal]

Script works great with ab-solution now!! Nice update adding the whitelist. Keep up the excellent work much appreciated by all us!!

 

[HRearden]

If anyone is interested I think I got the script to run in tomato by shibby without entware. The primary trouble is that the wget in busybox is not fully enabled and the -i didn't work. There is probably a much more elegant solution, but I hardcoded the urls into the script and used -O to append all the downloads to a temp file which I then piped into the rest of the script. I think the limited wget also did not like the --no-check-certificate

If anyone can tell me how to get a crippled busybox wget or curl to download a url list from a file, I would be gratetful. It is also missing xargs.

wget https://raw.githubusercontent.com/shounak-de/misc-scripts/master/telemetry_and_scanners.txt -O - > /tmp/ya-mal.txt
wget https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/firehol_level1.netset -O - >> /tmp/ya-mal.txt
wget https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/firehol_level2.netset -O - >> /tmp/ya-mal.txt
wget https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/firehol_level3.netset -O - >> /tmp/ya-mal.txt
cat /tmp/ya-mal.txt | nice -n 15 sed  (rest of the statement)

 

[redhat27]

@HRearden This script should be entware-agnostic on asuswrt, but wget is not from busybox:

admin@RT-AC66R-D700:/tmp/home/root# which wget
/usr/sbin/wget

You can probably just alter the script to say:

(while read -r url; do wget -qO- $url; done < $URLList) | nice -n 15 sed  (rest of the statement)

instead of cat /tmp/ya-mal.txt

I got the script to run in tomato by shibby without entware

If this works on other firmwares with other (limited) wget, I can make this change on the script itself, so that you'd not have to make this change on every update.

 

[redhat27]

Script works great with ab-solution now!! Nice update adding the whitelist. Keep up the excellent work much appreciated by all us!!

If this script is helpful, maybe it can be mentioned in the ipset scripts wiki. I know @shooter40sw had mentioned that I put it there, but I'm not sure if I should do so: There are other ipset related scripts in the forums, and it may seem improper to list just this one. If @Adamm @Martineau @spalife (I'm certain there would be others) also mention their work there, it would make sense.

@RMerlin any ideas?

 

[RMerlin]

I don't like the idea of having existing scripts listed under "Using ipset". It would be like listing a bunch of programs using VisualC++ under "How to use the VC++ APIs". These applications should be listed elsewhere on the wiki, not on the page explaining how to use ipset.

 

[redhat27]

I don't like the idea of having existing scripts listed under "Using ipset". It would be like listing a bunch of programs using VisualC++ under "How to use the VC++ APIs". These applications should be listed elsewhere on the wiki, not on the page explaining how to use ipset.

Makes sense. I do not think that page explains how to use ipsets very much now There are 3 bullet points and a link to the man page, and that's about it.
Would you like me to rename the page to something along the lines of "Firewall scripts using ipsets" or something similar?

Also, there are many scripts in this forum. Would each maintainer add their work there (if they wanted to)?

 

[thelonelycoder]

Makes sense. I do not think that page explains how to use ipsets very much now There are 3 bullet points and a link to the man page, and that's about it.
Would you like me to rename the page to something along the lines of "Firewall scripts using ipsets" or something similar?

Also, there are many scripts in this forum. Would each maintainer add their work there (if they wanted to)?

There's a section for 'How to block ads with pixelserv' where I added the parts how to install it.
Decoderman is my GitHub handle.
You could add such a section, then all of the script writers could add their links.
It's a Wiki for Asuswrt-Merlin but also helps how to use this firmware to the fullest.
AB gets many, many referrer hits from that page, it's read a lot.

 

[redhat27]

Could this then be an entry on the networking how-to? Something like "How to block scanners, bots, malware, ransomware, etc with published FireHOL blocking tiers" or something similar?

 

[thelonelycoder]

Could this then be an entry on the networking how-to? Something like "How to block scanners, bots, malware, ransomware, etc with published FireHOL blocking tiers" or something similar?

Thats a long title, How to block scanners, bots, malware, ransomware might just do for the curious minds.

 

[eclp]

The script is not loaded after a router reset and re installation and obviously doesn't load IPs. Otherwise only DNScrypt and AB-Solution are installed.

May 25 10:15:09 Firewall: /jffs/scripts/ya-malware-block.sh: Adding ya-malware-block rules to firewall...
May 25 10:15:11 Firewall: /jffs/scripts/ya-malware-block.sh: Loaded sets YAMalwareBlock1IP (0) and YAMalwareBlockCIDR (0) in 2 seconds

Help again would be nice ...

 

[redhat27]

@eclp Are you running the script from services-start?

Edit: Also, when you manually run it does it work?

 

[thelonelycoder]

update-hosts.add version 3.8.2 is now available for AB-Solution

Use 12 or cu in the AB UI to update to this latest addon version.

Changelog update-hosts.add v3.8.2:
- adds auto-whitelist support for @redhat27's iblocklist-loader.sh and ya-malware-block scripts.sh.
- auto-whitelists domains used in the above scripts to use full potential of them

Note that the auto-whitelisting of domains only works if the above scripts are installed in the default locations:
/jffs/scripts/iblocklist-loader.sh and/or ya-malware-block.sh
/jffs/ipset_lists/blacklist-domains.txt and/or ya-malware-block.urls

@Adamm's and @Martineau's scripts do not need whitelisting in AB.

 

[MarCoMLXXV]

Installed ya-malware-block.sh today and am wondering about two things:

First: why, after installing, the default sets return a count of zero?

marco@RT-AC68U:/tmp/home/root# wget --no-check-certificate -O /jffs/scripts/ya-malware-block.sh https://raw.githubusercontent.com/shounak-de/misc-scripts/master/ya-malware-block.sh
Will not apply HSTS. The HSTS database must be a regular and non-world-writable file.
ERROR: could not open HSTS store at '/root/.wget-hsts'. HSTS will be disabled.
--2017-05-25 12:44:17--  https://raw.githubusercontent.com/shounak-de/misc-scripts/master/ya-malware-block.sh
Resolving raw.githubusercontent.com... 151.101.128.133, 151.101.192.133, 151.101.0.133, ...
Connecting to raw.githubusercontent.com|151.101.128.133|:443... connected.
WARNING: cannot verify raw.githubusercontent.com's certificate, issued by 'CN=DigiCert SHA2 High Assurance Server CA,OU=www.digicert.com,O=DigiCert Inc,C=US':
  Unable to locally verify the issuer's authority.
HTTP request sent, awaiting response... 200 OK
Length: 3101 (3.0K) [text/plain]
Saving to: '/jffs/scripts/ya-malware-block.sh'
/jffs/scr   0%[                    ]       0  --.-KB/s           
/jffs/scripts/ya-ma 100%[===================>]   3.03K  --.-KB/s    in 0.001s
2017-05-25 12:44:17 (3.38 MB/s) - '/jffs/scripts/ya-malware-block.sh' saved [3101/3101]

marco@RT-AC68U:/tmp/home/root# chmod +x /jffs/scripts/ya-malware-block.sh
marco@RT-AC68U:/tmp/home/root# /jffs/scripts/ya-malware-block.sh
/jffs/scripts/ya-malware-block.sh: Adding ya-malware-block rules to firewall...
/jffs/scripts/ya-malware-block.sh: Loaded sets YAMalwareBlock1IP (0) and YAMalwareBlockCIDR (0) in 1 seconds

Not sure whether relevant, but I'm running dnscrypt, AB-solution by @thelonelycoder with pixelserv-tls and the Skynet firewall script by @Adamm too. I installed ya-malware-block.sh, then updated AB-solution to todays release. I checked whether the default lists are in ya-malware-block.urls ipset lists, which is the case. I just don't know wether they are actually imported, nor how I can check that.

And second, why are these two IP's whitelisted by default?

marco@RT-AC68U:/tmp/home/root# nslookup 213.230.210.230
Server:    127.0.0.1
Address 1: 127.0.0.1 localhost.localdomain

Name:      213.230.210.230
Address 1: 213.230.210.230 boo.yoyo.org
marco@RT-AC68U:/tmp/home/root# nslookup 192.124.249.10
Server:    127.0.0.1
Address 1: 127.0.0.1 localhost.localdomain

Name:      192.124.249.10
Address 1: 192.124.249.10 cloudproxy10010.sucuri.net

Thanks in advance.

 

[RMerlin]

Could this then be an entry on the networking how-to? Something like "How to block scanners, bots, malware, ransomware, etc with published FireHOL blocking tiers" or something similar?

I haven't given it much thought, but at a quick glance at the current Wiki layout, I'd suggest renaming "External Software Repositories" to "External Software" or "Software add-ons", and adding a new section to it called "Scripts" or something like that.

 

[redhat27]

First: why, after installing, the default sets return a count of zero?

I believe this is related to your current wget utility installed in your router.

Will not apply HSTS. The HSTS database must be a regular and non-world-writable file.
ERROR: could not open HSTS store at '/root/.wget-hsts'. HSTS will be disabled.

These are discussed here too. Even @RMerlin offers some advice there. Try using entware-ng wget if that works for you. (opkg install wget)
Can you post the output of

which wget

before you install entware wget?

And second, why are these two IP's whitelisted by default?

AB solution uses a hosts list provider from pgl.yoyo.org. That was being blocked on the default Level1 through Level3 blocking of FireHOL tiers
try hostip or nslookup of pgl.yoyo.org. That first IP is for pgl.yoyo.org

I use all 4 Levels and noticed that androidfilehost.com (needed by most files hosted in xda-developers.com) was blocked on Level4. That second IP is for androidfilehost.com

 

[redhat27]

I haven't given it much thought, but at a quick glance at the current Wiki layout, I'd suggest renaming "External Software Repositories" to "External Software" or "Software add-ons", and adding a new section to it called "Scripts" or something like that.

I had already added it on the networking how-to, but will make the change you mentioned.

 

[MarCoMLXXV]

I believe this is related to your current wget utility installed in your router.

Will not apply HSTS. The HSTS database must be a regular and non-world-writable file.
ERROR: could not open HSTS store at '/root/.wget-hsts'. HSTS will be disabled.

These are discussed here too. Even @RMerlin offers some advice there. Try using entware-ng wget if that works for you. (opkg install wget)
Can you post the output of

which wget

before you install entware wget?

I already had wget installed through entware-ng as one of the pre-requisites for running an NTP-daemon on Asuswrt-Merlin, so the output of which wget is, as expected:

marco@RT-AC68U:/tmp/home/root# which wget
/opt/bin/wget

Thanks for the clarification on the pre-whitelisted IP's.

 

[eclp]

@redhat27
Thank you! Now it's running ... (I Noob )

May 25 18:00:00 Firewall: /jffs/scripts/ya-malware-block.sh: Adding ya-malware-block rules to firewall...
May 25 18:00:11 Firewall: /jffs/scripts/ya-malware-block.sh: Loaded sets YAMalwareBlock1IP (65536) YAMalwareBlock2IP (12004) and YAMalwareBlockCIDR (8919) in 11 seconds