How to blacklist/block sites using MVPS Hosts file with DNSCrypt-Proxy v2

[kamoj]

MVPS Hosts file is a well known way to block ads, banners, and "bad" sites.
http://winhelp2002.mvps.org/hosts.htm

Voxel has add-ons for DNSCrypt-Proxy v2 for his Firmware for the R7800 and R9000 routers.
https://www.snbforums.com/threads/dnscrypt-proxy-version-2-and-stubby-add-ons-for-r7800-r9000.48445/

It's easy to combine these:

wget -O /tmp/mvps.zip http://winhelp2002.mvps.org/hosts.zip
unzip -o /tmp/mvps.zip HOSTS -d /tmp/
awk '/^0.0.0.0/ {print $2}' "/tmp/HOSTS" >/etc/blacklist.txt
sed -i "s/# blacklist_file = 'blacklist.txt'/blacklist_file = 'blacklist.txt'/g" /etc/dnscrypt-proxy-2.toml
/etc/init.d/dnscrypt-proxy-2 stop
/etc/init.d/dnscrypt-proxy-2 start

To "uninstall", e.g.:

\mv /etc/blacklist.txt /etc/blacklist.txt.bup
touch /etc/blacklist.txt
/etc/init.d/dnscrypt-proxy-2 stop
/etc/init.d/dnscrypt-proxy-2 start

 

[eevanskiteboards]

I am going to try this, thanks.

 

[ajp2k14]

MVPS Hosts file is a well known way to block block ads, banners, and "bad" sites.
http://winhelp2002.mvps.org/hosts.htm

Voxel has add-ons for DNSCrypt-Proxy v2 for his Firmware for the R7800 and R9000 routers.
https://www.snbforums.com/threads/dnscrypt-proxy-version-2-and-stubby-add-ons-for-r7800-r9000.48445/

It's easy to combine these:

wget -O /tmp/mvps.zip http://winhelp2002.mvps.org/hosts.zip
unzip -o /tmp/mvps.zip HOSTS -d /tmp/
awk '/^0.0.0.0/ {print $2}' "/tmp/HOSTS" >/etc/blacklist.txt
sed -i "s/# blacklist_file = 'blacklist.txt'/blacklist_file = 'blacklist.txt'/g" /etc/dnscrypt-proxy-2.toml
/etc/init.d/dnscrypt-proxy-2 stop
/etc/init.d/dnscrypt-proxy-2 start

To "uninstall", e.g.:

\mv /etc/blacklist.txt /etc/blacklist.txt.bup
touch /etc/blacklist.txt
/etc/init.d/dnscrypt-proxy-2 stop
/etc/init.d/dnscrypt-proxy-2 start

Do you think it would be possible to automate this and maybe combine several lists? Maybe not needed because I know Dnscrypt-proxy v2 has regex filtering too... and cloaking to redirect to google safesearch etc.

I would love to try and do this but my linux skills are limited and I'm not sure it's realistic...

 

[kamoj]

Automate? It's automatic in the sense it will survive reboot.
You need to repeat the commands only to update to a new hosts list.
All commands can be done by a cron-job.

Combine several host list? Yes, sure. There is one or two linux commands to merge and sort lists.

And as you already know, DNSCrypt-Proxy 2 support different filterings and lists.

Do you think it would be possible to automate this and maybe combine several lists? Maybe not needed because I know Dnscrypt-proxy v2 has regex filtering too... and cloaking to redirect to google safesearch etc.

I would love to try and do this but my linux skills are limited and I'm not sure it's realistic...

 

[kamoj]

How to install a cron job to update MVPS hosts file once a day:
2019-11-11 UPDATE: New dnscrypt2 need full path to blacklist.txt, so #4 below updated!

Pre-requisites:

• A telnet client program
  
  Spoiler: How to enable Telnet on Windows 10

  Hold down the Windows Key, then press the “R“ key.
  The Run dialog box appears. In the Open: window, type:
  pkgmgr /iu:”TelnetClient”
  Click OK
• Internet connection (To download the mvps hosts file)

#1. Logon (From your web-browser):
http://www.routerlogin.net

#2. Enable Telnet:
http://www.routerlogin.net/debug.htm
Check (Set a tick in the box for) "Enable Telnet". (No apply needed)

#3. Login to the router using telnet:

Spoiler: How to open a command window in Windows 10

Hold down the Windows Key, then press the “R“ key.
The Run dialog box appears. In the Open: window, type:
cmd
Click OK.

telnet www.routerlogin.net

The password is the same as from the normal router login page http://routerlogin.net

#4. Create a shell script containing the needed commands:

(The commands can be copied by marking them with the mouse, and pressing Ctrl+c,
and then pasted into the command window by using right mouse button.
Just make sure to press the Enter key after each paste)

[ ! -e /etc/dnscrypt-proxy-2.toml.mvps ] && \cp -p /etc/dnscrypt-proxy-2.toml /etc/dnscrypt-proxy-2.toml.mvps
echo '#!/bin/sh' >/usr/bin/update_mvps.sh
echo 'wget -O /tmp/mvps.zip http://winhelp2002.mvps.org/hosts.zip' >>/usr/bin/update_mvps.sh
echo '[ $? -ne 0 ] && exit 1' >>/usr/bin/update_mvps.sh
echo 'unzip -o /tmp/mvps.zip HOSTS -d /tmp/' >>/usr/bin/update_mvps.sh
echo "awk '/^0.0.0.0/ {print \$2}' "/tmp/HOSTS" >/etc/blacklist.txt" >>/usr/bin/update_mvps.sh
echo "sed -i \"s/# blacklist_file = 'blacklist.txt'/blacklist_file = '\/etc\/blacklist.txt'/g\" /etc/dnscrypt-proxy-2.toml" >>/usr/bin/update_mvps.sh
echo '/etc/init.d/dnscrypt-proxy-2 restart' >>/usr/bin/update_mvps.sh
chmod +x /usr/bin/update_mvps.sh
/usr/bin/update_mvps.sh

#5. Create your cron job, to update mvps list every morning at 4 am:

\cp -p /etc/rc.local /etc/rc.local.premvps
echo >>/etc/rc.local
echo 'mkdir -p /opt/tmp/cronmvps/crontabs && echo "0 4 * * * /usr/bin/update_mvps.sh" >/opt/tmp/cronmvps/crontabs/root && crond -c /opt/tmp/cronmvps/crontabs' >/tmp/x.mvps
sed -n -i -e '/^exit 0/r /tmp/x.mvps' -e 1x -e '2,${x;p}' -e '${x;p}' /etc/rc.local
sed -i '$ { /^$/ d}' /etc/rc.local
\rm /tmp/x.mvps

reboot

#6. You can "uninstall" the MVPS blacklist from DNSCrypt-Proxy 2 and the cron job with these commands:

\cp -p /etc/dnscrypt-proxy-2.toml.mvps /etc/dnscrypt-proxy-2.toml
\cp -p /etc/rc.local.premvps /etc/rc.local
/etc/init.d/dnscrypt-proxy-2 restart

reboot

 

[kc6108]

sed -n -i -e '/^exit 0/r /tmp/x.mvps' -e 1x -e '2,${x;p}' -e '${x;p}' /etc/rc.local
sed -i '$ { /^$/ d}' /etc/rc.local

What does this do?

 

[kc6108]

@kamoj

Would you look over my code please? This is based on your code, but to get the blacklist.txt file directly from the author of dnscrypt-proxy 2...

Create update_blacklist.sh:

echo '#!/bin/sh' >/usr/bin/update_blacklist.sh
echo 'wget -O /etc/blacklist.txt https://download.dnscrypt.info/blacklists/domains/mybase.txt' >>/usr/bin/update_blacklist.sh
echo '[ $? -ne 0 ] && exit 1' >>/usr/bin/update_blacklist.sh
echo '/etc/init.d/dnscrypt-proxy-2 restart' >>/usr/bin/update_blacklist.sh

Make update_blacklist.sh executable:

chmod +x /usr/bin/update_blacklist.sh

Run update_blacklist.sh:

/usr/bin/update_blacklist.sh

Cronjob:

\cp -p /etc/rc.local /etc/rc.local.preblacklist
echo >>/etc/rc.local
echo 'mkdir -p /opt/tmp/cronblacklist/crontabs && echo "0 4 * * * /usr/bin/update_blacklist.sh" >/opt/tmp/cronblacklist/crontabs/root && crond -c /opt/tmp/cronblacklist/crontabs' >/tmp/x.blacklist
sed -n -i -e '/^exit 0/r /tmp/x.blacklist' -e 1x -e '2,${x;p}' -e '${x;p}' /etc/rc.local
sed -i '$ { /^$/ d}' /etc/rc.local
\rm /tmp/x.blacklist

reboot

 

[kc6108]

I manually updated the dnscrypt-proxy-2.toml to uncomment the blacklist_file parameter, but adding this should work just as well, correct?

echo "sed -i \"s/# blacklist_file = 'blacklist.txt'/blacklist_file = 'blacklist.txt'/g\" /etc/dnscrypt-proxy-2.toml" >>/usr/bin/update_mvps.sh

 

[kc6108]

I'd like to combine the two... the MVPS Hosts and mybase.txt files. I don't think it's so easy.

1. It would need to be sorted
2. There would be duplicates
3. The MVPS Hosts file would need to be validated against the whitelists config file

I'm sure there's more that would need to be done, but am I correct in my thinking?

 

[kamoj]

1+2 and removing blank lines and comments can be done like this:

#Get mybase.txt:
wget -O /tmp/mybase.txt https://download.dnscrypt.info/blacklists/domains/mybase.txt

#Merge the files:
cat /tmp/mybase.txt /etc/blacklist.txt | sort -u >/etc/blacklist.txt

#Delete empty lines and comments:
sed -i -e '/^\s*#.*$/d' -e '/^\s*$/d' /etc/blacklist.txt

for 3 I don't understand what you mean.

I'd like to combine the two... the MVPS Hosts and mybase.txt files. I don't think it's so easy.

1. It would need to be sorted
2. There would be duplicates
3. The MVPS Hosts file would need to be validated against the whitelists config file

I'm sure there's more that would need to be done, but am I correct in my thinking?

 

[eevanskiteboards]

Hello. Just updated to Voxel' s new firmware(.66) And I was wondering when you upload new firmware and then factory reset the router is this ad blocking solution wiped away and I have to reinstall it? Thanks

 

[e38BimmerFN]

Yes add-ons need to be re-installed after a reset.

 

[eevanskiteboards]

Yes add-ons need to be re-installed after a reset.

Understood and thank you for your quick response

 

[KW.]

Hello again. I try to set up a adblocker on my router R9000 but when I try your commands I get "DNSCrypt-Proxy-2 is not enabled in nvram. Exit." How do I enable it?

MVPS Hosts file is a well known way to block ads, banners, and "bad" sites.
http://winhelp2002.mvps.org/hosts.htm

Voxel has add-ons for DNSCrypt-Proxy v2 for his Firmware for the R7800 and R9000 routers.
https://www.snbforums.com/threads/dnscrypt-proxy-version-2-and-stubby-add-ons-for-r7800-r9000.48445/

It's easy to combine these:

wget -O /tmp/mvps.zip http://winhelp2002.mvps.org/hosts.zip
unzip -o /tmp/mvps.zip HOSTS -d /tmp/
awk '/^0.0.0.0/ {print $2}' "/tmp/HOSTS" >/etc/blacklist.txt
sed -i "s/# blacklist_file = 'blacklist.txt'/blacklist_file = 'blacklist.txt'/g" /etc/dnscrypt-proxy-2.toml
/etc/init.d/dnscrypt-proxy-2 stop
/etc/init.d/dnscrypt-proxy-2 start

To "uninstall", e.g.:

\mv /etc/blacklist.txt /etc/blacklist.txt.bup
touch /etc/blacklist.txt
/etc/init.d/dnscrypt-proxy-2 stop
/etc/init.d/dnscrypt-proxy-2 start

 

[Voxel]

Hello again. I try to set up a adblocker on my router R9000 but when I try your commands I get "DNSCrypt-Proxy-2 is not enabled in nvram. Exit." How do I enable it?

From telnet:

nvram set dnscrypt2=1
nvram commit

After reboot it will be started automatically (DNSCrypt-Proxy 2).
Voxel.

 

[kamoj]

All this functionality and more are included in the klamoj add-on (just a tick in the box):
https://www.snbforums.com/threads/kamoj-add-on-v5-for-netgear-r7800-x4s-and-r9000-x10.60590/

 

[KW.]

You guys are great. Have installed your addon Kamoj. Great.

I have one problem thou. When i enable the dnscrypt i cant visit any websites anymore. I understand I in someway messed up the dns. I have on a usb created a file (as i understood in the Voxels readme) with the name "dnscrypt.conf". In it i have som DNS-names like "adguard-dns-ns1" and so on. Maybe that is were i do wrong. Put in wrong names?

Anyway as I said if i enable dnscrypt 2 everything messes up. Also in the the Kamoj mega addon I cant enable dnscrypt as it is now.

Thank you for your patience and cool with a person like me but i love your work and try to get most out of it. It is like a candy store.

 

[kamoj]

/etc/dnscrypt.conf is used for dnscrypt proxy v1 only, and is not supported any longer by neither Voxel or the Add-on.
(Chapter 6 in the @Voxel readme.docx should be discarded/removed.)

If you are not familiar with Linux I suggest you don't try manual settings to start with.
Try to reinstall Voxel FW, and then the add-on, and only use the add-on menus until you have the knowledge to tweak it further.

 

[KW.]

Okey.

But on your addon its three boxes I can click in "DNS encryption". None, Stubby and DNScrypt v.2. My problem is that i cant click on any of the boxes besides "none". I kill my internet when I use the command for enable dnscrypt "
nvram set dnscrypt2=1
nvram commit"

I have unplugged my usb with the proxy 1 commands.

 

[kamoj]

You should NOT have run any nvram command at all if using the add-on.

For the problem with the not selectable radio buttons:

• Try another browser
• Try "New Incognito Window" in the Google Chrome browser (Ctrl+Shift+N).