[How-to] Adblock Plus filters right on router

[waffles]

Hi Folks,

Since the last update 378.53/54 privoxy and DNScrypt doesn't start automatically. I'm forced to connect to the routeur to restart the 2 services. Anyone else with this issue?

Where you able to resolve it?

How do you start these (2) services manually?

My ad block set up is also broken as of late (Internet access is blocked for IP range specified in firewall-start).

I am not sure if this is related to my recent f/w upgrades. I could not find anything out of the ordinary in the Systems Log, though.

 

[WaVeR]

The System Log is your best friend. Have a look at it, it probably reports something to the effect that the router was asked to run a script, but you have custom scripts disabled. If that's the case, enable them under Administration -> System.

Thanks Merlin, I'll check next time when I reboot the router

 

[WaVeR]

Where you able to resolve it?

How do you start these (2) services manually?

My ad block set up is also broken as of late (Internet access is blocked for IP range specified in firewall-start).

I am not sure if this is related to my recent f/w upgrades. I could not find anything out of the ordinary in the Systems Log, though.

You can start your services using the command:
/opt/etc/init.d/SXXService start

As example, privoxy accept these parameters:
/opt/etc/init.d/S24privoxy
Usage: /opt/etc/init.d/S24privoxy (start|stop|restart|check|kill|reconfigure)

 

[shooter40sw]

Hi guys, is it possible with the dnscrypt-proxy implemented by ryzhov_al block ads? I have read on the dnscrypt page on the 1.5 version that it´s possible to use plugins and to use lists to block ads and malware sites, does somebody know if it can be implemented on the router? thanks

 

[spalife]

The easier way perhaps is to have a rudimentary ad and spam blocking if you are using opendns on top of dnscrypt.

Go to (if you have not already) dashboard.opendns.com and open an account at dashboard.opendns.com.

OpenDNS has rudimentary community updated lists for Malware/Botnet/Phishing and Web content filtering for Adblocking and Webspam.

You won't have to install any software or maintain or update or restart the machines.

Do note you would have to update your IP using DDNS if your ISP allocates you with Dynamic IP.

The above solution though will be rudimentary and won't be as effective as ryzhov_al block ads using Privoxy and Adblock Plus block lists.

 

[kvic]

Three Months after Using Privoxy+Adblock Plus filters

This really does magic to the user experience on smartphones, not only browsing but in almost every app (after all most are simply based on HTML5).

I believe most people using Privoxy may be like me who is also using dnsmasq+combined host files. Looking at the thread creation dates, Privoxy+Adblock Plus was (Jan 2013) a whole year earlier than dnsmasq+combined host files. This justifies the inclusion of Adblock Plus filters in the Privoxy setup back then.

Recently I asked myself if the Adblock Plus filters are necessary given I'm also running dnsmasq+combined host files. So I gave it a try by removing Adblock Plus filters and ran the vanilla Privoxy. Within my expectation, no adverse effect on user experience. Seems dnsmasq+combined host files is doing its job.

My conclusion (I won't quickly jump to it...) is that Adblock Plus filters seem redundant in the presence of dnsmasq+combined host files. However, vanilla Privoxy does make up in places where dnsmasq+combined host files can't do e.g. blocking in-app ad bars and pop-ups!

With Adblock Plus filters removed, I was hoping to see reduced memory footprint, increased service stability and speedier responses. In reality, to my surprise memory footprint at peak is not so correlated to number of filters but the complexity of webpages. Initial footprint on startup however does show a huge reduction. Since Privoxy does not have to load tons of filters on startup, it can serve requests almost instantly (unlike before a stall of a minute or two). The last point is a bonus as Privoxy crashes often (and if you have it auto restart).

Service stability improves actually without AdBlock Plus filters. I've yet to see a crash. I can quantify but I would think processing is faster and more efficient as many fewer filters to go through.

At the moment vanilla Privoxy + dnsmasq /w combined host files is the sweet spot for me.

 

[thelonelycoder]

Three Months after Using Privoxy+Adblock Plus filters

Interesting, thank you.
Two questions:
Do you limit to certain devices only or do you proxy all devices through it (e.g. computer as well)
Does the combined hosts file run separately from Privoxy and dnsmasq?

 

[spalife]

yep sounds really interesting and
your update is very informative and objective.

I am on privoxy + adblockplus filters since couple of days
for mobile devices.

It works so far, though the caveat being unable to filter https sites and ads within them and the startup time for privoxy with filters is high.

kvic
could you walk us through your installation & configuration for combined hosts file approach ?
i.e.,
1. router info
2. lists aka host files used
3. config files
4. installation and configuration steps
5. lists updation frequency
6. experience with https enabled sites/apps adverts

 

[kvic]

Interesting, thank you.
Two questions:
Do you limit to certain devices only or do you proxy all devices through it (e.g. computer as well)
Does the combined hosts file run separately from Privoxy and dnsmasq?

I only redirect my smartphones/tablets through Privoxy. Privacy is great but still runs slow on a router.

My current setup runs combined host files (literally I have one for worldwide and one for chinese specifically) through dnsmasq. In addition, running lightttpd to serve one pixel gif for blocked hosts.

I know various proxy software can filter at host level but performance wise I believe they cannot beat dnsmasq+lighttpd. At the moment I have about 26k hosts, memory footprint for dnsmasq is less than 3MiB. Average DNS query response (for hosts already cached by dnsmasq) is less than 10ms.

Also in this combined setup, unnecessary processing in Privoxy is stopped early on because Privoxy only see empty pages for tons of ad links.

yep sounds really interesting and
your update is very informative and objective.

I am on privoxy + adblockplus filters since couple of days
for mobile devices.

It works so far, though the caveat being unable to filter https sites and ads within them and the startup time for privoxy with filters is high.

kvic
could you walk us through your installation & configuration for combined hosts file approach ?
i.e.,
1. router info
2. lists aka host files used
3. config files
4. installation and configuration steps
5. lists updation frequency
6. experience with https enabled sites/apps adverts

Combined host files can help in cases of https site because when browsers/apps look up the host for ip address, it's being redirected to an empty page. You may refer to the first thread below for steps to create the combined host files, and the second thread for setting up lightttpd to serve one pixel gif.

http://www.snbforums.com/threads/adblocking-with-combined-hosts-file.15309/
http://www.snbforums.com/threads/adblocking-with-combined-hosts-file-with-lighttpd.16060/

They shall answer most of your questions. My router is RT-AC56U which is mostly idle. So I would love to put it to work harder.

Once you have the setup of combined host files. You can schedule to run the script for list update to your liking. Mine on a weekly cycle at the moment. It served me well in the past year. I like its simplicity after initial setup. Didn't recall any manual intervention. It benefits every device communicating through my router. Even adblock plugins on PC browsers don't have to work as hard as before. Smartphones on roads benefit through VPN as well.

 

[wuyanxu]

My current setup runs combined host files (literally I have one for worldwide and one for chinese specifically) through dnsmasq. In addition, running lightttpd to serve one pixel gif for blocked hosts.

I'd be interested in your hosts files, specifically the Chinese one.

I tried out Privoxy + adblock lists, but same as you, I'd concluded combined hosts files is the way to go on the router for multiple devices

 

[kvic]

I'd be interested in your hosts files, specifically the Chinese one.

I tried out Privoxy + adblock lists, but same as you, I'd concluded combined hosts files is the way to go on the router for multiple devices

Just looked at my script after a whole year. Didn't recall the china list actually from Adblock Plus. If some folks need the codes, here you go:

#!/bin/sh

WGET='/opt/bin/wget --no-check-certificate -qO-'

${WGET} \
https://easylist-downloads.adblockplus.org/easylistchina.txt \
| grep ^\|\|[0-9a-zA-Z.-]*\^[$]* | sed -e 's:\^.*$::' \
| sort -u | sed -e 's:||:192.168.1.1 :'  > /jffs/etc/hosts-cn.clean

In my setup I have all hosts resolving to 192.168.1.1 (which is my router running lighttpd on port 80) and created the host file "hosts-cn.clean" in "/jffs/etc" They have to be changed to suit individuals' setup.

My worldwide lists are those from e.g. winhelp2002.mvps.org, someonewhocares.org etc. I'm quite sure I got those from the first thread mentioned in my previous post.

Now...I couldn't figure why I didn't switch all lists to adblock plus back then..

 

[spalife]

Thanks kvic your reply was helpful.

I went through the forum posts you specified and they have been informative.

I do see issues with https ads and admin page slow loading being mentioned in those threads.

My concern is the lists.

The efficacy of the ad / malware blocking solution depends on the lists.
Adblock plus lists are well maintained.

The lists used in the forum threads of combined host blocking seem maintained too.

So far Privoxy with Adblock plus lists are working fine without any crashes or extreme memory usages, but they have an Achilles knee, the conversion script.
The script has to be maintained along with privoxy developmental updates.

The only thing where privoxy + adblock lists does not meet my requirement is with adverts in https enabled sites / apps.

It all comes down to the optimal use of router resources.
The solution you are using is very elegant, I might soon venture into it.

 

[Formel]

Hey guys, I did everything outlined in the guide but it doesn't work. I am currently running Asus-Merlin for r7000. I had no errors and the service is running. Am I missing something? Any help is appreciated. I did enable scripts for jffs also. I had no errors too.

 

[spalife]

Assuming that no errors during installation & configuration and you have rebooted the router after carrying out the steps.

::Troubleshoot by::

1. Check to see if Privoxy is up and running without issues by
Login through ssh/telnet into your router
ps -w | grep privoxy
cat /opt/var/log/privoxy

2. Go to your Router Administration Page. Lan->DHCP Server and see you have assigned static address to the device for which you want to route traffic through privoxy

3. Go back to your ssh/telnet session

4. Go to cd /jffs/scripts

5. cat firewall-start
Check you have added iptables routing for the static address you have given to the device

Also check by entering (the output should show your static address being forwarded to privoxy)
iptables -t nat -L -n -v

6. If all above checks out

7. In your ssh/telnet session enter
tail -f /opt/var/log/privoxy

8. On your static address assigned device i.e., phone / tablet / laptop
etc., surf the web through firefox / chrome (non https sites for beginning such as http://www.yahoo.com)

9. You should see in your ssh/telnet window privoxy logs being updated with your surfing details and privoxy ad block logging

10. As an additional step you can enter in your ssh / telnet window

 

[kvic]

Thanks kvic your reply was helpful.

I went through the forum posts you specified and they have been informative.

I do see issues with https ads and admin page slow loading being mentioned in those threads.

My concern is the lists.

The efficacy of the ad / malware blocking solution depends on the lists.
Adblock plus lists are well maintained.

The lists used in the forum threads of combined host blocking seem maintained too.

So far Privoxy with Adblock plus lists are working fine without any crashes or extreme memory usages, but they have an Achilles knee, the conversion script.
The script has to be maintained along with privoxy developmental updates.

The only thing where privoxy + adblock lists does not meet my requirement is with adverts in https enabled sites / apps.

It all comes down to the optimal use of router resources.
The solution you are using is very elegant, I might soon venture into it.

lighttpd had to run on port 80. People mentioned they had to run the WebUI on https. This indeed makes the WebUI half retarted when compared to http. Also it's an overkill in a home lan. I managed to run it over http on a non standard port. A workable solution though not of my liking.

Participation in this thread drives me to revisit my setup Thinking over again. lightttpd need not to run on port 80. It can run on a non-standard port and use iptables to redirect traffics. Then WebUI continues to run happily on port 80 - untouched. I might try this later.

 

[jyzhang]

Hi, I've installed privoxy and implemented adblock plus list. Everything is running. Thanks for the detailed instruction.

However, I've found I cannot use the "Apply" button anymore in the WebUI. Is this a know issue? How do I resolve it?

Also, Once I hard-reset the router, Entware & Privoxy went away. Is there a way to rescue what's already been install without having to re-install everything?

Thanks to the OP again for the great instruction.

 

[spalife]

Hi, I've installed privoxy and implemented adblock plus list. Everything is running. Thanks for the detailed instruction.

However, I've found I cannot use the "Apply" button anymore in the WebUI. Is this a know issue? How do I resolve it?

Also, Once I hard-reset the router, Entware & Privoxy went away. Is there a way to rescue what's already been install without having to re-install everything?

Thanks to the OP again for the great instruction.

Take a look at
http://www.snbforums.com/threads/user-nvram-save-restore-utility-r19.19521/page-15#post-193305


It will help you save and restore your settings.

 

[WaVeR]

Thanks Merlin, I'll check next time when I reboot the router

Hi RMerlin,

I don't know if I should open a new thread for this issue. As discussed when I reboot the router services are not started automatically. To be able to get everything running, I need to:

- Update the date from the command line with "date MMDDhhmm"
- Restart DNS-crypt
- Restart Privoxy

If the time is not correct, DNSCrypt can't get new certificate.

Attached the syslog

 

[ryzhov_al]

If the time is not correct, DNSCrypt can't get new certificate.

Which resolver are using? Default "opendsns" resolver doen't exists anymore, because OpenDNS was bought by Cisco.

 

[WaVeR]

Which resolver are using? Default "opendsns" resolver doen't exists anymore, because OpenDNS was bought by Cisco.

Hi ryzhov_al,

It seems that I'm using OpenDNS but the issue appeared on July

ARGS="--local-address=127.0.0.1:65053 --daemonize -R opendns"
#ARGS="--local-address=127.0.0.1:65053 --daemonize -R dnscrypt.eu-nl"

I'll change the resolver and check during the next reboot.

Should I change opendns to cisco or you'll update your package to include the new csv list?

cisco Cisco OpenDNS Remove your DNS blind spot

Thanks for the feedback