AdBlocking with combined hosts file with Lighttpd

[nhan72nn]

fail to do it...
would you like yo help me please

Hi everyone,

First post here so bare with me.

This builds on thelonelycoder's script for host based domain blocking. The main difference is a customizable HTML or 1x1 transparent GIF file will resolve and replace the blocked section of the page, instead of a 404-type message. This is essentially emulating pixelserv if you use a GIF file.

Prerequisites:
- Merlin's latest firmware
- Entware installed
- USB drive formatted to EXT2 (I used EASEUS Partition Master for Windows)
- Running thelonelycoder's script

For these instructions, I'm going to use 192.168.0.1 as my router's IP address. Replace with your router's IP as necessary.

FIRST STEP: Freeing port 80 on the router
1. Login to the your router via web browser

2. Go to Administration > System
- Enabled SSH > Checked to Yes (if not already done)
- Authentication Method > Change to HTTPS
- HTTPS Lan port > 8443

3. Click Apply

4. Verify you are able to login to your router by going to https://192.168.0.1:8443

5. Depending on the browser, it will give you a security warning, simply bypass or add the address to the exceptions list. There's also a possibility that you will get a long message that the IP/port has changed and will not let you login. Do not panic. Firefox or Chrome has enabled offline cache, you can simply turn that off (just do a search) or use Internet Explorer.

SECOND STEP: Modifying the hosts.clean file to route to 192.168.0.1
1. Login to the route via WinSCP > browse to /jffs/configs/ > right-click, edit dnsmasq.conf.add with changes:
remove this line as modifying it doesn't do anything:

address=/0.0.0.0/0.0.0.0

and ensure this shows as the first line:

addn-hosts=/tmp/mnt/sda1/hosts.clean

click the save icon

2. Browse to /jffs/scripts/ > right-click, edit post-mount file > copy and paste the code below:

#!/bin/sh
if [ $1 = "/tmp/mnt/SD2" ]
then
  ln -sf $1/entware /tmp/opt
fi
sleep 12
wget -qO- "http://winhelp2002.mvps.org/hosts.txt" "http://someonewhocares.org/hosts/zero/hosts" "http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=0&startdate[day]=&startdate[month]=&startdate[year]=&mimetype=plaintext&useip=0.0.0.0" | grep -w ^0.0.0.0 | sed $'s/\r$//;s/0.0.0.0/192.168.0.1/' | sort -u > /tmp/mnt/sda1/hosts.clean
sleep 5
service restart_dnsmasq

click the save icon

3. In the same directory > right -click, edit update-hosts.sh file > copy and paste the code below:

#!/bin/sh
wget -qO- "http://winhelp2002.mvps.org/hosts.txt" "http://someonewhocares.org/hosts/zero/hosts" "http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=0&startdate[day]=&startdate[month]=&startdate[year]=&mimetype=plaintext&useip=0.0.0.0" | grep -w ^0.0.0.0 | sed $'s/\r$//;s/0.0.0.0/192.168.0.1/' | sort -u > /tmp/mnt/sda1/hosts.clean
sleep 5
service restart_dnsmasq

click the save icon

THIRD STEP: Install and configure Lighttpd
1. Login to the router via PuTTY and enter the following commands:

opkg install lighttpd
opkg install lighttpd-mod-redirect
/opt/etc/init.d/lighttpd stop

2. Login to the router via WinSCP > browse to /opt/etc/lighttpd/ > right-click, edit lighttpd.conf with changes:
between the parenthesis of server.modules, add "mod_redirect" so it looks like:

server.modules = ( "mod_redirect" )

change server.port from 81 to 80

server.port = 80

comment out server.bind by adding # in front

#server.bind = "0.0.0.0"

change server.document-root path to /mnt/sda1 where sda1 is the name of your USB stick

server.document-root = "/mnt/sda1"

modify line to:

index-file.names = ( "index.html" )

add new line:

url.redirect = ( "^(.*?)(?<!\/index\.html)$" => "/index.html" )

click the save icon

3. Optional: the url.redirect is essentially a regex that redirects any blocked URL to /index.html. Once it reaches /index.html, it will stop redirecting. You can change index.html to blank.gif (for example) by replacing all instances of index with blank and all instance of html with gif for the url.redirect line only.

url.redirect = ( "^(.*?)(?<!\/blank\.gif)$" => "/blank.gif" )

4. Browse to /mnt/sda1/ and upload a simple index.html or blank.gif file. Ensure this file matches entries on lighttpd.conf

5. Go back to PuTTY and type reboot or click Reboot button from the browser.

6. Use a device connected to the router via WiFi or LAN port and browse to adchimp dot com and see if it redirects to your index.html or blank.gif file.

DONE!

 

[TPSDF]

Have been searching for something like this ever since I switched from Tomato to Merlin for my ASUS RT66U. I'm going to try this tonight. On Tomato I was using Haarps Adblock.sh + pixelsrv which worked great. This looks to be equivalent minus the front end Web-Gui for configuration and control. Any thoughts on adding something like that. Would be a nice addition.

 

[rogern]

This looks like dietpi i have on an raspberrypi2.
In conjunction with privoxy,e2guardian and urlfilter it is a killer.
No delay as i can see though i have fast internet 500/500Mbps and local LAN is 1Gbps.

 

[mstombs]

raspberrypi only has one port 10/100Mbps Ethernet internally connected by USB - it would definitely slow everything down if all traffic routed through it - good for just DNS poisoning based adblocking + pixleserv though!

 

[Jarnootje]

People, regarding Prerequisites in first post:
thelonelycoder stated that Entware is not needed: August 31, 2015: Removed requirement for Entware, it was never really needed. Does this also apply to this script with Lighttpd now?

For whitelisting:
@baltosml used cat /mnt/sda1/whitelist | while read line; do sed -i /${line}/d /mnt/sda1/hosts.clean ; done
Do we need the advanced script from thelonelycoder, or just firstpost scripts. Can you update first post with this? Does this line really need to be placed in the two scripts under line with host.clean? And can we use whitelist.txt instead?

Is it better to close port 80 and disable ssh after successfully running this adblock script for security reasons, or is it only open on LAN now and not accessible from the wan-side?

Thanks in advance!

 

[Kevin R]

I just got adblocking working. Thank you to everyone who contributes scripts and code to make this possible!

I thought entware would be needed to install lighttpd and other packages, but I might be wrong. I have not used a whitelist; I did however change the script to not use
"http://someonewhocares.org/hosts/zero/hosts" as a source for the hosts file. It was blocking NBC and CBS videos. After removing that source, CBS and NBC videos seem to be blocked. I used the index.html method and made a webpage that says "blocked." Now when I go to a page in the hosts file it returns "blocked"

I think port 80 is being used by lighttpd (if you use that method), so you probably don't want to close that. SSH can only be used on the LAN network, unless it is specifically enabled for WAN on the Administration GUI page.