AdBlocking with combined hosts file

[Frejoh466]

admin@RT-N66U-B308:/# /jffs/scripts/update-hosts.sh
/jffs/scripts/update-hosts.sh: line 1: -: not found

Done.
admin@RT-N66U-B308:/# cd /jffs/scripts/
admin@RT-N66U-B308:/jffs/scripts# ls
post-mount  services-start  update-hosts.sh
admin@RT-N66U-B308:/jffs/scripts#

 

[ColinTaylor]

Your update-hosts.sh file is corrupted somehow. What do you get with

cat /jffs/scripts/update-hosts.sh

 

[Frejoh466]

Your update-hosts.sh file is corrupted somehow. What do you get with

cat /jffs/scripts/update-hosts.sh

admin@RT-N66U-B308:/jffs/scripts# cat /jffs/scripts/update-hosts.sh
- "http://winhelp2002.mvps.org/hosts.txt" "http://someonewhocares.org/hosts/zero/hosts" "http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=0&startdate[day]=&startdate[month]=&startdate[year]=&mimetype=plaintext&useip=0.0.0.0" | grep -w ^0.0.0.0 | sed $'s/\r$//' | sort -u > /tmp/mnt/TEST/hosts.clean
sleep 5
service restart_dnsmasq

admin@RT-N66U-B308:/jffs/scripts#

 

[ColinTaylor]

There's your answer. You're missing the whole of the first line and the beginning of the second line.

 

[Frejoh466]

There's your answer. You're missing the whole of the first line and the beginning of the second line.

I wasn't able to edit the file with vi, as the file got corrupted every time, so I moved the files to the USB and cp -r them to the right place and now I get this.

admin@RT-N66U-B308:/jffs/scripts# /jffs/scripts/update-hosts.sh
-sh: /jffs/scripts/update-hosts.sh: not found
admin@RT-N66U-B308:/jffs/scripts# ls
post-mount  services-start  update-hosts.sh
admin@RT-N66U-B308:/jffs/scripts# cat /jffs/scripts/update-hosts.sh
#!/bin/sh
wget -qO- "http://winhelp2002.mvps.org/hosts.txt" "http://someonewhocares.org/hosts/zero/hosts" "http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=0&startdate[day]=&startdate[month]=&startdate[year]=&mimetype=plaintext&useip=0.0.0.0" | grep -w ^0.0.0.0 | sed $'s/\r$//' | sort -u > /tmp/mnt/TEST/hosts.clean
sleep 5
service restart_dnsmasqadmin@RT-N66U-B308:/jffs/scripts#

I created the files with Notepad++, as Notepad something save something not work with Linux. And I'm on Win10.

 

[ColinTaylor]

It looks like you've still got DOS/Windows end-of-lines instead of UNIX. You're also missing the end-of-line character for the last line.

Check your Notepad++ setting, Edit > EOL Conversion = "UNIX/OSX Format".

 

[Frejoh466]

So it seems like vi and Notepad++ was corrupting the files, reinstalled notepad++ and now it's working just fine. Thanks for the help @ColinTaylor

 

[nhan72nn]

Now I made it work, but I don't know how to add my own hosts file to the hosts.clean file to make it more effective because I have some list of ads and tracker sites when I open some websites but are not in the list of hosts.clean.
I tried to add it manually but after the reboot, all the url I added just disappear.
Also, how can I add these hosts files from these sites (127.0.0.1)
http://adaway.org/hosts.txt, http://hosts-file.net/ad_servers.txt, http://www.malwaredomainlist.com/hostslist/hosts.txt,
into my hosts.clean?
thanks

 

[ColinTaylor]

Now I made it work, but I don't know how to add my own hosts file to the hosts.clean file to make it more effective because I have some list of ads and tracker sites when I open some websites but are not in the list of hosts.clean.

You need to use the "Advanced" method in post #1 if you want to create blacklists and whitelists.

 

[nhan72nn]

thanks, it works perfectly

You need to use the "Advanced" method in post #1 if you want to create blacklists and whitelists.

 

[philmiami]

I just got a Rasberry Pi 2 for $35 and the files are free, set it as static IP on router, then set it as LAN>DNS and it blocks 1.6-2 million ad sites. Does not slow down network at all. Only use like .4-1% of it's 4 core processor and works way better than the AdTrap that I bought 2+ years ago for $149 (that is a linux box in line or in proxy but is locked down on scripting or changing).
I would have likes to use my RT-AC68U but doing it this way, it's a headless PC blocking the ads and it updates itself weekly
all for $35 I spent
and someone else did the programing
http://jacobsalmela.com/block-millions-ads-network-wide-with-a-raspberry-pi-hole-2-0/

 

[thelonelycoder]

I would have likes to use my RT-AC68U but doing it this way, it's a headless PC blocking the ads and it updates itself weekly
all for $35 I spent and someone else did the programing

The router is a headless device as well. My scripts update weekly too and I did the programming for free.
So tell me, what is your point?

 

[philmiami]

Just that if I had kept up with the forum, and seen this I would have tried this first
I am glad someone did it for the routers
My question is this
is it working on IPv6?
I had to disable the IPv6 because this Pi2 only works on IPv4 as it is scripted
and no, I have not read the entire post but I will later when I have time

The router is a headless device as well. My scripts update weekly too and I did the programming for free.
So tell me, what is your point?

 

[thelonelycoder]

@philmiami Ah, I see. And no, it's not currently working with IPv6 as I still wait for my ISP to turn it on...

 

[followmsi]

Thanks a lot ! Works really fine on my RT-AC68U.

 

[philmiami]

I have Crapcast Blast and it does deliver IPv6 BUT
to keep things blocking I disabled IPv6 on the Router RT-AC68U
and more and more sites do both IPv4 (which they KNOW will be blocked) and IPv6 (which they push first because it's harder to block at router/modem level, and some of the ads are as static ads, like CNN.com)
ALSO, if you use Google Chrome, and let it "compress" your data, what that does is go THRU Google servers (so they can track you even more) and it compresses it, makes it HTTPS and it's hard to block because they re-write webpages and hardcode the ads in
seen it

 

[Eet_46]

I've been using following options in "dnsmasq.conf.add":

log-facility=/tmp/mnt/Data/hosts/logs/dnsmasq.log
log-queries

But are now seeing the log file simply growing larger and larger. Nearing 100MB now.
Is there a command i can add, to limit the log file size? Was looking at "logrotate", but don't know enough about how to use it, so i'm unsure exactly how to set it up (if this is the way to go anyways..)
And where, in which file, would i put this command?

 

[thelonelycoder]

I've been using following options in "dnsmasq.conf.add":

log-facility=/tmp/mnt/Data/hosts/logs/dnsmasq.log
log-queries

But are now seeing the log file simply growing larger and larger. Nearing 100MB now.
Is there a command i can add, to limit the log file size? Was looking at "logrotate", but don't know enough about how to use it, so i'm unsure exactly how to set it up (if this is the way to go anyways..)
And where, in which file, would i put this command?

I did look into log rotate, but the things I found crashed the router.
I ended up running a script that is run weekly, just before the update-hosts.sh is triggered by cron.

Create /jffs/scripts/dnsmasq-logrotate.sh and add this:

#!/bin/sh

# set directory
dir=/tmp/mnt/sda1/logs

#rotate the logs
mv $dir/dnsmasq_1.log $dir/dnsmasq_2.log
mv $dir/dnsmasq.log $dir/dnsmasq_1.log

#restart dnsmasq to create a new log file
service restart_dnsmasq

This will rotate the log files. You'll end up with 3 log files, spanning 2+ weeks. It might miss a few entries when dnsmasq.log is moved to dnsmasq_1.log but I don't care...

Add to /jffs/scripts/services-start

cru a DnsmasqLogrotate "23 59 * * 5 /jffs/scripts/dnsmasq-logrotate.sh"

Then set all files in /jffs/scripts/ as executable

chmod a+rx /jffs/scripts/*

Reboot to add the cron job to the queue.
That's it.

 

[wallaby32]

Hey guys,

Not sure how many of you have an Xbox One, but, I just got the preview update to try out Win10 on xbox one. Anyway, I am having some interesting issues with the new dashboard and the domains that are blocked using the advanced method.

Simple explanation: When ad-block script is loaded into the router's dnsmasq, Xbox One will not stay at home dashboard, always reverts to previous app. Now, with ad blocking disabled from the dnsmasq, Xbox has no problem's staying at dashboard and even navigating like normal.

Now, I guess my question is, what would be the most efficient way to figure out what domains are being blocked and causing the issue with the xbox dashboard? I would like to find these so I can add them to the whitelist.

I have the Asus RT-N66R BTW.

 

[ColinTaylor]

Now, I guess my question is, what would be the most efficient way to figure out what domains are being blocked and causing the issue with the xbox dashboard? I would like to find these so I can add them to the whitelist.

I would suggest the following:

1) Disconnect all other devices from your network apart from your PC and the Xbox (to reduce superfluous log entries).
2) From the router, enable detailed logging

# killall dnsmasq
# dnsmasq --log-async --log-queries --log-facility=/tmp/mnt/sda1/dnsmasq.log

3) Go onto the Xbox and recreate the problem.
4) Re-enable normal logging

# service restart_dnsmasq

5) Examine the dnsmasq.log file and see what was blocked (IP address = 0.0.0.0)