AdBlocking with combined hosts file

[zekesdad]

Ok, so it's me again

I'm trying to set this up on my work router, RT-N66U, and now I'm not getting the hosts.blocked file to populate. I think I've gone over this about 90 times trying to figure out what I'm missing so that I wouldn't be here asking a dumb question, but I guess that's what I've come to.

admin@RT-N66U:/tmp/mnt/ADBLOCK# tail -f /tmp/mnt/ADBLOCK/logs/dnsmasq.log
Sep 14 15:29:37 dnsmasq[625]: read /mnt/ADBLOCK/hosts/blacklist.txt - 0 addresses
Sep 14 15:29:37 dnsmasq[625]: read /mnt/ADBLOCK/hosts/hosts.blocked - 0 addresses
Sep 14 15:29:37 dnsmasq[625]: read /etc/hosts.dnsmasq - 3 addresses
Sep 14 15:29:37 dnsmasq-dhcp[625]: read /etc/ethers - 3 addresses
Sep 14 15:29:37 dnsmasq[625]: using nameserver 192.168.254.254#53 for domain local
Sep 14 15:29:37 dnsmasq[625]: using nameserver 192.168.254.254#53 for domain domain.invalid
Sep 14 15:29:37 dnsmasq[625]: using nameserver 192.168.254.254#53 for domain local
Sep 14 15:29:37 dnsmasq[625]: using nameserver 192.168.254.254#53 for domain domain.invalid
Sep 14 15:29:37 dnsmasq[625]: using nameserver 192.168.254.254#53
Sep 14 15:29:37 dnsmasq[625]: using nameserver 192.168.254.254#53


i mean, it seems like everything SHOULD be working it just isn't. The scripts are executable, i don't know what else it could be.


EDIT: ok, weird, the USB stick is formatted with ADBLOCK as the volume label. sometimes it shows up under /tmp/mnt/ADBLOCK/hosts and somtimes it's /tmp/mnt/sda/ and there is no /hosts/ folder ....

 

[thelonelycoder]

@zekesdad Try not to give a name to that USB stick. Let the router decide on the name, it will default to sda1 or something like that.
Also, look at the system log for clues what it says about the path and name for it.

 

[robahearts]

i'm getting the following error when running the script

./update-hosts.sh: line 7: http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=0&startdate[day]=&startdate[month]=&startdate[year]=&mimetype=plaintext&useip=0.0.0.0: not found
./update-hosts.sh: line 8: syntax error: unexpected "|"

 

[ColinTaylor]

i'm getting the following error when running the script

./update-hosts.sh: line 7: http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=0&startdate[day]=&startdate[month]=&startdate[year]=&mimetype=plaintext&useip=0.0.0.0: not found
./update-hosts.sh: line 8: syntax error: unexpected "|"

It looks like you have line-wrapped line 7. Check your file and editor settings.

 

[robahearts]

It looks like you have line-wrapped line 7. Check your file and editor settings.

Yep that did it. Thank you.

 

[zekesdad]

@zekesdad Try not to give a name to that USB stick. Let the router decide on the name, it will default to sda1 or something like that.
Also, look at the system log for clues what it says about the path and name for it.

Thank you. I formatted it, it seems to reliably call it "sda" and I changed all the references (i believe) to the directory structure. Still no go.
Heres a pastebin of the system logs:

http://pastebin.com/JayJfxQ5

See anything in there that could be causing this?

 

[tijaune]

zekesdad, your dnsmasq is running before your usb drive get mounted...

Dec 31 19:00:13 dnsmasq[269]: cannot open log /tmp/mnt/sda/logs/dnsmasq.log: No such file or directory
Dec 31 19:00:13 dnsmasq[269]: FAILED to start up
...
Dec 31 19:00:19 usb: USB vfat fs at /dev/sda mounted on /tmp/mnt/sda.

I do not remember using dnsmasq.log on mine.

 

[zekesdad]

zekesdad, your dnsmasq is running before your usb drive get mounted...

Dec 31 19:00:13 dnsmasq[269]: cannot open log /tmp/mnt/sda/logs/dnsmasq.log: No such file or directory
Dec 31 19:00:13 dnsmasq[269]: FAILED to start up
...
Dec 31 19:00:19 usb: USB vfat fs at /dev/sda mounted on /tmp/mnt/sda.

I do not remember using dnsmasq.log on mine.

any clue on how to change that?

i just added the logging in case i needed to show what it was doing.

 

[bilboSNB]

Anyone had any luck getting this to work for the youtube app on android devices?

 

[tijaune]

any clue on how to change that?

i just added the logging in case i needed to show what it was doing.

I have this in /jffs/scripts/post-mount to ensure that my ASUS usb drive is ready before the adblock script run. Download host file and black / white list are saved there:

if [ $1 = "/tmp/mnt/ASUS" ]
then
. /jffs/scripts/adblock.sh
fi

Are you plugging your to a hub?

 

[zekesdad]

I have this in /jffs/scripts/post-mount to ensure that my ASUS usb drive is ready before the adblock script run. Download host file and black / white list are saved there:

if [ $1 = "/tmp/mnt/ASUS" ]
then
. /jffs/scripts/adblock.sh
fi

Are you plugging your to a hub?

No, no hub. Just one of the two usb ports. I've tried using different sticks, and as near as i can tell, it works about once every ten tries or so, so I'd like to try your script cause I think the usb stick not being mounted seems to be a big part of the problem. And please excuse me, im a linux baby, how exactly do I use that? does the /ASUS/ part have to be the exact path, in my case /tmp/mnt/sda1/hosts/ ?

Edit: also, i don't have an "adblock.sh" file anywhere...

 

[Mustard]

Hey guys,

I'm curious to see if anyone can replicate the issue of "pre-load" ads causing the player to infinitely load and not play the video.

Examples -

1) Any video on ESPN. Links always change but there are tons just on their homepage. If you click play, the preview image will disappear and be replaced with just the play button (no preview image).
2) Watching any show on Adult Swim's website. Direct URL to an example. This page just has their video loading animation infinitely repeating.

The easy solution is to white list both sites but ESPN's site is so unbearable now; I could live without videos. I'm sure this issue is also appearing on other sites with pre-load videos.

What's interesting is if I use the app for Adult Swim and ESPN, the videos play just fine. This is likely because they are using a different form of ad injection for mobile.

 

[tijaune]

No, no hub. Just one of the two usb ports. I've tried using different sticks, and as near as i can tell, it works about once every ten tries or so, so I'd like to try your script cause I think the usb stick not being mounted seems to be a big part of the problem. And please excuse me, im a linux baby, how exactly do I use that? does the /ASUS/ part have to be the exact path, in my case /tmp/mnt/sda1/hosts/ ?

Edit: also, i don't have an "adblock.sh" file anywhere...

mine /tmp/mnt/ASUS = your /tmp/mnt/sda1 or earlier /tmp/mnt/ADBLOCK.
adblock.sh = update-hosts.sh (thelonelycoder).

I had chance to re-examine my router log. dns-masq seems to be restarted at least 2-3 times during router reboot so you can probably ignore those first instances. Maybe you missed the post-mount script, as I understand it is executed after drive finished mounting.

try to add this to your /jffs/scripts/post-mount:

if [ $1 = "/tmp/mnt/sda1" ]
then
. /jffs/scripts/update-hosts.sh
fi

 

[zekesdad]

mine /tmp/mnt/ASUS = your /tmp/mnt/sda1 or earlier /tmp/mnt/ADBLOCK.
adblock.sh = update-hosts.sh (thelonelycoder).

I had chance to re-examine my router log. dns-masq seems to be restarted at least 2-3 times during router reboot so you can probably ignore those first instances. Maybe you missed the post-mount script, as I understand it is executed after drive finished mounting.

try to add this to your /jffs/scripts/post-mount:

if [ $1 = "/tmp/mnt/sda1" ]
then
. /jffs/scripts/update-hosts.sh
fi

im using the advanced, option 2, from the OP. I don't think there's anything on there about creating a post-mount script.

 

[tijaune]

im using the advanced, option 2, from the OP. I don't think there's anything on there about creating a post-mount script.

option 1 used post-mount with at least 17 secs before dns_masq restart.
option 2 used services-start with sleep 5, maybe try increase this to 25 to see.

regardless, those 2 recipes are just to run update-host.sh script when router is restart. You can run it manually...

cd /jffs/scripts/
./update-hosts.sh

... then observe router log. This way you know that error is not due to unmounted usb. It will be more likely minor typo or invalid path.

 

[TechLifeWeb]

Hey guys,

I'm curious to see if anyone can replicate the issue of "pre-load" ads causing the player to infinitely load and not play the video.

Examples -

1) Any video on ESPN. Links always change but there are tons just on their homepage. If you click play, the preview image will disappear and be replaced with just the play button (no preview image).
2) Watching any show on Adult Swim's website. Direct URL to an example. This page just has their video loading animation infinitely repeating.

The easy solution is to white list both sites but ESPN's site is so unbearable now; I could live without videos. I'm sure this issue is also appearing on other sites with pre-load videos.

What's interesting is if I use the app for Adult Swim and ESPN, the videos play just fine. This is likely because they are using a different form of ad injection for mobile.

Edit... I misread what you are looking for so I deleted my original reply. For ESPN, try whitelisting these:
adm.fwmrm.net
m.v.fwmrm.net

Haven't really tested it yet, however.

 

[Mustard]

Edit... I misread what you are looking for so I deleted my original reply. For ESPN, try whitelisting these:
adm.fwmrm.net
m.v.fwmrm.net

Haven't really tested it yet, however.

That works but unfortunately it also white lists every ad on ESPN's site.

I know that the way that Ublock/Adblock blocks ads is different than a hosts file block, but was curious if anyone had a work around for players that check if an ad has been played before initializing.

Unfortunately, combining the host block and using Ublock/Adblock doesn't solve the issue. The only way to fix, that I've come up with, this is to white list the ads (and then use Ublock/Adblock).

 

[Chrysalis]

On skysports website, if I have ublock activate videos wont play. If I disable ublock but leave the adblocking active on my router, the ads play without ads. Which is interesting but my adblock list on my router has a few domains whitelisted, I had to tweak to get it to work.

 

[Mustard]

I found a good work around (until you need Flash)

If you disable your Flash plugin in Chrome (chrome://plugins), then if a site also has an HTML5 player; it will use HTML5 instead of Flash.

On ESPN - They embed a short ~15 second ad before the actual video. Super easy to deal with.

On Adult Swim - No ads

Of course, if a site only offers a flash player, I'll have to re-enable plugin.

 

[TechLifeWeb]

Is Crackle.com working for everyone? I don't use it often but it seems I need to whitelist a few to get it to work.