ad blocking on the router

[dbt78]

I think there is something wrong with jffs scripts or configs. Let's try to run adsuck manually:

• please, remove adsuck autostart:
  
• reboot router and type:
  
  killall dnsmasq
  adsuck -d -l 127.0.0.1 -c /opt/etc/adsuck -f resolv.conf -p 65053 -u admin Hosts.sex Hosts.trc
  dnsmasq --no-resolv --server=127.0.0.1#65053

hi

I also tried to run adsuck with no success.
By following the instructions (manual start), I get this message :

adsuck: can't udp bind: Address already in use

Is there a way to run adsuck with asuswrt-merlin?

 

[sardelkin]

hello!
please post step by step adblock manual for rt-ac66u with Asuswrt-Merlin 3.0.0.4.374.33 and this hosts url http://winhelp2002.mvps.org/hosts.txt

 

[gijor33]

dsuck is a small DNS server, that returns NXDOMAIN answer for blacklisted sites, so browser/OS will not wait for answer from blocked hosts. It's main advantage over DD-WRT solution.

Requirements to run adsuck are:

• Asuswrt-Merlin driven RT-N16/RT-N66U/RT-AC66U router with USB-drive and jffs partition enabled.
• Working Entware environment. Please, refer HOW-TO for details.

Installation:

• Install adsuck package:
  
• copy provider's DNS list to adsuck folder:
  
• choose what type of sites you want to filter, all filter files is in the /opt/etc/adsuck/Hosts.*:
  • Hosts.blc - 1916 sites anti-phishing white list,
  • Hosts.mis - 566350 other potential dangerous sites (drugs, sects, spam, traps, violence)
  • Hosts.pub - 106338 advert. sites,
  • Hosts.rsk - 120033 sites with potencial dangerous contents, viruses mostly,
  • Hosts.sex - 905497 adult content sites,
  • Hosts.trc - 46584 anti tracker/counters list.
• let adsuck to run at boot time:
  

and reboot router. In example, i've added three filters: Hosts.pub + Hosts.sex + Hosts.trc which gives over a million hosts to be blacklisted.

Feel free to ask any details you want to know!

I finally got this working... I think. I see in the system log, that adsuck starts and so does dnsmasq. What is the best way to test it? I can still see ads at the start of youtube video's which the host based blocking removed. I had to stop the host based blocking on my router because a lot of web pages were breaking as mentioned in other threads.

Cheers,
Andy

 

[mromero]

I finally got this working... I think. I see in the system log, that adsuck starts and so does dnsmasq. What is the best way to test it? I can still see ads at the start of youtube video's which the host based blocking removed. I had to stop the host based blocking on my router because a lot of web pages were breaking as mentioned in other threads.

Cheers,
Andy

I think Merlin should take this thread and make it into a coherent How-To For Adsuck and post it on his wiki:

https://github.com/RMerl/asuswrt-merlin/wiki

 

[ryzhov_al]

Any suggestions for where I can get lists of useful IPs to feed into ipset?

I-BlockList.

I finally got this working... I think. I see in the system log, that adsuck starts and so does dnsmasq. What is the best way to test it?

Try to open some blacklisted site. Don't forget to flush DNS cache on PC first.

I think Merlin should take this thread and make it into a coherent How-To For Adsuck and post it on his wiki:

https://github.com/RMerl/asuswrt-merlin/wiki

Wiki is open for everyone. It's up to you.

 

[Wycleff]

Hi

Nice How-To but something is wrong with this lines:

echo \#!/bin/sh > /jffs/scripts/services-start
echo adsuck -d -l 127.0.0.1 -c /opt/etc/adsuck -f resolv.conf -p 65053 -u admin Hosts.sex Hosts.trc Hosts.pub >> /jffs/scripts/services-start
echo no-resolv > /jffs/configs/dnsmasq.conf.add
echo server=127.0.0.1\#65053 >> /jffs/configs/dnsmasq.conf.add

I tried this and i encounter the same problem as "amdoou":
No luck, configured as above, after reboot no internet.


starting adsuck manually works with these lines:

killall dnsmasq
adsuck -d -l 127.0.0.1 -c /opt/etc/adsuck -f resolv.conf -p 65053 -u admin Hosts.sex Hosts.trc
dnsmasq --no-resolv --server=127.0.0.1#65053

Now i tried several things from the Wiki and other Sources to let Adsuck start properly after reboot but nothing works... can someone help me with a hint or something?

Thanks for your help

dsuck is a small DNS server, that returns NXDOMAIN answer for blacklisted sites, so browser/OS will not wait for answer from blocked hosts. It's main advantage over DD-WRT solution.

Requirements to run adsuck are:

• Asuswrt-Merlin driven RT-N16/RT-N66U/RT-AC66U router with USB-drive and jffs partition enabled.
• Working Entware environment. Please, refer HOW-TO for details.

Installation:

• Install adsuck package:
  
• copy provider's DNS list to adsuck folder:
  
• choose what type of sites you want to filter, all filter files is in the /opt/etc/adsuck/Hosts.*:
  • Hosts.blc - 1916 sites anti-phishing white list,
  • Hosts.mis - 566350 other potential dangerous sites (drugs, sects, spam, traps, violence)
  • Hosts.pub - 106338 advert. sites,
  • Hosts.rsk - 120033 sites with potencial dangerous contents, viruses mostly,
  • Hosts.sex - 905497 adult content sites,
  • Hosts.trc - 46584 anti tracker/counters list.
• let adsuck to run at boot time:
  

and reboot router. In example, i've added three filters: Hosts.pub + Hosts.sex + Hosts.trc which gives over a million hosts to be blacklisted.

Feel free to ask any details you want to know!

 

[SN_]

I had the same problem to get adsuck running and found the following solution:

Creating '/jffs/configs/dnsmasq.conf.add' with content:

no-resolv
server=127.0.0.1#65053

and in /jffs/scripts adding a file 'wan-start' with content:

#!/bin/sh
 
# send SIGHUP to adsuck to reread resolv.conf file
#
#killall -s SIGHUP adsuck 

# a simple line of
#killall -s SIGHUP adsuck
# should be fine after WLAN connection. But i dont get
# adsuck to listen to /etc/resolv.conf
# Therfore we do it the hard way ;)

# kill any running adsuck
#
killall adsuck

# copy the updated name server file to the adsuck folder
#
cp /etc/resolv.conf /opt/etc/adsuck/
 
# start adsuck
#
adsuck -d -l 127.0.0.1 -c /opt/etc/adsuck -f resolv.conf -p 65053 -u admin Hosts.pub Hosts.trc

Doing it this way solves the issue for me. Not very elegant but it works

 

[gijor33]

No, doesn't seem to be working because I can browse to www.anxo.com which is a listed site. (I did a flushdns). I copied the resolv.conf file so it knows about the ISP DNS servers.

If I kill adsuck then I can't resolve any hostnames from my windows PC so it is being used to resolve DNS. When I enable adsuck again I get DNS back, but it's not blocking ad sites etc.

After a while I also notice that DNS stops working (can still ping 8.8.8.8) and I see these entries in the system log. Looks like adsuck crashes.

The first line was included to show the time that it loaded, and those entries were loaded.
The second entry I notice has weird time stamps as the adjacent log entries either side are hours after the cannot fork entry. I cut some of the log lines out because they were just repeated.
The third line is when I lose DNS. Only way to get it working is to kill adsuck and run it again

Oct 27 12:36:18 adsuck[2115]: total entries: 237974
Oct 28 07:13:31 adsuck[2115]: cannot fork: Cannot allocate memory
Oct 28 17:33:06 adsuck[2115]: can't convert wire packet to struct

I'm running version 3.0.0.4.374.33 Merlin Build and an RT-N66U

Any help would be great. I feel like it's almost working

Cheers.

 

[SN_]

The third line is when I lose DNS. Only way to get it working is to kill adsuck and run it again

Oct 27 12:36:18 adsuck[2115]: total entries: 237974
Oct 28 07:13:31 adsuck[2115]: cannot fork: Cannot allocate memory
Oct 28 17:33:06 adsuck[2115]: can't convert wire packet to struct

Blocking was working fine for me but after less than 1 day adsuck stopped to functioning properly.
Got also the 'can't convert wire packet to struct' messages.

Not listening to /etc/resolv.conf as it should and throwing error messages after such a short time.
I gave up from adsuck. Atleast this build or version is just buggy and a waste of time...

 

[ml70]

Is it possible to just edit the router's /etc/hosts and save the changes and have functional ad blocking this way? WITHOUT Merlin firmware etc stuff.

 

[RMerlin]

Is it possible to just edit the router's /etc/hosts and save the changes and have functional ad blocking this way? WITHOUT Merlin firmware etc stuff.

No. Asuswrt generates the host file dynamically at boot time.

 

[Grigsby]

I have adsuck running well, for the most part. When it works. About once a week or so it starts failing to respond to dns queries, so I have to kill the process and restart.

I'm getting tons of the same error message in the log:

Dec 3 03:23:21 adsuck[22976]: can't convert wire packet to struct

But I don't know if it's related to adsuck hanging. It's getting to be a pain, especially when it happens when I'm not home -- my family isn't going to be able to fix the problem. I'm thinking about switching to using the hosts file method, which, while less elegant, seems like it would be more robust.

 

[gijor33]

I have adsuck running well, for the most part. When it works. About once a week or so it starts failing to respond to dns queries, so I have to kill the process and restart.

I'm getting tons of the same error message in the log:

Dec 3 03:23:21 adsuck[22976]: can't convert wire packet to struct

But I don't know if it's related to adsuck hanging. It's getting to be a pain, especially when it happens when I'm not home -- my family isn't going to be able to fix the problem. I'm thinking about switching to using the hosts file method, which, while less elegant, seems like it would be more robust.

I didn't manage to get adsuck working, it would load fine, but didn't actually block any ads when I test it, then it would fail and stop resolving DNS altogether.

At the moment I've got it disabled and not using it. I would love to get it running but am a bit stuck with this problem

 

[Grigsby]

Also, the more hosts I tell adsuck to load, the less stable it seems. If I only load about a 15,000-line file (like the MVPS one), it'll work for a week or so before needing a restart. If I try to load those giant French files (e.g. Hosts.sex, which contains about 900,000 lines), adsuck craps out pretty quickly. It may not be realistic to expect adsuck to manage 25+ Mb of hosts files on a small embedded device, but I still think there's a memory leak or something in adsuck that makes it crap out after a while.

 

[Juggler]

I was thinking about privoxy to filter all ads and have found a working solution for ubuntu.
http://ubuntuforums.org/showthread.php?t=1974461
In short it downloads lists, parse them and convert to privoxy format.
And you can schedule it with cron with RMerlins great FW.
But AFAIK our busybox is very limited and script didn't work as is because of it.
Maybe some Gurus can have a look at it and fix it?
IMHO it is a little more elegant solution than adsuck. Or not?

 

[calculon]

As I suspected, no performance degradation revealed while using over a 6000 strings ipset filter on my 30Mbps internet connection. This is a first approach to new HOW-TO.

Ipset feature will be a part of next Merlin's release, if you wish to try it right now, please use this build for RT-N66U: RT-N66U_3.0.0.4_270.26_ipset.trx and please note:

There will be no any WEB UI ipset page or something, it's a command line tool for geeks


It's my working script, fast and dirty. This is NOT for ad-blocking, but for preventing hacker's/scanner's attacks by blocking incoming connections from certain IP addresses. Will add some fool-proof later. Script divided to three parts, started with (#) signs:

• loading ipset modules,
• defining rules to block traffic from Tor nodes,
• defining rules to block traffic from specific countries,

Please, enable JFFS form WEB UI, put this content to /jffs/scripts/firewall-start file:

#!/bin/sh
# load ipset modules
IPSET_PATH=/lib/modules/2.6.22.19/kernel/net/ipv4/netfilter
insmod $IPSET_PATH/ip_set.ko
insmod $IPSET_PATH/ip_set_nethash.ko
insmod $IPSET_PATH/ip_set_iphash.ko
insmod $IPSET_PATH/ipt_set.ko

# block traffic from Tor nodes
iptables -D INPUT -m set --set TorNodes src -j DROP
ipset --destroy TorNodes
ipset -N TorNodes iphash
for IP in $(wget -q -O - http://torstatus.blutmagie.de/ip_list_all.php/Tor_ip_list_ALL.csv)
do
    ipset -A TorNodes $IP
done
iptables -A INPUT -m set --set TorNodes src -j DROP

# block incoming traffic from some countries. 
iptables -D INPUT -m set --set BlockedCountries src -j DROP
ipset --destroy BlockedCountries
ipset -N BlockedCountries nethash
for country in pk cn
do
    for IP in $(wget -q -O - http://www.ipdeny.com/ipblocks/data/countries/$country.zone)
    do
        ipset -A BlockedCountries $IP
    done
done
iptables -A INPUT -m set --set BlockedCountries src -j DROP

and make it executable:

chmod +x /jffs/scripts/firewall-start

You may put (un)desired countries codes to this string

where pk is for Pakistan and cn is for Chine, just for example. Please refer to ipdeny.com/ipblocks/ for full countries list.

So I have this setup on my router now listed as the following:

/tmp/home/root# cat /jffs/scripts/firewall-start

ad ipset modules
IPSET_PATH=/lib/modules/2.6.22.19/kernel/net/ipv4/netfilter
insmod $IPSET_PATH/ip_set.ko
insmod $IPSET_PATH/ip_set_nethash.ko
insmod $IPSET_PATH/ip_set_iphash.ko
insmod $IPSET_PATH/ipt_set.ko

# block traffic from Tor nodes
iptables -D INPUT -m set --set TorNodes src -j DROP
ipset --destroy TorNodes
ipset -N TorNodes iphash
for IP in $(wget -q -O - http://torstatus.blutmagie.de/ip_list_all.php/Tor_ip_list_ALL.csv)
do
    ipset -A TorNodes $IP
    done
    iptables -A INPUT -m set --set TorNodes src -j DROP

# block incoming traffic from some countries.
iptables -D INPUT -m set --set BlockedCountries src -j DROP
ipset --destroy BlockedCountries
ipset -N BlockedCountries nethash
for country in af al dz as ad ao ai ag ar am aw az bs bd bb by bj bm bt bo ba bw br io bn bg bf bi kh cm ca ky cf cl cn co cd ck cr ci hr cu cy cz dj do ec eg sv er ee et fo gf pf ga gm ge gh gi gu gt gw gy gt gn gk in id ir iq il kz ke ki kr kg la lb ls lr ly  mo mk mg mw my mv ml mt mr mu fm mn mz mm na nr np nc ni ne ng nu mp om pk pw ps pa pg py pe ph pt pr qa rw kn lc ws sm sa sn sc sl za lk sd sr sz sy tj tz tg to tt tn tr tm tv ug ae uy uz vu ve ye zm zw
    do
        for IP in $(wget -q -O - http://www.ipdeny.com/ipblocks/data/countries/$country.zone)
            do
                    ipset -A BlockedCountries $IP
            done
done
iptables -A INPUT -m set --set BlockedCountries src -j DROP

I see it running through the tornodes, and it completes those (I think) but then it produces this error:

ipset v4.5: 192.71.245.137 is already in set TorNodes.
ipset v4.5: 193.10.227.195 is already in set TorNodes.
ipset v4.5: 195.154.167.111 is already in set TorNodes.
ipset v4.5: 199.119.100.32 is already in set TorNodes.
ipset v4.5: 199.217.118.169 is already in set TorNodes.
ipset v4.5: 204.124.83.130 is already in set TorNodes.
ipset v4.5: 204.124.83.134 is already in set TorNodes.
ipset v4.5: 205.206.190.23 is already in set TorNodes.
ipset v4.5: 212.17.91.221 is already in set TorNodes.
ipset v4.5: 212.83.144.179 is already in set TorNodes.
ipset v4.5: 212.83.144.180 is already in set TorNodes.
ipset v4.5: 212.83.151.15 is already in set TorNodes.
ipset v4.5: 212.83.151.18 is already in set TorNodes.
ipset v4.5: 212.83.151.26 is already in set TorNodes.
ipset v4.5: 212.83.154.33 is already in set TorNodes.
ipset v4.5: 213.163.64.74 is already in set TorNodes.
ipset v4.5: 213.163.71.84 is already in set TorNodes.
ipset v4.5: 213.163.71.85 is already in set TorNodes.
wget: server returned error: HTTP/1.1 404 Not Found

And I am pretty sure it is failing at the IPdeny countries lists.

EDIT: I put some echo's in to help debug why it is saying the wget error (I double checked my countries, I don't see any mistyped country codes), and it seems to complete. But the ipset -L list appears, VERY small.

 

[RMerlin]

And I am pretty sure it is failing at the IPdeny countries lists.

Possibly related to this post.

 

[calculon]

Possibly related to this post.

Dang, that is quite unfortunate. I wonder if there is any alternative site that has a comparable url string or countries lists. Guess it is off to google.

 

[calculon]

So after contacting IP Deny multiple times, without a response, I am wondering if there is a way to possibly use this with MaxMind's GeoIP?

 

[calculon]

So ipdeny.com has finally fixed their zone files, however, when I run the script mentioned in the wiki, the tornode.lst file is listed with IP being added. The country.lst file however is not being created and no IP addresses are being added to the list.

However, the iptables rule is created.

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
num pkts bytes target prot opt in out source destination
1 946 44559 logdrop all -- any any anywhere anywhere state INVALID
2 13468 1857K logaccept all -- any any anywhere anywhere state RELATED,ESTABLISHED
3 381 33387 ACCEPT all -- lo any anywhere anywhere state NEW
4 9811 964K ACCEPT all -- br0 any anywhere anywhere state NEW
5 1923 638K logaccept udp -- any any anywhere anywhere udp spt:bootps dpt:bootpc
6 0 0 SSHBFP tcp -- eth0 any anywhere anywhere tcp dpt:ftp-data state NEW
7 419 40514 logdrop all -- any any anywhere anywhere
8 0 0 DROP all -- any any anywhere anywhere set TorNodes src
9 0 0 DROP all -- any any anywhere anywhere set BlockedCountries src

:/jffs/scripts# cat firewall-start
#!/bin/sh

# Loading ipset modules
lsmod | grep "ipt_set" > /dev/null 2>&1 || \
for module in ip_set ip_set_nethash ip_set_iphash ipt_set
do
insmod $module
done

# Preparing folder to cache downloaded files
IPSET_LISTS_DIR=/jffs/ipset_lists
[ -d "$IPSET_LISTS_DIR" ] || mkdir -p $IPSET_LISTS_DIR

# Block traffic from Tor nodes
if [ "$(ipset --swap TorNodes TorNodes 2>&1 | grep 'Unknown set')" != "" ]
then
ipset -N TorNodes iphash
[ -e $IPSET_LISTS_DIR/tor.lst ] || wget -q -O $IPSET_LISTS_DIR/tor.lst http://torstatus.blutmagie.de/ip_list_all.php/Tor_ip_list_ALL.csv
for IP in $(cat $IPSET_LISTS_DIR/tor.lst)
do
ipset -A TorNodes $IP
done
fi
[ -z "$(iptables-save | grep TorNodes)" ] && iptables -I INPUT -m set --set TorNodes src -j DROP

# Block incoming traffic from some countries. cn and pk is for China and Pakistan. See other countries code at http://www.ipdeny.com/ipblocks/
if [ "$(ipset --swap BlockedCountries BlockedCountries 2>&1 | grep 'Unknown set')" != "" ]
then
ipset -N BlockedCountries nethash
for country in pk cn
do
[ -e $IPSET_LISTS_DIR/$country.lst ] || wget -q -O $IPSET_LISTS_DIR/$country.lst http://www.ipdeny.com/ipblocks/data/countries/$country.zone
for IP in $(cat $IPSET_LISTS_DIR/$country.lst)
do
ipset -A BlockedCountries $IP
done
done
fi
[ -z "$(iptables-save | grep BlockedCountries)" ] && iptables -I INPUT -m set --set BlockedCountries src -j DROP

:/jffs/ipset_lists# ls -al
drwxrwxrwx 2 user root 0 Aug 22 19:11 .
drwxr-xr-x 6 user root 0 Aug 22 19:11 ..
-rw-rw-rw- 1 user root 86105 Aug 22 19:11 tor.lst

/dev/mtdblock4 5.5M 488.0K 5.0M 9% /jffs

So my question is, is what is causing the functions to fail downloading the country lists?

EDIT: ipdeny.com also is offering aggregated zones, is there a good way to incorporate that? I would assume just changing the script file to be '$country'-aggregate.zone would suffice?