Diversion Diversion - the Router Ad-Blocker

[dave14305]

Interesting, Diversion maps up to 20 domains per line to the blocking IP in the blockinglist. At what position is that domain?

I can see similar behavior when a blocked domain is returned as a result of a CNAME. Is this a loophole for ad servers?

# dig look.udncoeln.com @127.0.0.1

; <<>> DiG 9.14.4 <<>> look.udncoeln.com @127.0.0.1
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 24611
;; flags: qr rd ra; QUERY: 1, ANSWER: 5, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;look.udncoeln.com.             IN      A

;; ANSWER SECTION:
look.udncoeln.com.      118     IN      CNAME   ads.ad-center.com.
ads.ad-center.com.      60      IN      A       184.73.206.80
ads.ad-center.com.      60      IN      A       107.22.210.53
ads.ad-center.com.      60      IN      A       54.204.2.119
ads.ad-center.com.      60      IN      A       184.73.225.11

;; Query time: 23 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Mon Nov 04 13:07:02 EST 2019
;; MSG SIZE  rcvd: 138

# dig ads.ad-center.com @127.0.0.1

; <<>> DiG 9.14.4 <<>> ads.ad-center.com @127.0.0.1
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 12000
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;ads.ad-center.com.             IN      A

;; ANSWER SECTION:
ads.ad-center.com.      0       IN      A       192.168.1.2

;; Query time: 1 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Mon Nov 04 13:07:13 EST 2019
;; MSG SIZE  rcvd: 62

 

[dugaduga]

Interesting, Diversion maps up to 20 domains per line to the blocking IP in the blockinglist. At what position is that domain?

@thelonelycoder position 6, here is my blocklist: https://ufile.io/jp6vblll

you need to block look.udncoeln.com; ive spoken to dnsmasq & dnscrypt maintainer about blocking incoming cname responses in the wildcard list, the latter was in an unfortunate and tragic blow to innovation, reluctant, as this will become more and more a problem over time. for example, dnscrypt can block only outgoing requests, but if a cname points from outgoing request look.ichlnk.com that responds as ads.ad-center.com it is incapable of parsing the response, the only way is to block the cname. and yea is a way for trackers & ads to sneak by. Esp if cnames use cname responses like analytics.google.com or whatever

Dnsmaq maintainer is clearly ahead of the evolutionary bell curve, but seems dnsmasq blocks cnames of a, but not aaaa. could be dnsmasq bug or diversion 20 line perhaps? a temporary solution is blocking look.udncoeln.com ; ive made a blocklist here, confirmed via threatcrowd, virustotal and robtex, many of these domains are only a few weeks old:

look.utnldn.com
look.kfioplkn.com

look.ichlnk.com
look.udncoeln.com
look.fjfiln.com
look.ufinkln.com
hlok.qertewrt.com
look.opskln.com
look.ichlnk.com
look.ufinkln.com
look.utndln.com
look.djfiln.com
look.kfiopkln.com
look.udncoeln.com

goa.camimbu.com
zulu.r867qq.net

ads.ad-center.com
cdn1.ad-center.com
cdn2.ad-center.com
ads4.ad-center.com
ads.ad-center.com
ads-script.ad-center.com
www.ad-center.com

#these are closely associated and or interacting directly with ad-center https://threatcrowd.org/domain.php?domain=pwwysydh.com

pwwysydh.com
piz7ohhujogi.com

prjcq.com
pocofh.com
npmpecd.com
pt21na.com
pwdplz.com
pohs2oom.com
ps4ux.com
jadserve.postrelease.com
docaz.net
prjcq.com
e.serverbid.com
d303resfoavrtt.cloudfront.net
payae8moon9.com
pohs2oom.com

#all ptrs for ads.ad-center.com https://www.robtex.com/dns-lookup/ads.ad-center.com

ec2-46-137-77-217.eu-west-1.compute.amazonaws.com
ec2-46-137-110-83.eu-west-1.compute.amazonaws.com
ec2-54-243-100-198.compute-1.amazonaws.com
ec2-175-41-141-63.ap-southeast-1.compute.amazonaws.com
ec2-175-41-150-101.ap-southeast-1.compute.amazonaws.com
ec2-176-34-182-24.eu-west-1.compute.amazonaws.com
ec2-184-72-41-50.us-west-1.compute.amazonaws.com
ec2-184-169-151-9.us-west-1.compute.amazonaws.com
ec2-184-169-178-22.us-west-1.compute.amazonaws.com
ec2-184-169-182-88.us-west-1.compute.amazonaws.com

 

[ColinTaylor]

I can see similar behavior when a blocked domain is returned as a result of a CNAME. Is this a loophole for ad servers?

Possibly, but I think there are two (or arguably 3) different queries happening here.

There's the initial query for look.udncoeln.com which retrieves the CNAME and its associated IP addresses. This is working correctly. It's difficult to see how dnsmasq could block this CNAME and the addresses because it would have to filter all the incoming replies. I would argue that this is not a role that dnsmasq should be performing.

Then there is a second query, separate from the first, that queries ads.ad-center.com. Presumably the client got this name from the previous CNAME response. Anyway, it does an IPv4 (A) and IPv6 (AAAA) query on that name. Only the IPv4 name is blocked because the router's hosts file only contains IPv4 address. There is no matching IPv6 address in the hosts file so the AAAA request is forwarded upstream.

To block the IPv6 lookups you would need to duplicate every line in the hosts file with an equivalent IPv6 entry.

 

[dugaduga]

It's difficult to see how dnsmasq could block this CNAME and the addresses because it would have to filter all the incoming replies. I would argue that this is not a role that dnsmasq should be performing.

This is exactly what I would pay him to do, and it is imperative that it is an optional feature if you care about security and choice.

I have not heard a single rational or valid arguments against it. I'm all ears.

"Yeah we don't want features that can block otherwise impossible to block zero day malware, tracking and ads, we want vanilla, same as always limited systems with fewer options so we cant chose for ourselves."

Then there is a second query, separate from the first, that queries ads.ad-center.com. Presumably the client got this name from the previous CNAME response. Anyway, it does an IPv4 (A) and IPv6 (AAAA) query on that name. Only the IPv4 name is blocked because the router's hosts file only contains IPv4 address. There is no matching IPv6 address in the hosts file so the AAAA request is forwarded upstream.

To block the IPv6 lookups you would need to duplicate every line in the hosts file with an equivalent IPv6 entry.

I see, this is another option that I requested from simon @dnsmaq, to block aaaa... there is a patch available for it but that is all. He didn't like the idea of user choice if I recall.

dnsmasq-aaaa-filter.patch

From 966471712184cfb3b067f2ae8dad9d8e2a896cae Mon Sep 17 00:00:00 2001
From: Bearice Ren <bearice@icybear.net>
Date: Tue, 20 Sep 2016 11:52:08 +0800
Subject: [PATCH] add filter-aaaa option

---
 src/dnsmasq.h | 3 ++-
 src/option.c  | 3 +++
 src/rfc1035.c | 9 +++++++++
 3 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/src/dnsmasq.h b/src/dnsmasq.h
index 1896a64..e10d6c4 100644
--- a/src/dnsmasq.h
+++ b/src/dnsmasq.h
@@ -238,7 +238,8 @@ struct event_desc {
 #define OPT_SCRIPT_ARP     53
 #define OPT_MAC_B64        54
 #define OPT_MAC_HEX        55
-#define OPT_LAST           56
+#define OPT_FILTER_AAAA    56
+#define OPT_LAST           57

 /* extra flags for my_syslog, we use a couple of facilities since they are known
    not to occupy the same bits as priorities, no matter how syslog.h is set up. */
diff --git a/src/option.c b/src/option.c
index d8c57d6..1cc65bf 100644
--- a/src/option.c
+++ b/src/option.c
@@ -159,6 +159,7 @@ struct myoption {
 #define LOPT_SCRIPT_ARP    347
 #define LOPT_DHCPTTL       348
 #define LOPT_TFTP_MTU      349
+#define LOPT_FILTER_AAAA   350

 #ifdef HAVE_GETOPT_LONG
 static const struct option opts[] =
@@ -323,6 +324,7 @@ static const struct myoption opts[] =
     { "dns-loop-detect", 0, 0, LOPT_LOOP_DETECT },
     { "script-arp", 0, 0, LOPT_SCRIPT_ARP },
     { "dhcp-ttl", 1, 0 , LOPT_DHCPTTL },
+    { "filter-aaaa", 0, 0, LOPT_FILTER_AAAA },
     { NULL, 0, 0, 0 }
   };

@@ -494,6 +496,7 @@ static struct {
   { LOPT_LOOP_DETECT, OPT_LOOP_DETECT, NULL, gettext_noop("Detect and remove DNS forwarding loops."), NULL },
   { LOPT_IGNORE_ADDR, ARG_DUP, "<ipaddr>", gettext_noop("Ignore DNS responses containing ipaddr."), NULL },
   { LOPT_DHCPTTL, ARG_ONE, "<ttl>", gettext_noop("Set TTL in DNS responses with DHCP-derived addresses."), NULL },
+  { LOPT_FILTER_AAAA, OPT_FILTER_AAAA, NULL, gettext_noop("Filter all AAAA requests."), NULL },
   { 0, 0, NULL, NULL, NULL }
 };

diff --git a/src/rfc1035.c b/src/rfc1035.c
index 24d08c1..1594962 100644
--- a/src/rfc1035.c
+++ b/src/rfc1035.c
@@ -1783,6 +1783,15 @@ size_t answer_request(struct dns_header *header, char *limit, size_t qlen,
            }
        }

+      //patch to filter aaaa forwards
+      if (qtype == T_AAAA && option_bool(OPT_FILTER_AAAA) ){
+        //return a null reply
+        ans = 1;
+        if (!dryrun) log_query(F_CONFIG | F_IPV6 | F_NEG, name, &addr, NULL);
+        break;
+      }
+      //end of patch
+
       if (!ans)
        return 0; /* failed to answer a question */
     }
--

I block all aaaa at the endpoint level with acrylic dns proxy for example, and network wide with dnscrypt. Acrylic does it right, see this, acrylic xonfig:

; In the following example only the requests for A, AAAA, MX and SRV query types get forwarded to the primary DNS
; server:
;
; PrimaryServerQueryTypeAffinityMask=A;AAAA;MX;SRV
;
; All query types are supported, either explicitly using A, AAAA, CNAME, MX, NS, PTR, SOA, SRV and TXT or implicitly
; using their decimal values.
;
; The supported query types are:
;
; A            NS           MD           MF           CNAME
; SOA          MB           MG           MR           NULL
; WKS          PTR          HINFO        MINFO        MX
; TXT          RP           AFSDB        X25          ISDN
; RT           NSAP         NSAPPTR      SIG          KEY
; PX           GPOS         AAAA         LOC          NXT
; EID          NIMLOC       SRV          ATMA         NAPTR
; KX           CERT         A6           DNAME        SINK
; OPT          APL          DS           SSHFP        IPSECKEY
; RRSIG        NSEC         DNSKEY       DHCID        NSEC3
; NSEC3PARAM   TLSA         HIP          NINFO        RKEY
; TALINK       CDS          CDNSKEY      OPENPGPKEY   CSYNC
; SPF          UINFO        UID          GID          UNSPEC
; NID          L32          L64          LP           EUI48
; EUI64        ADDRS        TKEY         TSIG         IXFR
; AXFR         MAILB        MAILA        ALL          URI
; CAA          TA           DLV          WINS         WINSR
;
PrimaryServerQueryTypeAffinityMask=A;CNAME

 

[unclebuk]

Please elaborate what you mean by "hack attempt"... and other checks, you could do that check after converting the host file to a compatible format.

You can do a hack attempt, and other checks, and that may be cool on your own private system, but I'm pretty sure most people would rather have the option whether or not they want to take that risk, and protect themselves against hackers with a more diverse set of hosts. A prior warning and option would suffice.

I have seven lists that I update manually adding the loopback, which I have custom added to my own git, most of the lists are updated daily, but not mine, its not an automated system... it has dramatically increased my hit list in blocklist log, privacy, and security. I am currently only able to use about 1/3 or so of the lists I would like to... there is just too many to update manually.

Question: I notice that AB solutions does not block ads on the Tor browser (at least not for me), any suggestions for a work-around for this anomaly I am experiencing?

Thanks,
buk

 

[DonCatoli]

What address(s) do I need to whitelist so that clicknupload.org downloads are not blocked, clicknupload.org itself is not in the Small+ list so something else is being blocked...I of course have to disable diversion while downloading.

 

[duffydack]

Large list:
http://support.it-mate.co.uk/downloads/hosts.txt
This page is not available !!!
Please fix it or use another page .

Been happening to me for a couple of weeks, I guess it has gone way of the dodo.

processing file 1 of 5
 remote file newer: getting new hosts file
 http://support.it-mate.co.uk/downloads/hosts.txt
Warning: Transient problem: timeout Will retry in 1 seconds. 3 retries left.                                     
Warning: Transient problem: timeout Will retry in 2 seconds. 2 retries left.                                     
Warning: Transient problem: timeout Will retry in 4 seconds. 1 retries left.             #      #     #    #     
-=O=-                                                                                                            ##
curl: (28) Connection timed out after 10001 milliseconds
 http://support.it-mate.co.uk/downloads/hosts.txt
 not hosts file, using backup file

 

[calzor suzay]

So I added both google ad urls but I still end up at a blank page cert error, when looking at the cert it refers to pixelsrv, Do I need to add the pixelsrv cert to my PC, I thought I had already.

 

[HairyA00]

So I added both google ad urls but I still end up at a blank page cert error, when looking at the cert it refers to pixelsrv, Do I need to add the pixelsrv cert to my PC, I thought I had already.

Let's temporarily disable pixelserv-tls and see if that works:
Open Diversion > ep > 1

After it's disabled, try to reproduce the problem (you can re-enable it the same way).

 

[calzor suzay]

Let's temporarily disable pixelserv-tls and see if that works:
Open Diversion > ep > 1

After it's disabled, try to reproduce the problem (you can re-enable it the same way).

Well doesn't make sense

So I disabled as you said and tried one of the google shopping links at top of a search and it worked.
I then turned it back on to prove the point and it carried on working...

Some links don't work but they seem to go dartsearch.net and I don't want to allow a whole load of stuff, the majority seem to work now, thanks

 

[Vulcano]

Can you please check this site https://www.andrerieu.com/ if diversion is enable, in the computer of my wife dont open (connected by wireless) and on my computer the site open (by wire), but if disable diversion the site open on both, I dont know why, any ideias?

 

[Butterfly Bones]

Can you please check this site https://www.andrerieu.com/ if diversion is enable, in the computer of my wife dont open (connected by wireless) and on my computer the site open (by wire), but if disable diversion the site open on both, I dont know why, any ideias?

The website opens fine in Chrome browser on Linux desktop and Chrome browser on iPhone. It does not open in Safari browser on iPhone, iPad, or Macbook. Looks like a Safari browser issue.

My router info in my signature below. Diversion Standard+ list.

 

[dugaduga]

Question: I notice that AB solutions does not block ads on the Tor browser (at least not for me), any suggestions for a work-around for this anomaly I am experiencing?

Thanks,
buk

add ublock origin extension/addon and enable all the lists under options, technically you could import your own list from ab-solution as well, if the list is not too big it shouldn't be an issue; you can upload a copy to github and import it in ubo, though that might give away some of your anonymity to microsoft who owns github.

What address(s) do I need to whitelist so that clicknupload.org downloads are not blocked, clicknupload.org itself is not in the Small+ list so something else is being blocked...I of course have to disable diversion while downloading.

Follow DNSMASQ, hit "F, 1, 3" in diversion and watch and see what pops up.

@thelonelycoder most of the time when following dnsmasq blocks only, the hits stop showing up after a few minutes. I have to exit and hit reload the follow tool you made to show new hits, they are showing up in dnsmasq.log.

 

[thelonelycoder]

@thelonelycoder most of the time when following dnsmasq blocks only, the hits stop showing up after a few minutes. I have to exit and hit reload the follow tool you made to show new hits, they are showing up in dnsmasq.log.

Something broke the pipe. Usually when Dnsmasq restarts. I may use the -F flag for tail instead of -f in the next release.

 

[dugaduga]

@thelonelycoder tx, its very annoying, lasts on average no more than 5 minutes.

What is the difference/benefit between using my own custom zero day blocklist that is equivalent in size, but more accurate with zero day hosts, vs the hphost plus list in terms of false positives with skynet, if any at all?

 

[thelonelycoder]

@thelonelycoder tx, its very annoying, lasts on average no more than 5 minutes.

What is the difference/benefit between using my own custom zero day blocklist that is equivalent in size, but more accurate with zero day hosts, vs the hphost plus list in terms of false positives with skynet, if any at all?

I don't know, that depends on what list you use.
Skynet offloads these domains in the Plus hosts specifically to Diversion so it does not have to add it to the blacklist as they are outbound blocks only.

 

[Vulcano]

The website opens fine in Chrome browser on Linux desktop and Chrome browser on iPhone. It does not open in Safari browser on iPhone, iPad, or Macbook. Looks like a Safari browser issue.

My router info in my signature below. Diversion Standard+ list.

Thank you, for now open with dnsfilter...

 

[dugaduga]

bug 1:

1. Edit whitelist
2. Edit blacklist
3. Edit wildcard-blacklist
4. Process all lists
5. List Settings: Colors on, Pagination off Auto-add www/non-www domain off (whitelist only) [Pagination is actually said to be on when you hit 5, not sure which one is accurate]

6. Delete backup files (81)

bug 2: whitelisting is not removing images-na.ssl-images-amazon.com from blacklist after processing whitelist or all lists; ****UPDATE****: its also in my custom blacklist, removing fixed this issue.

bug 3: tail -f breaks constantly

@thelonelycoder, thx

 

[Grisu]

Can you please check this site https://www.andrerieu.com/ if diversion is enable, in the computer of my wife dont open (connected by wireless) and on my computer the site open (by wire), but if disable diversion the site open on both, I dont know why, any ideias?

Cant open it too with FF or IE11 while ok with Edge, dont use any scripts or whatever at all only default provider modemrouter.
But all ok with language extension: https://www.andrerieu.com/en

 

[thelonelycoder]

bug 1: (...)
bug 2: whitelisting is not removing images-na.ssl-images-amazon.com from blacklist after processing whitelist or all lists; ****UPDATE****: its also in my custom blacklist, removing fixed this issue.
bug 3: tail -f breaks constantly
@thelonelycoder, thx

Bug 1: The wording may be confusing but the state is correct in both lines. Will be more clearer in the next Diversion release. Thanks for reporting!
Bug 2: When whitelisting a domain found in the blacklist, it will correctly be removed in the blacklist.
Bug 3: Check your Syslog, something is restarting Dnsmasq constantly and it's not Diversion unless you invoke it through it. Next Diversion release will have the tail -F flag which continues to try to tail after the pipe is broken.