[Beta] AB-Solution 3.x

[thelonelycoder]

Also, I started to add a comment for something in my whitelist by adding it again (intending to delete the uncommented one later), and I see you trap for whether it is in the blocking list. If not, you don't add it to the list. I wonder, though, whether one should have the ability to force the addition, in case the blocking list when updated adds it back, or you change to a bigger list.

Almost missed that remark.
I can't find where we discussed that, unless I read pages of pages of the private beta discussion threads we've had. The search function on this forum does not work for discussions and for google they are inaccessible.
Let's say we were over cautious at the time we discussed it.
I'll see if I can add an option to add whitelist entries anyway, even if not found in the blocking file.
Should not be too hard. This might actually be good for blocking file changes, along with Skynet auto-whitelisting all whitelist entries from Ab-Solution.

 

[yk101]

Beta has been running good, but I see the following in the report:

192.168.2.201, nas02:
 --------------------------------------------------------
grep: bad regex ' [2001$': Unmatched [ or [^
grep: bad regex ' [2001$': Unmatched [ or [^
grep: bad regex '[2001$': Unmatched [ or [^
 3089   checkip.synology.com                      
 1529   192-168-2-201.2a7a336574384a809edc274399e34afb.plex.direct  
 994    ddns.synology.com                         
 590    192.168.2.201                             
 227    [2001                                     
 223    checkipv6.synology.com                    
 217    cloudsync-tw.synology.com                 
 169    global.quickconnect.to                    
 72     plex.tv                                   
 57     www.googleapis.com

Is this something to be concern with?

Then there is this:

 The top 10 blocked ad domains were:
 --------------------------------------------------------
 1454   g.symcd.com                               blocked
 749    192-168-2-201.2a7a336574384a809edc274399e34afb.plex.direct  blocked
 746    192-168-2-210.2a7a336574384a809edc274399e34afb.plex.direct  blocked
 476    counter.yadro.ru                          blocked
 408    ssl.google-analytics.com                  blocked
 375    www.google-analytics.com                  blocked
 295    mb15                                      blocked

The plex.direct addresses seem to be legit and used by PlexPass. Any attempt to add them to the whitelist fail...
The mb15 is a local computer (mac book pro) that also seems to be deemed dangerous and blocked. Is that expected?

 

[elorimer]

Let's say we were over cautious at the time we discussed it.

Looking in the 3.7 beta thread, we had a conversation about removing whitelist entries from whitelist.txt, and tomsk noted that at the time, the removed whitelist entry was not added back into the blocking file. He also suggested that a whitelist not be added unless it was in the blocking file. So that feature was added: If you remove the whitelist entry, it is added back into the blocking file, to remain there until the blocking file is updated.

I'm thinking we might have figured that an entry shouldn't be whitelisted unless it was in the blocking file in the first place. If I forced the addition of a whitelist entry, that wasn't in the blocking file, and then if I deleted it it would go into the blocking file, as if it was in the blacklist, at least until the blocking file is updated.

Could that be what you recall?

If so, now that the whitelist is made available to skynet et al, ab-s becomes convenient one stop shopping for whitelists. Skynet may pull a domain into its list that isn't in your blocking file (like wordpress blogs). It may be better on deleting a whitelist entry to warn either that it will be blocked until the next update, or not add it to the blocklist and warn that it won't be blocked until the next update, assuming it is in one of the update lists.

 

[thelonelycoder]

227 [2001

This is what's throwing that error. Any idea where that name comes from? Do you have a device named "[2000" (and maybe someting more)?

But first, try setting rs option "4. Filter local client names" to on.

 

[thelonelycoder]

Could that be what you recall?

Exactly that one. What page is it on? Thanks.

Edit: found the discussion.

 

[yk101]

This is what's throwing that error. Any idea where that name comes from? Do you have a device named "[2000" (and maybe someting more)?

But first, try setting rs option "4. Filter local client names" to on.

I do not have anything named "[2001"! After setting the filter option to ON, I still see the same error:

 192.168.2.201, nas02:
 --------------------------------------------------------
grep: bad regex ' [2001$': Unmatched [ or [^
grep: bad regex ' [2001$': Unmatched [ or [^
grep: bad regex '[2001$': Unmatched [ or [^
 3136   checkip.synology.com                      
 1529   192-168-2-201.2a7a336574384a809edc274399e34afb.plex.direct  
 1008   ddns.synology.com                         
 237    [2001                                     
 229    checkipv6.synology.com                    
 222    cloudsync-tw.synology.com                 
 171    global.quickconnect.to                    
 72     plex.tv                                   
 57     www.googleapis.com                        
 39     retracker.local

I just noticed that dnsmasq.log2 size is 92.1M! Should it be that large?

File sizes:
 --------------------------------------------------------
 blocking_file  2.5M
 blacklist.txt  4.0K
 whitelist.txt  4.0K
 dnsmasq.log2   92.1M
 dnsmasq.log1   1.8M
 dnsmasq.log    1.6M
 --------------------------------------------------------

 Stats compiling times, in seconds:
 --------------------------------------------------------
 Ad-Blocking stats:                  2
 The top 10 requested domains:       16
 The top 10 blocked ad domains:      8
 The top 10 noisiest name clients:   84
 Top 10 domains for top 10 clients:  94

 Total time to compile stats:        204
 --------------------------------------------------------

 

[thelonelycoder]

I do not have anything named "[2001"! After setting the filter option to ON, I still see the same error:

In a SSH terminal enter this, replace <your device> with the USB device Ab-Solution is installed on:

grep "\[2001" /mnt/<your device>/adblocking/logs/dnsmasq.log*

I just noticed that dnsmasq.log2 size is 92.1M! Should it be that large?

It can get that large, it contains all log entries from this week, except yesterdays (dnsmasq.log1) and today's (dnsmasq.log).

BTW what router/firmware is AB running on?

 

[yk101]

In a SSH terminal enter this, replace <your device> with the USB device Ab-Solution is installed on:

grep "\[2001" /mnt/<your device>/adblocking/logs/dnsmasq.log*

I was just about to do that. Looks like one of my NAS servers is issuing a malformed DNS query...

/mnt/rt-ac88u/adblocking/logs/dnsmasq.log2:Nov 17 20:52:03 dnsmasq[30219]: query[AAAA] [2001 from 192.168.2.201
/mnt/rt-ac88u/adblocking/logs/dnsmasq.log2:Nov 17 20:52:03 dnsmasq[30219]: forwarded [2001 to 8.8.4.4
/mnt/rt-ac88u/adblocking/logs/dnsmasq.log2:Nov 17 21:02:20 dnsmasq[30219]: query[AAAA] [2001 from 192.168.2.201
/mnt/rt-ac88u/adblocking/logs/dnsmasq.log2:Nov 17 21:02:20 dnsmasq[30219]: forwarded [2001 to 8.8.4.4

BTW what router/firmware is AB running on?

RT-AC88U (RT-AC88U) Firmware-382.1 @ 192.168.2.1
Compiled by AB-Solution 3.9.3.1, Stats version 3.9.3.0

 

[thelonelycoder]

I was just about to do that. Looks like one of my NAS servers is issuing a malformed DNS query...

/mnt/rt-ac88u/adblocking/logs/dnsmasq.log2:Nov 17 20:52:03 dnsmasq[30219]: query[AAAA] [2001 from 192.168.2.201
/mnt/rt-ac88u/adblocking/logs/dnsmasq.log2:Nov 17 20:52:03 dnsmasq[30219]: forwarded [2001 to 8.8.4.4
/mnt/rt-ac88u/adblocking/logs/dnsmasq.log2:Nov 17 21:02:20 dnsmasq[30219]: query[AAAA] [2001 from 192.168.2.201
/mnt/rt-ac88u/adblocking/logs/dnsmasq.log2:Nov 17 21:02:20 dnsmasq[30219]: forwarded [2001 to 8.8.4.4

RT-AC88U (RT-AC88U) Firmware-382.1 @ 192.168.2.1
Compiled by AB-Solution 3.9.3.1, Stats version 3.9.3.0

query[AAAA] is IPv6, check what your IPv6 settings are on the router and the device. Google may help if you can't find an error.

295 mb15 blocked

Could the mb15 be a machine name or part of a entry in the blacklist.txt file?

 

[yk101]

query[AAAA] is IPv6, check what your IPv6 settings are on the router and the device. Google may help if you can't find an error.

Could the mb15 be a machine name or part of a entry in the blacklist.txt file?

Not likely!

# No IP address mapping (0.0.0.0 123.123.12.3) and
# no wildcards (*pricegrabber.com). This WILL NOT work.
# Run [el] to read changes into Dnsmasq.
# Add blacklisted domains as follows, without the leading #.
# 0.0.0.0 pricegrabber.com
# 0.0.0.0 www.pricegrabber.com

The funny thing is that IPv6 is OFF for all interfaces, including the one making the call:

ovs_eth0  Link encap:Ethernet  HWaddr 00:11:32:73:0A:2D  
          inet addr:192.168.2.201  Bcast:192.168.2.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:121644570 errors:0 dropped:0 overruns:0 frame:0
          TX packets:75299273 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:116368824719 (108.3 GiB)  TX bytes:119082661877 (110.9 GiB)

 

[thelonelycoder]

Not likely!

# No IP address mapping (0.0.0.0 123.123.12.3) and
# no wildcards (*pricegrabber.com). This WILL NOT work.
# Run [el] to read changes into Dnsmasq.
# Add blacklisted domains as follows, without the leading #.
# 0.0.0.0 pricegrabber.com
# 0.0.0.0 www.pricegrabber.com

The funny thing is that IPv6 is OFF for all interfaces, including the one making the call:

ovs_eth0  Link encap:Ethernet  HWaddr 00:11:32:73:0A:2D 
          inet addr:192.168.2.201  Bcast:192.168.2.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:121644570 errors:0 dropped:0 overruns:0 frame:0
          TX packets:75299273 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:116368824719 (108.3 GiB)  TX bytes:119082661877 (110.9 GiB)

So, IPv6 is off on both routers?
Could you run a grep for "mb15" on all dnsmasq log files?

grep "mb15" /mnt/<your device>/adblocking/logs/dnsmasq.log*

 

[yk101]

So, IPv6 is off on both routers?
Could you run a grep for "mb15" on all dnsmasq log files?

grep "mb15" /mnt/<your device>/adblocking/logs/dnsmasq.log*

Yes, IPv6 is off on both routers. My ISP doesn't support it, and using a tunnel to provide support kills my connection throughput.

mnt/rt-ac88u/adblocking/logs/dnsmasq.log2:Nov 18 00:42:29 dnsmasq[30219]: query[AAAA] mb15 from 192.168.2.20
/mnt/rt-ac88u/adblocking/logs/dnsmasq.log2:Nov 18 00:42:29 dnsmasq[30219]: forwarded mb15 to 8.8.4.4

Looks like mb15 may have IPv6 enabled. It is a laptop after all...
The thing I don't understand - why would dnsmasq forward local query to google? Wouldn't that be violating RFP?

 

[thelonelycoder]

Looks like mb15 may have IPv6 enabled. It is a laptop after all...
The thing I don't understand - why would dnsmasq forward local query to google? Wouldn't that be violating RFP?

That's why AB has built in options to suppress these.
In Experimenal settings es enable the dnsmasq settings, then set default (d) for the first two and set the third (domain-needed) to on.
These queries are now ignored by dnsmasq and the next full stats with this setting will no longer show the machine name mb15.

 

[yk101]

Thanks @thelonelycoder! I will do just that.

 

[thelonelycoder]

Thanks @thelonelycoder! I will do just that.

And I bet the mb15 is a Mac, possibly a MacBook Pro.

 

[yk101]

And I bet the mb15 is a Mac, possibly a MacBook Pro.

Indeed. it is a 15" MBP.

 

[thelonelycoder]

Indeed. it is a 15" MBP.

They have that unwelcome behaviour built in.

 

[thelonelycoder]

I spent today adding some new cool features to the el function.
But at the end of the day (literally) I noticed these plus the new comment function have an impact on the stats generation.
I'll have to ponder about the decisions I made and how best to get out of it without a loss of these most helpful features.
It's a beta version, things go wrong but I'll find a way, I'm sure of that.

 

[Butterfly Bones]

Watching this conversation, especially the IPv6 parts. I see this:

Nov 18 10:09:58 dnsmasq[32394]: query[AAAA] www.google.com from 192.168.1.5
Nov 18 10:09:58 dnsmasq[32394]: cached www.google.com is 2607:f8b0:4005:807::2004

I have IPv6 disabled in the router (AC68U). 192.168.1.5 is my ethernet connected linux laptop. I have disabled IPv6 in the laptop using both sysctl and grub, and verified no IPv6 address in the laptop via ifconfig. I've been trying to find why for a long time with no luck.

I know this is possibly off topic for ab-solution, but since using the follow log file function is where I keep seeing this, any idea? Is it a dns leak since I have a VPN tunnel for all clients, and using dnscrypt as well? All diagnostic sites I have tried show no dns leaks.

Thank you.

 

[thelonelycoder]

Watching this conversation, especially the IPv6 parts. I see this:

Nov 18 10:09:58 dnsmasq[32394]: query[AAAA] www.google.com from 192.168.1.5
Nov 18 10:09:58 dnsmasq[32394]: cached www.google.com is 2607:f8b0:4005:807::2004

I have IPv6 disabled in the router (AC68U). 192.168.1.5 is my ethernet connected linux laptop. I have disabled IPv6 in the laptop using both sysctl and grub, and verified no IPv6 address in the laptop via ifconfig. I've been trying to find why for a long time with no luck.

I know this is possibly off topic for ab-solution, but since using the follow log file function is where I keep seeing this, any idea? Is it a dns leak since I have a VPN tunnel for all clients, and using dnscrypt as well? All diagnostic sites I have tried show no dns leaks.

Thank you.

I'm no expert with these things. But I've seen IPv6 queries suddenly starting when one of the devices has it set to on.
Disabling it, and double checking that no device has v6 enabled and then shut down all devices and start them up again helps it.
I usually unplug my cable modem too, then plug it back in, next is the primary router and so forth.
This always makes the v6 queries go away.