Unbound - Authoritative Recursive Caching DNS Server

[Cam]

echo "Download root.zone file..."
curl -o /opt/var/lib/unbound/root.zone https://www.internic.net/domain/root.zone

I don't think you need this in your script as unbound downloads it for you if file doesn't exist.

 

[rgnldo]

I don't think you need this in your script as unbound downloads it for you if file doesn't exist.

In general, I use the unbound_manager installer script. This script is experimental. It generates the personal file unbound.conf that I am currently using. It is to avoid the work of redoing from the beginning.

 

[Cam]

Fair enough, but it's still not needed Anyway thought it was worth mentioning just in case auth-zone makes it into the default unbound.conf
Though if you do, then you also need an option with path to your ca-bundle option for this to work, so you have to also add...

tls-cert-bundle: "/opt/etc/ssl/certs.pem" (or /rom/etc/ssl/cert.pem as appropriate)

 

[rgnldo]

Fair enough, but it's still not needed Anyway thought it was worth mentioning just in case auth-zone makes it into the default unbound.conf

I didn't find documentation about SSL certificate? Is it working for you?
There are no plans for inclusion. I am organizing the auth-zone solution as a tcp transport together with other solutions.

 

[rgnldo]

Update *.conf file:
Improve efficiency and security in queries and responses with unbound.

Update adlist:
Improve ads tracker's:

A tracker is a script put on many websites to gather informations about the visitor. They can be used for multiple reasons: statistics, risk management, marketing, ads serving… In any case, they are a threat to Internet users' privacy and many may want to block them.

Traditionnaly, trackers are served from a third-party. For example, website1.com and website2.com both load their tracking script from https://trackercompany.com/trackerscript.js. In order to block those, one can simply block the hostname trackercompany.com, which is what most ad blockers do.

However, to circumvent this block, tracker companies made the websites using them load trackers from somestring.website1.com. The latter is a DNS redirection to website1.trackercompany.com, directly to an IP address belonging to the tracking company.

Those are called first-party trackers. On top of aforementionned privacy issues, they also cause some security issue, as websites usually trust those scripts more. For more information, learn about Content Security Policy, same-origin policy and Cross-Origin Resource Sharing.

In order to block those trackers, ad blockers would need to block every subdomain pointing to anything under trackercompany.com or to their network. Unfortunately, most don't support those blocking methods as they are not DNS-aware, e.g. they only see somestring.website1.com.

This list is an inventory of every somestring.website1.com found to allow non DNS-aware ad blocker to still block first-party trackers.
About:
https://hostfiles.frogeye.fr/#whats-a-first-party-tracker

Run unbound_manager
set update configurations

 

[Cam]

I didn't find documentation about SSL certificate? Is it working for you?.

Yes it works for me it's also needed if you want to turn on the DNS over TLS feature of unbound (be it direct is forwarded).

https://nlnetlabs.nl/documentation/unbound/unbound.conf/

Sent from my Nokia 7 plus using Tapatalk

 

[rgnldo]

DNS over TLS feature

Yes, with DNS forwarding over TLS it works. With this option set: forward-tls-upstream: yes

tls-cert-bundle: "/etc/ssl/certs/ca-certificates.crt"
    forward-zone:
        name: "."
        forward-tls-upstream: yes
        forward-addr: 185.222.222.222@853
        forward-addr: 185.184.222.222@853
        forward-addr: 2a09::@853
        forward-addr: 2a09::1@853

 

[rgnldo]

I don't think you need this in your script as unbound downloads it for you if file doesn't exist.

That's right. Checked. Once the location is correct, unbound does the rest with url setting. Keep collaborating. We all thank you!

 

[dave14305]

@Martineau I had also written a personal script to convert the Diversion blockinglist to an Unbound conf file for Pixelserv ad-block. I came to realize that using local-zone requires too much memory and it works well-enough to only use local-data. I read in the unbound-users maillist that each local-zone requires 8k of memory. Here was my script if anyone wants to compare memory utilization:

#!/bin/sh

if [ /opt/share/diversion/list/blockinglist -nt /opt/var/lib/unbound/ads.conf ]; then
        awk '{for (i=2; i<=NF; i++) print "local-data: \""$i". 0 A 192.168.1.2\""}' /opt/share/diversion/list/blockinglist > /opt/var/lib/unbound/ads.conf

        if $(grep -q "ads\.conf" /opt/var/lib/unbound/unbound.conf); then
                unbound-control reload
        fi
fi

In the end, with the Diversion Plus Hosts enabled with the Standard block list, the total memory required for Unbound to use Pixelserv IP was prohibitive on my AC68U. So I stuck with Diversion.

 

[Delusion]

Yes, with DNS forwarding over TLS it works. With this option set: forward-tls-upstream: yes

tls-cert-bundle: "/etc/ssl/certs/ca-certificates.crt"
    forward-zone:
        name: "."
        forward-tls-upstream: yes
        forward-addr: 185.222.222.222@853
        forward-addr: 185.184.222.222@853
        forward-addr: 2a09::@853
        forward-addr: 2a09::1@853

Maybe its better use DoT this way instead of running Stubby?

 

[rgnldo]

DoT this way instead of running Stubby?

It is a matter of benefits. Stubby is complete with TCP/ TLS and 443 support. I have no problems. Even the maintainer of Dnscrypt, jedisct1 aka Frank Denis, is following this same procedure. Soon we will have native TCP/TLS support without forwarding. I'm currently without stubby, implementing the auth-zone features.

 

[Martineau]

@Martineau I had also written a personal script to convert the Diversion blockinglist to an Unbound conf file for Pixelserv ad-block. I came to realize that using local-zone requires too much memory and it works well-enough to only use local-data. I read in the unbound-users maillist that each local-zone requires 8k of memory. Here was my script if anyone wants to compare memory utilization:

#!/bin/sh

if [ /opt/share/diversion/list/blockinglist -nt /opt/var/lib/unbound/ads.conf ]; then
        awk '{for (i=2; i<=NF; i++) print "local-data: \""$i". 0 A 192.168.1.2\""}' /opt/share/diversion/list/blockinglist > /opt/var/lib/unbound/ads.conf

        if $(grep -q "ads\.conf" /opt/var/lib/unbound/unbound.conf); then
                unbound-control reload
        fi
fi

In the end, with the Diversion Plus Hosts enabled with the Standard block list, the total memory required for Unbound to use Pixelserv IP was prohibitive on my AC68U. So I stuck with Diversion.

Hmmm....

So if you temporarily enable Ad Block

e  = Exit Script

Option ==> ?

    Version=1.25
  
<snip>

    Options:

    [✔] unbound Logging
    [✔] Ad and Tracker Blocking (No. of Adblock domains=100048,Blocked Hosts=832,Whitelist=21)

What results do you get from

e.g. I don't use Diversion Plus Hosts

e  = Exit Script

Option ==> ad

Analysed Diversion file: 'blockinglist'  Type=pixelserv, (Adblock Domains=100048) would add 497 entries
Analysed Diversion file: 'blacklist'  Type=pixelserv, (Adblock Domains=100048) would add 2 entries
Analysed Diversion file: 'whitelist'  Type=URL, (Adblock URLs=21) would add 51 entries

 

[dave14305]

Hmmm....

So if you temporarily enable Ad Block

e  = Exit Script

Option ==> ?

    Version=1.25
  
<snip>

    Options:
    [✔] unbound Logging
    [✔] Ad and Tracker Blocking (No. of Adblock domains=100048,Blocked Hosts=832,Whitelist=21)

What results do you get from

e.g. I don't use Diversion Plus Hosts

e  = Exit Script

Option ==> ad

Analysed Diversion file: 'blockinglist'  Type=pixelserv, (Adblock Domains=100048) would add 497 entries
Analysed Diversion file: 'blacklist'  Type=pixelserv, (Adblock Domains=100048) would add 2 entries
Analysed Diversion file: 'whitelist'  Type=URL, (Adblock URLs=21) would add 51 entries

I'm not running the installer script on John's fork. But I had gone through a lot of gyrations seeing how I could replicate the Pixelserv benefits using unbound. My resulting ads.conf looks like this (head of file):

# head ads.conf
local-data: "diversion-adblocking-ip.address. 0 A 192.168.1.2"
local-data: "-sso.anbtr.com. 0 A 192.168.1.2"
local-data: "0-07.ru. 0 A 192.168.1.2"
local-data: "0-day.us. 0 A 192.168.1.2"
local-data: "0.01.2.13.3.sydneypropertyinvestors.com. 0 A 192.168.1.2"
local-data: "0.01.2.23.3.sydneypropertyinvestors.com. 0 A 192.168.1.2"
local-data: "0.nextyourcontent.com. 0 A 192.168.1.2"
local-data: "0.r.msn.com. 0 A 192.168.1.2"
local-data: "000.9.009.09.ekurbani.com. 0 A 192.168.1.2"
local-data: "000.9.019.09.ekurbani.com. 0 A 192.168.1.2"

 

[dave14305]

Also, if using a private Pixelserv IP in Unbound, you might ensure via dnsmasq.postconf that "stop-dns-rebind" is deleted from dnsmasq.conf.

pc_delete "stop-dns-rebind" $CONFIG

 

[Martineau]

I'm not running the installer script on John's fork.

Well technically the scipt is now more of manager rather than a one-off installer , but are you saying the script won run on the LTS fork?

No matter, I was only curious to see the possible magnitude of the differences between the two lists (even with the additional 40K of 'first-party' Ad Block entries !)

P.S. I haven't seen any issue with unbound RAM usage, but I need to let the additional 40K entries settle.

 

[dave14305]

Well technically the scipt is now more of manager rather than a one-off installer , but are you saying the script won run on the LTS fork?

No matter, I was only curious to see the possible magnitude of the differences between the two lists (even with the additional 40K of 'first-party' Ad Block entries !)

P.S. I haven't seen any issue with unbound RAM usage, but I need to let the additional 40K entries settle.

With John’s fork, there are some dnsmasq.conf differences I added to dnsmasq.postconf (or unbound.postconf in your repo):

pc_delete "resolv-file" "$CONFIG"
pc_append "no-resolv" "$CONFIG"

John doesn’t use a server-file= parameter, nor the no-resolv that Merlin does.
If you can add those if uname -o = “ASUSWRT-Merlin-LTS” then it should work fine with both versions.

My memory was manageable with only the Steven Black list. If I added the Plus Hosts, unbound would take more than 382 MB RAM and bring things to a big slowdown, swapping. Interested in others’ experiences.

It was also important to remove the diversion addn-hosts conserve memory if using unbound Adblock, and disable DNS Rebind protection if using a private IP for Pixelserv:

pc_delete "addn-hosts=/opt/share/diversion/list/" "$CONFIG"
pc_delete "stop-dns-rebind" "$CONFIG"

EDIT: Also, John's fork has included haveged since 25E1j9527, so no need to install haveged on John's fork.

 

[Cam]

There is an unbound python module, which reads a python script with blocking guidelines.
https://github.com/cbuijs/unbound-dns-firewall/blob/master/dns-firewall.py

I was also keen on this, but unfortunately the python module is not compiled into entware unbound. For fun I tried to compile it in for my AC68U, but get stuck on python dev requirements so gave up (and frankly the python script is rather likely too complex for my setup anyway!)

I'm curious as to where you get formula for rrset-cache-size & mem-cache-size? Unbound have an optimised setting for forked operation that suggests rrset=msg*2.
Also FYI nproc is not installed by default (well not in Merlin, perhaps in Johns?) - would need to

opkg install coreutils-nproc

or you could try

grep -c processor /proc/cpuinfo

 

[rgnldo]

python script

He had noticed. This is the reason for not organizing the script.

opkg install coreutils-nproc

Yes.

grep -c processor /proc/cpuinfo

there is no need

rrset=msg*2.

calculated in bytes. reserved = 12582912 = 12m

availableMemory=$((1024 * $( (grep -F MemAvailable /proc/meminfo || grep -F MemTotal /proc/meminfo) | sed 's/[^0-9]//g')))
if [ $availableMemory -le $((reserved * 2)) ]; then

The available memory will be checked and calculated with the reference of 12582912.

 

[Martineau]

I'm curious as to where you get formula for rrset-cache-size & mem-cache-size? Unbound have an optimised setting for forked operation that suggests rrset=msg*2.
Also FYI nproc is not installed by default (well not in Merlin, perhaps in Johns?) - would need to

opkg install coreutils-nproc

or you could try

grep -c processor /proc/cpuinfo

Sigh he's been here before post #255 so I included Entware's 'nproc' then he changed his mind.

He is just guessing again (like last night's stupid addition of 40K Ad Block domains that he has now removed today) - his arrogant boasts of being the unbound expert are definitely unfounded.

NOTE: unbound-control cannot handle '59.42m' etc.

                             ad = Analyse Diversion White/Black lists ([ file_name [type=adblock] ])

                             ca = Cache Size Optimisation  ([ 'reset' ])

e  = Exit Script

[Enter] leave Advanced Tools Menu

Option ==> ca

unbound-control set_option 'msg-cache-size 62307328' ok
unbound-control set_option 'key-cache-size 62307328' ok
unbound-control set_option 'rrset-cache-size 83076437' ok

Option ==> ?

 Memory/Cache:

 'key-cache-size:' 62307328 (59.42m)
 'msg-cache-size:' 62307328 (59.42m)
 'rrset-cache-size:' 83076437 (79.23m)

 

[Martineau]

I understand your observation. The perception of those who organize from different sides is normal. I've used unbound since version 1.6.6. Many things have changed and will change again.
The concern is with the planning in the final installation, with the AMTM.
It is not what is desired, but it is very easy when planning alone. In a week, Skynet made several updates.
The 40k list is perfectly viable on my router, but on an AC68U router it is poor. You have to think as a whole.
I recognize your efforts, but you have shown disrespect towards me, including the leak of PM.
The unbound project is open source and available to everyone on the forum. The goal is just to share.
I give up.

On more than one occasion, I have asked you politely not to PM me with your your random thoughts about unbound - yet you persist because as you readily admitted in a PM that it 'suits you and is therefore more convenient' and it also conveniently hides your ineptitude from the forum?

Furthermore your attitude to other more learned forum members is appalling, particularly over the POST-CMD debacle, and have continued to ignore my suggestions to make the maintenance of the script easier such as version control on your 'unbound.conf'

Also why deliberately post a picture of your proposed menu layout, meaning I can't simply cut'n'paste the text?

Clearly you have no respect for others.