Unbound unbound_manager (Manager/Installer utility for unbound - Recursive DNS Server)

[bluepoint]

I disagree.

You are not forced to choose option 2, so hopefully most first-time installers would simply stick with option 1, but 'standing on the shoulders of giants' is why option 2 exists as a way of getting a simple step up the performance ladder (for free) in a controlled baby-steps manner.

Are you formally requesting that in 'easy' mode there shouldn't even be an option 2?

No, I think it's just my assumption is wrong. I just need to read more in the unbound thread.

The header box initially displayed (only once) the first time unbound_manager is run, does not show what options are currently installed (use '?' to see the true live status) it is effectively a 'help' screen to guide the user on what option 2 would do if chosen.

Please formally clarify your throw-away statement

"Looks like uninstall needs a little cleanup?" ​

I think what I am seeing is the result of two different way of accessing unbound manager which produces two different results. If I access the manager through amtm(opt. 7) the result is the picture I uploaded(it remembers what I used to install) but if you access the U manager by typing "unbound_manager" the result is what you've described.

 

[dave14305]

No citation provided. I've so far found no definitive answer as to whether is it really the case that your ISP can't just see what does your unbound setup communicate to those authoritative name servers. Not saying your ISP or other DNS providers are any better, but this shouldn't be stated as an advantage to using unbound. If you have sources that prove this, by all means do provide us with them, especially research papers. Those would greatly help to assure that our privacy is better protected as you mentioned above.

All Unbound recursive queries are performed in-the-clear with port 53. So yes, if your ISP is that interested in you, they could sniff all your traffic. The privacy benefit is in not intentionally sharing all your DNS query information with a third party like Cloudflare, Google, Quad9, Cleanbrowsing, OpenDNS, etc.

 

[skeal]

This is an update to the previous post about optimizing unbound.conf. Please read at the link below for further information.

https://www.snbforums.com/threads/r...recursive-dns-server.61669/page-3#post-548469

@Martineau are you up for this one?

These settings are specifically with my RT-AX88U and my 1Gbps up/down symmetrical Fibre connection to my ISP.

Proceed at your own risk!

While these settings give me an exceptionally fast and responsive network experience, test it thoroughly before implementing it into your normal workflows.

# no threads and no memory slabs for threads
num-threads: 8                    # L&LDv1.09 (Orig 1) RT-AX88U For RT-AC86U use (2)
msg-cache-slabs: 16                # L&LDv1.09 (Orig 1) RT-AX88U For RT-AC86U use (2)
rrset-cache-slabs: 16                # L&LDv1.09 (Orig 1) RT-AX88U For RT-AC86U use (2)
infra-cache-slabs: 16                # L&LDv1.09 (Orig 1) RT-AX88U For RT-AC86U use (2)
key-cache-slabs: 16                # L&LDv1.09 (Orig 1) RT-AX88U For RT-AC86U use (2)

# tiny memory cache
key-cache-size: 32m                # L&LDv1.09 (Orig 8m) RT-AX88U For RT-AC86U use (8m)
msg-cache-size: 32m                # L&LDv1.09 (Orig 8m) RT-AX88U For RT-AC86U use (8m)
rrset-cache-size: 64m                # L&LDv1.09 (Orig 16m) RT-AX88U For RT-AC86U use (16m)
cache-max-ttl: 21600             
cache-min-ttl: 0                # L&LDv1.09 (Orig 5) RT-AX88U or RT-AC86U
prefetch: yes
prefetch-key: yes
serve-expired: yes
serve-expired-ttl: 3600
incoming-num-tcp: 1024                # L&LDv1.09 (Orig 600) RT-AX88U For RT-AC86U use (1024)
outgoing-num-tcp: 128                # L&LDv1.09 (Orig 100) RT-AX88U For RT-AC86U use (512)
ip-ratelimit: 0                    # L&LDv1.09 (Orig 100) RT-AX88U For RT-AC86U use (0)
edns-buffer-size: 4096                # L&LDv1.09 RT-AX88U (Orig (1472) v1.01 as per @dave14305 minimal config)

I've upgraded to v2.09 and even with a 'zero' % hit ratio initially, this still is faster than the 'safe' defaults for me.

The 1GB RAM on the RT-AX88U has disappeared, there is between 16MB and 36MB depending on what I'm using the network for at the time. Asus, give us more RAM!

The 2GB swap file is at about 29.57 / 2048 MB currently (have seen it to almost 490MB).

Note the limitations that @skeal points out:



For the RT-AC86U? Someone else (who has the router in use) needs to test the limits and report back to us.

For anything below the two HND models (RT-AX88U and the RT-AC86U)? Who wants to be a pioneer?

Opening up a shortcut folder (Open all) of 25 or 40 links in Edge Chromium has never happened so quickly before.

Metrics on these kinds of improvements? Like driving a fine car and trying to explain it to someone who only drives a horse and buggy.

You just have to get behind the wheel to find out.

Ok nice side effect of the new settings bud. Channel changing is faster in my multicast TV stream app.
EDIT: I still don't alter the two settings I posted about above.

 

[Treadler]

I sent a pull request to Jack Yaz to remove that curl command entirely. Hasn't been merged yet.

While I think it's easier for everyone to maintain blocklists, blacklists and whitelists in Diversion, if you still want to offer adblock with Unbound, continue on the path you started when you compare the Diversion list and actually convert it into an Unbound equivalent.

I had tested this a while back but ultimately didn't want Unbound doing the adblocking due to memory consumption. This points everything to a hardcoded Pixelserv IP. You could adapt it to pull whatever IP is in the first field of each record.

#!/bin/sh

if [ /opt/share/diversion/list/blockinglist -nt /opt/var/lib/unbound/ads.conf ] || [ ! -f /opt/var/lib/unbound/ads.conf ]; then
        awk '{for (i=2; i<=NF; i++) print "local-data: \""$i". 0 A 192.168.1.2\""}' /opt/share/diversion/list/blockinglist > /opt/var/lib/unbound/ads.conf

        if $(grep -q "ads\.conf" /opt/var/lib/unbound/unbound.conf); then
                unbound-control reload
        fi
fi

Observation:

A few days ago I disabled Diversion, & began using the native Unbound Adblock.
AX88U memory used dropped from 900mb+ to 600mb.

No ads seen thus far......
Appears to work well.

 

[Martineau]

the picture I uploaded(it remembers what I used to install)

I reiterate, the info header box is the same for every one, i.e. there are two versions 'easy' and 'advanced' but they are STATIC (like the front title of a book), and cannot 'remember' anything.

i.e. I can start unbound_manager in 'advanced' mode and manually install all 5 of the available options.
If I then Exit unbound_manager and use amtm to start unbound_manager in 'easy' mode, the info header box will remain exactly the same as the one you posted.

So, I shall politely repeat my previous request

Please formally clarify your throw-away statement

"Looks like uninstall needs a little cleanup?"
​

to describe what needs fixing in the script.

 

[Treadler]

This is an update to the previous post about optimizing unbound.conf. Please read at the link below for further information.

https://www.snbforums.com/threads/r...recursive-dns-server.61669/page-3#post-548469

@Martineau are you up for this one?

These settings are specifically with my RT-AX88U and my 1Gbps up/down symmetrical Fibre connection to my ISP.

Proceed at your own risk!

While these settings give me an exceptionally fast and responsive network experience, test it thoroughly before implementing it into your normal workflows.

# no threads and no memory slabs for threads
num-threads: 8                    # L&LDv1.09 (Orig 1) RT-AX88U For RT-AC86U use (2)
msg-cache-slabs: 16                # L&LDv1.09 (Orig 1) RT-AX88U For RT-AC86U use (2)
rrset-cache-slabs: 16                # L&LDv1.09 (Orig 1) RT-AX88U For RT-AC86U use (2)
infra-cache-slabs: 16                # L&LDv1.09 (Orig 1) RT-AX88U For RT-AC86U use (2)
key-cache-slabs: 16                # L&LDv1.09 (Orig 1) RT-AX88U For RT-AC86U use (2)

# tiny memory cache
key-cache-size: 32m                # L&LDv1.09 (Orig 8m) RT-AX88U For RT-AC86U use (8m)
msg-cache-size: 32m                # L&LDv1.09 (Orig 8m) RT-AX88U For RT-AC86U use (8m)
rrset-cache-size: 64m                # L&LDv1.09 (Orig 16m) RT-AX88U For RT-AC86U use (16m)
cache-max-ttl: 21600              
cache-min-ttl: 0                # L&LDv1.09 (Orig 5) RT-AX88U or RT-AC86U
prefetch: yes
prefetch-key: yes
serve-expired: yes
serve-expired-ttl: 3600
incoming-num-tcp: 1024                # L&LDv1.09 (Orig 600) RT-AX88U For RT-AC86U use (1024)
outgoing-num-tcp: 128                # L&LDv1.09 (Orig 100) RT-AX88U For RT-AC86U use (512)
ip-ratelimit: 0                    # L&LDv1.09 (Orig 100) RT-AX88U For RT-AC86U use (0)
edns-buffer-size: 4096                # L&LDv1.09 RT-AX88U (Orig (1472) v1.01 as per @dave14305 minimal config)

I've upgraded to v2.09 and even with a 'zero' % hit ratio initially, this still is faster than the 'safe' defaults for me.

The 1GB RAM on the RT-AX88U has disappeared, there is between 16MB and 36MB depending on what I'm using the network for at the time. Asus, give us more RAM!

The 2GB swap file is at about 29.57 / 2048 MB currently (have seen it to almost 490MB).

Note the limitations that @skeal points out:



For the RT-AC86U? Someone else (who has the router in use) needs to test the limits and report back to us.

For anything below the two HND models (RT-AX88U and the RT-AC86U)? Who wants to be a pioneer?

Opening up a shortcut folder (Open all) of 25 or 40 links in Edge Chromium has never happened so quickly before.

Metrics on these kinds of improvements? Like driving a fine car and trying to explain it to someone who only drives a horse and buggy.

You just have to get behind the wheel to find out.

A few days ago, I reverted to the Unbound defaults on my AX88U, disabled Diversion, enabled the native Unbound Adblock, no unbound.config tweaks, just the ‘cpu optimise’ how it comes out of the box.
Memory use has gone from nearly maxed out, to 400mb free.
No ads seen, everything very ‘snappy’.

But of course, you’ve now suggested these .config tweaks, sigh, Winscp here I come.....

 

[Martineau]

Observation:

A few days ago I disabled Diversion, & began using the native Unbound Adblock.
AX88U memory used dropped from 900mb+ to 600mb.

No ads seen thus far......
Appears to work well.

Would you mind posting the output from

e  = Exit Script

A:Option ==> ad

e.g. mine currently reports

Analysed Diversion file: 'blockinglist'  Type=pixelserv, (Adblock Domains=51170) would add 579 entries
Analysed Diversion file: 'blacklist'     Type=pixelserv, (Adblock Domains=51170) would add 0 entries
Analysed Diversion file: 'whitelist'     Type=URL, (Adblock URLs=22) would add 0 entries

 

[Treadler]

Would you mind posting the output from

e  = Exit Script

A:Option ==> ad

e.g. mine currently reports

Analysed Diversion file: 'blockinglist'  Type=pixelserv, (Adblock Domains=51170) would add 579 entries
Analysed Diversion file: 'blacklist'     Type=pixelserv, (Adblock Domains=51170) would add 0 entries
Analysed Diversion file: 'whitelist'     Type=URL, (Adblock URLs=22) would add 0 entries

Hmmm, I was using the ‘large’ list + one other in Diversion. That may explain it.

‘ad’ output to follow....

With Diversion reenabled I get,

sort: standard output: No space left on device


After running that, disabling Diversion once more I get,

/opt/bin/unbound_manager: line 2433: arithmetic syntax error

& I can no longer get into unbound manager.

 

[Martineau]

EDIT: it also looks like there is a variable discrepancy in the script between TESTTHIS and THIS.

Whoops

Hotfix pushed to GitHub.

 

[joe scian]

Hi Martineau

The rl command dosent seem to work properly. Working case - I change the unbound.conf from 4 threads to 8 threads. Press rl - No change in balancing across 8 threads. However if I press rs - the load is across 8 threads.
Cheers

 

[Martineau]

Hmmm, I was using the ‘large’ list + one other in Diversion. That may explain it.

‘ad’ output to follow....

With Diversion reenabled I get,

sort: standard output: No space left on device


After running that, disabling Diversion once more I get,

/opt/bin/unbound_manager: line 2433: arithmetic syntax error

& I can no longer get into unbound manager.

There was no need to re-enable diversion, your diversion files are still available for analysis by unbound_manager.
Hopefully a REBOOT will restore access to unbound_manager.

 

[bluepoint]

I reiterate, the info header box is the same for every one, i.e. there are two versions 'easy' and 'advanced' but they are STATIC (like the front title of a book), and cannot 'remember' anything.

i.e. I can start unbound_manager in 'advanced' mode and manually install all 5 of the available options.
If I then Exit unbound_manager and use amtm to start unbound_manager in 'easy' mode, the info header box will remain exactly the same as the one you posted.

So, I shall politely repeat my previous request

Please formally clarify your throw-away statement

"Looks like uninstall needs a little cleanup?"
​

to describe what needs fixing in the script.

Never mind, I was just posting what I taught is not working properly. Since you said, what I'm seeing is normal then there is nothing to cleanup.

 

[Martineau]

Hi Martineau

The rl command dosent seem to work properly. Working case - I change the unbound.conf from 4 threads to 8 threads. Press rl - No change in balancing across 8 threads. However if I press rs - the load is across 8 threads.
Cheers

Sadly, I believe the 'rl' command has restrictions.

So I'd recommend 'vx' is always followed with 'rs' rather than the 'rl' command to reliably effect the 'unbound.config' reconfiguration.

NOTE: There is a post by the original SME that the 'rl' command is designed to eliminate the disruptive 'rs' but an argument ensued.

 

[Treadler]

There was no need to re-enable diversion, your diversion files are still available for analysis by unbound_manager.
Hopefully a REBOOT will restore access to unbound_manager.

A reboot fixed!
Disabled Diversion & result of ‘ad’ is


sort: standard output: No space left on device
sort: standard output: No space left on device
sort: standard output: No space left on device
cp: write error: No space left on device
Analysed Diversion file: 'blockinglist' Type=pixelserv, (Adblock Domains=51170) would add 0 entries
Analysed Diversion file: 'blacklist' Type=pixelserv, (Adblock Domains=51170) would add 0 entries
Analysed Diversion file: 'whitelist' Type=URL, (Adblock URLs=22) would add 0 entries

 

[joe scian]

This is an update to the previous post about optimizing unbound.conf. Please read at the link below for further information.

https://www.snbforums.com/threads/r...recursive-dns-server.61669/page-3#post-548469

@Martineau are you up for this one?

These settings are specifically with my RT-AX88U and my 1Gbps up/down symmetrical Fibre connection to my ISP.

Proceed at your own risk!

While these settings give me an exceptionally fast and responsive network experience, test it thoroughly before implementing it into your normal workflows.

# no threads and no memory slabs for threads
num-threads: 8                    # L&LDv1.09 (Orig 1) RT-AX88U For RT-AC86U use (2)
msg-cache-slabs: 16                # L&LDv1.09 (Orig 1) RT-AX88U For RT-AC86U use (2)
rrset-cache-slabs: 16                # L&LDv1.09 (Orig 1) RT-AX88U For RT-AC86U use (2)
infra-cache-slabs: 16                # L&LDv1.09 (Orig 1) RT-AX88U For RT-AC86U use (2)
key-cache-slabs: 16                # L&LDv1.09 (Orig 1) RT-AX88U For RT-AC86U use (2)

# tiny memory cache
key-cache-size: 32m                # L&LDv1.09 (Orig 8m) RT-AX88U For RT-AC86U use (8m)
msg-cache-size: 32m                # L&LDv1.09 (Orig 8m) RT-AX88U For RT-AC86U use (8m)
rrset-cache-size: 64m                # L&LDv1.09 (Orig 16m) RT-AX88U For RT-AC86U use (16m)
cache-max-ttl: 21600            
cache-min-ttl: 0                # L&LDv1.09 (Orig 5) RT-AX88U or RT-AC86U
prefetch: yes
prefetch-key: yes
serve-expired: yes
serve-expired-ttl: 3600
incoming-num-tcp: 1024                # L&LDv1.09 (Orig 600) RT-AX88U For RT-AC86U use (1024)
outgoing-num-tcp: 128                # L&LDv1.09 (Orig 100) RT-AX88U For RT-AC86U use (512)
ip-ratelimit: 0                    # L&LDv1.09 (Orig 100) RT-AX88U For RT-AC86U use (0)
edns-buffer-size: 4096                # L&LDv1.09 RT-AX88U (Orig (1472) v1.01 as per @dave14305 minimal config)

I've upgraded to v2.09 and even with a 'zero' % hit ratio initially, this still is faster than the 'safe' defaults for me.

The 1GB RAM on the RT-AX88U has disappeared, there is between 16MB and 36MB depending on what I'm using the network for at the time. Asus, give us more RAM!

The 2GB swap file is at about 29.57 / 2048 MB currently (have seen it to almost 490MB).

Note the limitations that @skeal points out:



For the RT-AC86U? Someone else (who has the router in use) needs to test the limits and report back to us.

For anything below the two HND models (RT-AX88U and the RT-AC86U)? Who wants to be a pioneer?

Opening up a shortcut folder (Open all) of 25 or 40 links in Edge Chromium has never happened so quickly before.

Metrics on these kinds of improvements? Like driving a fine car and trying to explain it to someone who only drives a horse and buggy.

You just have to get behind the wheel to find out.

I have enabled all the specified tweaks above on my AC-5300 ie 8 threads and increased memory parameters as for AX88 and all is working fine. I am accessing my Router over openvpn from a remote location and i still have 76% memory used and 123Mbyte free.

 

[Martineau]

A reboot fixed!
Disabled Diversion & result of ‘ad’ is


sort: standard output: No space left on device
sort: standard output: No space left on device
sort: standard output: No space left on device
cp: write error: No space left on device
Analysed Diversion file: 'blockinglist' Type=pixelserv, (Adblock Domains=51170) would add 0 entries
Analysed Diversion file: 'blacklist' Type=pixelserv, (Adblock Domains=51170) would add 0 entries
Analysed Diversion file: 'whitelist' Type=URL, (Adblock URLs=22) would add 0 entries

It appears you must have a very large blocking list in diversion?
What size swapfile do you use?

 

[Martineau]

Trying 2.09 on a 56U at 384.6 I get this glitch in the install:

i  = Begin unbound Installation Process ('/opt/var/lib/unbound/')
z  = Remove unbound/unbound_manager Installation   
3  = Advanced Tools                            rl = Reload Configuration (Doesn't halt unbound) e.g. 'rl test1[.conf]' (Recovery u)
?  = About Configuration               
               
e  = Exit Script
A:Option ==> i
    Router Configuration recommended pre-reqs status:
    [?] Swapfile=262140 kB
    [?] ***ERROR DNS Filter is OFF!                          see http://192.168.2.200:80/DNSFilter.asp LAN->DNSFilter Enable DNSg
[: 0: unknown operand
    [?] WAN: Use local caching DNS server as system resolver=NO
[: 0: unknown operand
    [?] Enable local NTP server=YES
    [?] Enable DNS Rebind protection=NO
    [?] Enable DNSSEC support=NO
    Options: unbound Advanced install - User will be prompted to install options
    The router does not currently meet ALL of the recommended pre-reqs as shown above.
    However, whilst they are recommended, you may proceed with the unbound INSTALL
    as the recommendations are NOT usually FATAL if they are NOT strictly followed.
    Press Y to continue unbound INSTALL  or press [Enter] to ABORT

Nevermind the DNS Filter, I was just noticing the unknown operands.

@dave14305 kindly suggested a Hotfix, for @john9527's LTS Release,

        if [ "$(uname -o)" == "ASUSWRT-Merlin-LTS" ];then               # v1.26 HotFix @dave14305
            [ $(nvram get ntpd_server) == "0" ] && { echo -e $cBRED"\a\t[✖] ***ERROR Enable local NTP server=NO $cRESET \t\t\t\t\tsee $HTTP_TYPE://$(nvram get lan_ipaddr):$HTTP_PORT/Advanced_System_Content.asp ->Basic Config"$cRESET 2>&1; ERROR_CNT=$((ERROR_CNT + 1)); } || echo -e $cBGRE"\t[✔] Enable local NTP server=YES" 2>&1
        else

which doesn't have the required NVRAM variable as expected, so your RT-AC56U presumably doesn't have all of the latest v384.xx NVRAM variable names either.

I suspect using 'uname -o' wouldn't be useful. Can you post the output of

nvram get productid
nvram get odmpid

uname -o

Alternatively flash the unit to @john9527's LTS Release

 

[Treadler]

It appears you must have a very large blocking list in diversion?
What size swapfile do you use?

2gb swap file.
Yes, I use the ‘large’ list in Diversion, + one extra custom one. (1.7m!).
So looks like I could use Diversion with a much smaller list, given the unbound one has 50k or so entries & appears to work well.
Or just keep using the Unbound ad block.

 

[Martineau]

2gb swap file.
Yes, I use the ‘large’ list in Diversion, + one extra custom one. (1.7m!).
So looks like I could use Diversion with a much smaller list, given the unbound one has 50k or so entries & appears to work well.
Or just keep using the Unbound ad block.

The 'ad' option is supposed to give you an idea of the impact of converting your diversion lists into Ad Block format.
i.e. My diversion 'blocklist=100,000' approx, so my 'ad' results show that perhaps due to using similar domain list sources, only a few hundred are unique to diversion.

@dave14305 was concerned that the memory consumed by unbound Ad Block lists could be worse than diversion, so caution would be advised - hence the discussion

"should someone take ownership of the current Ad Block script to give the same choice of lists as diversion"

 

[kfahoo]

For the RT-AC86U? Someone else (who has the router in use) needs to test the limits and report back to us.

I use those:

num-threads: 2
msg-cache-slabs: 4
rrset-cache-slabs: 4
infra-cache-slabs: 4
key-cache-slabs: 4

# tiny memory cache
key-cache-size: 16m
msg-cache-size: 16m
rrset-cache-size: 24m
cache-max-ttl: 21600
cache-min-ttl: 0
prefetch: yes
prefetch-key: yes
serve-expired: yes
serve-expired-ttl: 3600
incoming-num-tcp: 1024
outgoing-num-tcp: 512
ip-ratelimit: 0
edns-buffer-size: 4096

and used this https://www.grc.com/dns/benchmark.htm to test it

rt-ac86u |  Min  |  Avg  |  Max  |Std.Dev|Reliab%|
  ----------------+-------+-------+-------+-------+-------+
  + Cached Name   | 0,000 | 0,001 | 0,002 | 0,000 | 100,0 |
  + Uncached Name | 0,013 | 0,090 | 0,297 | 0,090 | 100,0 |
  + DotCom Lookup | 0,024 | 0,031 | 0,047 | 0,007 | 100,0 |
  ---<OOOO---->---+-------+-------+-------+-------+-------+