Unbound unbound_manager (Manager/Installer utility for unbound - Recursive DNS Server)

[glehel]

Add them to

'/opt/share/unbound/configs/rpzsites'

1589236320] unbound-checkconf[716:0] error: parse error /opt/var/lib/unbound/mobileadtrackers.zone 2:49: Syntax error, could not parse the RR's type
[1589236320] unbound-checkconf[716:0] error: error parsing zonefile /opt/var/lib/unbound/mobileadtrackers.zone for mobileadtrackers.
[1589236320] unbound-checkconf[716:0] fatal error: Could not setup authority zones

https://gitlab.com/my-privacy-dns/rpz-dns-firewall-tools/unbound/-/tree/master/nxdomain all list error, not add

 

[juched]

1589236320] unbound-checkconf[716:0] error: parse error /opt/var/lib/unbound/mobileadtrackers.zone 2:49: Syntax error, could not parse the RR's type
[1589236320] unbound-checkconf[716:0] error: error parsing zonefile /opt/var/lib/unbound/mobileadtrackers.zone for mobileadtrackers.
[1589236320] unbound-checkconf[716:0] fatal error: Could not setup authority zones

https://gitlab.com/my-privacy-dns/rpz-dns-firewall-tools/unbound/-/tree/master/nxdomain all list error, not add

That link takes me to a list of links on gitlab. You need the direct URL in RPZ format. Which one do you want to use?

— edit —

Looking into some of those folders and they are not in RPZ format but are in what appears to be unbound format. Those do not get downloaded by dns firewall.

 

[ttgapers]

@Martineau QQ. If I want to use Adblock, can I switch my source list to....?

I currently use the Large list below with Diversion and no issues.

https://hosts.oisd.nl/

Thanks!

 

[joe scian]

OK, what size blocking list do you use?

Hi Martineau

I was using Medium/Large with Fast Switch set to Medium in Diversion. I have since switched to Medium/Standard. Medium uses 92483 hosts and standard 55596. The large list was over 1 million hosts - but it was NOT the active list (blockinglist) in Diversion when getting those errors last night on 3.11.

Fast Forward to 3.12 - The ad command seems to work fine now after upgrading to 3.12. Many thanks.

 

[glehel]

That link takes me to a list of links on gitlab. You need the direct URL in RPZ format. Which one do you want to use?

— edit —

Looking into some of those folders and they are not in RPZ format but are in what appears to be unbound format. Those do not get downloaded by dns firewall.

https://gitlab.com/my-privacy-dns/rpz-dns-firewall-tools/unbound/-/tree/master/nxdomain/mypdns
from here i tried the raw link. Downloading sets but there is an error in the end. The other thing is sometimes the script when I refresh the list doesn't download only the first line. If I add the line 2x it will download.

 

[Martineau]

https://gitlab.com/my-privacy-dns/rpz-dns-firewall-tools/unbound/-/tree/master/nxdomain/mypdns
from here i tried the raw link. Downloading sets but there is an error in the end.

Apologies for my pervious post, but as @juched replied, I assumed you were using true RPZ format files which are not actually hosted on the site.

@juched will need to update 'gen_adblock.sh' to allow the native unbound 'local-data:' format files to be included from that site

e.g. I added two lists - 'adaway/mobileadtrackers.zone' and 'mypdns/mypdns.adware.zone'

Total domains for the two '.zone' files is 21316 of which 9133 are new/unique.

 _____   _ _   _         _
|  _  |_| | |_| |___ ___| |_
|     | . | . | | . |  _| '_|
|__|__|___|___|_|___|___|_,_|
(gen_adblock.sh): 13225 @juched - v1.0.7 Martineau HACK - Thanks to @SomeWhereOverTheRainBow


Removing possible temporary files..
Downloading list(s) from block site(s) configured...
Attempting to Download 1 of 6 from https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts.
######################################################################## 100.0%
Attempting to Download 2 of 6 from https://raw.githubusercontent.com/llacb47/mischosts/master/tiktok-hosts.
######################################################################## 100.0%
Attempting to Download 3 of 6 from https://raw.githubusercontent.com/mitchellkrogza/The-Big-List-of-Hacked-Malware-Web-Sites/master/hacked-domains.list.
######################################################################## 100.0%
Attempting to Download 4 of 6 from https://blocklist.cyberthreatcoalition.org/vetted/domain.txt.
######################################################################## 100.0%
Attempting to Download 5 of 6 from https://gitlab.com/my-privacy-dns/rpz-dns-firewall-tools/unbound/-/raw/master/nxdomain/adaway/mobileadtrackers.zone.
   # #=O=#   #                                                           
Attempting to Download 6 of 6 from https://gitlab.com/my-privacy-dns/rpz-dns-firewall-tools/unbound/-/raw/master/nxdomain/mypdns/mypdns.adware.zone.
 # #=O#-  #                                                               
Downloading list(s) from allow site(s) configured...
Adding user requested hosts to list...
Removing user requested hosts from list...
Removing required hosts from list...
Removing unnecessary formatting from the domain list...
Generating Unbound adservers file...
(gen_adblock.sh): 13225 Number of zoned hosts: 21316

(gen_adblock.sh): 13225 Number of New Unique zones: 9133

(gen_adblock.sh): 13225 Number of adblocked hosts: 77281

Generating Unbound unload/load lists...
Loading/Unload Unbound local-zones to take effect...
removed 85818 zones
added 86414 zones
Removing temporary files...
Adblock update complete!

The other thing is sometimes the script when I refresh the list doesn't download only the first line. If I add the line 2x it will download.

Not sure if it matters but ensure you use LF chars when adding the additional URLs.

 

[tomsk]

Apologies for my pervious post, but as @juched replied, I assumed you were using true RPZ format files which are not actually hosted on the site.

@juched will need to update 'gen_adblock.sh' to allow the native unbound 'local-data:' format files to be included from that site

e.g. I added two lists - 'adaway/mobileadtrackers.zone' and 'mypdns/mypdns.adware.zone'

Total domains for the two '.zone' files is 21316 of which 9133 are new/unique.

 _____   _ _   _         _
|  _  |_| | |_| |___ ___| |_
|     | . | . | | . |  _| '_|
|__|__|___|___|_|___|___|_,_|
(gen_adblock.sh): 13225 @juched - v1.0.7 Martineau HACK - Thanks to @SomeWhereOverTheRainBow


Removing possible temporary files..
Downloading list(s) from block site(s) configured...
Attempting to Download 1 of 6 from https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts.
######################################################################## 100.0%
Attempting to Download 2 of 6 from https://raw.githubusercontent.com/llacb47/mischosts/master/tiktok-hosts.
######################################################################## 100.0%
Attempting to Download 3 of 6 from https://raw.githubusercontent.com/mitchellkrogza/The-Big-List-of-Hacked-Malware-Web-Sites/master/hacked-domains.list.
######################################################################## 100.0%
Attempting to Download 4 of 6 from https://blocklist.cyberthreatcoalition.org/vetted/domain.txt.
######################################################################## 100.0%
Attempting to Download 5 of 6 from https://gitlab.com/my-privacy-dns/rpz-dns-firewall-tools/unbound/-/raw/master/nxdomain/adaway/mobileadtrackers.zone.
   # #=O=#   #                                                          
Attempting to Download 6 of 6 from https://gitlab.com/my-privacy-dns/rpz-dns-firewall-tools/unbound/-/raw/master/nxdomain/mypdns/mypdns.adware.zone.
 # #=O#-  #                                                              
Downloading list(s) from allow site(s) configured...
Adding user requested hosts to list...
Removing user requested hosts from list...
Removing required hosts from list...
Removing unnecessary formatting from the domain list...
Generating Unbound adservers file...
(gen_adblock.sh): 13225 Number of zoned hosts: 21316

(gen_adblock.sh): 13225 Number of New Unique zones: 9133

(gen_adblock.sh): 13225 Number of adblocked hosts: 77281

Generating Unbound unload/load lists...
Loading/Unload Unbound local-zones to take effect...
removed 85818 zones
added 86414 zones
Removing temporary files...
Adblock update complete!

Not sure if it matters but ensure you use LF chars when adding the additional URLs.

I see that some of these lists are quite static... not updated in months in some cases.... would you have these downloaded every 15 mins like the spamhaus list or do it daily like the steven black lists?​

 

[glehel]

Thank You! I try!
maybe another example from your rpzhost file?

 

[Martineau]

Thank You! I try!
maybe another example from your rpzhost file?

They are not RPZ files so I simply added the two test example URLs to 'blocksites'

 

[Martineau]

I see that some of these lists are quite static... not updated in months in some cases.... would you have these downloaded every 15 mins like the spamhaus list or do it daily like the steven black lists?​

Yes, I can see there are only a couple of feeds that are a few hours old, and several that are between 1-3 days old.

If the two .zone files are truly valid i.e. combined they appear to have 9133 domains not included in last night's Steven Black's list refresh, then I would assume a daily refresh is sufficient.

 

[tomsk]

Playing around with the menu options i tried

A:Option ==> ox verbosity 4

unbound-control set_option 'verbosity 4' ok

A:Option ==> oq verbosity

unbound-control 'verbosity' '4'

but nothing seemed to change much in the unbound.log So i had a search through the docs and found

set_option opt: val
Set the option to the given value without a reload. The cache
is therefore not flushed. The option must end with a ':' and
whitespace must be between the option and the value. Some val-
ues may not have an effect if set this way, the new values are
not written to the config file, not all options are supported.
This is different from the set_option call in libunbound, where
all values work because unbound has not been initialized.

The values that work are: statistics-interval, statistics-cumu-
lative, do-not-query-localhost, harden-short-bufsize,
harden-large-queries, harden-glue, harden-dnssec-stripped,
harden-below-nxdomain, harden-referral-path, prefetch,
prefetch-key, log-queries, hide-identity, hide-version, iden-
tity, version, val-log-level, val-log-squelch, ignore-cd-flag,
add-holddown, del-holddown, keep-missing, tcp-upstream,
ssl-upstream, max-udp-size, ratelimit, ip-ratelimit,
cache-max-ttl, cache-min-ttl, cache-max-negative-ttl.

ie verbosity not included in the list....... so now i'm confused ( but no more than my usual case of general incomprehension)

Is it working because the get_option value is changed and i'm not seeing any difference in the unbound.log output because that's expected ....

Either way... refinement suggestion...would it be a good idea to check for unsupported set_option values and throw up a " value xxx not supported" warning?

 

[Martineau]

Playing around with the menu options i tried

A:Option ==> ox verbosity 4

unbound-control set_option 'verbosity 4' ok

A:Option ==> oq verbosity

unbound-control 'verbosity' '4'

but nothing seemed to change much in the unbound.log So i had a search through the docs and found


ie verbosity not included in the list....... so now i'm confused ( but no more than my usual case of general incomprehension)

Is it working because the get_option value is changed and i'm not seeing any difference in the unbound.log output because that's expected ....

Either way... refinement suggestion...would it be a good idea to check for unsupported set_option values and throw up a " value xxx not supported" warning?

To change the 'verbosity:' i.e. the type of messages that appear in the log use the 'lo' command

e  = Exit Script [?]

A:Option ==> lo 4

and the change should be reflected on screen and in the log

As for your suggestion I think I'll pass.

 

[tomsk]

To change the 'verbosity:' i.e. the type of messages that appear in the log use the 'lo' command

e  = Exit Script [?]

A:Option ==> lo 4

and the change should be reflected on screen and in the log

View attachment 23440

As for your suggestions I think I'll pass.

Ah yes the "lo" command..... forgot about that... i was just messing with the "ox"command and verbosity was one i knew..... but the 'lo" command appears to use the "set_option verbosity X" too... i'm confused how that is different to doing it through "ox".

EDIT: I think i get it ... you issue the verbosity command directly for log level change

                                $UNBOUNCTRLCMD -q verbosity $LOGLEVEL                         # v3.08 v3.06 v2.05
                                $UNBOUNCTRLCMD -q set_option verbosity $LOGLEVEL

what is the 2nd line for... and should it need a colon stuck in there?

                                $UNBOUNCTRLCMD -q verbosity $LOGLEVEL                         # v3.08 v3.06 v2.05
                                $UNBOUNCTRLCMD -q set_option verbosity: $LOGLEVEL

Not that it seems to make any difference....

tOmsK@RT-AC68U-4690:/tmp/home/root# unbound-control set_option verbosity 2
ok
tOmsK@RT-AC68U-4690:/tmp/home/root# unbound-control set_option verbosity: 2
ok

all suggestion rejections humbly accepted...

 

[netware5]

Hi guys, I am considering to enable AdBlock in Unbound Manager. Till now I never used a router-based adblocking. Currently I am using browser-based uBlock Origin add-on on my PCs and AdAway app on my android devices. So I have some questions regarding AdBlock feature of Unbound Manager and Diversion.

1. What are the main differences between AdBlock in Unbound Manager and Diversion?
2. Currently the uBlock Origin allows to whitelist ads of particular site, i.e. ads on www.snbforums.com are allowed. I don't know how the browser add-on is doing this, but it seems to be not be based on blacklisted domains, but based on the site I am visiting. Is it possible to apply the same type of whitelisting under Unbound or Diversion?

 

[rafagomes]

On Android you can use 'dnspipe' by Frostnerd.com.
This allows you to use a dns address of your own choice.
[It works by setting up an internal dummy vpn which allows a 'New' dns address to be used.]

I have used it on Android for years and I have not found it to contain any 'Funnies' also no Ads etc.
If your devices are 'rooted' there are other ways such as 'DNS Switcher' that runs under 'Magisk'.

Thank you for your suggestion, but I want to take advantage of the built in private DNS option that android offers.

So I "almost" made it work. My android device is able to connect to the private DNS using my asus DDNS FQDN but only when I am connect to my LAN as soon I change to the mobile data it can no longer connect. It might be because I did something wrong when creating the iptables rules or forgot to set something so I count on your help with that.

I have a AC86U with asus merlin 384.17 and Skynet and unbound installed with amtm. I will describe what I have changed bellow, and I have a few question:

1. Is there a way to listen only to the WAN interface instead of all? i.e. 0.0.0.0
2. Is that a good security practice or is it better just leave listening to al?
3. How to correctly open port 853 on iptables?
4. Why all my local domain get DNS_PROBE_FINISHED_NXDOMAIN when I have the private DNS activated(I am connected to my local lan)?

The changes I made to the unbound.conf generated by unbound_manager was adding the "tls-port", additional interface 0.0.0.0:53, commented out the self-signed tls-cert-bundle and use tls-service-key and tls-service-pem using the files Lets Encrypt create to my asus DDNS. You could just replace the self-signed with the tls-cert-bundle that Lets Encrypt generate if you want to keep the file short.

#tls-cert-bundle: "/etc/ssl/certs/ca-certificates.crt"  # v1.01 as per @dave14305 minimal config
# Enable DoT to be used for devices outside of my LAN
tls-port: 853
interface: 0.0.0.0@853
tls-service-key: "/etc/key.pem" # I look through all the source code to see how the system copy the files from lets encrypt but could not find how. I just know that this file is the same as /jffs/.le/example.asuscomm.com/example.asuscomm.com.key because I diff them.
tls-service-pem: "/etc/cert.pem" # I look through all the source code to see how the system copy the files from lets encrypt but could not find how. I just know that this file is the same as /jffs/.le/example.asuscomm.com/fullchain.pem because I diff them.
#tls-cert-bundle: "/etc/server.pem" #You could use this file if you want to keep the config file as short as possible. This is the only one I could find on the source code and is basically key.pem + cert.pem

And these are the change I made to iptables:

iptables -I INPUT -p tcp --destination-port 853 -j ACCEPT
ptables -I INPUT -p udp --destination-port 853 -j ACCEPT

iptables -S | grep 853
-A INPUT -p udp -m udp --dport 853 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 853 -j ACCEPT
-A FORWARD -i br0 -p tcp -m tcp --dport 853 -j DNSFILTER_DOT

 

[ttgapers]

@Martineau - I use the following in my dnsmasq.conf.add currently, along with unbound. Has been working well for months. Now, I am wanting to test native Unbound with Adblock and notice the "dnsmasq disable" command only deals with the hosts found in the dnsmasq reservations.

Are you considering adding the following from dnsmasq.conf.add to unbound.conf.localhosts as well?

They are:

cname=
rebind-domain-ok=
server=

• cname is used for manage my internal nginx proxies.
• rebind-domain-ok for my site to site vpns i use subdomains and thus want to exclude them as they return internal IPs for those segment's name resolution.
• server points to the dns internal instances for said internal sub-domains

If not, I'll add my notes to manually add them to unbound.conf.localhosts.

Thanks and great work as per usual.

 

[Martineau]

@Martineau - I use the following in my dnsmasq.conf.add currently, along with unbound. Has been working well for months. Now, I am wanting to test native Unbound with Adblock and notice the "dnsmasq disable" command only deals with the hosts found in the dnsmasq reservations.

Are you considering adding the following from dnsmasq.conf.add to unbound.conf.localhosts as well?

They are:

cname=
rebind-domain-ok=
server=

• cname is used for manage my internal nginx proxies.
• rebind-domain-ok for my site to site vpns i use subdomains and thus want to exclude them as they return internal IPs for those segment's name resolution.
• server points to the dns internal instances for said internal sub-domains

If not, I'll add my notes to manually add them to unbound.conf.localhosts.

Thanks and great work as per usual.

Baby steps etc. i.e. how widespread is the use of unbound as the Primary DNS for the LAN?

In the interim, I suggest you place your custom mods in 'unbound.conf.add', rather than 'unbound.conf.localhosts' as when switching between unbound/dnsmasq, 'unbound.conf.localhosts' will get flushed/rebuilt, but the contents of 'unbound.conf.add' will not be altered but will be added on every unbound startup.

P.S. You're welcome to provide examples (or write the code and submit a pull request!)

 

[joe scian]

Hi Martineau
I issued the following command

A:Option ==> DoT

Do you want to ENABLE DoT with unbound?

        Warning: This will DISABLE being able to be your own trusted Recursive DNS Resolver

So, do you STILL want to ENABLE DoT with unbound?

        Reply 'y' or press [Enter]  to skip
y

        Enabling DoT with unbound now as a Forwarder.....
18:30:50 Checking 'unbound.conf' for syntax errors.....
18:30:51 Saving unbound cache to '/opt/share/unbound/configs/cache.txt'
18:30:52 Requesting unbound (S61unbound) restart.....

Done.
 Shutting down unbound...              done.
 Starting unbound...              done.
18:30:56 Checking status, please wait.....
18:30:58 Restoring unbound cache from '/opt/share/unbound/configs/cache.txt' (2020-05-13 18:30:52)
18:30:59 unbound OK

HOWEVER there is no change to unbound.conf

#@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ # v1.05 Martineau
#forward-zone:#DoT                                                    # v1.08 Add #DoT edit marker v1.05 DNS-Over-TLS support
#name: "."
#forward-tls-upstream: yes
#forward-addr: 1.1.1.1@853#cloudflare-dns.com
#forward-addr: 1.0.0.1@853#cloudflare-dns.com
#forward-addr: 9.9.9.9@853#dns.quad9.net
#forward-addr: 149.112.112.112@853#dns.quad9.net
#forward-addr: 2606:4700:4700::1111@853#cloudflare-dns.com
#forward-addr: 2606:4700:4700::1001@853#cloudflare-dns.com
#forward-addr: 2620:fe::fe@853#dns.quad9.net
#forward-addr: 2620:fe::9@853#dns.quad9.net
#@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

AND consequently I am still my own trusted Recursive DNS Resolver. Does one need to enable DoT in GUI for this option similar to Stubby integration below?

Do you want to integrate Stubby with unbound?

        Warning: This will DISABLE being able to be your own trusted Recursive DNS Resolver

        Click the link below, and read BEFORE answering!

        https://github.com/MartineauUK/Unbound-Asuswrt-Merlin/blob/master/Readme.md#a-very-succinct-description-of-the-implicationuse-of-the-option-stubby-integration

So, do you STILL want to integrate Stubby with unbound? (NO recommended)

        Reply 'y' or press [Enter]  to skip
y
Integrating Stubby with unbound.....

        ERROR: DNS Privacy (DoT) not enabled in GUI. see http://192.168.2.240:80/Advanced_WAN_Content.asp WAN->DNS Privacy Protocol

Restarting dnsmasq.....
Done.

 

[Martineau]

There is no change to unbound.conf (using command)

A:Option ==> DoT

Do you want to ENABLE DoT with unbound?

        Warning: This will DISABLE being able to be your own trusted Recursive DNS Resolver

So, do you STILL want to ENABLE DoT with unbound?

        Reply 'y' or press [Enter]  to skip
y

        Enabling DoT with unbound now as a Forwarder.....
18:30:50 Checking 'unbound.conf' for syntax errors.....
18:30:51 Saving unbound cache to '/opt/share/unbound/configs/cache.txt'
18:30:52 Requesting unbound (S61unbound) restart.....

Done.
 Shutting down unbound...              done.
 Starting unbound...              done.
18:30:56 Checking status, please wait.....
18:30:58 Restoring unbound cache from '/opt/share/unbound/configs/cache.txt' (2020-05-13 18:30:52)
18:30:59 unbound OK

#@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ # v1.05 Martineau
#forward-zone:#DoT                                                    # v1.08 Add #DoT edit marker v1.05 DNS-Over-TLS support
#name: "."
#forward-tls-upstream: yes
#forward-addr: 1.1.1.1@853#cloudflare-dns.com
#forward-addr: 1.0.0.1@853#cloudflare-dns.com
#forward-addr: 9.9.9.9@853#dns.quad9.net
#forward-addr: 149.112.112.112@853#dns.quad9.net
#forward-addr: 2606:4700:4700::1111@853#cloudflare-dns.com
#forward-addr: 2606:4700:4700::1001@853#cloudflare-dns.com
#forward-addr: 2620:fe::fe@853#dns.quad9.net
#forward-addr: 2620:fe::9@853#dns.quad9.net
#@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

AND consequently I am still my own trusted Recursive DNS Resolver. Does one need to enable DoT in GUI for this option similar to Stubby integration below?

Do you want to integrate Stubby with unbound?

        Warning: This will DISABLE being able to be your own trusted Recursive DNS Resolver

        Click the link below, and read BEFORE answering!

        https://github.com/MartineauUK/Unbound-Asuswrt-Merlin/blob/master/Readme.md#a-very-succinct-description-of-the-implicationuse-of-the-option-stubby-integration

So, do you STILL want to integrate Stubby with unbound? (NO recommended)

        Reply 'y' or press [Enter]  to skip
y
Integrating Stubby with unbound.....

        ERROR: DNS Privacy (DoT) not enabled in GUI. see http://192.168.2.240:80/Advanced_WAN_Content.asp WAN->DNS Privacy Protocol

Restarting dnsmasq.....
Done.

Whoops

I've pushed Hotfix

Version=3.12
Github md5=4a207524e455366859549c3cce137e95​

Does one need to enable DoT in GUI for this option similar to Stubby integration

One does not old chap!

 

[juched]

@juched will need to update 'gen_adblock.sh' to allow the native unbound 'local-data:' format files to be included from that site

Am considering the best way to handle this. I see two options:

1. Add to existing blacklist, and add code to strip out unbound "local-zone:" and "always_nxdomain" directives. Then handle with existing code to merge into one list and create unbound commands.
2. create a separate "zonesites" file which allows you to add .ZONE files in unbound format, without touching the file. This would allow other sorts of unbound commands and files to be used (not just NX-DOMAIN items.

Looking for input.