Unbound unbound_manager (Manager/Installer utility for unbound - Recursive DNS Server)

dave14305 · May 10, 2020

krgck said:
kasper@RT-AC88U-C440:/tmp# cat /jffs/scripts/dnsmasq.postconf sh /jffs/addons/unbound/unbound.postconf "$1"

You need this as the first line of dnsmasq.postconf:

Code:

#!/bin/sh

Run these 3 commands:

Code:

sed -i '1s~^~#!/bin/sh\n~' /jffs/scripts/dnsmasq.postconf
chmod 755 /jffs/scripts/dnsmasq.postconf
service restart_dnsmasq

krgck · May 10, 2020

dave14305 said:
You need this as the first line of dnsmasq.postconf:

Code:

#!/bin/sh

Run these 3 commands:

Code:

sed -i '1s~^~#!/bin/sh\n~' /jffs/scripts/dnsmasq.postconf chmod 755 /jffs/scripts/dnsmasq.postconf service restart_dnsmasq

That did the trick. Thanks for saving my time and sleep

tomsk · May 10, 2020

dave14305 said:
You need this as the first line of dnsmasq.postconf:

Code:

#!/bin/sh

Run these 3 commands:

Code:

sed -i '1s~^~#!/bin/sh\n~' /jffs/scripts/dnsmasq.postconf chmod 755 /jffs/scripts/dnsmasq.postconf service restart_dnsmasq

Oh the shebang is there for me because the dnsmasq.postconf was added by diversion ... but this install was written by the manager script?

dave14305 · May 10, 2020

tomsk said:
Oh the shebang is there for me because the dnsmasq.postconf was added by diversion ... but this unbound install was written by the manager script?

unbound.postconf is from Martineau. krgck must never have had a dnsmasq.postconf and unbound_manager doesn’t (yet) account for that scenario.

tomsk · May 11, 2020

Oh well

https://www.snbforums.com/threads/nextdns-installer.61002/page-20#post-582558

Twiglets · May 11, 2020

tomsk said:
Yes fairly close depending on that else you might have bolted onto diversion

Code:

A:Option ==> ad Analysed Diversion file: 'blockinglist' Type=pixelserv, (Adblock Domains=55204) would add 620 entries Analysed Diversion file: 'blacklist' Type=pixelserv, (Adblock Domains=55204) would add 2 entries Analysed Diversion file: 'whitelist' Type=URL, (Adblock URLs=19) would add 22 entries

I guess the ultimate if you wanted identical lists would to be to import them from diversion... the code is in the script to do the dnsmasq to unbound type lists conversion already ( the steven black one downloads in dnsmasq form and needs conversion)

The script already has a 'merge' option that is not tied to a menu option.
A small 'edit'

can run the 'merge' code which does appear to work !!!???

Not sure why it has been left out ?

tomsk · May 11, 2020

Twiglets said:
The script already has a 'merge' option that is not tied to a menu option.
A small 'edit' can run the 'merge' code which does appear to work !!!???

Not sure why it has been left out ?

Thats not just the icing... that's the cherry on top too.

joe scian · May 11, 2020

Hi Martineau

when I issue the command below I get

Code:

A:Option ==> ad


cp: write error: No space left on device
sed: write error

and now my syslog is filled up with. A router reboot fixes the syslog spam below but I wont issue that ad command again. !

Code:

May 11 21:08:09 RT-AC5300-0680 syslog: Error locking /var/lock/networkmap.lock: 28 No space left on device
May 11 21:08:09 RT-AC5300-0680 syslog: Error unlocking -1: 9 Bad file descriptor
May 11 21:08:09 RT-AC5300-0680 syslog: Error locking /var/lock/networkmap.lock: 28 No space left on device
May 11 21:08:09 RT-AC5300-0680 syslog: Error unlocking -1: 9 Bad file descriptor
May 11 21:08:09 RT-AC5300-0680 syslog: Error locking /var/lock/networkmap.lock: 28 No space left on device
May 11 21:08:09 RT-AC5300-0680 syslog: Error unlocking -1: 9 Bad file descriptor
May 11 21:08:09 RT-AC5300-0680 syslog: Error locking /var/lock/networkmap.lock: 28 No space left on device
May 11 21:08:09 RT-AC5300-0680 syslog: Error unlocking -1: 9 Bad file descriptor
May 11 21:08:09 RT-AC5300-0680 syslog: Error locking /var/lock/networkmap.lock: 28 No space left on device
May 11 21:08:09 RT-AC5300-0680 syslog: Error unlocking -1: 9 Bad file descriptor
May 11 21:08:14 RT-AC5300-0680 syslog: Error locking /var/lock/cfg_mnt.lock: 28 No space left on device

Martineau · May 11, 2020

tomsk said:

I just had a look at my unbound.conf just now ... With dnsmasq enabled shouldn't the unbound interface on 127.0.0.1 for port 53 be commented out?

Code:

#@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
port: 53535                                 # v1.08 If 53, requires 'port=0' in '/etc/dnsmasq.conf' to 'disable' dnsmasq to answer queries direct from LAN clients
interface: 127.0.0.1@53535                  # v1.01 As per @dave14305 minimal config; Will be overwritten by $(nvram get lan_ipaddr_rt) if dnsmasq 'disabled'
interface: 127.0.0.1@53                    # v1.10 Required by router if dnsmasq 'disabled'
#access-control: 0.0.0.0/0 allow            # v1.10 Will be overwritten by LAN subnet "${lan_ip_addr_rt}/24" if 'dnsmasq disabled' aka bypassed
#@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

Whoops!

In trying to tidy up 'unbound.conf' v1.10, I 'helpfully' added a comment....see this commit

Clearly the original code

Code:

[ -n "$(grep "^interface: 127.0.0.1@53$" /opt/var/lib/unbound/unbound.conf)" ] && sed   's/^interface: 127\.0\.0\.1@53$/#interface: 127\.0\.0\.1@53/' /opt/var/lib/unbound/unbound.conf

will now never match/find an uncommented line containing ONLY the text 'interface: 127.0.0.1@53'

I'll push a Hotfix later.

tomsk · May 11, 2020

I was having a look around for other organisations suppling RPZ data other than spamhaus . seems the other main player is SURBL .... the SURBL combined list is interesting in the way they use ip to show what list its from http://www.surbl.org/lists

multi.surbl.org - Combined SURBL list
All of the SURBL data sources are combined into a single, bitmasked list: multi.surbl.org.
Bitmasking means that there is only one entry per domain name or IP address, but that entry will resolve into an address (DNS A record) whose last octet indicates which lists it belongs to. The bit positions in that last octet for membership in the different lists are:
8 = listed on PH
16 = listed on MW
64 = listed on ABUSE
128 = listed on CR
If an entry belongs to just one list it will have an address where the last octet has that value. For example 127.0.0.8 means it's on the phishing list, while 127.0.0.64 means it's listed on the ABUSE list. An entry on multiple lists gets the sum of those list numbers as the last octet, so 127.0.0.80 means a record is on both MW and ABUSE (comes from: 16 + 64 = 80). In this way, membership in multiple lists is encoded into a single response. Octets other than the first and last one are reserved for future use and should be ignored.
/QUOTE]

Martineau · May 11, 2020

joe scian said:

Hi Martineau

when I issue the command below I get

Code:

A:Option ==> ad


cp: write error: No space left on device
sed: write error

and now my syslog is filled up with. A router reboot fixes the syslog spam below but I wont issue that ad command again. !

Code:

May 11 21:08:09 RT-AC5300-0680 syslog: Error locking /var/lock/networkmap.lock: 28 No space left on device
May 11 21:08:09 RT-AC5300-0680 syslog: Error unlocking -1: 9 Bad file descriptor
May 11 21:08:09 RT-AC5300-0680 syslog: Error locking /var/lock/networkmap.lock: 28 No space left on device
May 11 21:08:09 RT-AC5300-0680 syslog: Error unlocking -1: 9 Bad file descriptor
May 11 21:08:09 RT-AC5300-0680 syslog: Error locking /var/lock/networkmap.lock: 28 No space left on device
May 11 21:08:09 RT-AC5300-0680 syslog: Error unlocking -1: 9 Bad file descriptor
May 11 21:08:09 RT-AC5300-0680 syslog: Error locking /var/lock/networkmap.lock: 28 No space left on device
May 11 21:08:09 RT-AC5300-0680 syslog: Error unlocking -1: 9 Bad file descriptor
May 11 21:08:09 RT-AC5300-0680 syslog: Error locking /var/lock/networkmap.lock: 28 No space left on device
May 11 21:08:09 RT-AC5300-0680 syslog: Error unlocking -1: 9 Bad file descriptor
May 11 21:08:14 RT-AC5300-0680 syslog: Error locking /var/lock/cfg_mnt.lock: 28 No space left on device

OK, what size blocking list do you use?

juched · May 11, 2020

Martineau said:

@juched

FYI,,,

Code:

 _____   _ _   _         _ 
|  _  |_| | |_| |___ ___| |_
|     | . | . | | . |  _| '_|
|__|__|___|___|_|___|___|_,_|
(gen_adblock.sh): 28116 @juched - v1.0.6 - Thanks to @SomeWhereOverTheRainBow

Removing possible temporary files..
Downloading list(s) from block site(s) configured...
Attempting to Download 1 of 4 from https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts.
######################################################################## 100.0%
Attempting to Download 4 of 4 from https://raw.githubusercontent.com/llacb47/mischosts/master/tiktok-hosts.
######################################################################## 100.0%
Attempting to Download 7 of 4 from https://raw.githubusercontent.com/mitchellkrogza/The-Big-List-of-Hacked-Malware-Web-Sites/master/hacked-domains.list.
######################################################################## 100.0%
Attempting to Download 8 of 4 from https://blocklist.cyberthreatcoalition.org/vetted/domain.txt.
######################################################################## 100.0%
Downloading list(s) from allow site(s) configured...

should be

Code:

Removing possible temporary files..
Downloading list(s) from block site(s) configured...
Attempting to Download 1 of 4 from https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts.
######################################################################## 100.0%
Attempting to Download 2 of 4 from https://raw.githubusercontent.com/llacb47/mischosts/master/tiktok-hosts.
######################################################################## 100.0%
Attempting to Download 3 of 4 from https://raw.githubusercontent.com/mitchellkrogza/The-Big-List-of-Hacked-Malware-Web-Sites/master/hacked-domains.list.
######################################################################## 100.0%
Attempting to Download 4 of 4 from https://blocklist.cyberthreatcoalition.org/vetted/domain.txt.
######################################################################## 100.0%

'gen_adblock.sh' needs a minor tweak

Code:

    for url in $(echo $line); do
      [ "${url:0:1}" == "#" ] && continue # skip commented out lines - Thanks @Martineau
      echo "Attempting to Download $count of $(awk 'NF && !/^[:space:]*#/' $sites | wc -l) from $url."
      curl --progress-bar $url | grep -o '^[^#]*' | grep -v "::1" | grep -v "0.0.0.0 0.0.0.0" | sed '/^$/d' | sed 's/\ /\\ /g' | awk '{print $NF}' | grep -o '^[^\\]*' | grep -o '^[^\\$]*' | sort >> $list
      dos2unix $list
      count=$((count + 1)) 
    done

Thanks, fixed and pushed to master. I also pushed YT ad block script to master as well.

Martineau · May 11, 2020

joe scian said:
yes it should be perhaps an introduced "feature" since last hotfix.

What?

joe scian said:
Also noted that

Code:

# log-local-actions - yes

remains commented out during adblock installation when going back and forth enabling Dnsmasq and subsequently choosing Dnsmasq disable and Adblock install. Had to manually uncomment this.

The option is only enabled if explicitly requested, and logging to 'syslog-ng/scribe' is also explicitly ENABLED.

The design/reasoning was that if unbound logging was ENABLED, then previously as there was no housekeeping performed on the size of the log, it could silently fill the disk a lot quicker.

e.g. explicitly ENABLE the setting

Code:

e  = Exit Script [?]

A:Option ==> adblock track

Option Auto Reply 'y'    Installing Ads and Tracker (Ad Block) Blocking.....
    adblock/gen_adblock.sh downloaded successfully
    adblock/permlist downloaded successfully
Custom '/opt/share/unbound/configs/blocksites' already exists - 'adblock/blocksites' download skipped
Custom '/opt/share/unbound/configs/allowsites' already exists - 'adblock/allowsites' download skipped
Custom '/opt/share/unbound/configs/blockhost' already exists - 'adblock/blockhost' download skipped
Custom '/opt/share/unbound/configs/allowhost' already exists - 'adblock/allowhost' download skipped
Creating Daily cron job for Ad and Tracker update
Executing '/opt/var/lib/unbound/adblock/gen_adblock.sh'.....
                       
 _____   _ _   _         _
|  _  |_| | |_| |___ ___| |_
|     | . | . | | . |  _| '_|
|__|__|___|___|_|___|___|_,_|
(gen_adblock.sh): 3200 @juched - v1.0.6 - Thanks to @SomeWhereOverTheRainBow


Removing possible temporary files..
Downloading list(s) from block site(s) configured...
Attempting to Download 1 of 4 from https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts.
######################################################################## 100.0%
Attempting to Download 4 of 4 from https://raw.githubusercontent.com/llacb47/mischosts/master/tiktok-hosts.
######################################################################## 100.0%
Attempting to Download 7 of 4 from https://raw.githubusercontent.com/mitchellkrogza/The-Big-List-of-Hacked-Malware-Web-Sites/master/hacked-domains.list.
######################################################################## 100.0%
Attempting to Download 8 of 4 from https://blocklist.cyberthreatcoalition.org/vetted/domain.txt.
######################################################################## 100.0%
Downloading list(s) from allow site(s) configured...
Adding user requested hosts to list...
Removing user requested hosts from list...
Removing required hosts from list...
Removing unnecessary formatting from the domain list...
Generating Unbound adservers file...
(gen_adblock.sh): 3200 Number of adblocked hosts: 77143

Generating Unbound unload/load lists...
Loading/Unload Unbound local-zones to take effect...
removed 77143 zones
added 77143 zones
Removing temporary files...
Adblock update complete!

Logging Ad Block BLOCKED domains to scribe

I suppose now that logging housekeeping should no longer be an issue, as @juched's GUI stats can use the metrics, I suppose I could change the criteria to auto-enable if 'sgui' is ENABLED?

juched · May 11, 2020

Martineau said:

What?

The option is only enabled if explicitly requested, and logging to 'syslog-ng/scribe' is also explicitly ENABLED.

The design/reasoning was that if unbound logging was ENABLED, then previously as there was no housekeeping performed on the size of the log, it could silently fill the disk a lot quicker.

e.g. explicitly ENABLE the setting

Code:

e  = Exit Script [?]

A:Option ==> adblock track

Option Auto Reply 'y'    Installing Ads and Tracker (Ad Block) Blocking.....
    adblock/gen_adblock.sh downloaded successfully
    adblock/permlist downloaded successfully
Custom '/opt/share/unbound/configs/blocksites' already exists - 'adblock/blocksites' download skipped
Custom '/opt/share/unbound/configs/allowsites' already exists - 'adblock/allowsites' download skipped
Custom '/opt/share/unbound/configs/blockhost' already exists - 'adblock/blockhost' download skipped
Custom '/opt/share/unbound/configs/allowhost' already exists - 'adblock/allowhost' download skipped
Creating Daily cron job for Ad and Tracker update
Executing '/opt/var/lib/unbound/adblock/gen_adblock.sh'.....
                        
 _____   _ _   _         _
|  _  |_| | |_| |___ ___| |_
|     | . | . | | . |  _| '_|
|__|__|___|___|_|___|___|_,_|
(gen_adblock.sh): 3200 @juched - v1.0.6 - Thanks to @SomeWhereOverTheRainBow


Removing possible temporary files..
Downloading list(s) from block site(s) configured...
Attempting to Download 1 of 4 from https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts.
######################################################################## 100.0%
Attempting to Download 4 of 4 from https://raw.githubusercontent.com/llacb47/mischosts/master/tiktok-hosts.
######################################################################## 100.0%
Attempting to Download 7 of 4 from https://raw.githubusercontent.com/mitchellkrogza/The-Big-List-of-Hacked-Malware-Web-Sites/master/hacked-domains.list.
######################################################################## 100.0%
Attempting to Download 8 of 4 from https://blocklist.cyberthreatcoalition.org/vetted/domain.txt.
######################################################################## 100.0%
Downloading list(s) from allow site(s) configured...
Adding user requested hosts to list...
Removing user requested hosts from list...
Removing required hosts from list...
Removing unnecessary formatting from the domain list...
Generating Unbound adservers file...
(gen_adblock.sh): 3200 Number of adblocked hosts: 77143

Generating Unbound unload/load lists...
Loading/Unload Unbound local-zones to take effect...
removed 77143 zones
added 77143 zones
Removing temporary files...
Adblock update complete!

Logging Ad Block BLOCKED domains to scribe

I suppose now that logging housekeeping should no longer be an issue, as @juched's GUI stats can use the metrics, I suppose I could change the criteria to auto-enable if 'sgui' is enabled?

I think so, since every hour I flush those message as well, and support both scribe and non-scribe logging.

Would be good to have an "option" to use "sgui extended" to enable log-replies as well, so you can get extended stats showing all requested queries. Works even if dnsmasq disable is not used. No need to enable log-queries for extended stats, and my script will flush those lines from the log as well, so the disk should not fill up.

Thoughts?

Martineau · May 11, 2020

I've uploaded v3.12

Version=3.12
Github md5=4a207524e455366859549c3cce137e95

use 'u' to update when prompted on screen

Use of the 'i = Update unbound Installation' ** not required **

Code:

FIX:    If '/jffs/scripts/dnsmasq.postconf doesn't exist then create it with '#!/bin/sh' (and execute attributes) before adding 'sh /jffs/scripts/addons/unbound/unbound.postconf' - @dave14305
FIX:    dnsmasq bypass, 'unbound.conf' v1.10 added a comment, so switching back to dnsmasq as Primary DNS, unbound now correctly releases '127.0.0.1@53' - @tomsk
FIX:    'ad' to use '/opt/tmp/' rather than '/tmp/' - @joe scian
CHANGE: If 'sgui' ENABLED and 'adblock' requested then ENABLE tracking of blocked domains 'log-local-actions: yes' - @joe scian
CHANGE: 'youtube' YT Video Ad blocking will now retrieve script from @juched's Github master branch rather than his dev branch.
CHANGE: Enhance 'debug' diagnostics
ADD:    'youtube [view | edit]' to allow access (for the curious) to the current YT blocked domains
CHANGE: '?' command, include clickable URL link to unbound v1.10.0 manual

    About unbound: https://nlnetlabs.nl/projects/unbound/about/ , Manual https://nlnetlabs.nl/documentation/unbound/unbound.conf/

Thanks again to the usual suspects for proof reading/bug reporting.

Martineau · May 11, 2020

juched said:
I think so, since every hour I flush those message as well, and support both scribe and non-scribe logging.

Would be good to have an "option" to use "sgui extended" to enable log-replies as well, so you can get extended stats showing all requested queries. Works even if dnsmasq disable is not used. No need to enable log-queries for extended stats, and my script will flush those lines from the log as well, so the disk should not fill up.

Thoughts?

'lo = Enable Logging' currently ENABLEs both queries and replies, and 'lx Disable Logging' DISABLEs them.

I have added 'sgui all' which will now enable tracking of the Ad Blocking ('log-local-actions: yes') but if logging is DISABLED then it is obviously irrelevant until logging is ENABLED.

juched · May 11, 2020

Martineau said:
'lo = Enable Logging' currently ENABLEs both queries and replies, and 'lx Disable Logging' DISABLEs them.

I have added 'sgui all' which will now enable tracking of the Ad Blocking ('log-local-actions: yes') but if logging is DISABLED then it is obviously irrelevant until logging is ENABLED.

Log local actions should always work, even if log replies or log queries is disabled. So what do you mean that logging needs to be enabled? What does logging enabled or disabled mean to you? Verbosity?

Martineau · May 11, 2020

juched said:
Log local actions should always work, even if log replies or log queries is disabled. So what do you mean that logging needs to be enabled?

What does logging enabled or disabled mean to you? Verbosity?

Yes

glehel · May 11, 2020

how to add another rpz list? can I use multiple lists?

Martineau · May 11, 2020

glehel said:
how to add another rpz list? can I use multiple lists?

Add them to

'/opt/share/unbound/configs/rpzsites'

Thread starter	Title	Forum	Replies	Date
	Unbound Force all DNS requests through Unbound using iptables?	Asuswrt-Merlin AddOns	14	Oct 27, 2024
Z	Unbound Unbound + AdGuard Settings	Asuswrt-Merlin AddOns	1	Oct 19, 2024
Z	Unbound Unbound on subnet router	Asuswrt-Merlin AddOns	3	Oct 4, 2024
B	Unbound Does unbound update automatically?	Asuswrt-Merlin AddOns	8	Sep 8, 2024
F	Unbound Unbound stopped working	Asuswrt-Merlin AddOns	3	Sep 7, 2024
B	Unbound Unbound whitelist	Asuswrt-Merlin AddOns	15	Aug 30, 2024
	Is it possible to use nord dns with unbound or any way to add it?	Asuswrt-Merlin AddOns	9	Jul 21, 2024
	Guide to using vpn with unbound dns?	Asuswrt-Merlin AddOns	3	Jul 16, 2024
B	Unbound Use unbound/router as default DNS server	Asuswrt-Merlin AddOns	2	Jul 12, 2024
B	Unbound Test unbound	Asuswrt-Merlin AddOns	6	Jul 6, 2024

Unbound unbound_manager (Manager/Installer utility for unbound - Recursive DNS Server)

Part of the Furniture

Regular Contributor

Very Senior Member

Part of the Furniture

Very Senior Member

Senior Member

Very Senior Member

Very Senior Member

Part of the Furniture

Very Senior Member

Part of the Furniture

Very Senior Member

Part of the Furniture

Very Senior Member

Part of the Furniture

Part of the Furniture

Very Senior Member

Part of the Furniture

Regular Contributor

Part of the Furniture

Similar threads

Similar threads

Support SNBForums w/ Amazon

Sign Up For SNBForums Daily Digest