Unbound unbound_manager (Manager/Installer utility for unbound - Recursive DNS Server) - General questions / discussion thread 2

[juched]

P.S. I didn't write the GUI for unbound, so @juched will need to fix it to comply with the new enhanced serialisation process method used by the other Addon tabs.

Thanks for the heads up. I missed that this process has changed. Will need to review. Is there a thread with better details? or perhaps I just look at one of Jack's latest to update.
(Edit: Is this the change? https://github.com/jackyaz/ntpMerlin/commit/dfd47dd6cb1e7ef926c1e3d8fdbdff2cf6a6c03e)


On another note, given this new change in 386.2 coming, perhaps we want to remove the install of "haveged"?

- NEW: Added jittertimer-rngd daemon to HND routers. This will
ensure sufficient entropy is generated early on at
boot time, reducing boot stalls caused by insufficient
entropy for the kernel's random number generator,
and also generally improves security related to
crypto operations by the router.

 

[Martineau]

Thanks for the heads up. I missed that this process has changed. Will need to review. Is there a thread with better details? or perhaps I just look at one of Jack's latest to update.

Thanks

On another note, given this new change in 386.2 coming - NEW: Added jittertimer-rngd daemon to HND routers. , perhaps we want to remove the install of "haveged"?

So what about non-HND router owners e.g. RT-AC68U's?

plus...it may not actually make it into the final Release

[Alpha] 386.2

^^^ Updated 1 x RT-AC86U WAPs to 386.2_alpha1-gc52b410e89 just uploaded about 8 hours ago. Notify on issues. TY!

www.snbforums.com

 

[juched]

So what about non-HND router owners e.g. RT-AC68U's?

plus...it may not actually make it into the final Release

Good point, for non-HND it may be good to keep. It is optional in unbound today right? Wasn't sure if there is harm running both. We will see.

 

[nzwayne]

So you can see in PIC11.jpg there are two mount points for menutree.js ...so a new modified version of /tmp/menutree.js with a new Addon can never be created until one is manually removed.

You can see only two entries 'Flexqos' an 'scmerlin' were physically inserted into /tmp/menutree.js, although the .asp files for other tabs were created.

OK, so if you issue

umount  /www/require/modules/menuTree.js

df

then re-request the creation of the unbound Addon GUI using sgui does the tab immediately appear?

P.S. I didn't write the GUI for unbound, so @juched will need to fix it to comply with the new enhanced serialisation process method used by the other Addon tabs.

Those steps worked to restore the Unbound tab. Thanks again Martineau.

 

[nzwayne]

Now all the AddOn tabs back (yippee...), back to no data in the Unbound panels, just the text section. Is that directly related to the data overlay issue Juched is to solution?

 

[Martineau]

Now all the AddOn tabs back (yippee...), back to no data in the Unbound panels, just the text section. Is that directly related to the data overlay issue Juched is to solution?

View attachment 31602

Have you tried restarting unbound and then manually running both of @juchds's scripts? in the order shown in cron.

 

[nzwayne]

then manually running both of @juchds's scripts? in the order shown in cron.

Sorry, I have no programming knowledge. I couldn't spot which two of jucheds's scripts looking at page 1 header of this and original forum. I looked up "cron" so learnt something more today.

 

[nzwayne]

Sorry, I have no programming knowledge. I couldn't spot which two of jucheds's scripts looking at page 1 header of this and original forum. I looked up "cron" so learnt something more today.

The Data has returned...yippee...

 

[juched]

I have pushed an update to support the new standard locking mechanism for addon pages. It seems to work for me (but it also worked before too). Let me know if this fixes items for people with missing Unbound page.

To update simply run the sgui command again.

 

[steveh57]

I have pushed an update to support the new standard locking mechanism for addon pages. It seems to work for me (but it also worked before too). Let me know if this fixes items for people with missing Unbound page.

To update simply run the sgui command again.

Updated via sgui command, removed 3 second sleep in post-mount script and the Unbound tab disappears again on reboot

 

[Milan]

Updated via sgui command, removed 3 second sleep in post-mount script and the Unbound tab disappears again on reboot

I have reversed case - unbound tab is there but not others. In most cases a second restart will fix it ...

 

[juched]

Updated via sgui command, removed 3 second sleep in post-mount script and the Unbound tab disappears again on reboot

Can you verify that unbound_stats.sh is version 1.4?

 

[chongnt]

Can you verify that unbound_stats.sh is version 1.4?

Mine shows VERSION 1.12.0

+======================================================================+
|  Welcome to the unbound Manager/Installation script (Asuswrt-Merlin) |
|                                                                      |
|                      Version 3.22 by Martineau                       |
|                                                                      |
+======================================================================+


unbound (pid 11225) is running... uptime: 0 Days, 00:03:48 version: 1.12.0 # Version=v1.12 Martineau update (Date Loaded by unbound_manager Mon Mar 8 20:24:52 MYT 2021)

1  = Update unbound files and configuration                                             5  = Install Ad and Tracker blocker (Ad Block)
2  = Remove unbound/unbound_manager                                                     6  = Uninstall Graphical Statistics GUI Add-on TAB
3  = Stop unbound                                                                       7  = Disable   DNS Firewall [?]
4  = Show unbound statistics                                                            8  = Install YouTube Ad blocker
                                                                                        9  = Install Safe Search e.g. google.com->forcesafesearch.google.com

?  = About Configuration
v  = View ('/opt/var/lib/unbound/'unbound.conf)

e  = Exit Script [?]

E:Option ==> u

unbound_manager.sh is already the latest version. 6b4a500c071bcbb3f4a6e9596a178d43

 

[juched]

Mine shows VERSION 1.12.0

+======================================================================+
|  Welcome to the unbound Manager/Installation script (Asuswrt-Merlin) |
|                                                                      |
|                      Version 3.22 by Martineau                       |
|                                                                      |
+======================================================================+


unbound (pid 11225) is running... uptime: 0 Days, 00:00:08 version: 1.12.0 # Version=v1.12 Martineau update (Date Loaded by unbound_manager Mon Mar 8 20:24:52 MYT 2021)

The version is inside the unbound_stats.sh file under the /jffs/addons/unbound folder.
The version you are seeing is the config file. That is separate.

 

[steveh57]

Can you verify that unbound_stats.sh is version 1.4?

It is indeed version 1.4 , and it works perfectly with a 3 second sleep in post_mount prior to it being called

 

[chongnt]

The version is inside the unbound_stats.sh file under the /jffs/addons/unbound folder.
The version you are seeing is the config file. That is separate.

Thanks. Now I see the correct version 1.4

admin@RT-AC86U-DBA8:/jffs/addons/unbound# more unbound_stats.sh
#!/bin/sh
##
# ____ ___     ___.                            .___   _________ __          __         
#|    |   \____\_ |__   ____  __ __  ____    __| _/  /   _____//  |______ _/  |_  ______
#|    |   /    \| __ \ /  _ \|  |  \/    \  / __ |   \_____  \\   __\__  \\   __\/  ___/
#|    |  /   |  \ \_\ (  <_> )  |  /   |  \/ /_/ |   /        \|  |  / __ \|  |  \___ \
#|______/|___|  /___  /\____/|____/|___|  /\____ |  /_______  /|__| (____  /__| /____  >
#             \/    \/                  \/      \/          \/           \/          \/
## by @juched - Generate Stats for GUI tab
## with credit to @JackYaz for his shared scripts
## v1.0.0 - initial text based only UI items
## v1.1.0 - March 3 2020 - Added graphs for histogram and answers, fixed install to not create duplicate tabs
## v1.1.1 - March 8 2020 - Added new install of JackYaz shared graphing files (previously needed to have one of JackYaz's other plugins installed)
## v1.1.2 - March 9 2020 - Cleanup .db and .md5 files on uninstall, move startup to post-mount, fixed directory check
## v1.2.0 - March 23 2020 - Add output for top ad blocked graph top 10 and top domains - moved stats DB to USB
## v1.2.1 - March 26 2020 - Added daily replies table
## v1.2.2 - April 5 2020 - Added tracking of client ip
## v1.2.3 - April 10 2020 - Fixed issue with "" domain name in SQL, breaking JS
## v1.2.4 - April 12 2020 - Removed error message on clean install for missing md5 file
## v1.2.5 - April 13 2020 - During install, do not Generate stats if unbound is not running
## v1.3.0 - April 16 2020 - Show stats for DNS Firewall
## v1.4.0 - March 7 2021 - Introduce locking standard around mounting and unmouning, increase max pages to 20
readonly SCRIPT_VERSION="v1.4.0"

 

[juched]

It is indeed version 1.4 , and it works perfectly with a 3 second sleep in post_mount prior to it being called

Can you share the content of your post-mount file? Curious to see what is causing this timing issue. I am running on an External drive SSD, perhaps that is adjusting the timing as well.

This is mine:

#!/bin/sh
swapon /tmp/mnt/entware-ssd/myswap.swp # Skynet

. /jffs/addons/diversion/mount-entware.div # Added by amtm
cru a logrotate "5 0 * * * /opt/sbin/logrotate /opt/etc/logrotate.conf >> /opt/t

/jffs/addons/unbound/unbound_stats.sh startup # Unbound_Stats.sh

/jffs/scripts/ntpmerlin startup "$@" & # ntpMerlin
/jffs/scripts/spdmerlin startup "$@" & # spdMerlin
/jffs/scripts/uiScribe startup "$@" & # uiScribe
/jffs/scripts/scmerlin startup "$@" & # scMerlin

Edit: Perhaps try this. Change the Unbound_stats.sh startup line this this:
/jffs/addons/unbound/unbound_stats.sh startup > /jffs/ubound_stats.log


and lets see what is happening during boot?

 

[steveh57]

Can you share the content of your post-mount file? Curious to see what is causing this timing issue. I am running on an External drive SSD, perhaps that is adjusting the timing as well.

This is mine:


Edit: Perhaps try this. Change the Unbound_stats.sh startup line this this:
/jffs/addons/unbound/unbound_stats.sh startup > /jffs/ubound_stats.log


and lets see what is happening during boot?

post_mount

#!/bin/sh
swapon /tmp/mnt/asus/myswap.swp # Added by amtm
. /jffs/addons/diversion/mount-entware.div # Added by Diversion
/jffs/scripts/scmerlin startup "$@" & # scMerlin
/bin/sleep 3s  #give the disk 3 sec to really be ready
/jffs/addons/unbound/unbound_stats.sh startup # Unbound_Stats.sh
/jffs/scripts/ntpmerlin startup "$@" & # ntpMerlin

unbound_stats log

##
# ____ ___     ___.                            .___   _________ __          __         
#|    |   \____\_ |__   ____  __ __  ____    __| _/  /   _____//  |______ _/  |_  ______
#|    |   /    \| __ \ /  _ \|  |  \/    \  / __ |   \_____  \   __\__  \   __\/  ___/
#|    |  /   |  \ \_\ (  <_> )  |  /   |  \/ /_/ |   /        \|  |  / __ \|  |  \___ \
#|______/|___|  /___  /\____/|____/|___|  /\____ |  /_______  /|__| (____  /__| /____  >
#             \/    \/                  \/      \/          \/           \/          \/
## by @juched - Generate Stats for GUI tab - v1.4.0                                         
## with credit to @JackYaz for his shared scripts                                       

Mounting Unbound_Stats.sh WebUI page as user2.asp
Saving MD5 of installed file /jffs/addons/unbound/unboundstats_www.asp to /jffs/addons/unbound/www-installed.md5
Checking if Unbound is running to generate stats...
Calculated Cache Hit Percentage: 0.00
Adding new value to DB...
Calculating Daily data...
Calculating Weekly and Monthly data...
Outputting histogram performance data...
Outputting answers data...
Outputting top blocked domains...
Outputting top replies ...
Outputting daily replies ...
Calculating DNS Firewall data...
Outputting DNS Firewall Hits ...

As is life, the last reboot retained the Ubound GUI without the 3 second sleep, at a previous attempt the gui was not retained and I'm fairly certain the log was not appended

 

[juched]

post_mount

#!/bin/sh
swapon /tmp/mnt/asus/myswap.swp # Added by amtm
. /jffs/addons/diversion/mount-entware.div # Added by Diversion
/jffs/scripts/scmerlin startup "$@" & # scMerlin
/bin/sleep 3s  #give the disk 3 sec to really be ready
/jffs/addons/unbound/unbound_stats.sh startup # Unbound_Stats.sh
/jffs/scripts/ntpmerlin startup "$@" & # ntpMerlin

unbound_stats log

##
# ____ ___     ___.                            .___   _________ __          __        
#|    |   \____\_ |__   ____  __ __  ____    __| _/  /   _____//  |______ _/  |_  ______
#|    |   /    \| __ \ /  _ \|  |  \/    \  / __ |   \_____  \   __\__  \   __\/  ___/
#|    |  /   |  \ \_\ (  <_> )  |  /   |  \/ /_/ |   /        \|  |  / __ \|  |  \___ \
#|______/|___|  /___  /\____/|____/|___|  /\____ |  /_______  /|__| (____  /__| /____  >
#             \/    \/                  \/      \/          \/           \/          \/
## by @juched - Generate Stats for GUI tab - v1.4.0                                        
## with credit to @JackYaz for his shared scripts                                      

Mounting Unbound_Stats.sh WebUI page as user2.asp
Saving MD5 of installed file /jffs/addons/unbound/unboundstats_www.asp to /jffs/addons/unbound/www-installed.md5
Checking if Unbound is running to generate stats...
Calculated Cache Hit Percentage: 0.00
Adding new value to DB...
Calculating Daily data...
Calculating Weekly and Monthly data...
Outputting histogram performance data...
Outputting answers data...
Outputting top blocked domains...
Outputting top replies ...
Outputting daily replies ...
Calculating DNS Firewall data...
Outputting DNS Firewall Hits ...

As is life, the last reboot retained the Ubound GUI without the 3 second sleep, at a previous attempt the gui was not retained and I'm fairly certain the log was not appended

If you want the log to be appended, then we need to change to >> instead of >.

This will keep appending to the file for every boot. The one you sent is just the successful one. Not quite sure yet, but it seems there may be an issue with the disk not ready. There is a wait for Unbound to start, which is on the disk, so not sure how this could fail yet.

ie.:

/jffs/addons/unbound/unbound_stats.sh startup >> /jffs/ubound_stats.log

 

[chongnt]

If you want the log to be appended, then we need to change to >> instead of >.

This will keep appending to the file for every boot. The one you sent is just the successful one. Not quite sure yet, but it seems there may be an issue with the disk not ready. There is a wait for Unbound to start, which is on the disk, so not sure how this could fail yet.

ie.:

I did append but all the records seems to be the same. I did 4 times.
First I reboot using CLI, then twice by power cycle the physical power button. The unbound tab did not come up after reboot.
Last I put back the sleep and reboot using CLI.

Spoiler

***first startup by reboot***
##
# ____ ___     ___.                            .___   _________ __          __         
#|    |   \____\_ |__   ____  __ __  ____    __| _/  /   _____//  |______ _/  |_  ______
#|    |   /    \| __ \ /  _ \|  |  \/    \  / __ |   \_____  \   __\__  \   __\/  ___/
#|    |  /   |  \ \_\ (  <_> )  |  /   |  \/ /_/ |   /        \|  |  / __ \|  |  \___ \
#|______/|___|  /___  /\____/|____/|___|  /\____ |  /_______  /|__| (____  /__| /____  >
#             \/    \/                  \/      \/          \/           \/          \/
## by @juched - Generate Stats for GUI tab - v1.4.0                                        
## with credit to @JackYaz for his shared scripts                                      

Mounting Unbound_Stats.sh WebUI page as user1.asp
Saving MD5 of installed file /jffs/addons/unbound/unboundstats_www.asp to /jffs/addons/unbound/www-installed.md5
Checking if Unbound is running to generate stats...
Calculated Cache Hit Percentage: 0
Adding new value to DB...
Calculating Daily data...
Calculating Weekly and Monthly data...
Outputting histogram performance data...
Outputting answers data...
Outputting top blocked domains...
Outputting top replies ...
Outputting daily replies ...
Calculating DNS Firewall data...
Outputting DNS Firewall Hits ...

***second startup by power cycle***


##
# ____ ___     ___.                            .___   _________ __          __         
#|    |   \____\_ |__   ____  __ __  ____    __| _/  /   _____//  |______ _/  |_  ______
#|    |   /    \| __ \ /  _ \|  |  \/    \  / __ |   \_____  \   __\__  \   __\/  ___/
#|    |  /   |  \ \_\ (  <_> )  |  /   |  \/ /_/ |   /        \|  |  / __ \|  |  \___ \
#|______/|___|  /___  /\____/|____/|___|  /\____ |  /_______  /|__| (____  /__| /____  >
#             \/    \/                  \/      \/          \/           \/          \/
## by @juched - Generate Stats for GUI tab - v1.4.0                                        
## with credit to @JackYaz for his shared scripts                                      

Mounting Unbound_Stats.sh WebUI page as user1.asp
Saving MD5 of installed file /jffs/addons/unbound/unboundstats_www.asp to /jffs/addons/unbound/www-installed.md5
Checking if Unbound is running to generate stats...
Calculated Cache Hit Percentage: 0
Adding new value to DB...
Calculating Daily data...
Calculating Weekly and Monthly data...
Outputting histogram performance data...
Outputting answers data...
Outputting top blocked domains...
Outputting top replies ...
Outputting daily replies ...
Calculating DNS Firewall data...
Outputting DNS Firewall Hits ...

***third startup by power cycle***



##
# ____ ___     ___.                            .___   _________ __          __         
#|    |   \____\_ |__   ____  __ __  ____    __| _/  /   _____//  |______ _/  |_  ______
#|    |   /    \| __ \ /  _ \|  |  \/    \  / __ |   \_____  \   __\__  \   __\/  ___/
#|    |  /   |  \ \_\ (  <_> )  |  /   |  \/ /_/ |   /        \|  |  / __ \|  |  \___ \
#|______/|___|  /___  /\____/|____/|___|  /\____ |  /_______  /|__| (____  /__| /____  >
#             \/    \/                  \/      \/          \/           \/          \/
## by @juched - Generate Stats for GUI tab - v1.4.0                                        
## with credit to @JackYaz for his shared scripts                                      

Mounting Unbound_Stats.sh WebUI page as user1.asp
Saving MD5 of installed file /jffs/addons/unbound/unboundstats_www.asp to /jffs/addons/unbound/www-installed.md5
Checking if Unbound is running to generate stats...
Calculated Cache Hit Percentage: 0
Adding new value to DB...
Calculating Daily data...
Calculating Weekly and Monthly data...
Outputting histogram performance data...
Outputting answers data...
Outputting top blocked domains...
Outputting top replies ...
Outputting daily replies ...
Calculating DNS Firewall data...
Outputting DNS Firewall Hits ...


***put back sleep 1 sec, reboot by CLI*


##
# ____ ___     ___.                            .___   _________ __          __         
#|    |   \____\_ |__   ____  __ __  ____    __| _/  /   _____//  |______ _/  |_  ______
#|    |   /    \| __ \ /  _ \|  |  \/    \  / __ |   \_____  \   __\__  \   __\/  ___/
#|    |  /   |  \ \_\ (  <_> )  |  /   |  \/ /_/ |   /        \|  |  / __ \|  |  \___ \
#|______/|___|  /___  /\____/|____/|___|  /\____ |  /_______  /|__| (____  /__| /____  >
#             \/    \/                  \/      \/          \/           \/          \/
## by @juched - Generate Stats for GUI tab - v1.4.0                                        
## with credit to @JackYaz for his shared scripts                                      

Mounting Unbound_Stats.sh WebUI page as user2.asp
Saving MD5 of installed file /jffs/addons/unbound/unboundstats_www.asp to /jffs/addons/unbound/www-installed.md5
Checking if Unbound is running to generate stats...
Calculated Cache Hit Percentage: 0
Adding new value to DB...
Calculating Daily data...
Calculating Weekly and Monthly data...
Outputting histogram performance data...
Outputting answers data...
Outputting top blocked domains...
Outputting top replies ...
Outputting daily replies ...
Calculating DNS Firewall data...
Outputting DNS Firewall Hits ...

Here is my post-mount script. First I add 3 sec, later I found that even if I put 1 sec the effect are the same.

Spoiler

#!/bin/sh

# Turn on usage of swapfile
if [ -f "/tmp/mnt/amtm/myswap.swp" ];then
swapon /tmp/mnt/amtm/myswap.swp # Added by Diversion
echo "Turning Swapfile On"
fi

# Turn on uiDivStats, connmon, ntpMerlin, spdmerlin and other addons
. /jffs/addons/diversion/mount-entware.div # Added by Diversion
/jffs/scripts/scmerlin startup "$@" & # scMerlin
/jffs/scripts/uiDivStats startup "$@" & # uiDivStats
/jffs/scripts/uiScribe startup "$@" & # uiScribe
cru a logrotate "5 0 * * * /opt/sbin/logrotate /opt/etc/logrotate.conf >> /opt/tmp/logrotate.daily 2>&1" # added by scribe
/jffs/scripts/connmon startup "$@" & # connmon
/jffs/scripts/ntpmerlin startup "$@" & # ntpMerlin

/bin/sleep 1s    # give the disk 1 sec to really be ready
# Add cronjobs to cron
crontab -u admin /jffs/configs/cronjobs

/jffs/addons/unbound/unbound_stats.sh startup # Unbound_Stats.sh
/jffs/scripts/spdmerlin startup "$@" & # spdMerlin
/jffs/scripts/vpnmgr startup "$@" & # vpnmgr_startup