Mounting issues with add-ons

[Jack Yaz]

NTP "aggressive" syncing on boot while I thrash out a proper fix
Create /jffs/scripts/ntpbootwatchdog.sh

#!/bin/sh
if [ "$(nvram get ntp_ready)" -eq 1 ]; then
    /usr/bin/logger -st "ntpbootwatchdog" "NTP is synced, exiting"
else
    /usr/bin/logger -st "ntpbootwatchdog" "NTP boot watchdog started..."
    ntptimer=0
    while [ "$(nvram get ntp_ready)" -eq 0 ] && [ "$ntptimer" -lt 600 ]; do
        killall ntp
        killall ntpd
        service restart_ntpd
        ntptimer=$((ntptimer+30))
        sleep 30
        /usr/bin/logger -st "ntpbootwatchdog" "Still waiting for NTP to sync..."
    done
    
    if [ "$ntptimer" -ge 600 ]; then
        /usr/bin/logger -st "ntpbootwatchdog" "NTP failed to sync after 10 minutes - please check immediately!"
        exit 1
    else
        /usr/bin/logger -st "ntpbootwatchdog" "NTP has synced!"
    fi
fi

Create or edit /jffs/scripts/init-start

#!/bin/sh
sh /jffs/scripts/ntpbootwatchdog.sh & #make sure NTP starts

Make sure both are executable

chmod +x /jffs/scripts/ntpbootwatchdog.sh
chmod +x /jffs/scripts/init-start

 

[Jack Yaz]

Instead of the script above, I've made some changes on ntpmerlin's develop branch to work around the post-mount timeout whilst still being able to wait for NTP up to 5 minutes. The downside is Unbound will still block so would need a similar update to run async from post-mount/rc.unslung/the S startup script. The same holds true for any other S scripts that run a blocking wait loop

 

[dave14305]

Instead of the script above, I've made some changes on ntpmerlin's develop branch to work around the post-mount timeout whilst still being able to wait for NTP up to 5 minutes. The downside is Unbound will still block so would need a similar update to run async from post-mount/rc.unslung/the S startup script. The same holds true for any other S scripts that run a blocking wait loop

Just to be a troublemaker, I’ll suggest that all these wait loops for ntp can be replaced by a service-event-end hook for restart_diskmon which is always called by ntpd_synced. I used this in my own Unbound implementation. Just something to think about...

Jan 30 18:50:19 rc_service: ntpd_synced 4171:notify_rc restart_diskmon
Jan 30 18:50:19 custom_script: Running /jffs/scripts/service-event (args: restart diskmon)
Jan 30 18:50:19 custom_script: Running /jffs/scripts/service-event-end (args: restart diskmon)

 

[DanNuke86]

NTP "aggressive" syncing on boot while I thrash out a proper fix
Create /jffs/scripts/ntpbootwatchdog.sh

#!/bin/sh

logger -st "System" "NTP boot watchdog started..."

ntptimer=0
while [ "$(nvram get ntp_ready)" = 0 ] && [ "$ntptimer" -lt 300 ]; do
    killall ntp
    killall ntpd
    service restart_ntpd
    ntptimer=$((ntptimer+30))
    sleep 30
done

if [ "$ntptimer" -ge 300 ]; then
    logger -st "System" "NTP failed to sync after 5 minutes - please check immediately!"
    exit 1
fi

Create or edit /jffs/scripts/init-start

#!/bin/sh
sh /jffs/scripts/ntpbootwatchdog.sh & #make sure NTP starts

Make sure both are executable

chmod +x /jffs/scripts/ntpbootwatchdog.sh
chmod +x /jffs/scripts/init-start

I've managed to follow this, coupled with a quick read up on how to use vi. This was a success and following 3 reboots, the add-ons tab is now always present and 5 out of 6 add-on scripts are present. Just Unbound which is proving to be elusive now. I did try re-installing this and it mounted and was showing in the add-ons tab, but then didn't show again following a reboot.

I'll do some more testing tomorrow.

 

[DanNuke86]

Instead of the script above, I've made some changes on ntpmerlin's develop branch to work around the post-mount timeout whilst still being able to wait for NTP up to 5 minutes. The downside is Unbound will still block so would need a similar update to run async from post-mount/rc.unslung/the S startup script. The same holds true for any other S scripts that run a blocking wait loop

Jack, given I successfully (minus the fact unbound isn’t mounting) resolved the mount issue last night (many thanks for resolving/testing with me. My head was hurting this morning though..), is there any point in using the develop branch in addition? Or is it worth waiting for a permanent fix? I can live with unbound not mounting for now.

Just really happy the main scripts I like to use to monitor my ISP’s are mounting every time now.

 

[DanNuke86]

I've updated to the development branch now for ntpmerlin, but I'd had an issue since last night where something wasn't right with my connection. Carried a reboot out for this and there was a mounting issue again. I even had it an hour ago, where the syslog said they had mounted, but weren't showing, despite me clearing the cache, didn't change anything.

Typed the /jffs/scripts/ntpbootwatchdog.sh command in to SSH and the syslog said it had been started. did the 2 executable commands and rebooted, resulting in all but unbound were mounted again!

 

[Jack Yaz]

I've updated to the development branch now for ntpmerlin, but I'd had an issue since last night where something wasn't right with my connection. Carried a reboot out for this and there was a mounting issue again. I even had it an hour ago, where the syslog said they had mounted, but weren't showing, despite me clearing the cache, didn't change anything.

Typed the /jffs/scripts/ntpbootwatchdog.sh command in to SSH and the syslog said it had been started. did the 2 executable commands and rebooted, resulting in all but unbound were mounted again!

i;d be interested to see the failed syslog if you have it.

ntpmerlin develop fixes the NTP wait loop blocking post-mount, but Unbound in its current state could still block it. We'll need to ask @Martineau if he can update the S61unbound script to not run the ntp wait loop on the blocking call from post-mount/rc.unslung

 

[francovilar]

@Jack Yaz my post at the other topic that you kindly replied was for this issue... sorry about that.

this was my post:
I'm having the same problem. Have Diversion (webui), skynet, unbound, uidivstats, scmerlin and ntpmerlin (chrony) of this, after a reboot, I'm missing webui for:
Skynet, uidivstats and ntpmerlin, the rest are ok (the webui part)

edit: unbound webui is working but has an option to delete and install webui so I did try to unnistal and gave me this error:

user4.asp
sed: can't move '/tmp/menuTree.jsKTzQKi' to '/tmp/menuTree.js': Device or resource busy

I just add scribe to the list. Do you still need a log?
In my most recent reboot the list is complety different

 

[DanNuke86]

@Jack Yaz my post at the other topic that you kindly replied was for this issue... sorry about that.

this was my post:
I'm having the same problem. Have Diversion (webui), skynet, unbound, uidivstats, scmerlin and ntpmerlin (chrony) of this, after a reboot, I'm missing webui for:
Skynet, uidivstats and ntpmerlin, the rest are ok (the webui part)

edit: unbound webui is working but has an option to delete and install webui so I did try to unnistal and gave me this error:

user4.asp
sed: can't move '/tmp/menuTree.jsKTzQKi' to '/tmp/menuTree.js': Device or resource busy

I just add scribe to the list. Do you still need a log?
In my most recent reboot the list is complety different

Did you read through the previous posts? as he provided a fix and then said that ntpMerlin develop fixes the issue, but there’s still an issue with unbound (which needs the dev to tweak that one).

ntpmerlin develop in ssh and reboot, should resolve it.

 

[francovilar]

Did you read through the previous posts? as he provided a fix and then said that ntpMerlin develop fixes the issue, but there’s still an issue with unbound (which needs the dev to tweak that one).

ntpmerlin develop in ssh and reboot, should resolve it.

Yes, did you? I’m using the dev version. After that he asked for the log to other user...

edit: actually it was you... maybe I didn’t understand what he asked as the failed log

 

[Martineau]

edit: unbound webui is working but has an option to delete and install webui so I did try to unnistal and gave me this error:

user4.asp
sed: can't move '/tmp/menuTree.jsKTzQKi' to '/tmp/menuTree.js': Device or resource busy

Although it is impossible to now determine the exact reason of the above error, I understand there is no protection of the Addons tab management process used by @juched's 'unbound_stats.sh' script (or any other script?) when uninstalling, so there is no guarantee that the state/contents of '/tmp/menuTree.js' is as expected.

Perhaps a formal locking method should be adopted

e.g.

LOCKFILE="/tmp/Addons.LOCK"            # Global Lockfile name
Mount_WebUI(){

    FD=666                            # Unique File descriptor for $LOCKFILE to be used by ALL scripts.
    eval exec "$FD>$LOCKFILE"

    # Acquire EXCLUSIVE access to the critical resource (e.g. file) by only allowing one thread/script to lock the MUTEX semaphore                       
    flock -x $FD
   
        # Now we can Safely create/mangle the critical file as desired; but try to keep the amount of time the MUTEX is locked to a minimum.
       
        [ ! -f "/tmp/menuTree.js" ] && cp -f "/www/require/modules/menuTree.js" "/tmp/"

        sed '/do some modification to the resource file/d' /tmp/menuTree.js'
       
        umount /www/require/modules/menuTree.js 2>/dev/null
        mount -o bind /tmp/menuTree.js /www/require/modules/menuTree.js

    flock -u $FD

}

 

[Jack Yaz]

Although it is impossible to now determine the exact reason of the above error, I understand there is no protection of the Addons tab management process used by @juched's 'unbound_stats.sh' script (or any other script?) when uninstalling, so there is no guarantee that the state/contents of '/tmp/menuTree.js' is as expected.

Perhaps a formal locking method should be adopted

e.g.

LOCKFILE="/tmp/Addons.LOCK"            # Global Lockfile name
Mount_WebUI(){

    FD=666                            # Unique File descriptor for $LOCKFILE to be used by ALL scripts.
    eval exec "$FD>$LOCKFILE"

    # Acquire EXCLUSIVE access to the critical resource (e.g. file) by only allowing one thread/script to lock the MUTEX semaphore                    
    flock -x $FD

        # Now we can Safely create/mangle the critical file as desired; but try to keep the amount of time the MUTEX is locked to a minimum.
    
        [ ! -f "/tmp/menuTree.js" ] && cp -f "/www/require/modules/menuTree.js" "/tmp/"

        sed '/do some modification to the resource file/d' /tmp/menuTree.js'
    
        umount /www/require/modules/menuTree.js 2>/dev/null
        mount -o bind /tmp/menuTree.js /www/require/modules/menuTree.js

    flock -u $FD

}

This will solve collisions on MenuTree.js which is one source of webui tab mount failure. I'll look to implement it in my next dev cycle, thanks!

The other is causing a 2 minute timeout on post-mount to be exceeded thus terminating it. rc.unslung is called from post-mount which starts Entware services synchronously. Therefore S61unbound and S77* which run a wait loop for NTP to sync could breach this. I've implemented a bit of a hack to move the wait loop to a secondary daemon/script in ntpmerlin.

Another solution may be to call rc.unslung backgrounded from post-mount i.e. Add &
This could be considered by @thelonelycoder who handles Entware in amtm and Diversion

 

[Martineau]

This will solve collisions on MenuTree.js which is one source of webui tab mount failure. I'll look to implement it in my next dev cycle, thanks!

The other is causing a 2 minute timeout on post-mount to be exceeded thus terminating it. rc.unslung is called from post-mount which starts Entware services synchronously. Therefore S61unbound and S77* which run a wait loop for NTP to sync could breach this. I've implemented a bit of a hack to move the wait loop to a secondary daemon/script in ntpmerlin.

Another solution may be to call rc.unslung backgrounded from post-mount i.e. Add &
This could be considered by @thelonelycoder who handles Entware in amtm and Diversion

You are making an hubristic assumption that everyone is using ntpmerlin (or Diversion to install Entware)

i.e.
unbound_manager installs /init.d/S61unbound which contains a prudent spin-loop for NTP given unbound requires an accurate clock for its security certificate validation.

So if there are no other scripts and @juched's unbound_stats script is implemented, then I believe the unbound GUI statsistics tab is ALWAYS correctly applied.

If not then as noted in this post perhaps the 120 sec dead-man timer should be increased?

However, users using ntpmerlin should update '/init.d/S61unbound' as follows

e  = Exit Script [?]

A:Option ==> i s61unbound dev

Updating S61unbound
    S61unbound downloaded successfully Github 'dev/development' branch

and the next time post-mount runs (normally as part of the router boot process) the unbound NTP 'blocking' spin-loop should no longer be an issue.

If there are no reported issues with script S61unbound v1.02 then it will be either included in unbound_manager v3.23 or as a standalone Hotfix.

 

[Jack Yaz]

You are making an hubristic assumption that everyone is using ntpmerlin (or Diversion to install Entware)

i.e.
unbound_manager installs /init.d/S61unbound which contains a prudent spin-loop for NTP given unbound requires an accurate clock for its security certificate validation.

So if there are no other scripts and @juched's unbound_stats script is implemented, then I believe the unbound GUI statsistics tab is ALWAYS correctly applied.

If not then as noted in this post perhaps the 120 sec dead-man timer should be increased?

However, users using ntpmerlin should update '/init.d/S61unbound' as follows

e  = Exit Script [?]

A:Option ==> i s61unbound dev

Updating S61unbound
    S61unbound downloaded successfully Github 'dev/development' branch

and the next time post-mount runs (normally as part of the router boot process) the unbound NTP 'blocking' spin-loop should no longer be an issue.

If there are no reported issues with script S61unbound v1.02 then it will be included in unbound_manager v3.23

The current versions S61unbound and S77ntpd/S77chronyd (and any others I can't recall off the top of my head) block any S scripts after them that are called, and anything else after rc.unslung in post-mount, assuming NTP does not sync before the 2 minute is up. The 2 minute timer is a firmware limit and would need to be increased by @RMerlin I believe, if he considered increasing the limit as something that would be beneficial and not cause problems elsewhere that are not yet identified.

I'm not pointing the blame at you, as my scripts are equally at fault. It's a change we need to collaborate on making as simple as possible for users, and other addon developers, to not be bitten by.

Re. Entware installation, I believe the majority of the user base install Entware via amtm (which is baked into the firmware), and the "install-entware" script has actually been replaced by the amtm approach.

 

[francovilar]

@Martineau and @Jack Yaz thank you very much, for me it’s working. When I installed scribe support for unbound I had to apply Martineau fix again (or my installation was all over the place, I’ve reinstall all and all is ok).
thank you @DanNuke86 for bringing this issue up.

 

[DanNuke86]

I may not know a lot, but I’m a quick learner and finding faults is something I’m good at, spotting anomalies etc, so I’m happy to contribute to the fantastic community on this forum where so many people are helping each other!

 

[juched]

Let me know what needs to change once we have a standard and I will try to find time to make an update. Wasn't aware people were having Addon UI issues during install or uninstall. I have never seen any myself.

 

[thelonelycoder]

Re. Entware installation, I believe the majority of the user base install Entware via amtm (which is baked into the firmware), and the "install-entware" script has actually been replaced by the amtm approach.

The symlink entware-setup.sh to amtm - introduced in firmware 384.15 to replace the dated original Entware install script - has been removed in Asuswrt-Merlin 386.xx.
It died of old age

 

[zd59]

Update/Reset Mini Guide

formatting the USB drive in a PC with NTFS file format.

Is there any exact reason, to use NTFS formatted drive, since this is Window$ native, Router (read Linux) native is EXT4?
I use EXT4 ever since, and had zero problems (Linksys, Netgear and now Asus RT-AC86u).

 

[L&LD]

The reason to format in NTFS is so that there is no possibility that any previous amtm or script remnants are picked up by amtm the moment you insert the drive into the router.

This ensures the 'cleanest' install possible of Entware, scripts, etc.

But you also need to use amtm to format the USB drive properly before using it too. As mentioned in the amtm Step-by-Step guide in my signature (see link below).