Mounting issues with add-ons

[zd59]

The reason to format in NTFS is so that there is no possibility that any previous amtm or script remnants are picked up by amtm the moment you insert the drive into the router.

This ensures the 'cleanest' install possible of Entware, scripts, etc.

But you also need to use amtm to format the USB drive properly before using it too. As mentioned in the amtm Step-by-Step guide in my signature (see link below).

Thanks for explanation.
So I can safely format EXT4 on my Linux PC and use it for Entware. I installed it before amtm was included in RMerlins FW.
And I like that. I only use additions included in Entware and assume amtm as not needed router CPU overkill. See how many users complain about high temperature. This also could be the reason.

Users should be aware of that Routers are NOT regular computers, and so their processors (and other hardware) are much lower capability.
Routers are designed for use as routers and access points, all additional use is a risk to overload resources (routing speed drop, overheating, no free RAM, use of a swap file, to count some). So install and use only absolutely indispensable services.

To install Entware there is no need to use "man in the middle" (amtm), just fire install script on Entware repository.
Or maybe I'm wrong?

 

[thelonelycoder]

Thanks for explanation.
So I can safely format EXT4 on my Linux PC and use it for Entware. I installed it before amtm was included in RMerlins FW.
And I like that. I only use additions included in Entware and assume amtm as not needed router CPU overkill. See how many users complain about high temperature. This also could be the reason.

Users should be aware of that Routers are NOT regular computers, and so their processors (and other hardware) are much lower capability.
Routers are designed for use as routers and access points, all additional use is a risk to overload resources (routing speed drop, overheating, no free RAM, use of a swap file, to count some). So install and use only absolutely indispensable services.

To install Entware there is no need to use "man in the middle" (amtm), just fire install script on Entware repository.
Or maybe I'm wrong?

amtm certainly is no man in the middle. It comes with the firmware. Its Entware installer is the best you can get. The old Entware install script is outdated and would not be compatible with the way we handle scripts these days.

 

[zd59]

amtm certainly is no man in the middle. It comes with the firmware. Its Entware installer is the best you can get. The old Entware install script is outdated and would not be compatible with the way we handle scripts these days.

Thanks.

First I congratulate you for a great amtm. It offers a lot of router enhancements.

As I've noted in a previous post, I reused USB disk with Entware installed two years ago (regularly updated) on AC86u with RMerlin FW 386.1. I only restored backed up /jffs/scripts and it works flawless.
This is important info for users wondering to upgrade FW and fear to install Entware from the beginning. Its great shortcut.

 

[Deleted member 62525]

Just to be a troublemaker, I’ll suggest that all these wait loops for ntp can be replaced by a service-event-end hook for restart_diskmon which is always called by ntpd_synced. I used this in my own Unbound implementation. Just something to think about...

Jan 30 18:50:19 rc_service: ntpd_synced 4171:notify_rc restart_diskmon
Jan 30 18:50:19 custom_script: Running /jffs/scripts/service-event (args: restart diskmon)
Jan 30 18:50:19 custom_script: Running /jffs/scripts/service-event-end (args: restart diskmon)

Can you share how you implemented your solution for NTP sync and Unbound? Sounds very promising and intuitive.

 

[dave14305]

Can you share how you implemented your solution for NTP sync and Unbound? Sounds very promising and intuitive.

If NTP is not synced during Unbound startup, I add val-override-date: -1 to the unbound.conf to ignore bad dates. Once NTP is synced and restart_diskmon is triggered from service-event-end script, Unbound will restart and NOT add val-override-date: -1 to the config for DNSSEC. I can do this because in my approach, the unbound.conf is dynamically generated during every start. It's not necessarily compatible with the way Unbound_Manager works today. I don't encourage anyone to switch to my version, since it's not actively maintained in light of the Unbound_Manager proliferation.

The code is available to review here:

GitHub - dave14305/Unbound-Merlin-UI: ASUSWRT-Merlin Unbound User Interface

ASUSWRT-Merlin Unbound User Interface. Contribute to dave14305/Unbound-Merlin-UI development by creating an account on GitHub.

github.com

 

[Deleted member 62525]

I don't have a full understanding of all events that are triggered during the startup/reboot but what you have described begs a question. Would it be better to start Unbound when NTP is synced when restart_disckmon is triggered as you decribed? If I understand it correctly this would guarantee that NTP is already synchronized.

 

[dave14305]

I don't have a full understanding of all events that are triggered during the startup/reboot but what you have described begs a question. Would it be better to start Unbound when NTP is synced when restart_disckmon is triggered as you decribed? If I understand it correctly this would guarantee that NTP is already synchronized.

I don't know if it'd be "better" or just "different". We never have any guarantee that NTP will sync. So starting it up without the DNSSEC timestamp checks is an acceptable risk, which is usually temporary in most cases. Some time back, people would get themselves in a chicken-and-egg situation where DNS with DNSSEC wouldn't start without NTP and NTP wouldn't sync without DNS being able to resolve the NTP server hostname.

 

[Deleted member 62525]

If you can only confirm that indeed restart_diskmon is triggered upon full NTP sync than starting any NTP related/dependent services would make sense we do using this approach. Maybe @RMerlin can confirm this. It would be preferable to use this type of event based solution vs. waiting in code for NTP.

I wish Asus firmware would provide some sort of event when NTP is sync in order for all NTP dependent service to start at this point.

 

[dave14305]

If you can only confirm that indeed restart_diskmon is triggered upon full NTP sync than starting any NTP related/dependent services would make sense we do using this approach. Maybe @RMerlin can confirm this. It would be preferable to use this type of event based solution vs. waiting in code for NTP.

It is called unconditionally (assuming diskmon is included in the firmware build) from within the ntpd_synced command that ntp calls upon a sync, so I think it's reliable.

asuswrt-merlin.ng/release/src/router/rc/ntpd.c at master · RMerl/asuswrt-merlin.ng

Third party firmware for Asus routers (newer codebase) - RMerl/asuswrt-merlin.ng

github.com

I wish Asus firmware would provide some sort of event when NTP is sync in order for all NTP dependent service to start at this point.

I lobbied for something similar in Merlin's firmware a while back, but eventually settled on diskmon after being suggested by @Dabombber.

 

[Deleted member 62525]

I am going to test it today and start my Unbound from services-event and still leaving S61unbound.
Current implementation and starting Unbound purely with S61unbound is unreliable. Many times it does not start or waits for a long time. I examined the system-event arg: restart discmon and it it triggered at the very end of the boot process. It makes total sense to me to start Unbound from there.

I am leaving S61unbound untouched since it is used by unbound_manager for restarting. But at the very least the boot process will be more reliable in my opinion.

 

[dave14305]

I am going to test it today and start my Unbound from services-event and still leaving S61unbound.
Current implementation and starting Unbound purely with S61unbound is unreliable. Many times it does not start or waits for a long time. I examined the system-event arg: restart discmon and it it triggered at the very end of the boot process. It makes total sense to me to start Unbound from there.

I am leaving S61unbound untouched since it is used by unbound_manager for restarting. But at the very least the boot process will be more reliable in my opinion.

Use care not to lock up the rc service queue. I've found that the best practice is to use the service-event-end script and put your command in the background with & so that the restart_diskmon event will finish and allow subsequent events to fire without waiting for something potentially slow like Unbound to start. You'll know if it's happening if you see "waitting..." [sic] and "skipping event" messages in the syslog.

 

[MvW]

NTP "aggressive" syncing on boot while I thrash out a proper fix
Create /jffs/scripts/ntpbootwatchdog.sh

#!/bin/sh

logger -st "System" "NTP boot watchdog started..."

ntptimer=0
while [ "$(nvram get ntp_ready)" = 0 ] && [ "$ntptimer" -lt 300 ]; do
    killall ntp
    killall ntpd
    service restart_ntpd
    ntptimer=$((ntptimer+30))
    sleep 30
done

if [ "$ntptimer" -ge 300 ]; then
    logger -st "System" "NTP failed to sync after 5 minutes - please check immediately!"
    exit 1
fi

Create or edit /jffs/scripts/init-start

#!/bin/sh
sh /jffs/scripts/ntpbootwatchdog.sh & #make sure NTP starts

Make sure both are executable

chmod +x /jffs/scripts/ntpbootwatchdog.sh
chmod +x /jffs/scripts/init-start

@Jack Yaz: I re-created the ntpbootwatchdog.sh script and init-start (through ssh, in nano to make sure they're in the correct format and not using a windows LF), but I actually wonder whether it's (correctly) running. Both files are made executable with chmod u+x but the line

System: NTP boot watchdog started...

never shows up in my logfiles after a reboot.

I added:

touch /tmp/S00ntpbootwatchdog

to /jffs/scripts/ntpbootwatchdog.sh and /tmp/S00ntpbootwatchdog is actually created, so the script is called during boot, but the line logger should print to syslog doesn't show up. So now I'm not sure whether the script is running from start to end and my knowledge of shell scripting has become limited.

I tried with both syslogd and with syslog-ng/Scribe/ScribeUI, but to no avail.

When running sh /jffs/scripts/ntpbootwatchdog.sh manually it does print that specific line to the screen instead of the log. Is it supposed to do that? Shouldn't it print to syslog even when ran in a terminal?

The good news: I've rebooted 5 times since I re-created the script and init-start and I haven't had NTP issues anymore. It could be (but I can't be sure, because of short-term memory issues) that I've previously removed ntpbootwatchdog.sh and the changes to init-start because I never saw the logger line show up in syslog and therefore assumed your script wasn't working ...

 

[Jack Yaz]

@Jack Yaz: I re-created the ntpbootwatchdog.sh script and init-start (through ssh, in nano to make sure they're in the correct format and not using a windows LF), but I actually wonder whether it's (correctly) running. Both files are made executable with chmod u+x but the line

System: NTP boot watchdog started...

never shows up in my logfiles after a reboot.

I added:

touch /tmp/S00ntpbootwatchdog

to /jffs/scripts/ntpbootwatchdog.sh and /tmp/S00ntpbootwatchdog is actually created, so the script is called during boot, but the line logger should print to syslog doesn't show up. So now I'm not sure whether the script is running from start to end and my knowledge of shell scripting has become limited.

I tried with both syslogd and with syslog-ng/Scribe/ScribeUI, but to no avail.

When running sh /jffs/scripts/ntpbootwatchdog.sh manually it does print that specific line to the screen instead of the log. Is it supposed to do that? Shouldn't it print to syslog even when ran in a terminal?

The good news: I've rebooted 5 times since I re-created the script and init-start and I haven't had NTP issues anymore. It could be (but I can't be sure, because of short-term memory issues) that I've previously removed ntpbootwatchdog.sh and the changes to init-start because I never saw the logger line show up in syslog and therefore assumed your script wasn't working ...

Maybe init-start is too early for logger/i need to specify the full path. I'll do some testing

 

[MvW]

I just replaced the contents of the script with the new version, and rebooted, but two things don't add up:

Neither

ntbootbwatchdog: NTP is synced, exiting

nor

ntpbootwatchdog: NTP boot watchdog started..."

show up in syslog.

Furthermore, NTP gets synced

Mar 11 20:45:11 ntpd: Initial clock set

but ntpbootwatchdog reports:

Mar 11 20:45:38 RT-AC86U ntpbootwatchdog: Still waiting for NTP to sync...

NTP synced fine as the timestamp shows, but somehow ntpbootwatchdog is still waiting for it.

The last warning from your script:

NTP failed to sync after 10 minutes - please check immediately!

doesn't show up, so somewhere in between the script apparently decides ntp has synced in the meantime, I assume?

 

[Jack Yaz]

I just replaced the contents of the script with the new version, and rebooted, but two things don't add up:

Neither

ntbootbwatchdog: NTP is synced, exiting

nor

ntpbootwatchdog: NTP boot watchdog started..."

show up in syslog.

Furthermore, NTP gets synced

Mar 11 20:45:11 ntpd: Initial clock set

but ntpbootwatchdog reports:

Mar 11 20:45:38 RT-AC86U ntpbootwatchdog: Still waiting for NTP to sync...

NTP synced fine as the timestamp shows, but somehow ntpbootwatchdog is still waiting for it.

The last warning from your script:

NTP failed to sync after 10 minutes - please check immediately!

doesn't show up, so somewhere in between the script apparently decides ntp has synced in the meantime, I assume?

the loop should exit when

nvram get ntp_ready

is set to 1. can you check that nvram variable is actually being set?

logger is a firmware function. are you using scribe, if so, have you set an "ntp" log filter? script prints to syslog for me

 

[MvW]

the loop should exit when

nvram get ntp_ready

is set to 1. can you check that nvram variable is actually being set?

It is:

marco@RT-AC86U:/tmp/home/root# nvram get ntp_ready
1

logger is a firmware function. are you using scribe, if so, have you set an "ntp" log filter? script prints to syslog for me

I have not set an 'ntp' log filter. The lines

Mar 11 20:45:11 ntpd: Initial clock set
Mar 11 20:45:38 RT-AC86U ntpbootwatchdog: Still waiting for NTP to sync...

are directly copied from the (non-filtered) syslog-ng System Messages (where I see ntpd being started, syncing etc.). These would otherwise also be filtered out, I presume? I only installed syslog-ng, Scribe and Scribe UI to see if that would make a difference for the 'original' version of your script. I can just as easily uninstall them if you want me to?

 

[Jack Yaz]

It is:

marco@RT-AC86U:/tmp/home/root# nvram get ntp_ready
1

I have not set an 'ntp' log filter. The lines

       Mar 11 20:45:11 ntpd: Initial clock set
       Mar 11 20:45:38 RT-AC86U ntpbootwatchdog: Still waiting for NTP to sync...

arre directly copied from the (non-filtered) syslog-ng System Messages (where I see ntpd being started, syncing etc.). These would otherwise also be filtered out, I presume? I only installed syslog-ng, Scribe and Scribe UI to see if that would make a difference for the 'original' version of your script. I can just as easily uninstall them if you want me to?

no, i missed a line in my update to print a "success" line. my bad, sorry.

script updated in original post

 

[MvW]

The script works, NTP hasn't failed me a single time in 10 reboots. Thanks so much!

I have one last cosmetic change request:

Either change the text to

'ntpbootwatchdock: Verifying whether NTP has succesfully synced..."

(or something alike) or check the nvram variable ntp_ready prior to printing the message

ntpbootwatchdog: Still waiting for NTP to sync...

Because now, even though NTP has synced already seconds before

Mar 11 22:07:24 ntpd: Initial clock set

you get

Mar 11 22:07:48 ntpbootwatchdog: Still waiting for NTP to sync...
Mar 11 22:07:48 ntpbootwatchdog: NTP has synced!

Thanks for your effort, much appreciated.

Cheers,
Marco

 

[Jack Yaz]

The script works, NTP hasn't failed me a single time in 10 reboots. Thanks so much!

I have one last cosmetic change request:

Either change the text to

'ntpbootwatchdock: Verifying whether NTP has succesfully synced..."

(or something alike) or check the nvram variable ntp_ready prior to printing the message

ntpbootwatchdog: Still waiting for NTP to sync...

Because now, even though NTP has synced already seconds before

Mar 11 22:07:24 ntpd: Initial clock set

you get

ntpbootwatchdog: Still waiting for NTP to sync...
Mar 11 22:07:48 ntpbootwatchdog: NTP has synced!

Thanks for your effort, much appreciated.

Cheers,
Marco

done, thanks for putting up with my gaffes! i've made a dedicated post here for the script, for anyone else who'd like to use it: https://www.snbforums.com/threads/ntp-boot-watchdog-script.71069/

 

[JGrana]

Hmmm, our old nemesis is back...

Runnin 386.2 Beta 2. Addons installed are:
Unbound, scMerlin, spdMerlin, ntpMerlin, connmon and scribe/uiScribe.

After reboot last night, all addon tabs showed fine.
I added the alpha of Cake-qos 2.0 last night as well.
Did a reboot - addon tabs fine.

I needed to do a reboot this morning. Post boot, Unbound and 2 instances of scMerlin show up. It appears just the web page/tab is missing (i.e. chrony is running fine)

Here are some logs:

Mar 21 06:36:40 RT-AX88U-26A8 S77chronyd: Waiting for NTP to sync before starting...
Mar 21 06:36:48 RT-AX88U-26A8 ntpMerlin: /tmp/mnt/Entware contains Entware, starting ntpMerlin
Mar 21 06:36:55 RT-AX88U-26A8 ntpMerlin: Mounted ntpMerlin WebUI page as user6.asp

I see spdMerlin in cru and speedtests are being run/logged and spdMerlin mounting it's tab:
Mar 21 06:36:48 RT-AX88U-26A8 spdMerlin: /tmp/mnt/Entware contains Entware, starting spdMerlin
Mar 21 06:36:57 RT-AX88U-26A8 spdMerlin: Mounting spdMerlin WebUI page as user7.asp


I do see 2 WebUI mounts for scMerlin:
Mar 21 06:36:48 RT-AX88U-26A8 scMerlin: Mounting scMerlin WebUI page as user3.asp
Mar 21 06:36:48 RT-AX88U-26A8 scMerlin: Mounting scMerlin WebUI page as user3.asp

I also see a mount of connmon WebUI (but no tab displayed):
Mar 21 06:36:48 RT-AX88U-26A8 connmon: /tmp/mnt/Entware contains Entware, starting connmon
Mar 21 06:36:54 RT-AX88U-26A8 connmon: Mounted connmon WebUI page as user5.asp

I have not done a reboot (yet). If there is any other debugging, let me know.