Seeking Feedback & Contributions: Merlin Auto Update Solutions

[Yota]

Is there harm in keeping nohup? The reason I ask is if I call/run the script from SSH shell like the below and select option 2:

View attachment 53492

it won't work unless I have nohup right now, it does the download and then quits the sessons and fails, however if I have nohup and select 2, it does the download and completes the flash successfully.
it makes me wonder about any future implementations if we remove it.

Edit: i understand when the cron calls the run_now function directly it may not be needed, but when about when a user first configures? etc?

If the script relies on interaction over SSH (for example there is an API that allows you to log into SSH via a desktop script and then trigger checks and automatic updates/or as shown in your screenshot, it has a user interface) then retained nohup will be necessary.

But if it's going to be a purely non-interactive script, where updates will only be triggered by custom scripts and cron, then nohup may be redundant. and, I'm not sure when nohup became available again because it never worked in my memory (2017-2023).

nohup does nor work, am I doing something wrong?

So I am trying to run a script all the time, it is a simple thing just checking if a mobile is connected to wifi. It works as long as I am online (with putty). Logout and it stops direct. So I start with: nohup /bin/sh /jffs/scripts/CheckIfHome2 & Am I doing something wrong here or is nohup...

www.snbforums.com

In addition, I suggest splitting the user interaction part and the core functional part of the script into different scripts. However, it is worth pointing out that if one script calls another script in an SSH session, the parent process is still dropbear. Even if & is used, the script running in the child process will be killed when the SSH session is ended.

EDIT:

Edit: i understand when the cron calls the run_now function directly it may not be needed, but when about when a user first configures? etc?

That would be the first case, in which case nohup or cron jobs would be required. But I'm not sure how many users will log into SSH to trigger the update? Why don't they upload the firmware directly through the GUI? I think most people who use auto-update are trying to make it easier on themselves.

 

[ExtremeFiretop]

If the script relies on interaction over SSH (for example there is an API that allows you to log into SSH via a desktop script and then trigger checks and automatic updates/Or as shown in your screenshot, it has a user interface) then retained nohup will be necessary.

But if it's going to be a purely non-interactive script, where updates will only be triggered by custom scripts and cron, then nohup may be redundant. and, I'm not sure when nohup became available again because it never worked in my memory (2017-2020).

nohup does nor work, am I doing something wrong?

So I am trying to run a script all the time, it is a simple thing just checking if a mobile is connected to wifi. It works as long as I am online (with putty). Logout and it stops direct. So I start with: nohup /bin/sh /jffs/scripts/CheckIfHome2 & Am I doing something wrong here or is nohup...

www.snbforums.com

In addition, I suggest splitting the user interaction part and the core functional part of the script into different scripts. However, it is worth pointing out that if one script calls another script in an SSH session, the parent process is still dropbear. Even if & is used, the script running in the child process will be killed when the SSH session is ended.

Understood, fair enough I kinda pictured it being user interactive (for ease of use) to setup the cron schedule and credentials instead of having to modify text files manually, etc.
So if it had an interactive view, I may as well add access to check if their are "updates" or if you are "up to date" with the options 2 "run now" function.

However the cron it sets up simply directly calls the run_now function and skips the whole user UI stuff.
Not sure if this is the best implementation, while I have scripting experience in general (mostly for work), this is my first time doing it on a router lol.

Learning experience for all it seems!

 

[Yota]

Understood, fair enough I kinda pictured it being user interactive (for ease of use) to setup the cron schedule and credentials instead of having to modify text files manually, etc.
So if it had an interactive view, I may as well add access to check if their are "updates" or if you are "up to date" with the options 2 "run now" function.

Good idea. In that case, we do need nohup, but nohup will still need to be tested extensively because in my memory it never worked, it only happened in this thread. Maybe busybox has been updated recently?

Wonder what @RMerlin @thelonelycoder @sfx2000 and @ColinTaylor know about nohup working again?

However the cron it sets up simply directly calls the run_now function and skips the whole user UI stuff.

This is a good trick, I've used it in the past because nohup didn't work, and if the function you plan to flash firmware uses a cron call, then nohup will no longer be needed.

 

[ExtremeFiretop]

I actually got some Github pull requests? Wow.

Thanks everyone!

 

[ColinTaylor]

Wonder what @RMerlin @thelonelycoder @sfx2000 and @ColinTaylor know about nohup working again?

I don't believe the behaviour of the shell has changed. See this post but also this post.

 

[Yota]

I don't believe the behaviour of the shell has changed. See this post but also this post.

Thanks, very good explanation!

IIRC from my own experiments if you nohup / & a binary executable (like tail) it will keep running when you log out. The problem comes when you try to do the same with a script (like .sh or .awk). Because these are invoked from the busybox shell they are killed when the shell (terminal) session terminates. From what I remember of looking at the source code there wasn't an obvious was around this. -- Well there is, sort of. Don't invoke the commands/scripts from a terminal session. Invoke them from a cron job instead.

You are right because they are using nohup curl 'http://www.asusrouter.com/upgrade.cgi' \ in the script and that's why it works. Shell and scripts also don't survive, but curl as a binary does.

But this does remind us of the potential threat, although curl can run in the background, the rest of the script will be closed, so the script must have nothing to do next. @ExtremeFiretop

 

[ExtremeFiretop]

Thanks, very good explanation!



You are right because they are using nohup curl 'http://www.asusrouter.com/upgrade.cgi' \ in the script and that's why it works. Shell and scripts also don't survive, but curl as a binary does.

But this does remind us of the potential threat, although curl can run in the background, the rest of the script will be closed, so the script must have nothing to do next. @ExtremeFiretop

Correct I see the script only running to fetch the latest update, after that the curl is run in the background and the script can close

 

[Yota]

But I'm not sure how many users will log into SSH to trigger the update?

Answering my own stupid question, I've seen way too many feature requests on this forum about how to update routers using SSH (they may only have SSH access from the remote). Well, this script's SSH user interface will help with this request.

Correct I see the script only running to fetch the latest update, after that the curl is run in the background and the script can close

If curl is the last step of the update, then there's nothing to worry about.

 

[ExtremeFiretop]

If curl is the last step of the update, then there's nothing to worry about.

It is the last step currently and I'll make sure it stays that way for this reason.
It was still valuable information has been noted, infact I may even add a note directly to the code as a reminder of this fact.

Edit: DONE!

 

[Yota]

It is the last step currently and I'll make sure it stays that way for this reason.
It was still valuable information has been noted, infact I may even add a note directly to the code as a reminder of this fact.

Sounds great, don’t forget about the cute RGB lights!

 

[ExtremeFiretop]

Sounds great, don’t forget about the cute RGB lights!

Oh boy, will have to look into that lol. Do all routers handle RGB lights the same?
I may need to scrape some existing functions for guidance on that but I actually kinda like the idea lol.

Plus I still think there is value in backing up on a USB before the flash.
While the flash does unload the USB. (we can't hold the update there) nothing stopping us from backing up the .cfg first, dumping it on a USB, and then moving forwards with the update process, I would think?

 

[ExtremeFiretop]

I also have a note from RMerlin which is finding a way to handle firmware files that contain two separate images for separate hardware revisions. Like the RT-AX58U...
I still think user selection in the UI similar to how I handle the rog builds. If theres multiple .w's detected then prompt for which to user and record that in the settings file.

 

[dave14305]

It’s a good reference on how Merlin builds/zips the firmware.

asuswrt-merlin.ng/tools/build-all at master · RMerl/asuswrt-merlin.ng

Third party firmware for Asus routers (newer codebase) - RMerl/asuswrt-merlin.ng

github.com

 

[Yota]

Do all routers handle RGB lights the same?

Most Merlin supported models do not have RGB.

I may need to scrape some existing functions for guidance on that but I actually kinda like the idea lol.

Take a look at this wiki.

While the flash does unload the USB. (we can't hold the update there) nothing stopping us from backing up the .cfg first, dumping it on a USB, and then moving forwards with the update process, I would think?

Yes, the saving value is that if after a reboot the script still has access to the USB drive, if a quick check at the moment reveals a problem with the firmware, like the internet is not accessible, we will be able to downgrade to the older version via the legacy files saved in the USB drive (Of course it may need to be copied to RAM in advance), and then even restore nvram and jffs after the downgrade is complete to avoid post-downgrade incompatibilities.

Also if we screw up everything, for example the router won't boot and you need to flash the firmware via recovery mode, the configuration and jffs will not be lost. (Of course, messing up everything caused by us is almost impossible to happen in the way curl simulates)

 

[Yota]

I also have a note from RMerlin which is finding a way to handle firmware files that contain two separate images for separate hardware revisions. Like the RT-AX58U...
I still think user selection in the UI similar to how I handle the rog builds. If theres multiple .w's detected then prompt for which to user and record that in the settings file.

I just downloaded the file RT-AX58U_3004_388.4_0.zip for the RT-AX58U firmware and I don't see two separate images in one zip file.

Actually I think you are talking about the RT-AC68U, which merges 2 firmwares (40MB each) into one firmware (80MB) and then after uploading to the router, the update process will automatically split the file and select the correct firmware to flash it. We don't have to do anything, the update process will do it, unless you plan to bypass the GUI and update it on the backend.

 

[sfx2000]

All fun at the end of the day...

Some thoughts - rather than use all this shell scripting with curl (the app), I would do a small binary that would call libcurl...

REST/JSON over TLS makes the most sense, as it has to scale and this solution is already there - just pour in the request/responses...

REST API would be best - there the client side can send over a HW type/subtype, and the server side returns the current version.

We then compare the installed release to what was in the JSON response, and if installed != current, one can download current (putting the SHA256 in the JSON response).

Download into /tmp - do the hash check, and if successful, you can write the FW image into the appropriate MTD partitions...

Since we do this over TLS, the REST calls are secure end to end, and this can be a private/long term cert as this is not really a web broswer (public CA's usually only will issue one year certs now) - why a private cert vs. public - after a factory reset, one needs to get back to a known point in time - a short term cert may be expired, and you'll end up with a brick.

Anyways - common server cert, and client side certs are based on something unique (s/n or other UUID)

This makes things client driven, and this can be on a scheduled basis, and no server side push is required, so this removes the GDPR concerns.

For the actual FW upgrade - I would probably check out sysupgrade from openwrt, as it is very flexible, and the code already exists in a way that is GPL friendly...

Challenge is what if the FW update requires a hardware reset - that's a flag that could be set, if hw_reset is false, then upgrade, if hw_reset is true, don't upgrade and notifiy the upper layer that a FW update is available and needs additional actions

 

[ExtremeFiretop]

I just downloaded the file RT-AX58U_3004_388.4_0.zip for the RT-AX58U firmware and I don't see two separate images in one zip file.

Actually I think you are talking about the RT-AC68U, which merges 2 firmwares (40MB each) into one firmware (80MB) and then after uploading to the router, the update process will automatically split the file and select the correct firmware to flash it. We don't have to do anything, the update process will do it, unless you plan to bypass the GUI and update it on the backend.

If this is really true then that's one less thing on the list!
I took the RMerlins message is it meaning the .zip had 2 firmware images, and the user had to select which one for their revision.

if it's really one file and the GUI picks the correct version, then we are already ahead of the game!

The list of things I'd like to consider left is:

Prerequisites:

-Ensure the USB drive is connected to the router for storage.

Backup and Storage:

-Save the following to the USB drive:
-Configuration files.
-JFFS backup.
-Newer firmware version.
Note: New routers use UBIFS instead of JFFS2.
Be aware that JFFS partitions may not work post-upgrades in some cases.

Memory Management:

-Address the paradox of RAM usage:
-USB cache consumes significant RAM when writing files.
-Firmware updates require RAM to cache the entire firmware.
-Check RAM usage.
-If free RAM is less than the firmware file size, reboot the router.

Firmware Management:

-Implement a waiting period for firmware updates:
-Wait for a set duration after a new firmware release.
-If no newer firmware emerges, proceed with the update.
-If newer firmware appears, reset and reevaluate the waiting period.

System Notifications:

-Possibly modify the hardcoded notification in the GUI's upper right corner.
-Possibly Trigger a firmware update notification using a script.

LED Control:
-Possibly Setup blinking LEDs as a visual indicator before starting the firmware update.

I think I may need to open a beer tonight for the progress made today, I'll poke at this list more tomorrow, cheers to everyone in this thread!

 

[sfx2000]

Some thoughts - rather than use all this shell scripting with curl (the app), I would do a small binary that would call libcurl...

Anyways - I can't go deeper into what is a strawman design, as I do this as my jobby-job and that pays the bills...

There should be enough to follow in my post - and it's fairly lightweight by doing the task outside of the normal means of doing things since we're client-server...

Remember - everything inside runs as admin anyways at the lowest level, and firmware upgrades need to have root access in any case...

 

[ExtremeFiretop]

All fun at the end of the day...

Some thoughts - rather than use all this shell scripting with curl (the app), I would do a small binary that would call libcurl...

REST/JSON over TLS makes the most sense, as it has to scale and this solution is already there - just pour in the request/responses...

REST API would be best - there the client side can send over a HW type/subtype, and the server side returns the current version.

We then compare the installed release to what was in the JSON response, and if installed != current, one can download current (putting the SHA256 in the JSON response).

Download into /tmp - do the hash check, and if successful, you can write the FW image into the appropriate MTD partitions...

Since we do this over TLS, the REST calls are secure end to end, and this can be a private/long term cert as this is not really a web broswer (public CA's usually only will issue one year certs now) - why a private cert vs. public - after a factory reset, one needs to get back to a known point in time - a short term cert may be expired, and you'll end up with a brick.

Anyways - common server cert, and client side certs are based on something unique (s/n or other UUID)

This makes things client driven, and this can be on a scheduled basis, and no server side push is required, so this removes the GDPR concerns.

For the actual FW upgrade - I would probably check out sysupgrade from openwrt, as it is very flexible, and the code already exists in a way that is GPL friendly...

Challenge is what if the FW update requires a hardware reset - that's a flag that could be set, if hw_reset is false, then upgrade, if hw_reset is true, don't upgrade and notifiy the upper layer that a FW update is available and needs additional actions

I had quickly tried to poke at this with Powershells RestMethod but was unsuccessful and just wrote it off as a limitation of the cmdlets.

Seeking Feedback & Contributions: Merlin Auto Update Solutions

Lots and lots of valuable take away here. Thank you very much!!! Especially the SourceForge stuff and the idea on the backup to the USB. You've given both valuable points and ideas to alternative solutions and I like that, it's really what the discussion was aiming to achieve. @RMerlin as well...

www.snbforums.com

Now, coming back to the Powershell script, even though Invoke-RestMethod is a way to interact with RESTful APIs, I was not sending and receiving JSON-formatted data over HTTPS.
I was getting 'ByteArrayContent' I thought this may be due to passing a very large array of bytes to the constructor.

The script I had written is interacting with a specific URI on the router (http://www.asusrouter.com/upgrade.cgi) which may or may not be designed to handle RESTful requests?
I was sending a multi-part form data POST request to the router's upgrade endpoint, which is a different kind of HTTP interaction compared to a typical REST/JSON over TLS interaction.

If the router has a RESTful API for firmware upgrades. We could maybe restructure this script to use RESTful API?

Right now this is a working solution and I am above happy, but always open to more contributions

 

[sfx2000]

I took the RMerlins message is it meaning the .zip had 2 firmware images, and the user had to select which one for their revision.

Here's @RMerlin actual wording...

and also (for some models, like the RT-AC68U) checking for the version to prevent flashing firmwares older than a minimal version. That was done to prevent RT-AC68U V3 owners from flashing a very old version that didn`t support the newer hardware revisions. They've done it for other models, but I don't know the up-to-date list of these checks as last time I decompiled that code was years ago.

Anyways - that can be addressed in my proposal as hwtype/subtype - so one only downloads the image needed...

How I do things, and this is interesting perhaps - I have delta images - so we patch in place, and this is faciliated by the 2-bank solution I run - so worst case, if FW upgrade fails, we timeout and the watchdog (taak has expired, kick the dog, yipe yipe yipe) and we boot into the last good firmware.

What also makes this possible is that I have a kernel image, a rootfs image that contains all the core stuff, and an applications image which is specific - /data is user data for configs, etc...