Seeking Feedback & Contributions: Merlin Auto Update Solutions

[Yota]

Lots and lots of valuable take away here. Thank you very much!!! Especially the SourceForge stuff and the idea on the backup to the USB.
You've given both valuable points and ideas to alternative solutions and I like that, it's really what the discussion was aiming to achieve. @RMerlin as well, you've just stayed on track the entire time and never did anything but just provide straight up information to the best of your knowledge. It's much appreciated.

Like I mentioned I had actually though about the browser emulation (login and upload) before posting about this really when starting the project, but because I had already worked so far down this path, hnd-write was the easier choice to include in this all-in-one type solution.
However discussing openly the restrains around hnd-writre and the hesitation on actually relying on it (which I completely understand, it is unknowable what ASUS may change, just because it works today and in the last 8 months doesn't mean it will work forever).

These other alternatives of triggering the systems natural update process once you dump the file in the right location feels like the natural next step, however requires a bit more digging, I won't lie, I didn't click these links the first time they were posted and I'm a big enough man to admit when I was wrong.

For the desktop app, I would probably work as you said on splitting it from the rest of the script I have now, and really focus on the browser emulation part, find a way to prompt for credentials, cache those somewhere locally and login and do the update, if I can do that on the desktop (where I have extra tools and knowledge) then naturally I can move over lots of the logic to Shell and have the router try the steps itself of logging into it's own UI and doing the update that way.

I'll probably be going down a rabbit hole for a while, like Merlin said some of this stuff is fairly complex and closed source, but the whole point was the challenge of finding a reliable way to accomplish this update/flashing step, however possible.

Windows/macOS/Linux all support curl, and curl is a powerful command line tool that can simulate almost all behaviors of the browser through get/post. So your desktop app will most likely achieve cross-platform compatibility.

It's important to note however that Asuswrt's GUI has many protections against access by automated scripts, (even including browser header referrers) but don't worry curl can still simulate all this, you just need to spend some time understanding how to do it.

The browser console and curl's man page will be the most helpful places.


(In fact, this reminds me of a script I had planned to launch on the SNB forums a few years ago to automatically update the SNB forums signature after every router/add-on update, but after discussing it with Tim, it may cause DDOS to the forums, so this script has never been made public. that script will automatically log in to the SNB forums on the router, save the cookies, and use the post to update the signature.)

 

[Viktor Jaep]

The second is that scripts cannot be run after factory reset. I can't resolve this paradox. So the project was shelved.

Kinda unconventional thinking here, but would there be a way to alter the default nvram values that get applied to a new firmware install so that it would have JFFS enabled after it boots back up?

 

[Yota]

Kinda unconventional thinking here, but would there be a way to alter the default nvram values that get applied to a new firmware install so that it would have JFFS enabled after it boots back up?

There is a way that may be able to break this paradox, but requires a lot of research, and that is to create a clean nvram backup, including the default values for all settings, but the only thing changed is the setting of the jffs script.

The router then doesn't actually do a factory reset, but deletes the values that are not in the clean nvram and then rewrites them to the values of the clean nvram.

But I can never know what a "clean nvram" should look like, because the nvram after factory reset is different after every firmware update, they always add new nvram values, or change something, not to mention the nvram is different for every model, so the cost of maintaining a project like this is huge.

So I finally gave up on this solution as well.



Edit:
Your ideas are really inspiring and it made me think of another solution that I hadn't thought of before.

First of all, you need to know what the router does specifically to restore factory settings. Basically what the router does is to clear the nvram and then replace it with the "default nvram" stored in the firmware. However, the default nvram does not contain all the items and values, some content is automatically generated after boot, once the boot is complete, you get what I called before "clean nvram", (of course there is another step in the process including emptying jffs, but that is the simple thing, easily implemented via script).

New inspiration is, can we make a clean update after every firmware update with new default nvram + user defined changes?

this is possible.

I previously translated an article about where the default nvram value is saved, and how to decrypt it. You can see it here.


But it's best if we discuss it in another thread, because it's seriously off topic. Sorry to the OP.

 

[Viktor Jaep]

Edit:
Your ideas are really inspiring and it made me think of another solution that I hadn't thought of before.

I'm simply here to inspire and question the norms! LOL

 

[Yota]

Log analysis to determine the necessity of a factory reset within a specified date range...

Talking about log analysis, I have an idea. System logs have severity levels. you only need to regularly count the number of logs of each level to judge the operating stability of the router.

But you can also adopt what I mentioned before, automatic factory reset and automatic configuration. The above has found a way to break the paradox that prevents its development. Well, that will be a project as big as update firmware.

requiring a notification system.

As for system notifications, the notification in the upper right corner of the GUI is probably hardcoded (perhaps in the www directory), but you may be able to trigger the firmware update notification via a script, and then use the mount command to replace the notification content from firmware update to what you wish to display content.

Install and Uninstall features for AMTM... needing further research and assistance.

I don't think this is a priority, at least the script needs to be there for years, after it needs to be sure there are no problems and can be used safely by anyone without reading the guide and thread.

 

[Morris]

It does work today for an RMerlin-powered router. At no point did I not take into account valid concerns... Everytime a cocern was brought up it was met with:



and an explanation of why I think we can workaround/resolve that concern, etc... No one has said anything in regard to my counter-points once.

It's depressing when people simply want to fixate on how it's not something they want, instead of fixating on the questions at hand and in OP, like I have a script that works, help me address concerns to make it better.
Literally not a single person in 3 pages has offered any scripting advise to my questions in the OP.

Instead I get lots of "Don't do it, I don't want it" as if that was the question or the reason for this post.



I honestly have no plans on anyone using it but me and my friend, I think that's what people fail to understand.
But it's bad practice to ask for coding or scripting advise without showing your work, which I did.. Just ask stackoverflow forum guys what happens if you ask for help without showing your work.

And github is how I shared this project with my friend with the AC86U, everytime we make a discover or update, so honestly yes it's tiring to hear this over and over again:







On a thread specifically asking for "Contributions" in the title.
it's lots of defending a project I was interested in from a technical aspect since it DOES WORK. Lol. I feel like not a single person has even reviewed the script I shared or given points of improvement (scripting wise)

This is one of the only groups of people in my life, that are techy tinkers, that has seen evidence of something working for solid months and goes "Nah instead of making it better, lets tell this guy I don't need this or want this over and over again."
I'm tired of defending why I'm working on this, so you know what? I'm not anymore. Project is dropped, the funs been taken out of it and my friend can use my ordinal version without the GUI and be happy.

Thanks for the people that did offer feedback though.
The take aways were valuable, but clearly people without anything to contribute tech wise can't keep their personal opinions out.

The quote you attributed to me was not me. I stated "Why write all this custom code with all the issues that Merlin has mentioned as well as other unknowns? The router already knows how to update it's self so screen scrape and let the router do the work."

I had suggested auto update to Merlin a long time ago.

 

[Yota]

Another thing worth noting is that, unlike CMD, PowerShell will store all user input in the %AppData%\Microsoft\Windows\PowerShell\PSReadLine\ folder by default. So, when your script asks for a password, well, you're actually giving your password to any program installed on Windows, if your Windows has Microsoft's user experience improvements enabled (enabled by default), then there's a good chance that Microsoft also has access to your router password.


Honestly, this is riskier than saving password on the router because you never know what they have installed on their Windows computer. So two solutions, use a traditional cmd script (which is more difficult, but ssh and curl are still there), or run the script directly on the router.

 

[Yota]

Some suggestions on using curl to simulate user behavior (sorry my schedule is full until January next year, so I don't really have time to code with you, but here's some code logic)

Prerequisites:
1) All scripts are stored on the router because I honestly don't like having to rely on another computer to do this.
2) The script needs to know the router's login password. Due to Asus's security measures, the router's password is not stored in plain text on the router. During the initial configuration, the user needs to enter the router's username and password, which are then saved by the script, and improve the security of password storage as much as possible (you can consider encrypting it. In fact this would be the biggest risk with this script).
3) The USB drive must be plugged into the router to provide storage space. (The paradox here is that the USB cache takes up a lot of RAM when writing files, however firmware updates will also use RAM to cache the entire firmware, so there must be free RAM, below I will try to provide a solution to break the paradox).

Update script logic (excluding checking for updates):
1) Back up nvram and jffs to the USB drive.
2) Download the current version of firmware and the new version of firmware to the USB drive.
3) Check the RAM usage and if the free RAM is less than the firmware file size, reboot the router (do this at the right time of course).
4) Unzip and verify firmware SHA256 (older models do not have the sha256sum command, you can use OpenSSL to do this).
5) Use curl to log in to the router and obtain the cookie after login.
6) Use curl to upload firmware through this cookie to simulate user behavior.
Reboot the router....
7) After restarting, check the router function, Internet access, WiFi, etc.
8) If everything is normal, a log will be sent to inform user that "you have successfully upgraded automatically. Thank you for using the MAGICAL automatic upgrade function."
9) Log out of the router (invalidating cookie) otherwise the user may not be able to log into the router GUI.

If things get weird:
8) After the script is checked and it is found that there is a major problem with the firmware, the rollback will begin. First unzip and verify the SHA256 of the old firmware, (the firmware was previously downloaded and saved in the USB drive).
9) Repeat steps 5-8.
10) If all else fails, send a log saying "Congratulations, the SNB forum trolls are right, but it's not our fault ".

There you can also update the AiMesh model in step 6, but I think these features should be introduced after version 1.5 or 2.0, not earlier. In the early days, more focus should be placed on mature implementations of automatic updates.


Some suggestions about firmware checking for updates. Firmware updates should have a waiting period after the new firmware is launched to avoid any possible problems with the firmware itself. Once there is no new firmware after the waiting period, it will be automatically updated. If there is new firmware, the waiting period will be recalculated for the new firmware. The waiting period can be a week or a few weeks after the firmware is released, depending on the pros and cons based on stability and security.


Good luck to the OP!




EDIT:
There is something very important that I forgot to remind you.

There is a small challenge using curl on the router. First, the router's GUI only allows access from IPs within its subnet (LAN).

When using the router's built-in curl to access the GUI, the GUI detects that the access source is 0.0.0.0. This will of course be rejected by the firewall because it is an illegal access.

The simplest solution is to add 0.0.0.0 as a rule in Access Restrictions (Go to Administration - System, then Enable Access Restrictions. It can also be changed via SSH/Script, but please note its format, which can be found here).

The difficult solution is to create a new virtual interface, like br0:1, with another IP address in the LAN (because the IP address of router br0 cannot reach itself), and then let curl use the specified interface.

I just tested and I can access the router GUI smoothly using curl.

curl -k https://192.168.50.1/Main_Login.asp -O

-k option to ignore https self-signing
--interface br0:1 option to used to select the interface.

 

[dave14305]

I've spent more time that I'd like to admit trying to derive the proper curl commands to login and upload the firmware. It's "easy" to get the rough syntax by using the browser developer tools (F12) when on the login screen and on the firmware upgrade screen (look for login.cgi and upgrade.cgi in the list of network requests). Right click and "Copy as cURL" to get the syntax.

But I have trouble composing the proper multipart form that includes the binary firmware file, since copy as cURL separates the POST data from the command.

Login:

curl 'http://192.168.50.1/login.cgi' -X POST \
-H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0' \
-H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8' \
-H 'Accept-Language: en-US,en;q=0.5' \
-H 'Accept-Encoding: gzip, deflate' \
-H 'Content-Type: application/x-www-form-urlencoded' \
-H 'Origin: http://192.168.50.1' \
-H 'Connection: keep-alive' \
-H 'Referer: http://192.168.50.1/Main_Login.asp' \
-H 'Upgrade-Insecure-Requests: 1' \
--data-raw 'group_id=&action_mode=&action_script=&action_wait=5&current_page=Main_Login.asp&next_page=index.asp&login_authorization=abcdefghijklmnop' \
--cookie-jar /tmp/cookie.txt

login_authorization is the base64 encoded 'username:password' of your router. For example: echo 'admin:mypassword' | openssl base64. cookie-jar tells curl where to store the login token generated by the firmware, to pass to subsequent curl commands.

The upgrade upload command:

curl 'http://192.168.50.1/upgrade.cgi' -X POST \
-H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0' \
-H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8' \
-H 'Accept-Language: en-US,en;q=0.5' -H 'Accept-Encoding: gzip, deflate' \
-H 'Content-Type: multipart/form-data; boundary=---------------------------13252363599581883533344311780' \
-H 'Origin: http://192.168.50.1' \
-H 'DNT: 1' \
-H 'Connection: keep-alive' \
-H 'Referer: http://192.168.50.1/Advanced_FirmwareUpgrade_Content.asp' \
-H 'Upgrade-Insecure-Requests: 1' \
--data-binary $'-----------------------------13252363599581883533344311780\r\nContent-Disposition: form-data; name="current_page"\r\n\r\nAdvanced_FirmwareUpgrade_Content.asp\r\n-----------------------------13252363599581883533344311780\r\nContent-Disposition: form-data; name="next_page"\r\n\r\n\r\n-----------------------------13252363599581883533344311780\r\nContent-Disposition: form-data; name="action_mode"\r\n\r\n\r\n-----------------------------13252363599581883533344311780\r\nContent-Disposition: form-data; name="action_script"\r\n\r\n\r\n-----------------------------13252363599581883533344311780\r\nContent-Disposition: form-data; name="action_wait"\r\n\r\n\r\n-----------------------------13252363599581883533344311780\r\nContent-Disposition: form-data; name="preferred_lang"\r\n\r\nEN\r\n-----------------------------13252363599581883533344311780\r\nContent-Disposition: form-data; name="firmver"\r\n\r\n3.0.0.4\r\n-----------------------------13252363599581883533344311780\r\nContent-Disposition: form-data; name="file"; filename="RT-AC86U_3.0.0.4_386_51915-g43629d2_ubi.w"\r\nContent-Type: application/octet-stream\r\n\r\n-----------------------------13252363599581883533344311780--\r\n' \
--cookie /tmp/cookie.txt

This doesn't work since the binary data from the firmware file doesn't get sent. There are probably a lot of superfluous headers being sent from Firefox, but I didn't want to muck around too much. The use of data-binary feels problematic when the -F option is normally used to upload a file in a multipart form.

Just leaving this here in case it helps jumpstart anyone else's efforts.

 

[ExtremeFiretop]

I've spent more time that I'd like to admit trying to derive the proper curl commands to login and upload the firmware. It's "easy" to get the rough syntax by using the browser developer tools (F12) when on the login screen and on the firmware upgrade screen (look for login.cgi and upgrade.cgi in the list of network requests). Right click and "Copy as cURL" to get the syntax.

But I have trouble composing the proper multipart form that includes the binary firmware file, since copy as cURL separates the POST data from the command.

Login:

curl 'http://192.168.50.1/login.cgi' -X POST \
-H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0' \
-H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8' \
-H 'Accept-Language: en-US,en;q=0.5' \
-H 'Accept-Encoding: gzip, deflate' \
-H 'Content-Type: application/x-www-form-urlencoded' \
-H 'Origin: http://192.168.50.1' \
-H 'Connection: keep-alive' \
-H 'Referer: http://192.168.50.1/Main_Login.asp' \
-H 'Upgrade-Insecure-Requests: 1' \
--data-raw 'group_id=&action_mode=&action_script=&action_wait=5&current_page=Main_Login.asp&next_page=index.asp&login_authorization=abcdefghijklmnop' \
--cookie-jar /tmp/cookie.txt

login_authorization is the base64 encoded 'username:password' of your router. For example: echo 'admin:mypassword' | openssl base64. cookie-jar tells curl where to store the login token generated by the firmware, to pass to subsequent curl commands.

The upgrade upload command:

curl 'http://192.168.50.1/upgrade.cgi' -X POST \
-H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0' \
-H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8' \
-H 'Accept-Language: en-US,en;q=0.5' -H 'Accept-Encoding: gzip, deflate' \
-H 'Content-Type: multipart/form-data; boundary=---------------------------13252363599581883533344311780' \
-H 'Origin: http://192.168.50.1' \
-H 'DNT: 1' \
-H 'Connection: keep-alive' \
-H 'Referer: http://192.168.50.1/Advanced_FirmwareUpgrade_Content.asp' \
-H 'Upgrade-Insecure-Requests: 1' \
--data-binary $'-----------------------------13252363599581883533344311780\r\nContent-Disposition: form-data; name="current_page"\r\n\r\nAdvanced_FirmwareUpgrade_Content.asp\r\n-----------------------------13252363599581883533344311780\r\nContent-Disposition: form-data; name="next_page"\r\n\r\n\r\n-----------------------------13252363599581883533344311780\r\nContent-Disposition: form-data; name="action_mode"\r\n\r\n\r\n-----------------------------13252363599581883533344311780\r\nContent-Disposition: form-data; name="action_script"\r\n\r\n\r\n-----------------------------13252363599581883533344311780\r\nContent-Disposition: form-data; name="action_wait"\r\n\r\n\r\n-----------------------------13252363599581883533344311780\r\nContent-Disposition: form-data; name="preferred_lang"\r\n\r\nEN\r\n-----------------------------13252363599581883533344311780\r\nContent-Disposition: form-data; name="firmver"\r\n\r\n3.0.0.4\r\n-----------------------------13252363599581883533344311780\r\nContent-Disposition: form-data; name="file"; filename="RT-AC86U_3.0.0.4_386_51915-g43629d2_ubi.w"\r\nContent-Type: application/octet-stream\r\n\r\n-----------------------------13252363599581883533344311780--\r\n' \
--cookie /tmp/cookie.txt

This doesn't work since the binary data from the firmware file doesn't get sent. There are probably a lot of superfluous headers being sent from Firefox, but I didn't want to muck around too much. The use of data-binary feels problematic when the -F option is normally used to upload a file in a multipart form.

Just leaving this here in case it helps jumpstart anyone else's efforts.

I am presently neck deep in the same process, is fun.

 

[ExtremeFiretop]

I've spent more time that I'd like to admit trying to derive the proper curl commands to login and upload the firmware. It's "easy" to get the rough syntax by using the browser developer tools (F12) when on the login screen and on the firmware upgrade screen (look for login.cgi and upgrade.cgi in the list of network requests). Right click and "Copy as cURL" to get the syntax.

But I have trouble composing the proper multipart form that includes the binary firmware file, since copy as cURL separates the POST data from the command.

Login:

curl 'http://192.168.50.1/login.cgi' -X POST \
-H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0' \
-H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8' \
-H 'Accept-Language: en-US,en;q=0.5' \
-H 'Accept-Encoding: gzip, deflate' \
-H 'Content-Type: application/x-www-form-urlencoded' \
-H 'Origin: http://192.168.50.1' \
-H 'Connection: keep-alive' \
-H 'Referer: http://192.168.50.1/Main_Login.asp' \
-H 'Upgrade-Insecure-Requests: 1' \
--data-raw 'group_id=&action_mode=&action_script=&action_wait=5&current_page=Main_Login.asp&next_page=index.asp&login_authorization=abcdefghijklmnop' \
--cookie-jar /tmp/cookie.txt

login_authorization is the base64 encoded 'username:password' of your router. For example: echo 'admin:mypassword' | openssl base64. cookie-jar tells curl where to store the login token generated by the firmware, to pass to subsequent curl commands.

The upgrade upload command:

curl 'http://192.168.50.1/upgrade.cgi' -X POST \
-H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0' \
-H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8' \
-H 'Accept-Language: en-US,en;q=0.5' -H 'Accept-Encoding: gzip, deflate' \
-H 'Content-Type: multipart/form-data; boundary=---------------------------13252363599581883533344311780' \
-H 'Origin: http://192.168.50.1' \
-H 'DNT: 1' \
-H 'Connection: keep-alive' \
-H 'Referer: http://192.168.50.1/Advanced_FirmwareUpgrade_Content.asp' \
-H 'Upgrade-Insecure-Requests: 1' \
--data-binary $'-----------------------------13252363599581883533344311780\r\nContent-Disposition: form-data; name="current_page"\r\n\r\nAdvanced_FirmwareUpgrade_Content.asp\r\n-----------------------------13252363599581883533344311780\r\nContent-Disposition: form-data; name="next_page"\r\n\r\n\r\n-----------------------------13252363599581883533344311780\r\nContent-Disposition: form-data; name="action_mode"\r\n\r\n\r\n-----------------------------13252363599581883533344311780\r\nContent-Disposition: form-data; name="action_script"\r\n\r\n\r\n-----------------------------13252363599581883533344311780\r\nContent-Disposition: form-data; name="action_wait"\r\n\r\n\r\n-----------------------------13252363599581883533344311780\r\nContent-Disposition: form-data; name="preferred_lang"\r\n\r\nEN\r\n-----------------------------13252363599581883533344311780\r\nContent-Disposition: form-data; name="firmver"\r\n\r\n3.0.0.4\r\n-----------------------------13252363599581883533344311780\r\nContent-Disposition: form-data; name="file"; filename="RT-AC86U_3.0.0.4_386_51915-g43629d2_ubi.w"\r\nContent-Type: application/octet-stream\r\n\r\n-----------------------------13252363599581883533344311780--\r\n' \
--cookie /tmp/cookie.txt

This doesn't work since the binary data from the firmware file doesn't get sent. There are probably a lot of superfluous headers being sent from Firefox, but I didn't want to muck around too much. The use of data-binary feels problematic when the -F option is normally used to upload a file in a multipart form.

Just leaving this here in case it helps jumpstart anyone else's efforts.

Have you tried to replaced the --data-binary option? and the manually created multipart form data with a series of -F options.

curl 'http://192.168.50.1/upgrade.cgi' -X POST \
-H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0' \
-H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8' \
-H 'Accept-Language: en-US,en;q=0.5' \
-H 'Accept-Encoding: gzip, deflate' \
-H 'Origin: http://192.168.50.1' \
-H 'DNT: 1' \
-H 'Connection: keep-alive' \
-H 'Referer: http://192.168.50.1/Advanced_FirmwareUpgrade_Content.asp' \
-H 'Upgrade-Insecure-Requests: 1' \
-F 'current_page=Advanced_FirmwareUpgrade_Content.asp' \
-F 'next_page=' \
-F 'action_mode=' \
-F 'action_script=' \
-F 'action_wait=' \
-F 'preferred_lang=EN' \
-F 'firmver=3.0.0.4' \
-F 'file=@/path/to/your/firmware/RT-AC86U_3.0.0.4_386_51915-g43629d2_ubi.w' \
--cookie /tmp/cookie.txt

Each -F option creates a part in the multipart form data for one of the form fields?

 

[dave14305]

Have you tried to replaced the --data-binary option? and the manually created multipart form data with a series of -F options.

Yes, that's how I spent a few hours today in lieu of my real work.

I had a tcpdump of a real firmware upload, and I was unable to get the file content inserted in the octet-stream so that it looked similar to the original.

 

[ExtremeFiretop]

Yes, that's how I spent a few hours today in lieu of my real work.

I had a tcpdump of a real firmware upload, and I was unable to get the file content inserted in the octet-stream so that it looked similar to the original.

I'm playing with this as well: -F "file=@/path/to/file;type=application/octet-stream" just know you aren't alone in spending more hours than you want to Admit. Lol
I appreciated all the work and feedback since day one you put into trying to advance this!

 

[dave14305]

I'm playing with this as well: -F "file=@/path/to/file;type=application/octet-stream" just know you aren't alone in spending more hours than you want to Admit. Lol
I appreciated all the work and feedback since day one you put into trying to advance this!

OK, I did it, upgrading an AC86U from 386.11 to 386.12, from my Debian laptop. I had some issues because my firmware file had been truncated to 0 bytes at some point during my testing.

Login:

mypw="$(echo -n 'myusername:mypassword' | openssl base64)"
curl 'http://192.168.50.1/login.cgi' -X POST \
--referer  http://192.168.50.1/Main_Login.asp \
--user-agent 'Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0' \
-H 'Accept-Language: en-US,en;q=0.5' \
-H 'Content-Type: application/x-www-form-urlencoded' \
-H 'Origin: http://192.168.50.1' \
-H 'Connection: keep-alive' \
--data-raw "group_id=&action_mode=&action_script=&action_wait=5&current_page=Main_Login.asp&next_page=index.asp&login_authorization=${mypw}" \
--cookie-jar /tmp/cookie.txt

Successful login:

<HTML><HEAD>
<meta http-equiv="refresh" content="0; url=index.asp">
</HEAD></HTML>

Upload and Upgrade:

curl 'http://192.168.50.1/upgrade.cgi' \
--referer http://192.168.50.1/Advanced_FirmwareUpgrade_Content.asp \
--user-agent 'Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0' \
-H 'Accept-Language: en-US,en;q=0.5' \
-H 'Origin: http://192.168.50.1/' \
-F 'current_page=Advanced_FirmwareUpgrade_Content.asp' \
-F 'next_page=' \
-F 'action_mode=' \
-F 'action_script=' \
-F 'action_wait=' \
-F 'preferred_lang=EN' \
-F 'firmver=3.0.0.4' \
-F 'file=@RT-AC86U_386.12_0_ubi.w' \
--cookie /tmp/cookie.txt

Successful upload:

<html>
<head>
<title>ASUS Wireless Router Web Manager</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta HTTP-EQUIV="Pragma" CONTENT="no-cache">
<meta HTTP-EQUIV="Expires" CONTENT="-1">
<link rel="shortcut icon" href="images/favicon.png">
<link rel="icon" href="images/favicon.png">
</head>
<body>
<script>
var knv = "4.1.27";
var knv_threshold = (parseInt(knv[0])=="")?0:parseInt(knv[0]);
var reboot_needed_time = eval("90");
parent.document.getElementById("hiddenMask").style.visibility = "hidden";
if(parent.Bcmwifi_support && knv_threshold >= 4){
reboot_needed_time += 40;
parent.showLoadingBar(reboot_needed_time);
setTimeout("parent.detect_httpd();", (reboot_needed_time+2)*1000);
}
else if(parent.based_modelid == "RT-N11P"){
parent.showLoadingBar(160);
setTimeout("parent.detect_httpd();", 162000);
}
else{
parent.showLoadingBar(270);
setTimeout("parent.detect_httpd();", 272000);
}
</script>
</body>
</html>

* Closing connection 0

 

[ExtremeFiretop]

OK, I did it, upgrading an AC86U from 386.11 to 386.12, from my Debian laptop. I had some issues because my firmware file had been truncated to 0 bytes at some point during my testing.

Login:

mypw="$(echo -n 'myusername:mypassword' | openssl base64)"
curl 'http://192.168.50.1/login.cgi' -X POST \
--referer  http://192.168.50.1/Main_Login.asp \
--user-agent 'Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0' \
-H 'Accept-Language: en-US,en;q=0.5' \
-H 'Content-Type: application/x-www-form-urlencoded' \
-H 'Origin: http://192.168.50.1' \
-H 'Connection: keep-alive' \
--data-raw "group_id=&action_mode=&action_script=&action_wait=5&current_page=Main_Login.asp&next_page=index.asp&login_authorization=${mypw}" \
--cookie-jar /tmp/cookie.txt

Successful login:

<HTML><HEAD>
<meta http-equiv="refresh" content="0; url=index.asp">
</HEAD></HTML>

Upload and Upgrade:

curl 'http://192.168.50.1/upgrade.cgi' \
--referer http://192.168.50.1/Advanced_FirmwareUpgrade_Content.asp \
--user-agent 'Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0' \
-H 'Accept-Language: en-US,en;q=0.5' \
-H 'Origin: http://192.168.50.1/' \
-F 'current_page=Advanced_FirmwareUpgrade_Content.asp' \
-F 'next_page=' \
-F 'action_mode=' \
-F 'action_script=' \
-F 'action_wait=' \
-F 'preferred_lang=EN' \
-F 'firmver=3.0.0.4' \
-F 'file=@RT-AC86U_386.12_0_ubi.w' \
--cookie /tmp/cookie.txt

Successful upload:

<html>
<head>
<title>ASUS Wireless Router Web Manager</title>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta HTTP-EQUIV="Pragma" CONTENT="no-cache">
<meta HTTP-EQUIV="Expires" CONTENT="-1">
<link rel="shortcut icon" href="images/favicon.png">
<link rel="icon" href="images/favicon.png">
</head>
<body>
<script>
var knv = "4.1.27";
var knv_threshold = (parseInt(knv[0])=="")?0:parseInt(knv[0]);
var reboot_needed_time = eval("90");
parent.document.getElementById("hiddenMask").style.visibility = "hidden";
if(parent.Bcmwifi_support && knv_threshold >= 4){
reboot_needed_time += 40;
parent.showLoadingBar(reboot_needed_time);
setTimeout("parent.detect_httpd();", (reboot_needed_time+2)*1000);
}
else if(parent.based_modelid == "RT-N11P"){
parent.showLoadingBar(160);
setTimeout("parent.detect_httpd();", 162000);
}
else{
parent.showLoadingBar(270);
setTimeout("parent.detect_httpd();", 272000);
}
</script>
</body>
</html>

* Closing connection 0

WHOOOHOOO! Did we just confirm a theory? Confirmed on my side. Just replicated the results on Windows. A bit longer and I would of had it, you beat me to it
I had issues first with my cookie, then had issues with the syntax since it changes a bit on Windows but here is the Windows Powershell version and Output with debugging on:

# Define your credentials
$username = 'Admin'
$password = 'Your Password'


# Base64 encode the credentials in the format username:password
$mypw = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes("${username}:${password}"))
curl.exe -i -v 'http://www.asusrouter.com/login.cgi' -X POST `
--referer http://www.asusrouter.com/Main_Login.asp `
--user-agent 'Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0' `
-H 'Accept-Language: en-US,en;q=0.5' `
-H 'Content-Type: application/x-www-form-urlencoded' `
-H 'Origin: http://192.168.50.1' `
-H 'Connection: keep-alive' `
--data-raw "group_id=&action_mode=&action_script=&action_wait=5&current_page=Main_Login.asp&next_page=index.asp&login_authorization=$mypw" `
--cookie-jar 'C:\tmp\cookie.txt'

curl.exe : Note: Unnecessary use of -X or --request, POST is already inferred.
At line:7 char:1
+ curl.exe -i -v 'http://www.asusrouter.com/login.cgi' -X POST `
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (Note: Unnecessa...ready inferred.:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError
 
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 10.254.0.1:80...
* Connected to www.asusrouter.com (10.254.0.1) port 80 (#0)
> POST /login.cgi HTTP/1.1
> Host: www.asusrouter.com
> User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0
> Accept: */*
> Referer: http://www.asusrouter.com/Main_Login.asp
> Accept-Language: en-US,en;q=0.5
> Content-Type: application/x-www-form-urlencoded
> Origin: http://www.asusrouter.com
> Connection: keep-alive
> Content-Length: 140
>
} [140 bytes data]
* HTTP 1.0, assume close after body
< HTTP/1.0 200 OK
< Server: httpd/2.0
< Date: Thu, 05 Oct 2023 03:48:16 GMT
< Content-Type: text/html
< Connection: close
<
{ [84 bytes data]
100   224    0    84  100   140   3782   6303 --:--:-- --:--:-- --:--:-- 10666
* Closing connection 0
HTTP/1.0 200 OK
Server: httpd/2.0
Date: Thu, 05 Oct 2023 03:48:16 GMT
Content-Type: text/html
Connection: close


<HTML><HEAD>
<meta http-equiv="refresh" content="0; url=index.asp">
</HEAD></HTML>

 

[dave14305]

here is the Windows Powershell version

By tomorrow this needs to run with Invoke-WebRequest.

 

[ExtremeFiretop]

By tomorrow this needs to run with Invoke-WebRequest.

How about tonight? Drum roll:

# Define your credentials
$username = 'Admin'
$password = 'Your Password'

# Base64 encode the credentials in the format username:password
$mypw = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes("${username}:${password}"))

# Create a header for the web request
$headers = @{
    'Accept-Language' = 'en-US,en;q=0.5'
    'Content-Type' = 'application/x-www-form-urlencoded'
    'Origin' = 'http://www.asusrouter.com'
    'Referer' = 'http://www.asusrouter.com/Main_Login.asp'
}

# Create a session variable to store cookies
$session = New-Object Microsoft.PowerShell.Commands.WebRequestSession

# Send a POST request to the login URL
$response = Invoke-WebRequest -Uri 'http://www.asusrouter.com/login.cgi' -Method Post `
-Body "group_id=&action_mode=&action_script=&action_wait=5&current_page=Main_Login.asp&next_page=index.asp&login_authorization=$mypw" `
-Headers $headers -WebSession $session -UserAgent 'Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0'

# Save the cookies to a file
$session.Cookies.GetCookies('http://www.asusrouter.com') | Export-Clixml -Path 'C:\tmp\cookie.txt'

Response:

StatusCode        : 200
StatusDescription : OK
Content           : <HTML><HEAD>
                    <meta http-equiv="refresh" content="0; url=index.asp">
                    </HEAD></HTML>
                  
RawContent        : HTTP/1.0 200 OK
                    Connection: close
                    Content-Type: text/html
                    Date: Thu, 05 Oct 2023 04:13:39 GMT
                    Server: httpd/2.0
                  
                    <HTML><HEAD>
                    <...
Forms             : {}
Headers           : {[Connection, close], [Content-Type, text/html], [Date, Thu, 05 Oct 2023 04:13:39 GMT],; HttpOnly;]...}
Images            : {}
InputFields       : {}
Links             : {}
ParsedHtml        : mshtml.HTMLDocumentClass
RawContentLength  : 84

 

[Yota]

How about tonight? Drum roll:

OK, I did it, upgrading an AC86U from 386.11 to 386.12, from my Debian laptop. I had some issues because my firmware file had been truncated to 0 bytes at some point during my testing.

Congratulations guys, you did it!


I expect within the next few weeks you guys will create a relatively mature curl-based router script.


Just a little suggestion, Asuswrt can control the LEDs light on and off through commands. Please create a blinking LEDs before preparing to update and after the update is completed to remind the user to avoid power interruption during the process, and to make this script look cooler.

If you want to control RGB, go for it, players will love the automatic updates.



EDIT:

Another suggestion is to assume that SourceForge's CDN was taken over by a malicious person and hosted malicious firmware. Of course this is very unlikely.

One mitigation is to not use the SHA256 in the firmware zip file, but to obtain the firmware SHA256 from RMerlin's website, as apparently this would significantly reduce the risk of attack.

However, it should still be noted that this may cause some kind of DDOS attack on RMerlin's website, so you need to obtain @RMerlin's consent before you can do this.

The script also does not need to access RMerlin's website to get SHA256 every time it checks for updates, it only needs to do so before preparing to flash it.

 

[ExtremeFiretop]

I'm still struggling to get PowerShells Webrequest or Invoke-RestMethod to do the upload.
With the below, I can start the upload to the router it seems, but it then fails with references to the 'ByteArrayContent' This may be due to passing a very large array of bytes to the constructor. Seems like some kinda PowerShell cmdlet limitation on that front.

# Read the file into a byte array
$fileBytes = [System.IO.File]::ReadAllBytes('H:\$env:USERPROFILE\Downloads\GT-AXE11000 Firmware Release\Production\GT-AXE11000_3004_388.4_0_pureubi.w')

# Create a ByteArrayContent object with the file bytes
$fileContent = New-Object System.Net.Http.ByteArrayContent -ArgumentList $fileBytes

# Define your credentials (this part remains unchanged)
$username = 'Admin'
$password = 'YOUR PASSWORD'
$mypw = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes("${username}:${password}"))

# Define headers
$headers = @{
    'Accept-Language' = 'en-US,en;q=0.5'
    'Origin' = 'http://www.asusrouter.com'
    'Referer' = 'http://www.asusrouter.com/Advanced_FirmwareUpgrade_Content.asp'
}

# Import cookies
$cookieJar = Import-Clixml -Path 'C:\tmp\cookie.txt'
$webSession = New-Object Microsoft.PowerShell.Commands.WebRequestSession
$webSession.Cookies = $cookieJar

# Create form data content
$form = New-Object System.Net.Http.MultipartFormDataContent
$form.Add((New-Object System.Net.Http.StringContent('Advanced_FirmwareUpgrade_Content.asp')), 'current_page')
$form.Add((New-Object System.Net.Http.StringContent('')), 'next_page')
$form.Add((New-Object System.Net.Http.StringContent('')), 'action_mode')
$form.Add((New-Object System.Net.Http.StringContent('')), 'action_script')
$form.Add((New-Object System.Net.Http.StringContent('')), 'action_wait')
$form.Add((New-Object System.Net.Http.StringContent('EN')), 'preferred_lang')
$form.Add((New-Object System.Net.Http.StringContent('3.0.0.4')), 'firmver')
$form.Add($fileContent, 'file', 'GT-AXE11000_3004_388.4_0_pureubi.w')

# Send the request
$response = Invoke-RestMethod -Uri 'http://www.asusrouter.com/upgrade.cgi' -Method Post `
-Headers $headers -WebSession $webSession -UserAgent 'Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0' `
-Body $form

# Check the response
$response.StatusCode

# Print the response body
$responseContent = $response.Content
Write-Output $responseContent

I tried doing a File Stream to pass more data (The firmware file is about 80MB) but it still failed:

# Define your credentials (this part remains unchanged)
$username = 'Admin'
$password = 'YOUR PASSWORD'
$mypw = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes("${username}:${password}"))

# Define headers
$headers = @{
    'Accept-Language' = 'en-US,en;q=0.5'
    'Origin' = 'http://www.asusrouter.com'
    'Referer' = 'http://www.asusrouter.com/Advanced_FirmwareUpgrade_Content.asp'
}

# Import cookies
$cookieJar = Import-Clixml -Path 'C:\tmp\cookie.txt'
$webSession = New-Object Microsoft.PowerShell.Commands.WebRequestSession
$webSession.Cookies = $cookieJar

# Create form data content
$form = New-Object System.Net.Http.MultipartFormDataContent
$form.Add((New-Object System.Net.Http.StringContent('Advanced_FirmwareUpgrade_Content.asp')), 'current_page')
$form.Add((New-Object System.Net.Http.StringContent('')), 'next_page')
$form.Add((New-Object System.Net.Http.StringContent('')), 'action_mode')
$form.Add((New-Object System.Net.Http.StringContent('')), 'action_script')
$form.Add((New-Object System.Net.Http.StringContent('')), 'action_wait')
$form.Add((New-Object System.Net.Http.StringContent('EN')), 'preferred_lang')
$form.Add((New-Object System.Net.Http.StringContent('3.0.0.4')), 'firmver')

# Open a file stream for the file to be uploaded
$fileStream = [System.IO.File]::OpenRead('H:\$env:USERPROFILE\Downloads\GT-AXE11000 Firmware Release\Production\GT-AXE11000_3004_388.4_0_pureubi.w')
$fileContent = New-Object System.Net.Http.StreamContent -ArgumentList $fileStream
$form.Add($fileContent, 'file', 'GT-AXE11000_3004_388.4_0_pureubi.w')

# Send the request
$response = Invoke-RestMethod -Uri 'http://www.asusrouter.com/upgrade.cgi' -Method Post `
-Headers $headers -WebSession $webSession -UserAgent 'Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0' `
-Body $form

# Close the file stream
$fileStream.Close()

# Check the response
$response.StatusCode

# Print the response body
$responseContent = $response.Content
Write-Output $responseContent

But we can do it with Curl anyways which is probably more universal

 

[ExtremeFiretop]

Good morning Folks!

Good news, seems I'm done the basic script setup for the router using the new method identified here:

MerlinAutoUpdate-Router/MerlinAutoUpdate.sh at main · Firetop/MerlinAutoUpdate-Router

MerlinAutoUpdate-Router is a Merlin router script to allow you to remotely identify a stable firmware update for an ASUS Merlin router, and automatically download and update via an unattended metho...

github.com

The script does the same basic idea as before, parses SourceForge for the latest release for that model, and downloads it to it's own local storage.
Then the router actually logs into it's own web GUI using Curl to simulate the browser, and starts an upload using it's own Web UI.

Just tested on my own router (GT-AXE11000) and it seems to work for me.
Naturally I feel this a good point to pause and reflect, before we go too deep the rabbit hole and start developing more tools and scripts based on this method.

Anymore concerns with using this method? In theory this should be much safer than hnd-write since it's for sure triggering the normal update process over the Web UI.
My only instant thoughts is if ASUS changes stuff in regards to the Web UI or any of the introduced security measures to avoid storing clear text passwords in the router.

Naturally we can re-asses if they ever do that, luckily with this method if it fails, it just fails, no risk of breaking anything.
Just wanted to put this thought out there before I really go ham on trying to implement this at the router and my desktop app.