Seeking Feedback & Contributions: Merlin Auto Update Solutions

[jj22038]

Definitely outside of country. We actually do most our recycling efforts in space, near-earth orbit. There's a lot of junk out there, but Crazy Viktor's Electronics Recycling, Inc. takes care of business!

You actually got me to search for "Crazy Viktor's Electronics Recycling"

 

[Viktor Jaep]

Apologies to the [OP] and participants for my blantant off-topic advertisements for near-earth orbit recycling efforts by Crazy Viktor's Electronics Recycling, Inc... back to the fantastic developments happening in this thread!!

 

[ExtremeFiretop]

Morning Team!

You actually got me to search for "Crazy Viktor's Electronics Recycling"

Apologies to the [OP] and participants for my blantant off-topic advertisements for near-earth orbit recycling efforts by Crazy Viktor's Electronics Recycling, Inc... back to the fantastic developments happening in this thread!!

Thank you, but I won't lie, those last responses did get a laugh out of me XD

My office is actually renewing their fleet of printers, but i don't think the plan was to send the old ones near-orbit lol! (working ones go to schools, etc, garbage ones are tossed for free, no cost recover)

I did make note though of some needed hard coded min version.

Added modifications for a hard coded min version of: 386.11, this can be easily changed I just didn't want to start too aggressive, let me know if you have thoughts or concerns about this min version

 

[KevTech]

I really do not think this is a good idea and feel a warning on the first post is not enough.
The GitHub links should be removed from public view before someone bricks their parents 300+ dollar router.
Let the links to the files be provided by PM and not just in public view.
You may see a lawsuit for a bricked router or you could even see someone try to sue SNB.
I would never update firmware without being right there while it is happening in case there is an issue.

 

[Yota]

Guys, RT-AC68U testing is not going well. I just uploaded the firmware (zip) from sourceforge to tmpfs (via scp), the decompression seemed to cause a lack of RAM and eventually I couldn't enter any commands and the GUI was inaccessible and I had to do a hard reboot.



EDIT:
I have tested it several times with the same result. It may be that the RAM is not enough.


The firmware zip size of RT-AC68U is 90MB and the RAM is 256MB. Usually the RAM usage does not exceed 70MB without USB. So decompressing the firmware is a bit troublesome because it can easily consume all the RAM.



EDIT2:
New test:
Test environment: RT-AC68U connected to USB
Test purpose: Copy the decompressed firmware from USB to tmpfs, and then use curl to flash.
Issue: Unable to copy, various system tools and GUI errors or crashes again.

This makes me wonder, why can't I copy the firmware from USB to tmpfs? At this time, the RAM is more than 120MB free, but a 90MB firmware cannot be saved?





EDIT3:

I'll do some more testing later If it is determined that RAM bottleneck is the cause of this, then we will have to restrict the use of this script on some low RAM models, otherwise it may really cause glitches. In my case, when SSH showed "-sh: can't fork", it seemed that the shell crashed, and then the GUI also crashed and failed to load or part of the content could not be displayed after loading. Anyway the router started acting weird and had glitchy functionality.

Since this crash is not self-recoverable and can only be resolved by a power restart, we have to assume that such an issue may occur on other routers and try to find the trigger (probably RAM) and avoid it before it happens.

Can you deliberately fill up your RAM and reproduce this issue? @ExtremeFiretop @dave14305

 

[Tech9]

I can predict trouble with RT-AX86S as well.

 

[sfx2000]

This makes me wonder, why can't I copy the firmware from USB to tmpfs? At this time, the RAM is more than 120MB free, but a 90MB firmware cannot be saved?

You mean /tmp right?

tmpfs is a filesystem only...

To copy from the USB mount over to /tmp, as long as permissions are ok, you should be able to if there is enough space in the filesystem

 

[Yota]

You mean /tmp right?

Yes,

To copy from the USB mount over to /tmp, as long as permissions are ok, you should be able to if there is enough space in the filesystem

That's the problem, I think when I was copying from the USB, the router read it into cache first, causing the tmp space to decrease, and ultimately causing the copy to fail. But this caused the system tools to crash and I don't know why.

 

[ExtremeFiretop]

Guys, RT-AC68U testing is not going well. I just uploaded the firmware (zip) from sourceforge to tmpfs (via scp), the decompression seemed to cause a lack of RAM and eventually I couldn't enter any commands and the GUI was inaccessible and I had to do a hard reboot.

How much is the total space of the zip plus the decompressed file? My quick test shows me the zip is 91.2MB and the decompressed file is 93 so your looking at about 185MB for storage needed for the process.

EDIT2:
This makes me wonder, why can't I copy the firmware from USB to tmpfs? At this time, the RAM is more than 120MB free, but a 90MB firmware cannot be saved?

I had a similar issue when experimenting with decompressing, I had to decompress in the home/root for me, other system locations would give errors of either no space or no permission.

tmpfs is a filesystem only...

I don't use tmpfs in my testing, but as per sfx above, if it is a filesystem directory I would expect similar issues to what I was seeing.

Can you deliberately fill up your RAM and reproduce this issue? @ExtremeFiretop @dave14305

Let us know the results of your continued tests on memory!
I can try to fill my memory on purpose to re-produce, and memory monitoring and basic management (i.e reboot) is planned already. But i'm not 100% positive what your seeing is strictly related to memory, so keep us in the loop of your tests.

On both my AX88U and AXE11000, I have 256MB of storage and 1GB of memory, and both had no issues but both also weren't using tmpfs or tmp, I have a feeling this may need to be ruled out before we dive much deeper into memory.

However that being said, lower memory models are a thing, so I may as well start on some type of memory solution earlier than later.

 

[Yota]

How much is the total space of the zip plus the decompressed file? My quick test shows me the zip is 91.2MB and the decompressed file is 93 so your looking at about 185MB for storage needed for the process.

You're right about the file size, don't forget the system takes up 70MB (without USB), so 256-185-70, so we're running low.

I had a similar issue when experimenting with decompressing, I had to decompress in the home/root for me, other system locations would give errors of either no space or no permission.

I read the source code, so I also put my test in the /tmp/home/root directory. I want to manually enter the commands to see if it works, and then test the script next. But I didn't expect to encounter such an error at such an early stage.

Let us know the results of your continued tests on memory!
I can try to fill my memory on purpose to re-produce, and memory monitoring and basic management (i.e reboot) is planned already. But i'm not 100% positive what your seeing is strictly related to memory, so keep us in the loop of your tests.

I actually have no idea what's going on right now, everything looks broken, the SSH session is not disconnected, but typing anything just returns "-sh: can't fork" which seems to me to be evidence of a shell crash . The GUI is also inaccessible, or cannot read the memory usage after accessing it, because that part crashes as well.

Can you think of a test method? How do I diagnose the problem?

 

[ExtremeFiretop]

Yes,

That's the problem, I think when I was copying from the USB, the router read it into cache first, causing the tmp space to decrease, and ultimately causing the copy to fail. But this caused the system tools to crash and I don't know why.

When you do the curl commands manually; you have no issue with the login process? You get the expected result of: index.asp?

If so, the possibility of it being memory related is higher if you only have issues when running the flash and it appears to freeze.
Just making sure to rule out a problem with curl/nohup itself in the environment.

 

[Yota]

After the shell crashes, no commands can be executed.
When you do the curl commands manually; you have no issue with the login process? You get the expected result of: index.asp?

If so, the possibility of it being memory related is higher if you only have issues when running the flash and it appears to freeze.
Just making sure to rule out a problem with curl/nohup itself in the environment.

No, I haven't gone that far yet, all I have done now is put the firmware in /tmp/home/root and unzip it. I just want to do manually step by step what the script does.

But I did test curl before, curl can log in to the GUI correctly to get the cookie.

Now both tests I'm doing are getting the same error: The first test was to upload the zip firmware to /tmp/home/root using scp. There was no problem after uploading, no exceptions, but the decompression failed and it reported insufficient space.

After that everything crashed. I just had to turn the power off and restart it.

RT-AC68U was specially used for debugging for this project, so it was restored to factory settings and almost no other settings were made except turning on SSH/turning off WiFi.

The second test was to put the unzipped firmware on a USB drive and then copy it from the USB to /tmp/home/root in the router. As in the second screenshot above, the copy again reported insufficient space.


After the shell crashes, no commands can be executed. But please note that the SSH session is not disconnected at this time.

EDIT:

I tend to suspect lack of RAM for this error, but I could be wrong, as with the previous nohup, I also initially blamed it on lack of RAM. In fact I care so much about available memory that I might derive the wrong evaluation.

If there's any way you guys can help me determine the cause of the error, I'd love to try it tomorrow. I'll run more tests tomorrow and provide more details, sorry I can't test anymore today.

 

[ExtremeFiretop]

No, I haven't gone that far yet, all I have done now is put the firmware in /tmp/home/root and unzip it. I just want to do manually step by step what the script does.

But I did test curl before, curl can log in to the GUI correctly to get the cookie.

Now both tests I'm doing are getting the same error: The first test was to upload the zip firmware to /tmp/home/root using scp. There was no problem after uploading, no exceptions, but the decompression failed and it reported insufficient space.

After that everything crashed. I just had to turn the power off and restart it.

RT-AC68U was specially used for debugging for this project, so it was restored to factory settings and almost no other settings were made except turning on SSH/turning off WiFi.

The second test was to put the unzipped firmware on a USB drive and then copy it from the USB to /tmp/home/root in the router. As in the second screenshot above, the copy again reported insufficient space.


After the shell crashes, no commands can be executed. But please note that the SSH session is not disconnected at this time.
View attachment 53516

EDIT:

I tend to suspect lack of RAM for this error, but I could be wrong, as with the previous nohup, I also initially blamed it on lack of RAM. In fact I care so much about available memory that I might derive the wrong evaluation.

If there's any way you guys can help me determine the cause of the error, I'd love to try it tomorrow. I'll run more tests tomorrow and provide more details, sorry I can't test anymore today.

First of all, I find this all very valuable data, because already we have a baseline of hardware specs we can expect to function correctly, and a baseline where things can get "weird." as you say.

Now, I understand that you were just going through the steps one by one as described in the script, however now that we have an issue we can break it down.

To troubleshoot, what if you simply skipped the unzipping and even the zip files all together? Don't have them on the USB, or home/root or anywhere on the router.

Start by unzipping at your computer, and copy it via SCP to home/root. The same way you did with the zip the first time successfully. Make sure there is no zip, if there is delete it first.

What I may need clarification on, is you say the zip file copied no problem via SCP, but the unzipped firmware didn't copy via USB successfully due to storage issues. (Did we attempt the same method via SCP?)

The different between the two in size is negligible (91MB vs 93MB) so this lends me to believe that you left the zip there while you attempted to copy the unzipped firmware, this is good for testing the script since this is what it does, but what if we skip the zipped file all together to break it down now that we have an issue.

Put the firmware file as the only file in home/root and test the curl commands manually?

What this will do is narrow the issue done specifically to the unzip (either storage or RAM).

If the single firmware file is already there and unzipped, and the curl login and flashes is successful, we know that RAM is not a concern for the actual flash. But maybe still for the unzip.

I don't think storage is the issue since our models have the same amount, however to narrow down from there, we can test having the zipped firmware on the USB ( like it will be as per our expected backup solution) and simply unzip the firmware to home/root via command, where it should have enough storage (alone) and we will know that the rest of the process should work successfully since you already tested it manually.

If that works, then we know the issue is storage related, and we would maybe need to download the zip the the USB and unzip to the home for the flash instead of doing it all on the home directory.

If it doesn't work, then we know the issue is related to memory while unzipping since the root/home is empty and the difference is basically nothing between zipped and unzipped.

 

[ExtremeFiretop]

By the way, feel free to do those troubleshooting steps in whichever order you rather the manual flash test, or the copy tests.

The only reason I suggested that order, is that if the flash fails, there is not much point in troubleshooting the flash prep, especially when you handle all the pre-flash prep manually and the setup is ideal for the flash.

However if we know the flash can succeed manually with an ideal setup, then there is still hope and we can then further troubleshoot the unzip with storage and copy tests.

 

[dave14305]

You may see a lawsuit for a bricked router or you could even see someone try to sue SNB.

Someone will spend thousands on attorney fees to recover damages on a $100 - $600 home router?

The GitHub links should be removed from public view before someone bricks their parents 300+ dollar router.

Most of this litigious, vulnerable population of Merlin users can’t or won’t install a script without it being in the amtm menu. They aren’t at risk of bricking their router. And if they are reckless enough to install an experimental script on their poor parents’ router, they deserve to be cut out of the parents’ will.

There is a lot of overreaction in this thread. I may be under reacting myself, but where is the forum’s sense of adventure and “let’s try it” mentality?

Does anyone wonder why it gets so dull around here these days? Not many people want to take risks or have fun trying new things.

Odds are very high that this project will die for one valid reason or another. But “not trying” shouldn’t be one of them.

 

[KevTech]

Someone will spend thousands on attorney fees to recover damages on a $100 - $600 home router?

Why would you need an attorney for a civil dispute?

 

[dave14305]

Why would you need an attorney for a civil dispute?

Punitive damages! Sue everybody!

 

[ExtremeFiretop]

Why would you need an attorney for a civil dispute?

Why would anyone bother with a civil dispute on stuff that is clearly identified as experimental and in testing/development?

Let me tell you something, it's a laughing matter if someone thinks they can win a civil dispute over some development code that they ran without reading the documentation, and clear warnings that it's in development and in testing.

Would be dropped like a hot potato,

It would be one thing if I advertised this as a ready solution and it blows everything up, but ever heard of assumption of risk?

Why do you think people put wet floor signs up and stuff? So they can't be sued... I did the digital equivalent, so go ahead and try and sue me over a project clearly identified as such.

But where is the forum’s sense of adventure and “let’s try it” mentality?

Does anyone wonder why it gets so dull around here these days? Not many people want to take risks or have fun trying new things.

Odds are very high that this project will die for one valid reason or another. But “not trying” shouldn’t be one of them.

So so so agreed with every single line.

 

[JGrana]

Punitive damages! Sue everybody!

That does it @dave14305 ! Even though you are a gifted and knowledgeable developer, saying I “don’t want to have fun” has crossed the line.
My lawyers will be in contact first thing tomorrow.
;-)

Disclosure. The “wink” above implies humor, sarcasm, “just kidding”. The views and opinions of this reply are mine and not of snbforums, RMerlin, Asus or any other human or ChatGPT.

 

[JGrana]

On a more serious note, I do view this as an interesting and technically challenging project. Even if limited to a subset of routers, based on the flexable nature of Asuswrt-merlin - getting it correctly working all the time is a noble goal.
I wish the developers well - really.
Personally I would try it once, experiment a bit (spare router of course) and follow along.

I wont be using it though. It takes all the fun and excitement of the download, the unzipping, the hope that the upgrade actually succeeds the first time, the anticipation of the reboot and seeing the new firmware version string on the WebUI.

Im a retired engineer/engineering manager and need some technical challenges once in a while!