Solved Replacing USB (Contains Entware) - Safest & Most Efficient Way?

[SpicyLimes]

Greetings,

I have been using ASUS Merlin firmware on my RT-AX58U for roughly 2 years now, and along with that, I have also been benefiting from AMTM and Entware Software packages for around the same amount of time.

As we all know, using these third-party add-ons requires a storage device (in my case, a 16GB USB Flash Drive). And because it has been two years of using the same USB Flash Drive, I would like to replace it due to my own trust issues with such devices (I won't go into detail about why I don't trust the longevity of USB Flash Drives, but basically they CAN fail, and replacing one is super cheap anyway).

With all that being said, I would like to get some opinions on the best practices of replacing the USB Flash Drive, as well as how to transfer all data on the old USB Flash Drive to the new one without causing issues on the router-side of things. I assume a simple copy and paste of the files won't work, so I created a partition image of the drive (to have as a backup as well as to potentially flash the partition image onto the new drive) - although I am not sure that flashing the partition image onto the new drive would be my best option.

So, what is the best method to accomplish this? Copy and paste? Flash the parition image? Something else...?

 

[ColinTaylor]

For copying it probably depends on a number of factors.

Does your current USB drive have a label?
How many partitions does it have?
What is the filesystem type of those partitions?
What is your PC operating system of choice (e.g. Windows, macOS, Linux)?

 

[SpicyLimes]

Does your current USB drive have a label?

Yes - currently labeled as ASUSUSB.

How many partitions does it have?

Only 1 as it is only used for AMTM/Entware Software and a 2GB Swap File.

What is the filesystem type of those partitions?

Ext4, as recommended to use with ASUS Merlin (and Linux in general).

What is your PC operating system of choice (e.g. Windows, macOS, Linux)?

I am fluent with Windows, and am well versed in Linux. I have access to multiple Windows and Linux PC's (most of the Linux PC's are running my Home Lab Servers but I do have access to two with GUI's - I used a Linux PC via the 'Gnome Disks App' to create the partition image since Windows won't read Ext4). I am OK with Linux CLI, but by no means am I an expert.

Edit: I forgot to include in my original post that I am using the USB Flash Drive in USB 2.0 "mode" since USB 3.0 basically crippled my WLAN when I tried it. The new USB Flash Drive is ONLY USB 2.0 and will be 32GB instead of 16GB. If I go down the route of flashing the partition image onto the new drive, I will either expand the file system of the parition to utilize the entire drive, or partition the additional space as a secondary storage/backup partition.

 

[ColinTaylor]

OK. As you currently only have a single partition (labelled ASUSUSB) I think I would take this approach.

1. Disable custom scripts and configs in the GUI and reboot the router.
2. Insert the new USB device and create a single partition using amtm with a different label (we'll change this later). The router will reboot.
3. SSH into the router and stop USB activity

service stop_nasapps

4. Copy the contents of the old drive to the new one:

cp -a /tmp/mnt/ASUSUSB/* /tmp/mnt/new_drive_label

5. Do a df and make a note of the new drive's device name (e.g. /dev/sdb1).
6. Unmount both drives:

umount /dev/sda1
umount /dev/sdb1

7. Label the new drive:

tune2fs -L ASUSUSB /dev/sdb1

8. Physically remove the old drive and reboot.
9. If everything looks OK re-enable custom scripts and configs in the GUI and reboot the router.

EDIT: Added final reboot step. Thanks @jksmurf

 

[SpicyLimes]

Unfortunately I only have a single USB port on my router but I assume that I can simply copy the contents of the old USB Flash Drive to some other folder on the router and move them to the new drive - however, I wouldn't be able to move the Swap File to the router so I would need to re-enable that later.

Another issue I can see popping up is all of the scripts and software I have running to keep internet access. Specifically, if I disable Custom Scripts and Configs, the router will (for lack of a better phrase) lose it's #*!%.

I currently have AdGuard Home, Unbound, Skynet, and Wireguard Manager (Client) running on the router (AGH + Unbound handling the local DNS before going out through Surfshark's VPN tunnel - I have Unbound using the tunnel as well).

Thoughts?

Edit: Forgot to say thanks for responding to quickly - I am hoping to get this done before this evening, but it's not absolutely required since it looks like I will be doing an update to 388.1 as well. Maybe it's just better to start clean with a factory reset, but I'd prefer to not since I have quite a few static IP's setup, not to mention I've renamed almost every device in the GUI (I have around 75 devices connected at any given time, and a decent chunk of those are Kasa products that use the standard "TP-Link Kasa" hostname).

 

[ColinTaylor]

Disabling custom scripts is only a temporary change while you copy the contents of the USB drive. The router won't "lose it's #*!%".

But as you only have a single USB port this technique work work (I guess it might if you used a USB hub).

There is no "other folder" on the router that you can copy the contents of the drive to. So you'll have to use some other technique.

 

[SpicyLimes]

Why can't I simply create a temporary directory somewhere to temporarily house the files? The storage requirement for the items on the USB Flash Drive is minimal (since I won't be copying the swap file)... Maybe you know something that I don't.

I don't suppose putting both USB Flash Drives in one of my Linux PCs and doing your method would work because of permissioning?

 

[ColinTaylor]

Why can't I simply create a temporary directory somewhere to temporarily house the files? The storage requirement for the items on the USB Flash Drive is minimal (since I won't be copying the swap file)... Maybe you know something that I don't.

The internal flash memory on the router is fairly small. I'd guess you'd have less than 50MB free space. I'm assuming the contents of your USB drive is much bigger.

I don't suppose putting both USB Flash Drives in one of my Linux PCs and doing your method would work because of permissioning?

That should work and would probably be simpler. But I strongly suggest that you use amtm on the router to partition and format the new drive to avoid compatibility issues.

 

[SpicyLimes]

The internal flash memory on the router is fairly small. I'd guess you'd have less than 50MB free space. I'm assuming the contents of your USB drive is much bigger.

Understood - probably best not to risk it then.

That should work and would probably be simpler. But I strongly suggest that you use amtm on the router to partition and format the new drive to avoid compatibility issues.

Will do.

As a side note (because it seems like a factory reset is needed to clear out potentially conflicting routing code in dnsmasq and other files that has built up over the years with all my tinkering), is there anyway to backup the Static IPs and custom device names? I made a backup of dnsmasq.conf since it seems to store the Static IPs, but am not 100% sure that is the correct method. I assume that after a factory reset I would need to add these entries back into dnsmasq.conf using the dnsmasq.conf.add command?

 

[skeal]

Understood - probably best not to risk it then.


Will do.

As a side note (because it seems like a factory reset is needed to clear out potentially conflicting routing code in dnsmasq and other files that has built up over the years with all my tinkering), is there anyway to backup the Static IPs and custom device names? I made a backup of dnsmasq.conf since it seems to store the Static IPs, but am not 100% sure that is the correct method. I assume that after a factory reset I would need to add these entries back into dnsmasq.conf using the dnsmasq.conf.add command?

JFFS backup stores lots of stuff. Static IP's, Device names, VPN director, DNSDirector, etc

 

[SpicyLimes]

JFFS backup stores lots of stuff. Static IP's, Device names, VPN director, DNSDirector, etc

Good to know - thank you (I wasn't sure what was stored in the 'jffs backup' or the 'router settings' backup).

Although I do not think this will help me since I want to start with a clean slate (minus the 'Device Names' and 'Static IPs' that I mentioned).

It's not the end of the world to not have these restored, but it would have been nice.

 

[Piotrek]

Look: https://www.snbforums.com/threads/release-nvram-save-restore-utility.61722/
or save the following files from /jffs/nvram:
custom_clientlist
custom_usericon
dhcp_staticlist
After reset, enable Manual Assignment in DHCP, restore the files, reboot.

 

[SpicyLimes]

Look: https://www.snbforums.com/threads/release-nvram-save-restore-utility.61722/
or save the following files from /jffs/nvram:
custom_clientlist
custom_usericon
dhcp_staticlist
After reset, enable Manual Assignment in DHCP, restore the files, reboot.

This is excellent! Thank you. I downloaded the 'custom_usericon' file but it was empty which makes sense since I didn't upload my own icons.

I noticed a package in AMTM called 'nsrum - NVRAM Save/Restore Utility' but it shows "deprecated" next to it so I didn't want to install it, but I assume it can still be used?

 

[Crimliar]

I noticed a package in AMTM called 'nsrum - NVRAM Save/Restore Utility' but it shows "deprecated" next to it so I didn't want to install it, but I assume it can still be used?

It's no longer needed as the same function can be found in the WEBUI in Administration > Restore/Save/Upload Setting

 

[SpicyLimes]

It's no longer needed as the same function can be found in the WEBUI in Administration > Restore/Save/Upload Setting

Ah, got it. It's all starting to click now.

Well, I appreciate everyone that chimed in to help. I just did a dirty upgrade to 388.1 without any issues and all is working well.

We can consider this inquiry CLOSED.

Thanks again!

 

[JGrana]

It's no longer needed as the same function can be found in the WEBUI in Administration > Restore/Save/Upload Setting

Technically they are different. The built in function copies the entire NVRAM contents and stores them in a file. This includes many variables that are not typically changed by users or settings in the WebUI. Some of these variables relate to internal hardware settings.
As a result, when moving to a different firmware release - using a different file created by the Restore/Save/Upload setting can cause problems - and should be avoided.

The AMTM utility is selective in what it saves. It attempts to read user settings from the nvram and creates a script that then puts them back. Things like SSID, VPN settings, user changeable WiFi settings, etc. Essentially most any setting you can change in the WebUI.
As a result, it doesn’t touch nvram settings that are not usually user changeable. In fact, that’s the reason it was originally developed (long ago by). As a result, the script file it creates could be used for different firmware releases. in fact it even attempted to allow you to move settings from one type of Asus router to another.

Over time though, as Asus (and @RMerlin ) added features that created new nvram variables, it become thougher and tougher to maintiain and keep up to date.

It still works surprisingly well for older settings. But, for newer firmware releases it doesn’t know about things like AiMesh, VPN Director, the fact that static DNS is now in jffs, etc.

 

[inakaya]

I think I would take this approach.

1. Disable custom scripts and configs in the GUI and reboot the router.
2. Insert the new USB device and create a single partition using amtm with a different label (we'll change this later). The router will reboot.
3. SSH into the router and stop USB activity

service stop_nasapps

4. Copy the contents of the old drive to the new one:

cp -a /tmp/mnt/ASUSUSB/* /tmp/mnt/new_drive_label

5. Do a df and make a note of the new drive's device name (e.g. /dev/sdb1).
6. Unmount both drives:

umount /dev/sda1
umount /dev/sdb1

7. Label the new drive:

tune2fs -L ASUSUSB /dev/sdb1

8. Physically remove the old drive and reboot.
9. If everything looks OK re-enable custom scripts and configs in the GUI.

I just followed these instructions to switch from a USB to SDD on my RT-AX86U and worked perfectly keeping all my stats for conmon etc.

Thanks ColinTaylor!

 

[bibikalka]

I just followed these instructions to switch from a USB to SDD on my RT-AX86U and worked perfectly keeping all my stats for conmon etc.

Thanks ColinTaylor!

Here is a slightly different procedure, but gets to the same place fast.

amtm ext4 partition with journaling - 15 GB size, 1150 GB of writes in 20 months

As I was slightly re-partitioning my external AMTM drive (a very low end dram-less 240GB SSD hanging on a USB2/SATA ~10 inch adapter cable), I noticed something really interesting. In 20 months this partition with Entware and a 2 GB swap file had 1152 GB of disk writes. So this is about 2GB of...

www.snbforums.com

 

[jksmurf]

Maybe I’m missing something here but if all you need is to clone your USB drive, and you are ‘fluent in Windows’, what is stopping folks just cloning the USB drive in Windows?

Doing a quick google search suggests there are Freeware programs out there for doing this and they handle ext4. Is there something that would prevent this achieving what I understood the OP asked about?

example (no affiliation whatsoever) https://en.taiwebs.com/windows/download-usb-drive-clone-pro-7160.html

There are even Programs that clone USB to SSD, whether they support ext4, I am not sure.

 

[ColinTaylor]

Maybe I’m missing something here but if all you need is to clone your USB drive, and you are ‘fluent in Windows’, what is stopping folks just cloning the USB drive in Windows?

In most cases you don't want to clone (i.e. make a bit for bit copy) one drive to another because they have different capacities. If that were the case you could use dd or many other methods. Recreating the filesystem and copying the files allows you to optimise, or even change, the filesystem layout. It also avoids the problem of copying over a corrupted filesystem. But like you said, there are plenty of other ways of doing that but in my experience the only way to guarantee compatibility with the router is to make the filesystem using the router's own commands. You can then copy the files using any method you like, either on the router or on a PC. I've tried various Windows tools in the past and in the majority of cases they had one problem or another (especially with ext4), but YMMV. For the purposes of these forums we can't assume the user has access to a PC running Windows, Linux, macOS, etc. and an appropriate utility.

If you can recommend some Windows utilities that have worked for you then that would be helpful. Not everyone is comfortable using a fairly clunky command line method.