External disk advice

[Xandrios]

Hello all, thanks for accepting my registration to this forum.

I'm running an AX58U with the latest stable Merlin firmware. Recently I purchased a 4TB USB drive with Black Friday, thinking to hook it up to the router and use NFS/SMB (Internal network) and rsync-over-SSH (external backup). However that hasn't been working too well.

The drive was probably formatted FAT32. Not expecting compatibility issues I just plugged it in and copied a bunch of data to it over the network. Noticed the 'disk check' option, ran that...and it fully corrupted the drive and data. Looking back the issue seems to be that the router only supports up to 2TB drives with FAT32. Whoops. Reset and formatted the drive using NTFS and all seemed well.

Wanting to run rsync and SSH/SFTP I read that I should install the download center option to get access to a package manager. Tried that...and the install process hangs after a while. Seems to be a common issue. Most likely cause: NTFS not supporting symlinks. Ok, so that is not the way to go either.

So it looks like I need a FAT32 drive to be able to install/run a package manager to install rsync/sftp. But my disk is too large for FAT32. And this Asus router only has one single USB port.

There are a few options that I can imagine that may, possibly, work. Your advise would be greatly appreciated.

Option 1: Re-format again as HFS. A bit of an odd format but it may work. Would this support the full 4TB and also allow to install a package manager to install the tools I need?

Option 2: Buy some USB hub and add a small FAT32 USB flash drive to run the package manager stuff. Leave the 4TB disk in NTFS format. May cause power draw issues?

Option 3: Format the drive as EXT2/EXT4. I'm reading on this board, at various places, that only 2TB is supported that way. Yet the Asus disk support page (If only I knew about that page earlier ) seems to show that 4TB is supported. The web GUI does not allow formatting in EXT2/4 though. Would I be able to use the full 4TB if I format the disk on an external system (e.g. macbook)? Do we know which options are supported?

To be honest I'm not sure why I didn't initially format the disk as HFS, and chose NTFS instead. My impression is that HFS wasn't available initially. Perhaps it was added with the latest Merlin firmware (I recently upgraded). Would HFS be the recommended way forward?

In order to install rsync/sftp which package manager would you recommend? Is the download-station still the most efficient way to get access to such tool?

Thanks!

 

[ColinTaylor]

Don't use Download Manager with Merlin's firmware. Uninstall it.

Then SSH into the router and run amtm. From there you can format the USB drive and install Entware.

Correction: amtm can't partition drives over 2TB. So you will have to do that on another device.

 

[Xandrios]

Thanks. Some good pointers there. amtm looks like a great set of tools/scripts, thanks for that tip.

And indeed, running the format option it bails out saying that it cannot format drives beyond 2TB. Would the HFS option (via the GUI) be recommended? I have access to a windows and apple device so should be able to format to most filesystems using those... but judging by the (many) threads on here the exact options/flags do matter. Is there some consensus on which filesystem and options would be the best suitable to use with the merlin firmware?

Thanks!

 

[ColinTaylor]

ext4 would be the preferred format.

The only issue with the router is that it can't create the GPT partition layout (it only supports creating MBR). If you can use another PC to create the GPT then the router can format the partition(s) with your preferred file system.

Disk formatting

Third party firmware for Asus routers (newer codebase) - RMerl/asuswrt-merlin.ng

github.com

 

[Xandrios]

Got it, thanks. As long as the partitioning takes place outside of the router the actual format can be done on the router itself. The drive is currently already using GTP, and I should be able to re-partition using Windows or MacOs - both support GTP.

Would it be recommended to split the drive up into multiple partitions, at the least a separate partition for swap storage? Or is it fine to just use one single partition for everything?

 

[ColinTaylor]

I wouldn't bother creating a separate partition just for swap. I'd create an ext4 partition and put a swap file in there together with Entware and your packages.

As your drive is so large personally I'd split it in two, but it depends on your use case. I have one partition for Entware stuff and another used as a dumping ground for random data. This second partition is NTFS so that it can be plugged into my Windows PC without problem if I need to.

P.S. Is your drive an SSD or HDD?

 

[eibgrad]

In general, I don't recommend using the router as a NAS device anyway, but only for transient files. And I consider anything greater than ~256GB more of a NAS. The router cannot support RAID and other features commonly found w/ a full-sized NAS. And performance is typically far worse than w/ a true NAS.

So I would think carefully about why you need that much space on the router. If it's for transient files, it's overkill. If it's for a NAS, the router is inadequate.

 

[dosborne]

Definitely consider getting a proper NAS if you plan on doing anything serious. The USB drive can still be used for backups, but is best as a local device, or harvest the drive out of it and put it in a proper NAS chassis. Just my opinion.

 

[Xandrios]

Thanks. It's an HDD. I'm planning to use it mainly for TimeMachine backups (for two macbooks), and semi-cold file storage.

I've noticed my workstation SSD's filling up with files that I don't really need to access frequently. Large downloads, installation media, VM backups, etc. Stuff that I probably only need to access a few times a year at most. Performance isn't really important, but it is important that I'd be able to access all the data from within the local LAN, and when away from home through a VPN. SFTP would work fine to retrieve a file when out of the house. Won't need access often, but a few times a year I may be looking for a specific file.

In the past I used to have a Synology NAS but had mixed experiences. Their UI looks nice and flashy, but performance wasn't great (This was their old ARM platform) and the unix tools I was able to install were somewhat limited. This Asus router probably has more resources than the Synology did. In the end I started treating that NAS pretty much like what I'm planning to do here - just for storing files that don't need frequent access. And not having to run a NAS is of course better for power consumption too.

 

[A Purohit]

I started out doing similar stuff as you - using an old 2TB drive as a "NAS" through the router. It worked ok, but when copying large amounts of data, it would hiccup and have issues. It really isn't suited to doing this job. That said, it sounds like you're looking for the same general usage... so it could work.

That said, I bought a QNAP TS-231K last year, and this device is much better. Doesn't have any hiccups (and it's an ARM based setup, too)... and works great for true backups. If you're using the drive for backups, you should always remember the 3-2-1 guideline for backups, which means you should have an online location for the data as well... which the router won't do reliably anyway, hence a NAS makes more sense. For "transient" files, it's probably fine... if you don't run into problems along the way. It's worth a shot, I guess...

 

[Tech9]

I'm running an AX58U

With larger transfers your router will break WAN and drop Wi-Fi. USB storage in Asuswrt is not what you think it is.

 

[rborth]

That said, I bought a QNAP TS-231K last year, and this device is much better. Doesn't have any hiccups (and it's an ARM based setup, too)... and works great for true backups. If you're using the drive for backups, you should always remember the 3-2-1 guideline for backups, which means you should have an online location for the data as well... which the router won't do reliably anyway, hence a NAS makes more sense. For "transient" files, it's probably fine... if you don't run into problems along the way. It's worth a shot, I guess...

What is the 3-2-1 Backup Rule?

The 3-2-1 backup rule is a best practice for backing up your data by saving multiple copies of data on different storage devices and locations. Learn more in this guide.

www.uschamber.com

Here’s what the 3-2-1 backup rule involves:

• 3: Create one primary backup and two copies of your data.
• 2: Save your backups to two different types of media.
• 1: Keep at least one backup file offsite.

I distrust the cloud's security. I prefer to have physical media offsite. https://techcrunch.com/2023/04/13/h... breached data,not publishing the stolen data.

 

[heysoundude]

What is the 3-2-1 Backup Rule?

The 3-2-1 backup rule is a best practice for backing up your data by saving multiple copies of data on different storage devices and locations. Learn more in this guide.

www.uschamber.com

Here’s what the 3-2-1 backup rule involves:

• 3: Create one primary backup and two copies of your data.
• 2: Save your backups to two different types of media.
• 1: Keep at least one backup file offsite.

I distrust the cloud's security. I prefer to have physical media offsite. https://techcrunch.com/2023/04/13/h... breached data,not publishing the stolen data.

This is The Way

 

[A Purohit]

What is the 3-2-1 Backup Rule?

The 3-2-1 backup rule is a best practice for backing up your data by saving multiple copies of data on different storage devices and locations. Learn more in this guide.

www.uschamber.com

Here’s what the 3-2-1 backup rule involves:

• 3: Create one primary backup and two copies of your data.
• 2: Save your backups to two different types of media.
• 1: Keep at least one backup file offsite.

I distrust the cloud's security. I prefer to have physical media offsite. https://techcrunch.com/2023/04/13/h... breached data,not publishing the stolen data.

Yes, I already have 3-2-1. Technically I have 4-3-1 at the moment. I may have 4-3-2 if I add another cloud provider.

I hear you about the insecurity of the cloud, but QNAP can encrypt it before pushing it... and/or you can use a system on the computer to encrypt before it gets to the QNAP. I do the latter already... so even the QNAP doesn't see what's inside the cloud-sent data.

 

[rborth]

Yes, I already have 3-2-1. Technically I have 4-3-1 at the moment. I may have 4-3-2 if I add another cloud provider.

I hear you about the insecurity of the cloud, but QNAP can encrypt it before pushing it... and/or you can use a system on the computer to encrypt before it gets to the QNAP. I do the latter already... so even the QNAP doesn't see what's inside the cloud-sent data.

Thanks, I learned something. I guess I'll have to research the security of file encryption and it's possible downsides.

heysoundude:​

as a funny coincidence side bar, I was a sound guy for 51 years at retiring and my grand kids call me dude ;>)

 

[A Purohit]

Thanks, I learned something. I guess I'll have to research the security of file encryption and it's possible downsides.

In the end, this is really your only defense on your backups. Yes, shipping a disk around is theoretically "safe" but if it ever gets into anyone else's hands, then the unencrypted data is fully accessible.

Most backup software will encrypt before the data is sent to the target. If you do this before sending it to a NAS, then even the NAS doesn't have access to the data. From there, at least on QNAP, you can then push the data to the online system you want... and QNAP can apply an extra layer of encryption during that process as well. In such a situation, you'd have 2 layers of encryption on the cloud.

I suppose you could also encrypt the data on your shipped-drive. That would be better than unencrypted, of course. It might be more safe, in theory, versus a cloud folder (which has a larger attack surface than your shipped drive)... but there's something to be said about the convenience of the cloud, and I think (at least in my case) the encryption makes it safe enough for my needs.

 

[rborth]

Thanks, I learned something. I guess I'll have to research the security of file encryption and it's possible downsides.

heysoundude:​

as a funny coincidence side bar, I was a sound guy for 51 years at retiring and my grand kids call me dude ;>)

In the end, this is really your only defense on your backups. Yes, shipping a disk around is theoretically "safe" but if it ever gets into anyone else's hands, then the unencrypted data is fully accessible.

Most backup software will encrypt before the data is sent to the target. If you do this before sending it to a NAS, then even the NAS doesn't have access to the data. From there, at least on QNAP, you can then push the data to the online system you want... and QNAP can apply an extra layer of encryption during that process as well. In such a situation, you'd have 2 layers of encryption on the cloud.

I suppose you could also encrypt the data on your shipped-drive. That would be better than unencrypted, of course. It might be more safe, in theory, versus a cloud folder (which has a larger attack surface than your shipped drive)... but there's something to be said about the convenience of the cloud, and I think (at least in my case) the encryption makes it safe enough for my needs.

OK, I did some R&D but there is a possibility you are not as secure that you think you are.

And my older NAS devices are WD, but glad I did not expose then to the Internet. https://techcrunch.com/2023/04/13/hackers-claim-vast-access-to-western-digital-systems/

Just data points I found but not my confirmed analysis at this time.

 

[elorimer]

Depending on how much data you need to back up, a free account at Storj is a possibility too.

 

[A Purohit]

OK, I did some R&D but there is a possibility you are not as secure that you think you are.

And my older NAS devices are WD, but glad I did not expose then to the Internet. https://techcrunch.com/2023/04/13/hackers-claim-vast-access-to-western-digital-systems/

Sure, this is one guy's opinion, and a guy who is far more familiar with Synology than QNAP. That being said, he is correct that QNAP has had issues when it's internet exposed. Frankly, I have no idea why someone would do this at all, and I certainly don't. It never even occurred to me to do that... seems kinda dumb.

As he also noted, if you're a tinkerer and are familiar with VPN, then the QNAP is fine. I figure we're all tinkerers here, otherwise we wouldn't be talking about Merlin FW anyway. What's more, many of us use Merlin for the VPN capabiities (as do I)... so, again, not an issue.

So, I'm safer than you think. And, again, encryption is what we are using to keep us safe (VPN, etc). And as I specifically mentioned, the primary backups I do offline to GDrive are from a third party backup software, to the NAS... and then the NAS just pushes THAT encrypted data to GDrive.

I'm glad to hear you didn't put your WDs exposed either. Bottom line, why would anyone want to do that with their NAS?! But, if you were smart/lucky enough not to do that, then you should have no issue doing the same with a QNAP.

EDIT: I should also point out that there is ransomware on Synology as well, as the YT post mentions. And while most are due to insecure user/pass, there has been a "massive" brute force effort against Synology devices going on. That doesn't give me a warm/fuzzy in general, and again speaks to keeping ANY NAS off the public internet.