3TB drive on Asuswrt-Merlin??

[Quiller]

WD My Book Essential 3TB

So I got the Western Digital My Book Essential 3TB (usb 2 and 3 version) and it works right out of the box on my RT-AC66U. It's formatted as NTFS on a single GPT partition. No problems at all.

I have an old OCZ vertex 2 (56GB) on the other usb port through a SATA to USB adapter and that SSD is formatted to ext3, so my DLNA database and entware run from that SSD.

Running Asuswrt-Merlin 3.0.0.4.374.33 Beta 1 at the moment and runs great.

 

[oldgringo]

I'm using 1.8 GB ext3 partition. Disk is almost empty and without errors but e2fsck scans it a few minutes then ends up with following messages:

-Pass 1: Checking inodes, blocks, and sizes
-Pass 2: Checking directory structure
-Error allocating icount structure: Memory allocation failed
-e2fsck: aborted

 

[RMerlin]

That's normal. e2fsck require a lot of RAM to handle large partitions - nothing that can be done about it.

 

[Quiller]

Moving files on 3TB usb drive

I downloaded a large file of 13GB today with transmission on the 3TB drive that's attached to my RT-AC66U. When it was ready I moved the file to its permanent folder on the same drive, same partition but different folder. I used windows explorer to move the file.

Now when I do that with a file on my computer, windows only moves the directory entry to the new folder location and leaves the actual file on the same location on the hard disk. So that's an instant file move. The router however moves the entire file to the new folder on that same drive and takes around an hour.

My question is, can the merlin fw be changed so that a file move on the same attached usb drive and partition works like the file move in windows? That would save a lot of reads and writes to that disk and thus a lot of time.

 

[RMerlin]

I downloaded a large file of 13GB today with transmission on the 3TB drive that's attached to my RT-AC66U. When it was ready I moved the file to its permanent folder on the same drive, same partition but different folder. I used windows explorer to move the file.

Now when I do that with a file on my computer, windows only moves the directory entry to the new folder location and leaves the actual file on the same location on the hard disk. So that's an instant file move. The router however moves the entire file to the new folder on that same drive and takes around an hour.

My question is, can the merlin fw be changed so that a file move on the same attached usb drive and partition works like the file move in windows? That would save a lot of reads and writes to that disk and thus a lot of time.

You are working over a network share, this is why it has to be a full read/write operation. The router sees a read request and a write request coming from the network client, it has no way of knowing this is actually a move request.

 

[oldgringo]

That's normal. e2fsck require a lot of RAM to handle large partitions - nothing that can be done about it.

I wouldn't be so sure because I'm able to check the same disk on rt-n16 with latest oleg fw without any hassle. It took almost 15 minutes but successfully finished. Even ext4 with the same size. I can see there some space for the improvement. At least adding cache directory config into the e2fsck.conf file might be a little workaround for that situation. If I had a little free time I will try to compile such modified fw and I will let you know the result.

 

[RMerlin]

I wouldn't be so sure because I'm able to check the same disk on rt-n16 with latest oleg fw without any hassle. It took almost 15 minutes but successfully finished. Even ext4 with the same size. I can see there some space for the improvement. At least adding cache directory config into the e2fsck.conf file might be a little workaround for that situation. If I had a little free time I will try to compile such modified fw and I will let you know the result.

We might not both use the same e2fsck implementation. I use the one from e2fsprogs, for improved compatibility over the old ones implemented in Busybox.

 

[Nerre]

You are working over a network share, this is why it has to be a full read/write operation. The router sees a read request and a write request coming from the network client, it has no way of knowing this is actually a move request.

As far as I know samba has support for "local move", but it probabaly needs client side support (the client must perform a "move" and not a "read-write-delete").

As I saw someone saying it: Moving a file from one location to another within the same network share is essentially a "full path rename".

 

[bpearse]

Success with NTFS 3TB drive

Yes, but only with ext3. The closed-source NTFS driver that we use had troubles with large NTFS filesystems when I tested it, but ext3 was fine.

I am using 3TB USB Seagate Expansion Desktop disk (USB2/3) and am able to copy > 4GB files just fine and am using the entire 3TB drive size. I am running latest 3.0.0.4.374.33. Also getting quite fast copy speeds, around 15MB/s. Drive is MBR (using 4096 sector size). Router is RT-N16.

I was thinking I would have to convert to EXT3, but apparently NTFS works just fine. I am curious what issues you were seeing with large NTFS file systems? I am just curious what problems might crop up with this config.

 

[RMerlin]

I am using 3TB USB Seagate Expansion Desktop disk (USB2/3) and am able to copy > 4GB files just fine and am using the entire 3TB drive size. I am running latest 3.0.0.4.374.33. Also getting quite fast copy speeds, around 15MB/s. Drive is MBR (using 4096 sector size). Router is RT-N16.

I was thinking I would have to convert to EXT3, but apparently NTFS works just fine. I am curious what issues you were seeing with large NTFS file systems? I am just curious what problems might crop up with this config.

In my case, the router was unable to mount it at all. The NTFS driver was reporting being unable to replay the native journal, and was giving up. I've heard of at least one other user who was also unable to mount his NTFS-formatted large HDD, but had no problem once he had reformated it under ext3.

 

[bpearse]

In my case, the router was unable to mount it at all. The NTFS driver was reporting being unable to replay the native journal, and was giving up. I've heard of at least one other user who was also unable to mount his NTFS-formatted large HDD, but had no problem once he had reformated it under ext3.

There are so many variables, it is very hard to say why it works in one case and not in another. Perhaps GPT vs MBR (I used minitool to convert to MBR), sector size (4096 in my case) and drive brand/firmware all play a role. I am very happy with the result I have gotten, though! And after running only DD-WRT for years, I am more impressed with your Merlin firmware than any other firmware I have tried.

Thanks again for the incredible work!

 

[F5ing]

There's quite a bit involved here regarding MBR vs GPT, bit platform, OS, sector size, booting/nonbooting, etc. Good Wikipedia page on MBR:

https://en.wikipedia.org/wiki/Master_boot_record#Partition_table_entries

Maybe you're getting away with using MBR with a 4096 sector size for that 3TB drive simply because you're not trying to boot from it and the OS can handle it. A 512 sector size wouldn't have cut it (without a loss of access to anything beyond the 2.2TB mark (unless you used multiple partitions)).

 

[bpearse]

Maybe you're getting away with using MBR with a 4096 sector size for that 3TB drive simply because you're not trying to boot from it and the OS can handle it. A 512 sector size wouldn't have cut it (without a loss of access to anything beyond the 2.2TB mark (unless you used multiple partitions)).

Correct. My MBR 3TB drive would not work as boot, for sure. But it is working great so far as a data drive on my RT-N16. I have not stored more than 2.2TB of data yet, but I doubt any issues will arise at that point. Fingers crossed.

 

[cdysthe]

Yes, but only with ext3. The closed-source NTFS driver that we use had troubles with large NTFS filesystems when I tested it, but ext3 was fine.

I tried mine with ntfs and it didn't work very well. ext3 works flawlessly so far (except for the long time it takes to format the 3 TB). Kind of strange that ext4 isn't supported. It has been around for quite some time. Is it the kernel that doesn't support it?

 

[RMerlin]

I tried mine with ntfs and it didn't work very well. ext3 works flawlessly so far (except for the long time it takes to format the 3 TB). Kind of strange that ext4 isn't supported. It has been around for quite some time. Is it the kernel that doesn't support it?

Correct. N16/N66/AC66 run 2.6.22.19, it was only very experimental back then. The RT-AC56 and AC68 both run 2.6.36.4, which is why these two do support ext4.

 

[cdysthe]

Correct. N16/N66/AC66 run 2.6.22.19, it was only very experimental back then. The RT-AC56 and AC68 both run 2.6.36.4, which is why these two do support ext4.

Do you think we'll see firmware for the RT-N66U with an updated kernel?

 

[RMerlin]

Do you think we'll see firmware for the RT-N66U with an updated kernel?

Highly unlikely. Broadcom offers support for 2.6.22.19 for that hardware platform, so all manufacturers are sticking to it. A kernel change would imply not only a driver change, but also the loss of HW acceleration, since that part of the code is proprietary and strictly developed by Broadcom.

 

[shpankey]

Out of curiosity, why not just use exFAT and save any issues? That way PS3, Router, Windows, Mac, Linux and pretty much everything can see use it and not have any limitations? It's what I'm doing, but with a Seagate 500GB drive so I'm not sure I've really tested any limits. Just going off a post I read somewhere to use exFAT and be done with it so that's what I did. So far so good, but should I expect a problem somewhere for some reason? Again, I don't actually know anything, just reporting what I read and did and what's happened for me so far.

 

[RMerlin]

Asuswrt does not support exFat. And since this is yet another proprietary filesystem that technically requires you to pay Microsoft for a license, I refuse to support it even if it could be backported to this kernel

 

[krabs]

what about exfat-nofuse based on samsung exFAT filesystem.
https://github.com/dorimanx/exfat-nofuse
Samsung has released their exFAT filesystem under GPL.
http://www.phoronix.com/scan.php?page=news_item&px=MTQzODQ
http://opensource.samsung.com/reception/receptionSub.do?method=search&searchValue=exfat
Tomato has already support for it.
http://repo.or.cz/w/tomato.git?a=commit;h=97b2e3f8071efd5dcc3b0b06567099a10502ec8a
Seems legit.