USB 3.0 HD write permissions

[Zonkd]

When I don't understand something, I try to research as much as possible. Here's an interesting web page which shows "hard drive size" vs. "actual capacity". Seems I have a 2 TB HD as advertised, with an actual capacity of 1.862 TB. If I understand right, it's because there's 1,024 as the base instead of 1,000 when converting bytes to GB.

http://www.ussscctv.com/hdccalc.html

I'm glad you enjoy the challenge.

Yes all manufacturers technically misadvertise the true storage capacity of the drives (in the mind of the buyer). They argue it's to avoid confusing the public. Note 1 Tebibyte is not the same as 1 Terabyte. If you think you're buying a 2 Terabyte HDD, no, you're actually buying a 2 Tebibyte HDD (the manufacturer does state this in the fine print on the box in-store). Windows measures disk capacity in Tebibytes. So when you check the disk size in Windows your 2TB harddrive will actually be smaller than you thought it'd be. Ultimately you're still getting what you pay for. 2 Tebibytes is 2000 Gibibytes. If you want more Gigibytes you should buy more Tebibytes.

The fact is a “terabyte”, according to standards, is a trillion bytes or a thousand gigabytes. Although many may use 1 TB to mean 1024 GB, the standard (which helps alleviate confusion for all of us) states that only 1 TiB is 1024 GiB and that 1 TB is 1000 GB. - Source

 

[Zonkd]

Not TOO frustrated. I like these type of projects. I just don't like when I'm not really in control of what/when I can continue.

Plugged the drive back into Windows. It didn't register there either. Formatted NTFS. Says 1.862 GB. Included web page view of what it shows. Won't web format FAT.

Putty terminal to Asus to run amtm. 2TB problem again. Posted picture here too.

Next?

? I could be mistaken, so far as I can see your HDD is an anomaly with a greater storage capacity than it should have.

The AMTM fd feature won't ever work because it will never pass the disk size check. You would need to modify the AMTM script yourself (easily done) to make it pass. I wouldn't do that since the HDD is too large to support the MBR partition table that AMTM would try create. You're better of using GPT. Make a GPT on the disk manually with Windows, then use steps 5 8 and 9 to format it ext with the router.

Step by step here is what I'd be doing if I were in your situation:

1. Use Windows Disk Management tools to make 1 full disk partition and format it NTFS (if there is an option select GPT to make it have a GUID partition table, otherwise lets just hope windows is smart enough to do that automatically).
2. Plug it into the router
3. Hard reboot the router (optional)
4. Connect to router via ssh
   
5. Skip right to step 5 and unmount the disk
6. Skip to step 8 and format the disk to ext as desired
7. Skip to step 9 and reboot/remount

 

[ColinTaylor]

Yes all manufacturers technically misadvertise the true storage capacity of the drives (in the mind of the buyer).

That's not quite true IMHO. Disk storage has always been defined in 1000's so that is not misadvertising. However, memory (i.e. RAM) has traditionally been defined in 1024's. The class action case you linked to is interesting because it's a cross-over of both technologies - memory manufacturers were selling RAM that was emulating storage (but using the "storage" number base).

Side note: If you look at files/disks in Windows the sizes are reported multiples of 1024 for MB, GB, etc. If you look at the same things in MacOS you'll see that they're reported in multiples of 1000.

 

[L&LD]

ColinTaylor, that just means that disk storage manufacturers have always been lying to us.

Don't buy storage devices for my Windows computers to have to need to convert between what the os reports as available and what I thought I purchased. It is mis-advertising.

MacOS is a simpler os and can't count proper by base 2! Their take on 'simple' makes my head hurt, on all aspects of os control.

 

[David Cavalli]

Step by step here is what I'd be doing if I were in your situation:

1. Use Windows Disk Management tools to make 1 full disk partition and format it NTFS (if there is an option select GPT to make it have a GUID partition table, otherwise lets just hope windows is smart enough to do that automatically).
2. Plug it into the router
3. Hard reboot the router (optional)
4. Connect to router via ssh
   
5. Skip right to step 5 and unmount the disk
6. Skip to step 8 and format the disk to ext as desired
7. Skip to step 9 and reboot/remount

Done. The specific steps:
1-5. Done, no problem.
6. Step 8, no recommendation on has/!has journal. Chose has journal (because I saw tune2fs can undo that later). Named drive "Asus" on /dev/sda1
7. Rebooted/remounted automatically.

Asus web server configuration:

+USB Application
  *Media Server
  *Network Place (Samba) Share / Cloud Disk
   ^USB Application
    Enable Share - On
    Allow guest login - Off
    Device Name - Asus
    Work Group - <home workgroup>
    Simpler share naming - Yes
   ^<Permissions>
    cavalli        <login>        Backup, Media, Public (R/W)
    David        <win10pass>    Backup, Media, Public (R/W)
    Media        <keypass>    Media (R), Public (R/W)
    pi        <keypass>    Media, Public (R/W)
  *NFS Exports
   ^NFS Exports
    Enable NFSD - On
   ^Exported filesystems
    Path            Access    Options
    /mnt/Asus/Media        *    rw,sync   
    /mnt/Asus/Public    *    rw,sync

Think this configuration simulates what I had before. The David account helped it instantly connect with my David/Windows 10 account to copy. That was the good news. Created directories and simple files and it worked perfectly. Then I copied my first media file with consternation. It paused, asked me for admin rights. Unmounted? It said "unreachable", which I've never seen before. Asked me for admin again. Paused, slow, started, stopped. Drama. After it finished the mess, then it was rock solid and is copying my main TV (/tmp/mnt/Media/My Video/TV) directory. When that's done, I will turn on Plex and Sonarr to see how much they like this.

Figured I'd post a status in case anyone has any known issues they want to warn now, before the voluminous copy takes place.

So far so good. Not sure if it's faster or slower yet. Seems to be going faster, if I had to guess, when it's going.

Advice, help, news, etc. welcome, while I'm doing the conversion. Even thought I might put the NTFS drive into the USB2 port on the hope that the bulk copy might go faster... Staying the course for now, but that's my thought...

Updates as I have them! Thanks!

 

[David Cavalli]

cavalli@Asus:/tmp/mnt# df -h
Filesystem Size Used Available Use% Mounted on
/dev/root 34.6M 34.6M 0 100% /
devtmpfs 124.8M 0 124.8M 0% /dev
tmpfs 124.9M 2.4M 122.5M 2% /tmp
/dev/mtdblock4 62.8M 9.9M 52.8M 16% /jffs
/dev/sdb1 1.8T 3.3G 1.7T 0% /tmp/mnt/Asus

 

[Zonkd]

Done. The specific steps:
1-5. Done, no problem.
6. Step 8, no recommendation on has/!has journal. Chose has journal (because I saw tune2fs can undo that later). Named drive "Asus" on /dev/sda1
7. Rebooted/remounted automatically.

Asus web server configuration:

+USB Application
  *Media Server
  *Network Place (Samba) Share / Cloud Disk
   ^USB Application
    Enable Share - On
    Allow guest login - Off
    Device Name - Asus
    Work Group - <home workgroup>
    Simpler share naming - Yes
   ^<Permissions>
    cavalli        <login>        Backup, Media, Public (R/W)
    David        <win10pass>    Backup, Media, Public (R/W)
    Media        <keypass>    Media (R), Public (R/W)
    pi        <keypass>    Media, Public (R/W)
  *NFS Exports
   ^NFS Exports
    Enable NFSD - On
   ^Exported filesystems
    Path            Access    Options
    /mnt/Asus/Media        *    rw,sync  
    /mnt/Asus/Public    *    rw,sync

Think this configuration simulates what I had before. The David account helped it instantly connect with my David/Windows 10 account to copy. That was the good news. Created directories and simple files and it worked perfectly. Then I copied my first media file with consternation. It paused, asked me for admin rights. Unmounted? It said "unreachable", which I've never seen before. Asked me for admin again. Paused, slow, started, stopped. Drama. After it finished the mess, then it was rock solid and is copying my main TV (/tmp/mnt/Media/My Video/TV) directory. When that's done, I will turn on Plex and Sonarr to see how much they like this.

Figured I'd post a status in case anyone has any known issues they want to warn now, before the voluminous copy takes place.

So far so good. Not sure if it's faster or slower yet. Seems to be going faster, if I had to guess, when it's going.

Advice, help, news, etc. welcome, while I'm doing the conversion. Even thought I might put the NTFS drive into the USB2 port on the hope that the bulk copy might go faster... Staying the course for now, but that's my thought...

Updates as I have them! Thanks!

Great. Once data has copied keep data backup (as you always should) in case the disk has problem with these changes and crashes. Personally I just use MBR ext4 for small disks always plugged into router and GPT NTFS for large disks so I have the option to plug into any pc and rapidly copy gigabytes of media files instead of by network transfer which is always going to be slower.

 

[ColinTaylor]

@David Cavalli Looks promising. If, after a suitable period of testing, the SMB sharing is working for you I suggest that you turn off the NFS and Media servers to save resources. Particularly the Media Server as that can be a resource hog.

 

[David Cavalli]

VERY promising. For some reason, it was a disaster when it first "worked". It made unreasonable pauses, failed and the first 5-10 minutes were bad. Now it seems to be rock solid. Spent all night copying from the old NFS drive to the current Asus ext4 mount.

Best news was that it worked out of the box. My Sonarr copy instantly took root on the mount. All was happy again. All copy / permission issues seem to be gone. The drive seems to be as fast, if not theoretically faster. Yay!

Bad news was that my Samba share from the RPi to my Windows computer has stopped working. So far, I'll assume I've done something to my Pi and work on it on my own for a while. Not sure if that has/had anything to do with the work on the Asus. Don't think so (yet), but will test.

You mentioned that I might not want to use the NFS mount. I'm more than happy to switch it out, but always had problems getting the RPi, but not my Pivos Aios (Busybox media player) to mount the Asus. I think it has something to do with my configuration. I'll include pictures of my web config, my current Pi fstab file and my Aios load script.

This is the failing mount command on the RPi for the Asus.
sudo mount -t cifs 192.168.1.1:/mnt/Media -o ro,domain=Servalan,username=David,password=fakepass /mnt/Media

If I can get this command to mount, then make an equivalent fstab entry, then I should be able to disable NFS for good, and process better.

Thanks for getting me this far guys!

 

[ColinTaylor]

This is the failing mount command on the RPi for the Asus.
sudo mount -t cifs 192.168.1.1:/mnt/Media -o ro,domain=Servalan,username=David,password=fakepass /mnt/Media

You haven't included the vers option that I talked about earlier.

 

[David Cavalli]

AHA! You're right! Forest from the trees... Works with the credential option, but read only. Here's the line so far:

 sudo mount -t cifs //Asus/Media /mnt/Media -o "rw,vers=2.0,credentials=/home/cavalli/.smbcredentials"

The mount works fine and /mnt/Media is readable. However,

pi@PIMC:/mnt $ cd /mnt/Media
pi@PIMC:/mnt/Media $ ls
My Digital/  My Music/  My Photo/  My Video/  zDL/
pi@PIMC:/mnt/Media $ touch a
touch: cannot touch 'a': Permission denied

The .smbcredentials file looks like this:

domain=Servalan
username=David
password=workingwindowspass

The Media account/password is fine for read access on my Aios, because it is just a player (no update/delete). I'm using my David account on the RPi, because it acquires the media and organizes it, so it needs r/w access. I added "rw," at the start of your command and thought the David permissions were the prerequisit for that. Included the David portion of the web page in case that's the issue.

The mount /mnt/Media is working for Plex, so getting real close. Turned off the NFS and the mount is running just fine. Sonarr won't need write access until tonight, so leaving it as-is while playing with other ideas for the write access.

Almost there! Thanks!

 

[ColinTaylor]

Can you SSH into the Asus router and post the output of these commands please:

cd /mnt/Asus/
ls -l
ls -l Media

 

[David Cavalli]

Can you SSH into the Asus router and post the output of these commands please:

cd /mnt/Asus/
ls -l
ls -l Media

Done. Not sure about the '1000' and 'has' portions, but maybe that's what you wanted to see.

ASUSWRT-Merlin RT-AC68U 384.9-0 Sat Feb  2 18:16:52 UTC 2019
cavalli@Asus:/tmp/home/root# cd /mnt/Asus
cavalli@Asus:/tmp/mnt/Asus# ls -l
drwxrwxrwx   20 cavalli  root          4096 Feb 18 09:40 Backup
drwxrwxrwx    7 1000     nas           4096 Feb 18 07:14 Media
drwxrwxrwx    3 1000     nas           4096 Feb 17 14:49 Public
drwx------    2 cavalli  root         16384 Feb 17 08:46 lost+found
cavalli@Asus:/tmp/mnt/Asus# ls -l Media
drwxrwxrwx   16 cavalli  root          4096 Feb 17 19:21 My Digital
drwxrwxrwx   15 cavalli  root          4096 Feb 18 07:14 My Music
drwxrwxrwx    5 cavalli  root          4096 Dec  6 14:03 My Photo
drwxrwxrwx    9 cavalli  root          4096 Feb 17 22:37 My Video
drwxrwxrwx    2 cavalli  root          4096 Feb 17 06:43 zDL
cavalli@Asus:/tmp/mnt/Asus#

 

[David Cavalli]

FYI, when I was recreating the "Media" and "Public" directories, it's possible that I did it in Windows, and NOT in the GUI, which I suspect is potentially important. Didn't know that was a requirement, but with my newfound insight into all these permission issues, I see this possibility! Might create test directories while awaiting the response to see what I can learn on my own...

 

[David Cavalli]

Operated on my theory to no success. Moved data to temp locations. Deleted / recreated Media and Public, to get cavalli/root there. Moved data back in place. Mounted on RPi to no success. Oh well. Tried.

ASUSWRT-Merlin RT-AC68U 384.9-0 Sat Feb  2 18:16:52 UTC 2019
cavalli@Asus:/tmp/home/root# cd /mnt/Asus
cavalli@Asus:/tmp/mnt/Asus# ls -l
drwxrwxrwx   20 cavalli  root          4096 Feb 18 09:40 Backup
drwxrwxrwx    7 cavalli  root          4096 Feb 18 10:39 Media
drwxrwxrwx    3 cavalli  root          4096 Feb 18 10:42 Public
drwx------    2 cavalli  root         16384 Feb 17 08:46 lost+found
cavalli@Asus:/tmp/mnt/Asus# ls -l Media
drwxrwxrwx   16 cavalli  root          4096 Feb 17 19:21 My Digital
drwxrwxrwx   15 cavalli  root          4096 Feb 18 07:14 My Music
drwxrwxrwx    5 cavalli  root          4096 Dec  6 14:03 My Photo
drwxrwxrwx    9 cavalli  root          4096 Feb 17 22:37 My Video
drwxrwxrwx    2 cavalli  root          4096 Feb 17 06:43 zDL
cavalli@Asus:/tmp/mnt/Asus#

 

[ColinTaylor]

FYI, when I was recreating the "Media" and "Public" directories, it's possible that I did it in Windows, and NOT in the GUI, which I suspect is potentially important.

Yes the "shares" need to be created through the GUI otherwise the permissions are wrong as you have seen. But I suspect that isn't actually the problem in this case. Anyway, correct the ownership with the following:

cd /mnt/Asus
chown cavalli:root Media
chown cavalli:root Public
chmod -R 777 *

 

[ColinTaylor]

I can recreate the problem here, I just need to think what the answer is. The problem is that when the share is mounted on the Pi all the files and directories are owned by root. So your "pi" user doesn't have write permission.

 

[David Cavalli]

The chown was already done through the recreated directory recreation. Did it anyway, and also did a chmod 777 on the wildcard, as requested. No success.

pi@PIMC:/mnt/Media/zDL $ touch a
touch: cannot touch 'a': Permission denied

 

[David Cavalli]

I can recreate the problem here, I just need to think what the answer is. The problem is that when the share is mounted on the Pi all the files and directories are owned by root. So your "pi" user doesn't have write permission.

Aha! Ok, good! Not that it's good for me right now, but now the problem is in hand on your side. That will make my wife happy. She wants to take me shopping. Will turn on the NFS in the short term and put it back the way it was until the next round of testing.

Thanks Colin!

 

[ColinTaylor]

OK we'll have to try a couple of things to narrow down what options are available on the Pi.

First try this and see if it works:

sudo umount /mnt/Media
sudo mount -t cifs //Asus/Media /mnt/Media -o "vers=2.0,file_mode=0777,dir_mode=0777,credentials=/home/cavalli/.smbcredentials"

EDIT: BTW the only reason I suggested that you turn off NFS was because I thought you were using it only because you couldn't get SMB to work. If you prefer to use NFS then by all means do so.