QTS 4.2.2 disk expansion

[RMerlin]

I will be upgrading a customer's pair of 1 TB HDD (in RAID 1) to 3 TB in the coming days, on a TS-259Pro.

Has anyone done the process? I'm wondering if the NAS's services are available during the process? QNAP's documentation does not mention it. One user said the NAS's file services weren't availalable, but he didn't mention if the unavailability was during the disk swapping process, or during the final expansion process. I would expect the data to be unavailable only during the final expansion, since the first two stages (disk swaps) are merely RAID rebuilds.

If it's only during the expand process, then I could just start the last step remotely at the end of the day, after the two disks have been swapped and resynced.

Thanks!

 

[sfx2000]

That's a very good question...

QTS uses LVM under the hood, and MDADM to manage RAID sets - my best guess is that the Expansion Chassis would gather it's disks as a separate RAID set, adding it to the LVM as a separate PV set, and then either create a new volume group...

I'm thinking maybe it could do it hot, but I would definitely reach out to QNAP support, and plan this work in the evening when the NAS isn't in use...

Sounds scary - make sure a full backup is done...

 

[RMerlin]

That's a very good question...

QTS uses LVM under the hood, and MDADM to manage RAID sets - my best guess is that the Expansion Chassis would gather it's disks as a separate RAID set, adding it to the LVM as a separate PV set, and then either create a new volume group...

I'm thinking maybe it could do it hot, but I would definitely reach out to QNAP support, and plan this work in the evening when the NAS isn't in use...

Already opened a support ticket with them asking about it, will let you guys know what they answer (unless someone else has the answer).

My personal feeling is that, probably, the NAS stays available during the disk syncs (when you swap each disks, one at a time), but I'm really not sure about the expand section. If it's purely done through LVM, then the expand shouldn't bring the NAS down for more than a few minutes. However reports seem to indicate that this expand process can take multiple hours, so I'm not quite sure how they actually accomplish it.

Sounds scary - make sure a full backup is done...

Backups are run daily, so it's not a problem. The disk swaps would be done in the morning, after the backup has been made, provided of course that the NAS services to stay up during the disk syncs. Otherwise, I might end up just sharing their backup HDD from a PC as a read-only share for the duration of the process. This is an office of only around 10 users, and out of those 800 GB currently used, a few hundred gigabytes are backups taken from their Kerio Connect server + the network recycle bin.

Their "active" data is perhaps 100-200 GB at the very most (lots of photos taking space as well - they are architects/consultants in accessibility). Having them work off a read-only backup for a few hours is a reasonable alternative (been working with them for 18 years now, so I'm quite familiar with how they work).

I wish I had a few spare disks in the workshop to test it myself (I have a spare TS-669 Pro in stock, to be used in case of a hardware failure for any of my QNAP-based customers).

 

[sfx2000]

One thing to be careful of - I think it might be easy to get into a JBOD situation (the original RAID as one disk, the expansion as another) if not careful - if I recall there is an article on the main site that discussed this topic from a data recovery perspective some time back...

 

[sfx2000]

Already opened a support ticket with them asking about it, will let you guys know what they answer (unless someone else has the answer).

My personal feeling is that, probably, the NAS stays available during the disk syncs (when you swap each disks, one at a time), but I'm really not sure about the expand section. If it's purely done through LVM, then the expand shouldn't bring the NAS down for more than a few minutes. However reports seem to indicate that this expand process can take multiple hours, so I'm not quite sure how they actually accomplish it.

That makes sense - if one looks at a four-bay, one can do a hot "rolling" update by swapping out one disk, and let the array sync, and then rinse/lather/repeat, but that can take a very long time...

But I'm thinking with the expansion chassis, storage manager should recognize the chassis and drives, and once initialized, it should let you add the chassis to the storage pool - this is one of the areas that QNAP has made quite a few changes in 4.2.2 that might not be reflected in their online docs...

Hopefully support has a good "Method and Procedure" document that outlines how to do this (and how to revert if things go wrong, which does happen from time to time).

 

[RMerlin]

But I'm thinking with the expansion chassis, storage manager should recognize the chassis and drives, and once initialized, it should let you add the chassis to the storage pool - this is one of the areas that QNAP has made quite a few changes in 4.2.2 that might not be reflected in their online docs...

Hopefully support has a good "Method and Procedure" document that outlines how to do this (and how to revert if things go wrong, which does happen from time to time).

I won't be using an expansion chassis, I'm replacing the two existing disks with two of larger capacity, using QNAP OS's Online RAID Capacity Expansion:

http://docs.qnap.com/nas/4.1/Home/en/index.html?raid_management.htm

This also means that, in addition to the USB backup, the first disk I will remove will also contain a complete copy of all the data (it's a RAID 1).

 

[sfx2000]

I won't be using an expansion chassis, I'm replacing the two existing disks with two of larger capacity, using QNAP OS's Online RAID Capacity Expansion:

http://docs.qnap.com/nas/4.1/Home/en/index.html?raid_management.htm

This also means that, in addition to the USB backup, the first disk I will remove will also contain a complete copy of all the data (it's a RAID 1).

Yep, and that's a much easier solution - but being a two disk unit, I would expect to basically down the unit during the operation - just to keep things simple... hence the recommendation to do this during evening hours when folks are not actively using it.

 

[RMerlin]

Yep, and that's a much easier solution - but being a two disk unit, I would expect to basically down the unit during the operation - just to keep things simple... hence the recommendation to do this during evening hours when folks are not actively using it.

Not really an option. The length of doing this during the evening would basically cost them more in service than buying a new NAS (the process will take many hours). That's why I need to know if it brings the whole unit down or not. If it does, I'll either make them work off a read-only USB disk, or do a complete restore from a backup on the new raid array. A lot of the data does not need to be restored since they're backups from their mail servers.

 

[sfx2000]

That's a good approach - do a full backup, copy that over to a read-only image, that'll keep them up and running...

Bears repeating that Backup Station just backs up data - not the configuration - that's another file in another location...

Shutdown the unit - pull the old drives and set them aside (noting which bay is which) - turn the NAS on, insert one drive at a time, letting the NAS find it and start building the RAID set - and then once that's done, do the restore on configuration and data...

That way, if something goes wrong, you still have the previous disks (and RAID set) and configuration files - so restoring back to a known_good configuration should be easy..

Did you hear back from QNAP Support yet?

 

[RMerlin]

QNAP's tech support just came in.

1) NAS remains available during the disk replacement + resync
2) NAS will NOT be available during the filesystem expand, as the filesystem will need to be unmounted for this operation (I found a forum post where someone mentioned that the process involves running fs2ck and resize2fs based on his observations)

So I'm looking at two quick on-site calls to swap disks and initiate the resync (they're not that far from my home, so I can stop by on my way to the office), then I will remotely launch the filesystem expand either at the end of a day or the end of a week (latter would probably be safer).

Tech support also mentioned what to do in case the filesystem expansion fails: reboot the NAS, it should try again at boot time.

A pretty complete answer overall. I'll post the results in the coming 1-2 weeks once I've completed the operation.

 

[RMerlin]

I just completed the volume expansion. It took about 20-30 minutes to do the expansion portion of the process, going from 1 TB to 3 TB.