Can you add more drives using standard RAID?

[Armornone]

I was wondering if it possible to add more hard drives to an existing raid allowing the array to then incorporate the new drives to the array without deleting your data?

I know that Netgear( XRAID) is able to do this and Drobo( Beyond RAID), but what about standard raid?

The reason I ask is because I read this from Thecus N7700 page " Users can add larger hard disks to available slots and join an existing RAID volume. The N7700+ will automatically rebuild the RAID array and increase storage capacity."

I value the ability to start out with a small system and add larger hard drives as needed over the years vs having to buy everything when I start out. I was wondering if any standard level of RAID or current RAID cards on the market allow you to do this.

I was thinking of building a 12 RAID system but only starting out with maybe 4 drives and just adding more hard drives as I need more space)

Thanks.

 

[Jesse B]

What what I recall, standard RAID systems require rebuilding of the array, but I may be mistaken.

 

[thiggins]

Some RAID NASes allow this. Look for support for RAID level migration and online RAID expansion (or similar terms) in their specs.

When expanding, the RAID volume must be rebuilt and data is at risk while the process is running, i.e. if a drive fails you may lose the volume.

Rebuilds can take overnight or longer depending on the volume size and NAS rebuild speed. So a backup of your essential data is strongly recommended before doing an expansion.

 

[SimonD]

Also to add to Tims post, during the rebuild process access to the array is denied, this can obviously lead to problems depending on the type of data and the required access.

Usually you would add additional disks to an array and create them as an additional lun and present the new storage as a new volume.

One thing to also take into consideration when doing this is the kind of partitioning that's done on the existing array, if you haven't created the partition with the GPT style partition but left it as MBR based then you are stuck with a 2TB limit to the size of the array. If you're expanding beyond 2tb you may well be better off having a large disk (or 2) to migrate the data to initially, rebuild the array (with a gpt partition table) and starting again.

Rebuilding an array (even from a failed disk) has it's potential to destroy all your data, I would strongly advise that you have a complete backup of all your data prior to doing it (SAN\NAS storage is not a replacement for backing up, SAN replication\mirroring can however help the risk of data loss due to hardware failure be reduced).

 

[Armornone]

Also to add to Tims post, during the rebuild process access to the array is denied, this can obviously lead to problems depending on the type of data and the required access.

Usually you would add additional disks to an array and create them as an additional lun and present the new storage as a new volume.

One thing to also take into consideration when doing this is the kind of partitioning that's done on the existing array, if you haven't created the partition with the GPT style partition but left it as MBR based then you are stuck with a 2TB limit to the size of the array. If you're expanding beyond 2tb you may well be better off having a large disk (or 2) to migrate the data to initially, rebuild the array (with a gpt partition table) and starting again.

Rebuilding an array (even from a failed disk) has it's potential to destroy all your data, I would strongly advise that you have a complete backup of all your data prior to doing it (SAN\NAS storage is not a replacement for backing up, SAN replication\mirroring can however help the risk of data loss due to hardware failure be reduced).

How are you suppose to backup a 8TB system without getting another 8TB system? Its not like you can buy a 8TB drive to backup your array. The reason I use raid 5 in these arrays is to provide some protection against having these drives randomly fail. Its one thing to back up a few files but how are can you possible backup your mega RAID system?

 

[Armornone]

Some RAID NASes allow this. Look for support for RAID level migration and online RAID expansion (or similar terms) in their specs.

When expanding, the RAID volume must be rebuilt and data is at risk while the process is running, i.e. if a drive fails you may lose the volume.

Rebuilds can take overnight or longer depending on the volume size and NAS rebuild speed. So a backup of your essential data is strongly recommended before doing an expansion.

How does " RAID level migration and online RAID expansion work" ?

It was my understanding that only customized version of RAID would allow for this such as with Drobo and Netgear/Infrant XRAID type of RAIDS) which were kind of proprietary raid formats unique to these companies.

How can standard raid 5 support this type of expansion unless these other companies have their own modified RAID5 software?

Thanks.

 

[memilanuk]

How are you suppose to backup a 8TB system without getting another 8TB system? Its not like you can buy a 8TB drive to backup your array. The reason I use raid 5 in these arrays is to provide some protection against having these drives randomly fail. Its one thing to back up a few files but how are can you possible backup your mega RAID system?

That... is a very good question, one that has been on my mind quite a bit as of late

Shortly after I started backing up things to my FreeNAS box (three 500GB drives in a ZFS RAIDZ-1 (aka RAID 5) pool, I realized that I seriously underestimated the size/scope of my storage space needs and will probably need to expand again in the near future (i.e. next six months).

Given the mantra of 'RAID is not backup' (which I agree with, btw) I started looking around and trying to understand how these people who have the big fancy 6, 8, 12+ TB RAID arrays are backing that data. There seem to be two camps - a) build a second NAS that is as big or bigger and back up NAS-2-NAS, so not only do you get to 'eat' a drive or two worth of storage for RAID 5 or 6 to protect against one array failing, you get to do it *twice*) and b) only back up the really important stuff, which for most people seems to still fit on a single 2TB drive and write off the other stuff as replaceable i.e. stuff they've ripped (or pirated?) and can do so again.

Personally, the second option seems like an awful lot of work and/or annoyance, and the first sounds like it could get awful expensive real fast. For personal/family stuff plan b might the only workable way, but for soho business needs I'm guessing plan a might be necessary.

Unless anyone has a better option out there?

Monte

 

[thiggins]

Given the mantra of 'RAID is not backup' (which I agree with, btw) I started looking around and trying to understand how these people who have the big fancy 6, 8, 12+ TB RAID arrays are backing that data. There seem to be two camps - a) build a second NAS that is as big or bigger and back up NAS-2-NAS, so not only do you get to 'eat' a drive or two worth of storage for RAID 5 or 6 to protect against one array failing, you get to do it *twice*) and b) only back up the really important stuff, which for most people seems to still fit on a single 2TB drive and write off the other stuff as replaceable i.e. stuff they've ripped (or pirated?) and can do so again.

There are many more backup strategies including archiving to DVD, tape for some or all content.
It boils down to a cost / risk / benefit tradeoff, which only you can make.

 

[thiggins]

How does " RAID level migration and online RAID expansion work" ?

It was my understanding that only customized version of RAID would allow for this such as with Drobo and Netgear/Infrant XRAID type of RAIDS) which were kind of proprietary raid formats unique to these companies.

How can standard raid 5 support this type of expansion unless these other companies have their own modified RAID5 software?

RAID expansion and migration are supported by the Linux mdadm utility most consumer NASes use for RAID volume management.

The "advantage" that Drobo and NETGEAR have with their systems is that they are automatic. The RAID expansion / migration method used by other vendors must be initiated manually.

 

[memilanuk]

Are DVDs really feasible for backup with individual HDD capacity many times the size of an individual disc? If critical backup only constitutes <20GB, would the same amount of money spent on media be better put towards an online storage plan (at far less time and hassle than shuffling discs)?

Does anyone even use tape backup for home use anymore? Not being sarcastic, honestly asking. It seems like with the increase in HDD capacity and low HDD prices that people have gone away from tapes to just having spare HDD space and skipping having to manually shuffle tapes - though you could backup a lot more 'critical' data on one tape than on one dvd...

Monte

 

[thiggins]

Are DVDs really feasible for backup with individual HDD capacity many times the size of an individual disc...

As I said, there are many options. You pays your money and takes your choice.

 

[teknojnky]

How are you suppose to backup a 8TB system without getting another 8TB system? Its not like you can buy a 8TB drive to backup your array. The reason I use raid 5 in these arrays is to provide some protection against having these drives randomly fail. Its one thing to back up a few files but how are can you possible backup your mega RAID system?

that is exactly what you do, if you value your data.

if you think you can't afford to back it up, consider the cost of losing it all.

if you can afford to lose it all, then I guess you can afford to not back it up.

 

[Armornone]

Are DVDs really feasible for backup with individual HDD capacity many times the size of an individual disc? If critical backup only constitutes <20GB, would the same amount of money spent on media be better put towards an online storage plan (at far less time and hassle than shuffling discs)?

Does anyone even use tape backup for home use anymore? Not being sarcastic, honestly asking. It seems like with the increase in HDD capacity and low HDD prices that people have gone away from tapes to just having spare HDD space and skipping having to manually shuffle tapes - though you could backup a lot more 'critical' data on one tape than on one dvd...

Monte

DVD were only feasible for me when backing up songs and pictures back in 1998-2000 time period. Now that I have much my High def video cameras, super megapixel cameras, etc.. DVD seem like they are like floppy disk use to be.

Tape backup technology, is that 70's tech? My company used tape backup maybe 12 years ago until we had our raid servers contain all our valuable company information stored on a 12 disk RAID 0 system by our idiot so called IT manager from some south America country.( Yes, Raid 0, as in 1 disk fails, you have lost everything)
The tape backups never worked! They are about as advance as tape you use to record shows on your VCR. We almost lost the entire company because of that. Since that time, we at least do Raid5 and we threw out the tape equipment and started working a hard drive backup( placed in a form padded, shock resistance case and taken off site)

I have purchased several docking station which sit on your desk and allow you pop in and out hard drives as if they were USB drives. I have a stack of hard drives I put data on. So, instead of having a stack of CD/DVD, I have a stack of hard drives of all different sizes, 250GB, 500GB, 750GB,etc..

I also have a NAS system in a vacation house which I try to backup many pictures and documents in both location. However, trying to keep track of everything is a nightmare. I have tons of duplicate pictures, folders organized differently on each NAS, probably accidentally deleted things I was not suppose to. It is not easy. However, even with all that, its impossible to really backup everything unless you are running RAID 1 or have build 2 identical NAS system and use one as a primary and one has the backup .

I can actually use an unlimited amount of space. I wish I could go into the future and get some type of Petabyte system.

As far as the online storage plan, there is a few problem with that I can see.

1. How must privacy do they give your data? You don't want a bunch of weirdos looking at all your pictures and video do you.
2. Comcast has this data limit now of like 200GB a month. I have terabytes of info I need to backup. I am suppose to spend years and years backup up a little per month because comcast think its God over the internet.


Anyway, backing up is much easier said than done. Realistically, unless you have someone who full time job is backing up, this RAID5 better work!

 

[Armornone]

RAID expansion and migration are supported by the Linux mdadm utility most consumer NASes use for RAID volume management.

The "advantage" that Drobo and NETGEAR have with their systems is that they are automatic. The RAID expansion / migration method used by other vendors must be initiated manually.

Do you know how Drobo and Netgear work? Do they break up data by sector instead of by drive? Like raid 5 on a smaller level, so they can shuffle data across to different places and expand the array?

Second question, when these company say they have raid migration/expansion, what exactly do they mean? You said they use this "mdadm" program, but how exactly does that work within their system?

Do they expect you to go into telnet under the command line interface and try to configure the expansion like this type of thing" mdadm --add /dev/md1 /dev/sdd1 omdadm --grow /dev/md1 --raid-devices=4" or is this incorporated into some kind of GUI interface that you can do with the click on a button?

If you have to type commands to configure the raid expansion, I would rather not deal with it at all and just get a Readynas.

Please let me know

Thanks.

 

[teknojnky]

most consumer linux based nas use mdadm as its part of linux itself

https://encrypted.google.com/search?q=mdadm

normally they provide a web based GUI to manage the device and its arrays, so there typically no need to worry about command line stuff, however web based gui's tend to be fairly simplistic and rarely provide the full capabilities that command lines can achieve.

drobo uses some type of proprietry method, and unless you get a network model, appears as a direct attached storage (DAS) ie a usb/firewire drive.

netgear (at least the x86 models) use linux mdadm based raid, raid works at the block level.
Most normal raid can only expand when you replace all disks with larger ones (one at a time). Netgears x-raid2 achieves a little more flexible expansion by being able to stack 2 raid layars together when using 2 different size disks.


http://en.wikipedia.org/wiki/RAID
http://en.wikipedia.org/wiki/Standard_RAID_levels

there is a literal crapton of information on raid on the internet, and all your questions have been asked and answered by hundreds of other people before you.

 

[Armornone]

most consumer linux based nas use mdadm as its part of linux itself

https://encrypted.google.com/search?q=mdadm

normally they provide a web based GUI to manage the device and its arrays, so there typically no need to worry about command line stuff, however web based gui's tend to be fairly simplistic and rarely provide the full capabilities that command lines can achieve.

drobo uses some type of proprietry method, and unless you get a network model, appears as a direct attached storage (DAS) ie a usb/firewire drive.

netgear (at least the x86 models) use linux mdadm based raid, raid works at the block level.
Most normal raid can only expand when you replace all disks with larger ones (one at a time). Netgears x-raid2 achieves a little more flexible expansion by being able to stack 2 raid layars together when using 2 different size disks.


http://en.wikipedia.org/wiki/RAID
http://en.wikipedia.org/wiki/Standard_RAID_levels

there is a literal crapton of information on raid on the internet, and all your questions have been asked and answered by hundreds of other people before you.

It does not say that Xraid uses Mdadm, It said that its expansion abilities are also possible using Mdadm. Perhaps they accomplished the same outcome using 2 entirely different methods?

QUOTE:
Infrant (now part of Netgear) X-RAID offers dynamic expansion of a RAID5 volume without having to back up or restore the existing content. Just add larger drives one at a time, let it resync, then add the next drive until all drives are installed. The resulting volume capacity is increased without user downtime. (It should be noted that this is also possible in Linux, when utilizing Mdadm utility ( end quote)

How do you know that netgear's xraid uses mdadm for its " dynamic expansion"?

Anyway, Netgear has made millions from the readynas because it allows you to expand in this manner. Do you mean to say that the only thing they did was add a GUI to an open source linux utility named Mdadm?

All the awards, editors choices, 5-stars reviews the readynas has received is just for a GUI on the linux utility mdadm ?

I sure hope it is more too it than that.

I wonder how long until Buffalo, and the rest of the NAS device makers catch on ?

 

[SoftDux-Rudi]

I can't vouch for Drobo or Netgear, but I dot have some Thecus 8800PRO's in the office.

I setup the one with 2x RAID 6 arrays with 4 drives each - to see how this works. At any given time I could take out a drive and put a new one in (either replacement, or larger) and it would automatically rebuild the RAID array. If the drive was bigger, then I would need to migrate the RAID set to enlarge it. This is called "online migration" and the raid array (as well as shares and NFS mounts) is still accessible while doing this, but the system is sometimes a bit slower.

For example, I had 4x160GB HDD's in and migrated them one-by-one with 250GB HDD's, which took me about a day or so todo and we could still access data on the system.

Then, later on I destroyed the one RAID set and added the empty drives to the other RAID set, so now it has 8x drives setup in RAID6 - and there's 1x 80GB, 3x 6GB, 2x 500GB & 2x 1TB drives in the set. This isn't ideal, I know, but I wanted to see if this was indeed possible. The whole raid set was available throughout the migration process. I'll soon be migrating them all to 1TB drives.


Re: your question about how you backup 8TB data - RAID is not a backup solution. At all! RAID is merely an option to give you protection against hard drive failure, and allowing you to have one larger volume from a few smaller drives. If you only have one copy of your data, irrespective if it's on your PC, on a $1000 NAS or even on a $100000 SAN you still need to make a 2nd copy as backup if you value your data. Even on those multi-million dollar SAN's you could loose your data if you don't back it up. In this case your best bet is to get the cheapest NAS solution available, and then get another one to backup this one.

 

[teknojnky]

It does not say that Xraid uses Mdadm, It said that its expansion abilities are also possible using Mdadm. Perhaps they accomplished the same outcome using 2 entirely different methods?

The article in question refers to x-raid (the first generation, sparc based devices). This used a customized hardware controller of some type along with whatever netgear programming/software customization.

X-raid2 uses mdadm along with readynas specific scripting/tweaking/kernel as has been posted by netgear engineers on the readynas forums.