Backing up your NAS/Media Collection

[MoralDelima]

So you've just spent $900 on a TS-509 and another $400-$530 in 1-1.5TB hard drives (tack on another hundred if you were smart and got a spare drive) and you're all set up with three terabytes of media (BD isos, lossless music, etc). But everyone says you can't depend on your RAID5 rig - you need a backup.

So how do you securely backup 3TB worth of files? Maybe if you're a business you can turn to some really exspensive enterprise solution, but if you're a home power user, what do you turn to? A BD burner ($250) and $672 in 25GB BD-R discs? A 4.16TB Tape drive for $2,699.99? Upload it all to some online storage depot (how do you even get 3TB worth of hosting without paying a fortune and a custom hosting solution?) over your DSL 400KB upstream or 2MB cable upstream? Take it to a colo and ask for a rate to store it in their data center? Build ANOTHER Raid 5 array?

How do you back up your multi-terabyte data? What's the most cost effective way?

[[EDIT: Sorry, should have gone in the General questions section - can a mod please move? ]]

 

[valentin]

It seems the only possible way to do it at home is to have another NAS to backup to, or if your NAS allows - to attach to it external drives for backup.

So you would have been better with buying two cheaper NASes (even w/o RAID) instead of one expensive NAS.
________
BUY VAPORGENIE

 

[thiggins]

It's actually a very good and real question with no easy or "right" answer. It's also one of the reasons I advocate RAID 1 vs. RAID 5. With today's large drives, you can get a lot in a box and have money to buy two RAID 1's and have them mirror than one RAID 5.

Online storage can be an option, actually. Some online services like MozyHome offer "unlimited" backup for not a lot ($5/mo. for Mozy Home). First backup will take awhile, but they run in the background.

 

[MoralDelima]

It's actually a very good and real question with no easy or "right" answer. It's also one of the reasons I advocate RAID 1 vs. RAID 5. With today's large drives, you can get a lot in a box and have money to buy two RAID 1's and have them mirror than one RAID 5.

Online storage can be an option, actually. Some online services like MozyHome offer "unlimited" backup for not a lot ($5/mo. for Mozy Home). First backup will take awhile, but they run in the background.

The only problem with this method of backup is that you don't really have control of your files. I have hosting in a few places and backed up ISOs I had made of software/movies I would like to keep. I logged in one day to find that my files had been deleted, blah blah blah piracy. Never mind the fact the directory wasn't reachable for the web. Of course, MozyHome uses 448bit encryption automatically, but still. And no one guarentees your data - if they accidentally have a glitch and your backup was corrupted, oh well.

I know I'd be extremely pissed if I logged in after taking the time (Assuming a 1MB upload stream, which is fairly generous considering DSL usage and is also the cap that MozyHome will allow you to upload at, if I'm getting good Google Info) it would take 30 day(s), 10 hr(s), 4 min(s), 26.67 sec(s) to upload, doing nothing but maxing out your connection uploading. If you had a DSL connection (say, 6000KB/400KB like mine) it'd take you 69 day(s), 12 hr(s), 44 min(s), 26.67 sec(s). source. 70 days of straight uploading, no torrenting, no gaming just straight uploading. Good QoS would help, but you'd be taking a major hit to performance and be left swinging in the wind if anything happened. And then you'd have to re-download it if somethign did happen, which would take at least 6 or so days, ASSUMING you could get maximum sustained download rates from MozyHome. Actual time to get your info back would probably be much longer.

I guess Colo 'bring in your own rig and transfer over eSata' would be best, but you'd still have to worry abour encryption and the like, and not having control of your data.

 

[thiggins]

As I said. There is no easy answer...

 

[00Roush]

How important is your data to you?

As thiggins mentioned RAID 1 provides a bit more data security than RAID 5. RAID 10 would provide better performance with the higher data security but I don't think the TS-509 supports that mode.

I think the cheapest solution would be to see if you can't use a eSATA (or USB?) 2 drive enclosure with the TS-509. Then see if you could use two 1.5 TB drives to create a backup. To be safer you would want to have these drives connected to a separate computer and not the TS-509 as you would then have less risk of a complete failure.

Another option would be to build a very cheap backup NAS box. Should be able to build one for under $200. Use Freenas as the OS and with a couple of 1.5 TB drives you could backup that much data. Well strike that... You could use something like a D-link 323 which is all ready to go and might cost less.

Basically it really depends on how safe you want you data to be and how much of it. (and how much you want to spend)

I would think it starts with making your primary disk setup safer by using some type RAID 1 setup. Then it would go to using external drives. Next a completely separate backup box. Last would probably be an off site backup (Mozy?), optical backup or tape backup, in combination with a separate backup box.

Just throwing stuff out there that I might consider if I was in your position.

Like thiggins said, "There is no easy answer..."

00Roush

 

[MoralDelima]

I might consider doing a Raid 5 primary usage box and then having a Raid 0 2x1.5TB backup box if it weren't for the fact that the 1.5TB hard drives are entirely unstable and unreliable, even after the firmware update. Maybe I will just wait until the 2TB comes out.. don't know, the 1TB Hitachis are on sale for $80 after rebate right now.

 

[00Roush]

I might consider doing a Raid 5 primary usage box and then having a Raid 0 2x1.5TB backup box if it weren't for the fact that the 1.5TB hard drives are entirely unstable and unreliable, even after the firmware update. Maybe I will just wait until the 2TB comes out.. don't know, the 1TB Hitachis are on sale for $80 after rebate right now.

I don't think you would want to do RAID 0 on the backup box as this would mean if one drive failed all of the data on the backup would be lost. JBOD mode would be what I would use. Then if one drive failed you would at least have half of the data.

Just my opinion though.

00Roush

 

[Madwand]

Build ANOTHER Raid 5 array?

You can debate about RAID 0 or JBOD or what have you as the backup, but when you think about it, adding another drive for a separate RAID 5 backup is better.

 

[MoralDelima]

You can debate about RAID 0 or JBOD or what have you as the backup, but when you think about it, adding another drive for a separate RAID 5 backup is better.

You mean taking it from RAID5 to RAID6? Because I couldn't just add one drive to a seperate RAID 5 backup, I'd have to add at least three drives, an enclosure, etc...

 

[Madwand]

No what I meant was that instead of just the RAID 0 / JBOD backup some others have suggested, add another drive to that and make it a full RAID 5 backup.

Of course this is in addition to a complete computer, OS, and ideally gigabit networking and isn't the cheapest or simplest option to set up, but when your primary storage gets large, you run short of options.

Hard drives are the most economical way to store large amounts of data, and so the question becomes how will the hard drives be organized / accessed. I say that a separate server is best.

The cheapest alternative would be an eSATA/USB enclosure/dock such as the Vantec or Thermaltake. For this, you'd have to break up the backup into parts which could be stored in separate drives and mess around with manual drive swapping. This is far from ideal, hence the separate server recommendation, but a dock is still useful for smaller backups and one-off operations when you're careful with handling.

Ref: http://www.vantecusa.com/front/product/view_detail/295

There is another alternative -- an external eSATA port multiplier enclosure running a RAID or JBOD setup. I don't have any experience with this class of device though.

 

[MoralDelima]

stuff

Custom NAS Box
=========================================================
HP Procurve 1800-24G

ASUS P5BV-C/4L LGA 775 Intel 3200 ATX Server Motherboard - $209

LSI LSI00005-F 64-bit, 133MHz PCI-X SATA II MegaRAID 300-8X Kit 8 Port SATA/300 128MB RAID 0/1/5/10/50 - Retail $399.99

Athena Power CA-SWH01BH8 Black Steel Pedestal Server Case 2 External 5.25" Drive Bays - Retail $241.99

Of course, you'd have to toss in some ram (FBDIMMS, $90 for 2x 2GB) and a proc (2.0ghz dual core for about $70 if you were cheapo). Other then that, slap in 8 or so drives, configure a raid 5 setup with a linux distro and then hook up all four ports to your Procurve and away you go with Link Aggregation. If the chipset supports it, which I can't find but it'd be retarded if it didn't.
==========================================================


DAS Rig 1 (Multilane)
======================
MST4ML-B = Mini Storage Tower with 4X Multilane interface, black - Connects up to 4 drives to system via 1 Infiniband Multilane 4X cable. Includes 4 Internal SATA data cables (AASAIDC18I), power cord, 4 4P-15 pins SATA power connectors (AA4PSAPC), mounting screws, and user guide. - $129.95

4x Multilane bridge for SATA controller = Designed to convert 4 internal SATA ports to external multilane connector for Infiniband cabling system. Support connection of 4 devices via 1 cable. (model: AD4SAML-PCI) click to check price and shop online - $29
More Info...http://www.addonics.com/products/multilane/connector.asp

Hard Drives.

Drive Bay: 3 x 5.25Bay for 4 x 3.5" HDD hot swapable. $99.99 http://www.newegg.com/Product/Product.aspx?Item=N82E16816119011

AAIB4C150 1Infiniband 4X MultiLane eSATA cable, 150 cm $59.95

areca ARC-1210 PCI-Express x8 SATA II Controller Card RAID 0/1/1E/3/5 JBOD - Retail $299.99
http://www.newegg.com/Product/Product.aspx?Item=N82E16816131003

$619.87, Max of 4 Drives, must use 1.5 TB to get max cap ($4190.95 GB). 1TB cap results in 2793.96 useable GB. Highest possible transfer rates? Not portable in any sense.
==========================================================

Those are two builds I'm considering. They're fault tolerant up to a point - but if I get some data corruption going on or a write hole screws me, then I lose everything - these setups are geared towards speedy file serves first. How do you back up your three terabyte NAS??

That's what I'm trying to figure out - after you've already built a nice box for a grand and a half, how do you preserve your data further?

 

[bliko]

I used to have a massive media collection, however after a couple of years, i decided that its a constant up hill struggle that can't be won on a budget. I had to constantly buy disks + pay for power usage).

Now I just download, keep for a month or two then delete. If i really like the movie I may keep it, however my internet connection is fast enough to be able to pretty much download anything through bt and watch it within a couple of mins.

After giving up on a home server, but before giving up alltogether I got together with a couple of friends and rented two dedicated servers with 12X500GB hdds each (one for backup). We used it to downlaod and store movies. We all connected to it via vpn or ssh and streamed movies from it as in when we wanted to watch. It was EUR 500/month and between 9 of us (~55 EUR each) - ~£38 for me at the time... about the price of a high-speed internet connection. I have to admit, this arrangement was pretty sweet, however after about a year people started 2 pull out for one reason or another...so it wasnt sustainable.

I think the most cost effective solution is to use the internet as your backup. Just have a bunch of disks not in a raid array. Download your stuff to them...use a cataloging progrm of your choice to keep track of whats on the disks...if a disk fails...so what?...just download it again. If there are some movies which you have spent ages trying to find which have like 2 seeds, and you think may not be available again...back those few up to an esata single drive.

If there is anyone here who has genuinely bought serveral TB worth of dvds and blu-rays, i dont think the cost of a backup solution would be such an issue in comparison to the cost of the movies.