Best Practices? Migrating from WHS, need advice what to buy

[symboint]

I posted this over at the Synology forum but have not yet received a response. Perhaps some people here in this forum could help me out?

----

Hi -- I currently have a very small business where we have a Windows Home Server (WHS) acting as the main backup server and fileserver. We also use it to archive all old projects, for easy recall later on. For the number of computers we use, it works pretty well (less than 10 on the network). That server has folder duplication (so the shares are mirrored) and then all that gets backed up to an external fireproof hard-drive, so we have multiple copies. This has been a pretty good solution for us so far.

However, due to the death of Windows Drive Extender (Microsoft canceled this feature in the next version of Windows Home Server, which was one of the dumbest decisions I've seen in this market), I've decided to slowly migrate off this server onto one or more NAS devices -- no need or desire to invest any more time or money into Windows Home Server. The biggest thing that bothers me about Windows Home Server (besides the fact that Microsoft canceled Drive Extender), is that if the boot drive fails, then the whole server is down and needs to be re-installed from scratch. THAT is a major pain in the neck for me. Right now, the server occasionally behaves a little strangely and also has inconsistent performance, so all those things make me a little uneasy and I know have to take some action.

Synology jumps out as the obvious candidate to replace the WHS box. I've also looked at QNAP, but I like the Synology software better, so I'm pretty sure I'll be buying Synology.

So, this is what I need to figure out, and I'm hoping some experts here can help. I don't consider myself an IT expert, so any advice you can give will be greatly appreciated.

First, there are three classes of data we have:

1) Daily computer backups (often an image of the boot drive of various computers on the network)

2) Active project data (shares which hold tons of video, images, documents, etc.) so we need a lot of space

3) Long-term ARCHIVE data (which is data we never want to delete, like old projects and documents), also need ton of reliable, cheap, long-term space, but optical media isn't large enough any more and is too much of a hassle. So we're sticking with copies of archive materials on hard drives. Plus, it's quite common that we need to resurrect a project from 1-2-3 years ago and bring it back online quickly, so hard drives make sense.

So, what type of configuration should I get, and what type of migration plan should I implement?

I am not opposed to getting 2 separate NAS boxes for different tasks.

For the tasks above, 1+3 don't need high performance, since they are most often one-way storage and backups/archives can happen unattended.

For task 2, I would like slightly faster performance since we access that data frequently.

So I was thinking of getting two NAS boxes. One for tasks 1+3 and one for task 2. This could be part of the "migration" plan so I could buy one NAS box now, and migrate one or two tasks off the WHS, and then buy another NAS box later to finish the transition.

Now, I already realize I need to backup both NAS boxes if I buy two. So I'm wondering what the best approach would be? Should I use something like rsync and just sync the two boxes together? Or should I use an external USB drive as the NAS backup? A combination of the two? And frankly, all that data needs to eventually go on one or more fireproof hard drives for extra protection.

Thoughts? Recommendations?

Can I do all this on one larger NAS? Or should I get just two smaller ones?

Also, I'm not convinced that RAID-5 is a good solution. It seems like RAID 1 or RAID 6 would be better for my case for data protection... with RAID 5, one drive failure would be dangerous and put me at risk until I rebuild the array. I'd rather minimize the impact of a single drive failure by doing straight mirrors or at least allowing for two points of failure (RAID 6). ..... OR should I not worry about using any RAID 1, 5 or 6 AT ALL if I'm planning on rsyncing, or using external USB drives as backup? Why not skip RAID altogether and just do single drives, as long as I have backups/rsyncs?

Also, do you have any suggestions of how to gradually migrate off my current WHS system... maybe do it in steps? Move one task at a time to one or more NAS boxes?

Finally, keep in mind this is a very small business, which means I need to control the costs carefully! I wish I had a large budget, and I'd just buy 3-4 big NAS boxes for different tasks, but I have to keep things consolidated, simple and cost-effective.

Thanks in advance for any advice you can share.

 

[thiggins]

You didn't say how much storage you are looking for. That would help recommendations.

A lot of what you are trying to decide is personal preference. Here are mine:

- Unless you have frequent need to expand storage, there really isn't a
pressing need to migrate off WHS. And there is no reason for you to lose
Drive Extender; just don't upgrade.
- Don't use RAID if you don't have to. It just gives you a different set of hassles
(than WHS).
- If the NAS is primary store, it needs to be backed up. If it is backup storage, it doesn't need to be backed up, as long as you're ok with the time it takes to redo full backups of the clients it serves.
- Always have your NAS on a UPS.
- Backup speed to USB 2 drives will be around 25 MB/s maximum. eSATA or USB 3 backup can be > 2X that. Rsync will be somewhere in between. Check the NAS Charts.

 

[symboint]

Thanks, Tim!

Storage needs are:

2-3TB for daily backups of all networked computers. I'm okay with slower access speed for backups. (Also wondering about a good, intelligent incremental backup solution. WHS uses incremental, single instance storage for backups, so I might need more space for this if I move off the WHS.)

1-2TB for "active" projects at decent access speed -- lots of small, medium and large files, moving around all day.

Once the projects are complete, I compress them and move them into the...

3-5TB long-term "archive" which can be slower access speed.

The above space requirements should suit us fine for this year. I can add hard drive(s) to increase capacity next year if needed.

Total size needed: guessing 6TB - 10TB estimate by end of year. Shouldn't be more... could be less.

Right NOW, the WHS we currently use is going to get a little crowded fairly soon with 6.5TB total capacity.

Like I mentioned in my OP, this server is making me feel a little uneasy due to some inconsistent performance, occasional strangeness (could be normal with WHS), the lack of protection of the boot drive (which will bring the whole server down for a while if it dies) and since there won't be Drive Extender in the next version of WHS, there's no incentive for me to keep feeding/upgrading this machine. I might as well let it live its life and die in peace without upgrading it. I'm also out of SATA ports for further expansion of the WHS, which means I need to install a compatible SATA card, etc... again, that's putting more time and money into this WHS device, which is basically EOL for me.

So, I COULD continue to use the WHS for backup purposes for another year or so (since the WHS backup single-instance storage is actually pretty decent), but then set up a Synology or QNAP NAS for the "active" and/or "archive" data.

If I did it that way, I really only need 1-2TB for "active" projects (decent access speed) and 3-5TB for long term "archives" (slow access speed), each with their own, separate backup drives.

All that space could be split into 2 NAS devices, or should I put it all into 1 larger NAS device?

As for backups of the NAS boxes, my plan was to continue to backup to external USB drives... but if I buy 2 NAS units, I could perhaps rsync them together, so each serves as a backup of the other?

Thoughts?

- Don't use RAID if you don't have to. It just gives you a different set of hassles
(than WHS).

BTW, I'm very curious about this statement. I'm inclined to agree with it on the principle that my NAS unit(s) would be backed up or mirrored anyway without RAID coming into the picture. I definitely want to keep this as simple and streamlined as possible. I'd also like to "spread out the risk" so that one single drive failure doesn't risk taking down EVERYTHING. So are you saying that RAID 1 or 6 wouldn't really be helpful for my situation? I've pretty much ruled out RAID 5 due to very little benefit for our needs.

 

[symboint]

More thoughts, possible scenarios below. Would love any thoughts/recommendations.

Option 1:

a) Keep WHS for computer backups.

b) Buy faster single-bay or dual-bay NAS for "Active" projects (1-2TB is enough) - Possible models: Synology DS111, DS211, DS211+

c) Buy slower dual-bay or quad-bay NAS for "Archive" (3-5TB) - possible models: Synology DS211j, DS411j

d) Buy external USB drives to backup b) and c).


Option 2:

a) Keep WHS for computer backups AND archive (5-8TB) - will need to upgrade WHS with compatible SATA card since I've run out of free SATA ports.

b) Buy faster single-bay or dual-bay NAS for active projects (1-2TB)

c) Buy external USB drive to backup b).


Option 3:

a) Keep WHS for computer backups

b) Buy faster quad-bay NAS for active projects - DS410, DS411+

c) Buy slower quad-bay NAS for archive - DS411j

d) create shares on both b) and c) that serve as rsync mirrors of each other for backup purposes.


Option 4:

Same as Option 3, but ditch the WHS and move the backups and archive to the slower NAS (DS411j)


Option 5:

a) Buy a 5-bay NAS to do it all -- DS1511+? If so, how would I configure this to take into account the three classes of data I have?

b) Buy various external USB drives to backup the NAS.

 

[thiggins]

BTW, I'm very curious about this statement. I'm inclined to agree with it on the principle that my NAS unit(s) would be backed up or mirrored anyway without RAID coming into the picture. I definitely want to keep this as simple and streamlined as possible. I'd also like to "spread out the risk" so that one single drive failure doesn't risk taking down EVERYTHING. So are you saying that RAID 1 or 6 wouldn't really be helpful for my situation? I've pretty much ruled out RAID 5 due to very little benefit for our needs.

With RAID 5 or 10, a single drive failure will not take down the server. The server should continue to run as you replace the failed drive and then start a resync. However, until the array has completely resync'd a second drive failure will kill the volume.

With RAID 6, you can have two drive failures, but the same caveat applies. Any volume that is resyncing is a volume at risk.

RAID volumes have been known to go missing entirely for no good reason. Not an every day occurrence. But it happens. Again, backup is essential because RAID is not backup.

Backups on separate devices are essential for any data you can't afford to lose.

With two sync'd NASes (and frequent resyncs) you have the option of switching to a second NAS in case there is a catastrophic failure of the first (main board, power supply, theft). With attached drive backup, you'd need to do a restore first, or share it via a networked computer.

Since your requirements are > 2TB, you're going to need RAID since 3 TB drives aren't yet common. RAID 1 doesn't buy you anything since it doesn't add storage and you'll be backing the NAS up. Choice of RAID 5, 6 or 10 depends on how many drives you want to throw at reliability vs. the storage you achieve.

For cost effectiveness, I'd go with a four or 5 bay primary NAS and widen your scope to look at vendors other than Synology / QNAP, who charge a premium for a boatload of features that you're not going to use. NETGEAR also usually gets a premium for its ReadyNASes. But you may be able to find a deal in older models.

The advantage of a NETGEAR as primary NAS is that it can back up to virtually anything, FTP, rsync, even SMB shares. So it would back up to the WHS machine nicely, or even an old desktop/nettop with a bunch of drives stuck on it.

 

[symboint]

Thank you again, Tim!

I agree completely on the RAID backup issue, etc...

I will widen my scope to look at other vendors and look at Netgear. You're right that I won't be using most of the features of Synology and QNAP -- I had just narrowed it down to them due to good reviews. I preferred the Synology software from the demos/reviews I saw. But in the end, I need something simple, reliable.

Re: RAID levels, thank you for that clarification, and I agree, it all makes sense. Re: RAID 5 -- my concern (which you confirmed) is that the system is at greater risk when rebuilding after one drive failure. RAID 6 (or anything with more points of failure) definitely seems worth the expense if I'm going that way... or just stick with single drive-sized volumes/shares with no RAID at all. Either way, I need a separate backup.

As for my RAID needs, does it make any difference if I don't mind if a volume/share is limited to the size of a hard drive? 2TB chunks are fine with me, so maybe I don't need to use RAID at all. See my other post above with 5 options if any of those make sense, knowing that I can live with 2TB chunks.

 

[thiggins]

If you can live with a volume per drive, then make sure that the NAS you chose supports this mode, which Synology and QNAP both do. NETGEAR by default comes set to XRAID, which combines all drives into a RAID 5ish volume. But you can reset to normal RAID mode and, I think, use the drives individually. Volume per drive is the most space efficient.

 

[symboint]

Thanks again, Tim! Your help here has given much food for thought! Very much appreciated and I owe you a drink!

 

[symboint]

Quick update: I've been reading the manuals of various NAS devices, and I've been very surprised by the flexibility of the backup feature of the ReadyNAS devices. Does anyone have any thoughts/experience on the ReadyNAS backup capabilities? How stable, reliable they are? All those features are pretty amazing -- do they actually work?

For example, I've been surprised to see that the ReadyNAS can backup FROM some other external source (such as a remote share on the network, or even FTP server!) TO pretty much any destination you can think of... which could theoretically make the ReadyNAS the best NAS for backup purposes.

With the ReadyNAS, I could theoretically backup the ReadyNAS to my WHS (and vice-versa!)... which from my readings, nothing else on the market can currently do. That's pretty amazing flexibility... if it actually works.

I could also theoretically have the ReadyNAS backup a remote web server via FTP if I wanted to.

Anyway, if anyone has experience with things like this, I'd love to know your opinions if they actually work.

 

[thiggins]

Net gear backup does work. I used an NV+ for a few years that reliably backed up to and from multiple SMB shares multiple times per day.

The newer Iomega ix2 and ix4 also back to to SMB shares.

 

[symboint]

Thanks, Tim! I checked out the iomega offerings but I'm honestly scared off by the lousy reviews.

However, the more I read about the ReadyNAS Atom-based units, I think I've found the ideal combination for what I need. The extraordinary backup capabilities really shine, and make them much more valuable to me in terms of usable capabilities and bang for the buck. The idea that it can back itself up to FTP or other remote shares on the network (and vice-versa) is awesome. I wouldn't have to attach a USB drive to it, and it would still take care of itself.

I'm quite surprised that Synology does not allow this yet. Synology allows for rsync, yes, but not backup to a remote share or FTP.

From the ReadyNAS manual, I can also see how you can use Flex-RAID and create individual volumes per drive if you want to... so that makes it about perfect. Synology seems a little more flexible in that regard, but the ReadyNAS should still do the trick. The performance of the Atom-powered units isn't quite to the level of the comparable Synology, but is more than adequate for what I need...

So, at this point, I'm heavily leaning to ReadyNAS.

 

[symboint]

For the record, this is what I decided to do:

1) We will transition from WHS over the course of this year to TWO NAS servers... 1 ReadyNAS 4-bay and 1 Synology 4- or 5-bay.

2) I just purchased the ReadyNAS Ultra 4, looking forward for it to arrive. I went with ReadyNAS first since it has the ability to backup to different remote shares on the network, including to a share on the WHS. So we'll have back-and-forth backups between devices (in theory, I haven't set this up yet). The ReadyNAS will also backup to an external USB drive. So we'll have three copies of everything.

3) In a few months, we'll buy either the Synology DS411+ or 1511+ as the final replacement for the WHS. If the ReadyNAS Ultra 4 goes well, I might be open to buying the ReadyNAS Ultra 6 instead. But I think spreading the duties out over two vendors is probably better.

4) We'll let the WHS gracefully die by the end of this year, or simply phase it out, as we get the ReadyNAS and Synology working together perfectly.

5) The final configuration will consist of two separate backups running to both NAS devices at different times, rsyncing between the two for various shares (active projects), plus external USB backup. Minimum of three copies.

Thanks to everyone here for their help -- in this thread and in other threads. I really appreciate this website for all its valuable information. A fantastic resource!

 

[thiggins]

Thanks for reporting back. Good luck with your transition.

 

[symboint]

Thanks, Tim!

Quick update -- I got sick and tired of waiting for the shipment from Amazon to arrive. I can't believe how long it was taking just to ship! Not impressed at all. Anyway, while I was waiting I had more time to do some research on the ReadyNAS and I got cold feet. The more reviews I read, the less excited I was... and the more encouraged (read: "brave") I felt to build my own server. I realized that what I needed would fit Ubuntu or FreeNAS perfectly, and I liked the idea of buying each part and configuring the system myself (your mileage may vary!). Ultimately, I just got fed up with Amazon and canceled the order for the ReadyNAS.

So my plan is the same, except I decided to roll my own server for the first server -- and so I built a nice little 5-drive Ubuntu-based fileserver (1 boot drive + 4 X 2TB drives). So far, it has gone extremely well, and the performance with SAMBA has been nothing short of excellent. My network is the least optimized network on earth, and I'm getting steady 45-65 MB/sec transfer rates, depending on filesizes with NO tweaking at all. The larger the file, the faster the transfer. If I spent time tweaking this, I'm sure I could get better transfer rates.

(By way of comparison, my custom-built WHS manages perhaps 30-55 MB/sec transfer rates, but slows to a crawl (5-15 MB/sec) when it's cleaning the database or doing its hourly folder rebalancing act. So this Ubuntu server is literally crushing the WHS left and right.)

I just had to figure out some permission issues with SAMBA, Linux and user/group accounts, but a good Google search straightened me out. So far this machine far surpasses what I expected, and the cost has been very affordable. I'm in the process of setting up users/shares and transferring data. So far, so good. Hope it continues this direction!

BTW, the Hitachi 7K3000 2TB disks seem insanely fast. I can't believe such power and performance is available for such a low price. What a crazy world we live in.

 

[stevech]

Wondering the rationale for paying the premium (Hitachi 7K3000) for 6Gbps SATA in a small NAS where GbE LAN constrains the speed? Rebuild speed?

An issue for me, in *inx based NAS, is that some file attributes in NTFS for windows PCs are lost in the translation. This breaks some software that does real time file backups and cloning.

 

[symboint]

Wondering the rationale for paying the premium (Hitachi 7K3000) for 6Gbps SATA in a small NAS where GbE LAN constrains the speed? Rebuild speed?

Sheer luck and availability at my local store when I purchased. I wanted to have a couple different types of drives (i.e.: tried to select from different batches, different vendors). As luck would have it, one of the retail boxes that was labeled 7K2000 was actually a 7K3000 drive inside. Went back to pick up another, got another 7K3000. So I thought I'd keep them, since they seem quiet and cool (and fast). Hitachi must be making the transition over to the newer model in their retail packaging and used old packaging. I have been impressed with speed and temps, so I thought why not? Also picked up some Seagate drives.

At this point I'm not running RAID for my needs. I don't see an advantage in my case. These are individual EXT4 volumes of 2TB each. Each drive's contents are getting backed up externally 2 additional times (to USB and to WHS for a total of 3 copies). The extra speed actually has come in handy if I move things around internally on the server. But I could see them coming in handy for rebuild if I went RAID at some point.

An issue for me, in *inx based NAS, is that some file attributes in NTFS for windows PCs are lost in the translation. This breaks some software that does real time file backups and cloning.

I'm keeping it very simple. So far, things seem to work fine -- I'm experimenting with a few Linux backup tools like rsync, Grsync, Simple Backup, Back in Time and Deja Dup.

On the Windows side, I use Acronis, SyncBack and for now at least, the WHS backup. So far so good, no issues I know of (yet). As I get more experienced and do more testing, I'll streamline.

 

[stevech]

Acronis on Windows XP and Wnk 7. It has rescued me from a corrupted or broken drive several times - most with the 2009 version. The 2010 version was not to be trusted, I've heard. I just upgraded to 2011 on Win 7. I use the drive clone to dupe my SSD to hard disk in case it fails, I just boot and go. Also do drive images of several drives around here. And SecondCopy is my hour by hour backup for files it would be costly to lose.