QNAP TS509 benchmarks

[SirBC]

I benchmarked my TS509 Pro using the Intel NAS Performance Toolkit.

My Setup
NAS: TS509 Pro, 3 WD Green 1TB in Raid 5, 2.0.3 Build 1016T firmware.
Computer: Intel DX48BT2 mobo with 4GB, XP Pro SP3, WD Raptor 150GB
Network: CAT6, Netgear GS724T ProSafe Gigabit Switch

I have to say I'm a little disappointed with the results. My throughput seems lower than I would have expected. I ran the test 3 times and below is the average:

From the above Toms Hardware article, the tests are:

* HD Video Playback, 2x HD Video Playback, 3x HD Video Playback, 4x HD Video Playback

These benchmarks examine the behavior of the NAS unit while (simultaneously) playing one or more HD video files at 720p using Windows Media Player. Intel gives a percentage rate for the sequential reading of data in these tests, which lies at 99.5% for the HD Video Playback Test. With 2x HD Video Playback, it lies at 18.1%. The result is 6.6% with 3x HD Video Playback and 9.6 % with 4x HD Video Playback.

* HD Video Record

This test writes an HD Video file in 720p format to the NAS unit. This test is made of up of mostly sequentially transferred data.

* HD Video Playback & Recording

HD Video Play & Record examines the behavior of the NAS unit when simultaneously reading and writing an HD Video file in the 720p format. The sequentially-transferred data in this test is approximately 18% of the test.

* HD 2x Playback 2x Record

This benchmark is similar to the one above, but the proportion of sequential file operations is 3%.

* HD Playback With Office

This metric measures the data transfer rates when an HD Video file is read from the NAS unit while working with the Office applications. This test is made up of 608 files. The proportion of sequential file operations is 53.2%.

* HD Playback With Backup

Like the previous test, but this time an HD Video file is played while simultaneously carrying out a backup on the NAS unit.

* Content Creation

This benchmark is made up of 95% write operations to the NAS unit. This simulates the creation of files on the NAS unit such as is the case when, for example, using video editing programs.

* Backup / Restore

These tests are used to determine the performance data using a 4 GB file for data backup and a 4 GB file for restoration. The proportion of sequential data transfers for both of these tests is over 99%. When backing up, write processes of 8 KB are used.

* File Copy To NAS / File Copy From NAS

These tests determine the data transfer rate when copying files to or from the NAS unit. In both of these test processes, a 4 GB file is copied. Unlike with Backup / Restore, 64 KB is read and written.

* Directory Copy From NAS / Directory Copy To NAS

Similar to the previous test, files are copied to and from the NAS unit. A total of 126 files with a total size of approximately 188 MB are written and read across the network.

* Photo Album

This test determines how the NAS unit handles the supply of a multitude of small files—for example, viewing digital photographs stored on the NAS unit. It simulates the viewing of a total of 169 photographs with an overall size of approximately 1.2 GB.

I would be interested to see what others are getting. I don't have the latest firmware installed so I will rerun the benchmark once I update.

- Dave

 

[Dennis Wood]

Dave, I was not aware of this test program. I'll try this on the five drive TS509 with the 7200rpm drives. The WD "green" drive unit is slower. Also, I see you used multi-io settings at 3. All of the tests from iozone, file copying etc. are basically serial IO...one at a time. What you're doing is load testing the NAS in a limited way from one workstation with 3 IO operations. I stand to be corrected on this point, but I'll try some tests. Your results will get a lot better with a 3 drive RAID 0 in the test workstation...also SMB2 and Vista SP1 enhancements will take it up a few notches.

If this tool works as advertised, you just made my day. If I didn't know better I'd say some Intel engineer had looked at my batch file (tests a bunch of the same things) and polished it up. Either way, this is awesome.

Edit: If someone read my mind, and wrote a NAS testing program..this would be it. Multiple loads, simultaneous backups...perfect. Running two workstations simultaneously choosing the appropriate test suites would do a great job load testing 802.3ad configurations. The fact that you can create and analyze your own traces is simply awesome. A test suite like this for free is simply amazing!

Btw, Dave I think you've got version 1.6 there...1.7 is available for download and is the version I'll be using.

 

[Dennis Wood]

Here's run 1 of 5 (the "Batch Run" button runs five test sets and averages them) from the "slow" WD green equipped NAS. Note this unit is running the beta code from QNAP and uses 4GB of RAM. Don't get too choked up over your results as for example the file copy from the NAS is 47MB/s in this test, but our measured large file tests show that large files are being read at closer to 75MB/s from this WD green equipped unit.

 

[SirBC]

Dennis, thanks for the posting your numbers. It looks like they did away with a number of the tests in the latest version. I guess there was too much redundancy b/w the backup/restore and the file copy tests.

I'm not worrying too much about the numbers as I am using the NAS solely as a media center file server, I was just curious to see how it stacked up with what others were seeing. It seems odd that the file copy tests seem so out of whack though.

-Dave

 

[Dennis Wood]

and now the faster config...

This is from the other TS509 using the Seagate 7200rpm ES2 drives, same firmware (2.1.0 build 1119) and 4GB RAM. These test results are still straying from real measurements as a 2.5GB file copy from the NAS runs from 103 to 115 MB/s from this unit using Vista's cut/paste.

 

[SirBC]

Dennis, what do you think is causing the discrepancy between the benchmark scores and what you are seeing in real file copy operations?

-Dave

 

[Dennis Wood]

Good question. If someone with some time and expertise was to analyze the workload trace in the intel NAS trace analyzer, we'd have our answer.

My sense is that recreating the workload (and it's somewhat synthetic) is creating another issue here that is not being considered. That said, I can post up my batch file, excel files and brief how-to so you can try the same thing on your unit.

 

[SirBC]

Thanks Dennis, I'm not too concerned with my results as I'm just using my NAS as a media center file server so throughput isn't that much of a concern. I was just curious to see how my results compared to others. It looks like having a fast RAID system on the host machine makes a big difference.

 

[Dennis Wood]

Actually I'm pretty sure the drives in the NAS are making the largest difference. The reason I'm so sure of this is that when using 2 virtually identical test workstations using 3 drive RAID 0 (onboard Intel ICH7) the NAS using the faster 7200rpm drives is consistently posting numbers very similar to what you're seeing above. We've found that workstations using 3 drive RAID 0 using onboard controllers (Nvidia, Intel) is all you need to max out gigabit, assuming the NAS unit is capable.

 

[thiggins]

Dennis, been meaning to ask you, why 3 drives in RAID 0 vs 2 to get the desired performance. What performance did you get with 2 drives?

 

[Dennis Wood]

This is going back a few months but what we found with the 2 drive raid 0 (keep in mind ICH7) is that very large file transfers were being limited over gigabit by the array's speed. This probably has more to do with local access by various processes knocking 2 drive RAID 0 down below 100MB/s..with 3 you've got some headroom. The other deciding factor in not using 2 drive RAID 0 was the desire to get at least burst transfers in the 175 MB/s range for HD video editing.

Tim, didn't one of the vendors in your reviews suggest RAID0 at the workstation to maximize NAS performance...was it 2 or 3 drives? It's interesting to observe based on the above tests that the drives you use in the NAS unit can dramatically change it's performance. WD green drives run cooler, and no doubt use less power....and that's why they're only used in the backup NAS. For any NAS under load, they'd be a poor choice.

 

[thiggins]

Yes, Dennis. NETGEAR said it had to use 3 drive RAID 0 in order to not limit Vista SP1 filecopy performance.

Interesting that you see a difference with the WD green drives. Do they have less cache than the higher-performing drives you use? I'd assume both are 7200RPM 3.0 Gb/s SATA.

 

[Dennis Wood]

Tim, I was thinking too that 3 drives in RAID0 ensures a data delivery rate of 112 MB/s as the discs fill up or become fragmented. Makes sense.

The WD drives I'm pretty sure are 5400rpm. No question that they run cooler too based on the NAS SMART data. I was thinking that with 5 drives installed, and the gigabit ~120MB/s limit, that slower drives wouldn't matter. Based on all of our tests at least on the TS509 (and a 6 drive unit we're playing with) I'm pretty convinced now that faster drives = faster NAS.

 

[thiggins]

Tim, I was thinking too that 3 drives in RAID0 ensures a data delivery rate of 112 MB/s as the discs fill up or become fragmented. Makes sense.

How do you calculate the 112 MB/s? Just curious.

The WD drives I'm pretty sure are 5400rpm. No question that they run cooler too based on the NAS SMART data. I was thinking that with 5 drives installed, and the gigabit ~120MB/s limit, that slower drives wouldn't matter. Based on all of our tests at least on the TS509 (and a 6 drive unit we're playing with) I'm pretty convinced now that faster drives = faster NAS.

I would think that spindle speed would have a big effect on performance and it appears that it does.

 

[Dennis Wood]

Tim, 112 is not a calculated figure at all. I just haven't seen anything much quicker here than that over gigabit ever. Writes of files under 3 GB or so are being written to the TS509s 4GB of RAM at speeds between 115 and 100MB/s.

I'm guestimating the theoretical max at about 1000/8...125MB/s.

 

[yoh-dah]

Dennis, with 4 GB of RAM on the NAS, you're essentially measuring network and RAM speed. The results you posted would apply only if you're depending on your data being in RAM all the time. To really get a true measure of the device's performance, force it to hit the RAID 5 volume so you can see the overall performance characteristic of the NAS.

 

[Dennis Wood]

100% correct. I've alluded to this in most of my posts as the gain of 4GB of RAM is mitigated with file writes larger than 3.5 GB. In other words, if you're not interested in increasing speed of writes to the NAS with files, or file sets under 3.5 GB, the RAM update is irrelevant. Once you start loading a NAS like this with several workstations, the increased RAM cache makes a lot of sense. On reads from the NAS unit, unless the files are sitting in cache, you see no difference. With regard to the Intel test, you can only use it as a repeatable comparative test..the file copy numbers are much slower than what we measure using batched/timed file transfers and multi-stream encodes. The TS509 in load balancing mode has shown itself entirely capable of sustaining large file (over 100GB) aggregate write rates (two workstations simultaneously) of 75 MB/s, so really with the RAM update you're gaining about ~30MB/s on the write side until the file size exceeds 3.5GB. Regardless, these numbers pretty much put your average internal single SATA drive to shame

I'm generally only interested in very large file read/writes as that's what's required for video or photo editing directly from the NAS. To that end, there's a larger performance gain in just updating to the latest firmware (updated SAMBA), using a decent LACP switch, and using the NAS in load balancing mode. RAM updates are not as simple as they sound in terms of performance gains as the underlying OS can be configured by the vendors in a myriad of ways in terms of how the RAM is used. In most cases, the larger the RAM cache, the greater the overall performance. For example a write request can potentially go to RAM while a READ request is being served to the LAN. In this case the fact that the physical drives can address the READ request while the WRITE request is going to cache, would logically speed up the READ request from the array. If the NAS unit was continually hammered with an aggregate load of over 75MB/s in terms of write requests, RAM cache size would not increase performance much, as the disk array would never catch up. I introduced a 5 second delay between tests in my automated batch test and the individual results tended to increase a great deal...which makes sense as I'm giving the cache a chance to empty to disk.