DIY NAS high performance vs. Vendor NAS low power consumption

[sammynl]

After running a DIY NAS for almost a year now with free Ubuntu Server x64 (Linux RAID5 6*1TB) high performance (see image below) and low implementation costs, I am planning to switch to a vendor NAS for one reason only: low power consumption.

Because a NAS usually runs 24/7, for a home user power consumption is perhaps more critical than the sum of implementation costs + performance; as long as a NAS has RAID and can stream FullHD 1080P media it's fine!

Building a NAS yourself is fun in the startup fase (relatively low hardware costs + much higher performance) but in the end it gets expensive due to high energy costs. Maybe idle power consumption is what truly matters since a home NAS is accessed just a couple of hours per day. Can your mediaplayer (or windowsclient trying to map networkdrives) wake-up your idle samba-share DIY NAS if the idle NAS doesn't advertise itself on the network?

Is > 100 MB/sec a performance must have (it's nice to break the barrier for fun) for a NAS home user?

Shouldn't the focus for a DIY home NAS be more on low power consumption and less on performance? At least 50-70 MB/sec read and write is enough anyway and all vendor NAS can do that.

For me this raises the question: how to build a low-cost fairly fast but yet very energy-efficient DIY NAS?

 

[msl]

If overall cost is your motivation then implementation cost is just as important as running costs. For example, there is not much value in spending an extra $50 on a device that will save you $10 in electricity each year.

 

[00Roush]

MSL brings up a good point. Have you gotten a chance to test the actual power consumption on your server? My guess is your idle consumption isn't that bad. Maybe 60-80 watts. As Jay_S pointed out in this thread you can get very low power consumption with an Atom setup. But generally I think around 30 watts at idle (say 4 hard drives) would be obtainable. So for a worst case in cost I will use a difference of 50 watts for comparison. 50 * 24 (hrs) * 365 (days) = 438000 watts or 438 kilowatt/hours. 438 * .15 (national average kilowatt hour cost) = $65.70 for a year. But this would only be if you run the server 24/7. My current server idles around 100 watts from what I recall but I have it setup to go into standby mode when it hasn't been accessed for 20 minutes. It wakes up when accessed by any computer on the network which results in around a 15-20 second access delay.

Something else to consider is the fact that you can have a low power consuming NAS and have high performance. I have seen some idle power numbers for the Intel i3 setups around the 30-40 watt range. That setup should provide low idle consumption while still allowing for 100+ MB/sec transfer speeds.

00Roush

 

[Jay_S]

Though I love that my atom server idles at 15W, I still suspend it when not in use. It's basically just a back-end for my SqueezeBox music streamer, and I can sleep/wake the server using the SqueezeBox's remote control. It was considerably less expensive than most NAS devices (especially rack-mount ones) - just under $300 for everything.

@sammynl: I scanned your build thread; are you still using this gear?
* Intel Celeron Dual Core 2 Ghz.
* Intel DG965OT microATX motherboard. (6 o/b SATA ports)
* 6 x Samsung Spinpoint F1 HD103UJ - 1TB.
* 4Gb DDR2 667 Mhz RAM.
* Promise TX 2300 controller for 2.5" 160GB OS harddrive.

If I read that right, you have 7 total spindles (6x 3.5", 1x2.5").

Those HD103UJ samsungs are pretty typical for 7200rpm drives:

• Spin-up Current (Max) 2.4A (=28.8W each!)
• Seek (Typical) 7.6W
• Read/Write (Typical) 8.4W
• Idle (Typical) 6.7W

At idle you're looking at ~40W for the RAID array alone. Assuming you switch to a NAS appliance using the same 6 drives, those 40W are coming with!

First thing you should do is measure your existing power consumption. You'll need a base line figure to compare against. I own a Kill-a-Watt, and it's accurate enough for how inexpensive it was. If/when you get one, check it against some known values - light bulbs are a good "standard". Test against incandescents and CFLs (CFL's are closer to what PC PCU's "look" like).

I tried making a watt meter with a current clamp and a hacked PSU cable, but never got reliable readings from it. The Kill-a-Watt is much more convenient.

My UPS's LCD display shows real-time kW usage. But it reads quite a bit higher than it should, so I don't trust it.

There's likely a few things you can do with your existing server to lower power consumption. For example, disable EVERYTHING you're not using in BIOS. Serial, Parallel, on-board audio, etc. Pull a stick of RAM (do you need 4GB?). Underclock the on-board GPU (probably not possible with your Intel board). Sleep your HDD's when idle (maybe impossible w/RAID - I don't know).

My video server runs unRAID from a USB stick. With three WD green 1TB drives it idles at 39W. unRAID will spin down unused drives, which is one of the features that sold me on it. It's NOT a high-performance server OS, though. Reads are limited to whatever single disk performance is capable of. Writes suffer - about 1/4 of normal single-drive write performance - because of how unRAID does parity calcs. This is an acceptable trade off for the other benefits of unRAID.

By the way, is that a NETBURST Celeron D? As in, based on the old Pentium architecture? I checked Intel's "supported CPU" list for the DG965OT, and it looks like this is the case. But maybe I'm mistaken.

What PSU are you using? The EA430 that comes with the NSK6580? As an 80+ certified PSU, it's supposed to be 80% efficient between 20% and 100% of rated output. And tests show that it is (pdf). This is a good PSU, and is probably a good fit for your system.

 

[sammynl]

First of all I really want to thank you guys for the reply's they were very useful!

I followed your advice and bought a power consumption meter for 17 euro's. Powered on the NAS, afraid to see large values but... normal running state around 105 Watt and when writing large amounts of data about 115 Watt, not too bad but 105 Watt is way too much for 24/7 operation.

The processsor is the Intel Celeron E1400.

Reading the reply's it became clear to me that 6 SATA disks running @ 7200rpm all the time use a lot of power (thank you Jay_S for detailed data!).

Google is your best friend again: I found some articles describing how to idle disks in Ubuntu. Unlike Windows, in Ubuntu you can specify idle settings for each disk. Yet another reason to run the OS on a seperate 2.5" disk, I don't want the OS disk to idle, but the RAID-array disks I do want to idle.

First logon to the NAS (telnet or console) and run the fdisk -l command, you will see all disks listed in my case the RAID-array disks are sda, sdb, sdc, sdd, sde, sdf and finally sdg is the OS disk. (By the way, I did not create Linux Raid partitions on the disks, it is not necessary for a RAID array. If you did you will see sda1 sdb1 sdc1 etc. instead of sda sdb sdc etc.)

Now in Webmin use File Manager browse to etc/hdparm.conf and edit the file hdparm.conf by adding the following lines:

/dev/sda {
spindown_time = 240
}

/dev/sdb {
spindown_time = 240
}

/dev/sdc {
spindown_time = 240
}

/dev/sdd {
spindown_time = 240
}

/dev/sde {
spindown_time = 240
}

/dev/sdf {
spindown_time = 240
}

Let me explain the 240, a value of zero means "timeouts are
disabled": the device will not automatically enter standby mode. Values
from 1 to 240 specify multiples of 5 seconds, yielding timeouts from 5
seconds to 20 minutes. Values from 241 to 251 specify from 1 to 11 units
of 30 minutes, yielding timeouts from 30 minutes to 5.5 hours, so 242 is 1 hour.

Now reboot your Ubuntu machine. After the time specified (I tested it first with 1 minute timeout) all the RAID drives go into idle mode. A quick look on the power consumption meter tells me it has dropped to 45 Watt !! That is great it saves a lot of power and money compared to 105 Watt 24/7! Why didn't I think of this a year ago?

Now another test, will the drives spin-up again? In Windows I tried to access the Samba share and after about 15 seconds all drives were running and data could be written or read again. Since the OS disk is not idled, Samba is still running and shares can be seen all the time.

Drives in stand-by mode after 20 minutes

I hope other Ubunty DIY folks find this useful.

Regards, Sammy.

 

[Jay_S]

Sammy - excellent results! 45W is terrific considering your high-performance.

I've always been curious how raid-ed drives would react to hdparm spindown commands - for example, would it break the array if only 1 spun down? or if they all spun down at different times? Maybe not in your case because your raid "controller" is on your OS drive?

 

[sammynl]

I've always been curious how raid-ed drives would react to hdparm spindown commands - for example, would it break the array if only 1 spun down? or if they all spun down at different times? Maybe not in your case because your raid "controller" is on your OS drive?

Hi Jay,

I was wondering about that also but it seems to work smoothly, the RAID device (virtual device) /dev/md0 is not being paused (always mounted) indeed, I guess because the OS disks is still running. It's just the physical disks in the array that spin down. The RAID config is on the physical disks itself in the Superblocks metadata, hence if one reinstalls the OS or replaces the OS disks, the array is still intact. Each physical disk "knows" it's a member of a specific array regardless of the OS disk so that's safe.

I'm happy it all works fine, my HD Mediaplayer sees the Samba shares immediately if the array disks have spinned down and after about 15 seconds it can access data and start streaming, disks running again. Same for Windows clients.

Regards, Sammy.

 

[00Roush]

That is awesome.

So I guess that means you are not going to switch to a different NAS setup...

00Roush

 

[sammynl]

That is awesome.

So I guess that means you are not going to switch to a different NAS setup...

00Roush

No I'm very happy with this config now

I find out that spindown values > 240 don't work, up to 240 (=20 min) does work but larger values like 242 (1 hour) don't.

Spindown will not work that easy on the OS disk for many reasons, like syslog entry's being written every once in a while or samba.dat entry's being written every 5 minutes. So a seperate harddisk is for the OS is more a less a must.

Also during boot-up if the array is not 100% clean and synchronized, the spindown settings are ignored. You can overrule this with the following line in the init script: FORCE_RUN=yes /etc/init.d/hdparm start but I prefer not to do that.

So far the Ubuntu power saving quest, always nice to get it working...

-Sammy

 

[00Roush]

So Sammynl any chance you could do a "how to" for all of us DIY guys? Basically what hardware you have, what OS you are running, and any special things you had to do to get the performance you have. I am pretty sure you already have all of this in previous threads but I was thinking it would be awesome to have it all in one place.

I think Jay_S already summed up the hardware you are running above. You also already mentioned above you are running Ubuntu Server x64. So what about the special sauce!?

00Roush

 

[sammynl]

So Sammynl any chance you could do a "how to" for all of us DIY guys? Basically what hardware you have, what OS you are running, and any special things you had to do to get the performance you have. I am pretty sure you already have all of this in previous threads but I was thinking it would be awesome to have it all in one place.

I think Jay_S already summed up the hardware you are running above. You also already mentioned above you are running Ubuntu Server x64. So what about the special sauce!?

00Roush

Hi Roush,

Im happy to do that I guess it's best to start a new thread for this? As soon as I've got some time left I'll see what I can do, all together it's a quite a lot of info if it has to be detailed.

Regards, Sammy.

 

[00Roush]

It wouldn't have to be too detailed. Just an overview of your hardware and OS you are running. Then any special changes you did besides just a basic OS install. Things like installing Webmin, setting up your hard drives to spin down at idle, or special RAID settings you used.

Heck just a real basic hardware/OS performance summary with links to your other threads would be good to have.

00Roush

 

[cfishy]

My setup

Hi

First, thanks for this thread. I read it before buying my set up, after returning the Synology ds200+.

I just plugged in the kill-a-watt and I'm pleased with the boot up results:

My setup:

three hard drives:

- Seagate Barracuda LP ST32000542AS 2TB 5900 RPM SATA 3.0Gb/s
- Hitachi HDS722020ALA330
- for boot and system: one old IDE/ATA 250G western digital WDC WD2500JB-00GVC0, 08.02D08, max UDMA/100

RAM: one stick of Corsair XMS3 2G DDR3 (tried Kingston ValueRam server 1G ECC but it didn't work.)

motherboard with CPU: GIGABYTE GA-D525TUD Intel Atom D525@ 1.8GHz 1M L2 cache BGA559 Intel NM10 Mini ITX Motherboard/CPU Combo

SeaSonic SS-300ET Bronze 300W ATX12V V2.3 80 PLUS BRONZE

Case: Just wanna mention it's LIAN LI PC-Q08B because it's so awesome. 7 drive cages mini-itx that you never have to get your hands inside the tiny space.

After trying a few things, I decided to go with Ubuntu Desktop 10.0.4 LTS.

Boot up with kill-a-watt in place: 44 - 50 watts during the entire boot process. peaking to 50 watts only one time for a second; it's been more around 45 watts idle.

Shut down the system had a bigger peak, 53 watts.
disk spin up from idle costs a short peak to 57 watts.

when the two drives went into idle, the entire system cost 38-39 watts according to kill-a-watt.

File transfer: 46-48 watts, 40-43 mB/s over my router's network switch. router is WND3700 supposedly gigabit. I don't know how to get better results. ethernet is on-board, too.


I have been indexing a large collection of music using squeezebox server and it never got anywhere near 50 watts. So I think I can safely assume that the normal operating power consumption is roughly 47 watts.

I have not turned off all the motherboard features, os services, etc. yet. I suspect the difference won't be too big.


Thanks for the tips guys!

 

[cfishy]

Sammy - excellent results! 45W is terrific considering your high-performance.

I've always been curious how raid-ed drives would react to hdparm spindown commands - for example, would it break the array if only 1 spun down? or if they all spun down at different times? Maybe not in your case because your raid "controller" is on your OS drive?

I want to comment on one observation: After both drives spun down for a while, it suddenly started to spin one of the two (mirrored) drives. I'm unable to tell why, but I figure it might be a read only op that only requires one of the two mirrors.

 

[00Roush]

Hi

First, thanks for this thread. I read it before buying my set up, after returning the Synology ds200+.

I just plugged in the kill-a-watt and I'm pleased with the boot up results:

My setup:

three hard drives:

- Seagate Barracuda LP ST32000542AS 2TB 5900 RPM SATA 3.0Gb/s
- Hitachi HDS722020ALA330
- for boot and system: one old IDE/ATA 250G western digital WDC WD2500JB-00GVC0, 08.02D08, max UDMA/100

RAM: one stick of Corsair XMS3 2G DDR3 (tried Kingston ValueRam server 1G ECC but it didn't work.)

motherboard with CPU: GIGABYTE GA-D525TUD Intel Atom D525@ 1.8GHz 1M L2 cache BGA559 Intel NM10 Mini ITX Motherboard/CPU Combo

SeaSonic SS-300ET Bronze 300W ATX12V V2.3 80 PLUS BRONZE

Case: Just wanna mention it's LIAN LI PC-Q08B because it's so awesome. 7 drive cages mini-itx that you never have to get your hands inside the tiny space.

After trying a few things, I decided to go with Ubuntu Desktop 10.0.4 LTS.

Boot up with kill-a-watt in place: 44 - 50 watts during the entire boot process. peaking to 50 watts only one time for a second; it's been more around 45 watts idle.

Shut down the system had a bigger peak, 53 watts.
disk spin up from idle costs a short peak to 57 watts.

when the two drives went into idle, the entire system cost 38-39 watts according to kill-a-watt.

File transfer: 46-48 watts, 40-43 mB/s over my router's network switch. router is WND3700 supposedly gigabit. I don't know how to get better results. ethernet is on-board, too.


I have been indexing a large collection of music using squeezebox server and it never got anywhere near 50 watts. So I think I can safely assume that the normal operating power consumption is roughly 47 watts.

I have not turned off all the motherboard features, os services, etc. yet. I suspect the difference won't be too big.


Thanks for the tips guys!

Just wondering why you didn't go with Ubuntu Server 10.04. I don't know for sure but Samba might be tuned differently on the Desktop version versus the Server version.

As for your speeds... what OS and hardware (HD, CPU, RAM) does the client have? My guess is you should be able to see higher speeds than that.

00Roush

 

[cfishy]

Just wondering why you didn't go with Ubuntu Server 10.04. I don't know for sure but Samba might be tuned differently on the Desktop version versus the Server version.

As for your speeds... what OS and hardware (HD, CPU, RAM) does the client have? My guess is you should be able to see higher speeds than that.

00Roush

I tried to install ubuntu server about 4-6 times, eventually gave up on it. The distro is just poorly maintained. If I recall, first problem was the memory stick distro insist that I put a CD in the CDROM. so I tried to mount an image as an ISO device separately to get installation working. Then I ran into problems with packages (forgot what problem it was.) making me think that the support is also inferior. Then it would not reboot because the installer insist on installing GRUB on the MBR of /dev/sda1 instead of the drive that I installed the OS on. what a stupid idea. the same version number ubuntu desktop would let me choose which drive to install GRUB on MBR.

I have bad experiences with Netbook remix, too. I think Ubuntu Desktop is most supported, less buggy than any of their other distros. There are problems with Ubuntu Desktop if you try to use it as headless server, too. For example, if your RAID fails, the graphical interface will be stuck asking you if you want to ignore the error during boot, so you need a keyboard for that.

But I'm very happy with it, overall. I just decommissioned my old DNS323, and put in another two drives in this new box.

My client is Phenom II x4 955, Windows 7, Samba, Western Digital 1.5T hard drive, WDC WD15EARS-00Z5B1. The NAS serves mostly small MP3 files, so I figure that has something to do with the performance. Larger files write to the RAID at about 60MB/s, read sometimes go up to 70MB/s.

The drive I just added are 1T drives, and they seem much better performing during raid re-sync. So I think I might not have had optimal format options on the pervious two drives. I just used defaults. Power consumption when all five drives syncking is just under 66 watts. I might change to using a solid state drive for system to cut some power. More tweaks on the mobo to decrease power usage follows.

Edit: Kill-A-Watt shows just under 35 watts for this rig idle.

 

[kendallp]

First logon to the NAS (telnet or console) and run the fdisk -l command, you will see all disks listed in my case the RAID-array disks are sda, sdb, sdc, sdd, sde, sdf and finally sdg is the OS disk. (By the way, I did not create Linux Raid partitions on the disks, it is not necessary for a RAID array. If you did you will see sda1 sdb1 sdc1 etc. instead of sda sdb sdc etc.)

Now in Webmin use File Manager browse to etc/hdparm.conf and edit the file hdparm.conf by adding the following lines:

/dev/sda {
spindown_time = 240
}

Just so no one else makes the dumb mistake I did. Even if you did partition the drives so they show sd*1 in the fdisk -l output, you do not include the "1" in the hdparm.conf file. The hdparm.conf file should look exactly like Sammy has in his example.


-Kendall