From a consumer NAS to DIY

[ddaenen1]

I have started this thread to document my journey in setting up a DIY NAS.

I currently have a ReadyNAS RN212 which is a desktop model with 2x2Tb WD red drives. The sole purpose of this NAS is to host my movie and music collection which i stream via Plex to all home devices. There is nothing else on it and it does run flawless. All other docs, pics and other home stuff are housed on a separate Synology RS214 with 2x4Tb which is old, but perfect to host these type files as speed is not really a requirement.

The issue with the RN212 is though that it is a desktop model currently standing on a tray in the rack taking up 3U of space whereas i want all my equipment in my server rack (which not very deep - 500mm) to be rackequipment. Fact is though that in my research to replace the RN212 with a rack NAS, i always bumped into several issues that bothered me:

1. The NAS specs for the majority of the affordable rack NAS's would allow me to stream @ 1080P using Plex but no encoding or anything else, let alone being ready to stream 4K and even 8K in the future.
2. Many rackNAS models were too deep for my server rack
3. RackNAS models that did meet the criteria were way more expensive than i wanted to spend on a hobby.

So, i decided to look what other options i had.

Option 1: Install Plex on my 1U Supermicro server running Ubuntu and some server apps which has a Xeon L3426 and 16Gb ECC (expandable to 32GB ECC) which could do the trick but the chassis is ecurrently quipped with 2 2.5" hot swap drive bays housing 2 x 240GB SSD's. This would be difficult as the max size for 2.5" HDD's at the moment is 2TB and whilst this would be enough for right now, it would not be future proof.

Option 2: Install Plex on my office server running Windows 10 which is a tower but away from the rack and would not take up any rackspace as the RN212 is right now. The Asus motherboard has RAID and currently has 1 120Gb SSD (Win 10) and 2 x 500GB WD HDD's in there which are virtually empty so i could easily replace the 2 WD 500Gb disks with the 2Tb disks from the RN212 and get it going. Disadvantage is that with this server, i virtually run my network and do all the trials and tinkering before i move anything in my network so i t can crash if something really goes wrong. Secondly, my rack has an APC UPS to which all devices are connected and obviously, the office server is not. Not a place i would want Plex on.

Option 3: find another 2nd hand 1U server which has decent specs and that is short enough to fit my rack and install some NAS software on it that is capable of running Plex Media Server.

So eventually, i chose option 3 and have been searching since to find the specs i would need to get this on the road. This search ended today when i was able to buy another Supermicro 1U short server with following specs: supermicro chassis capable of housing 2 x 3.5" HDD's with X8SIL-F board (with IPMI and onboard USB slot), Intel Xeon X3430 CPU, 16GB ECC RAM and currently 1Tb Enterprise WD HDD onboard. All of that for 100 USD.

The plan is simple: remove the 1Tb drive from the Supermicro and install it in the officeserver. Remove the 2 500Gb HDD's from the office server. Backup the entire content from the RN212 onto the 1Tb drive. Install the 2 500Gb WD drives in the supermicro. Install FreeNAS from a thumb drive onto a 64Gb sandisk thumbdrive which is installed in the onboard USB slot. Configure FreeNAS in RAID 1 for the 500Gb drives and install Plex. Copy part of the content from the 1Tb drive onto the 500Gb RAID 1 FreeNAS volume and launch Plex. Test for some time to detect any potential flaws and fix. Once confidence has been achieved, replace one of the 500Gb with one of the 2Tb drives from the RN212 and mirror. Once completed, replace the other 500Gb drive with the other 2Tb drive and mirror again. This way, at all points in time, i have 2 copies of my media content in case something goes south.

In the end, this should do the trick and allow me to transition from the RN212 doing Plex media streaming to running a Plexserver that supports encoding using FreeNAS on the Supermicro rackNAS. All requirements fulfilled. Sounds simple, right?

Tomorrow i will start the operation and post progress on a daily basis. To be continued...

 

[ddaenen1]

3 days down the path a lot happened. I went along as laid out before. Disassembled the 1TB drive from the server, removed the 2 500Gb from my officeserver and installed the 1TB instead and started copying the entire content from my readyNAS to the drive. That went flawless. The 2 500Gb went in the server and in the mean time i prepared a bootable USB thumdrive with the latest release from FreeNAS on it.

FreeNAS recommends an embedded installation on a USD or SSD. I went for a 64Gb Sandisk slim fit USB drive that would fit the onboard USB header on the Supermicro motherboard with no interference with the housing cover as it is very short.

The installation of FreeNAS went pretty smooth. It would have been completely hassle free if i would have hooked the server up to my network during the installation. Would have prevented me from having to set up the network connection manually afterwards.

Once i booted the server, FreeNAS did detect that i had a couple of bad sectors on one of the drives. Something that Windows had never notified me about but not much of an issue.

Setting up the pool and the share also went pretty straight forward and in no time, I had a running NAS and i could access the share from my Windows officeserver. So far, so good.

Next step. Get Plex set up. This turned out to be a pretty cumbersome process and kind of made me understand this post https://www.snbforums.com/threads/looking-to-ditch-freenas.58906/ better. There was an IXsystems youtube tutorial which i followed step by step.

First, the Plex app didn't install successfully. A second attempt wasn't successful either although now FreeNAS listed the app as installed. A third attempt after removing it worked. The installation is a hassle though setting up mount points, users, etc...

Once done, i couldn't access the web interface via Firefox on my windows server. It turned out this wasn't a FreeNAS issue but a Firefox cache issue. Next problem is that when i wanted to configure the shares in plex, i could not access the FreeNAS shares from within the plex interface, they were greyed out. Lots of trial and error here but when i finally added "everyone" to the permissions of both folders through windows explorer, the could be accessed and i could continue to set up plex.

Once i completed the set up and done some trials with some media files, i decided it was time to swap the 500GB drives with the 2TB from the ReadyNAS. I had a full backup on the 1TB so i wasn't too worried. The swap went exactly as described. Made one 500Gb offline, replaced it with a 2Tb, started up the server and selected "replace". That went very smooth. After the 2nd swap and completion of the "resilvering", i had a redundant 2Tb.

I configured some more stuff such as email notifications using my gmail account and syslog connection to my Synology NAS and also the APC UPS configuration, all that went pretty well.

So this is where i am now. In the mean time i modified the rack rails as they were too long and next thing up is installing it in my server rack with which i want to configure LACP and after that, trying to figure out the whole users/groups configuration. I tried deploying this on the pool to have some access restrictions but for one reason or another, that doesn't to be working well, or i just don't know what i am doing (and that might well be the case )

 

[ddaenen1]

A couple of days down the road, FreeNAS is running well. The fact that the Supermicro server has IPMI which works like a charm, makes it very handy to reboot in case of connection issues (like experimenting with LACP) and also to look for any error messages that are posted on the shell screen. Up until now, i have encountered 2 main issues. I still can't get LACP to work. Everytime i set up a LAG, my connection drops and i have to delete the LAG and reboot for it to come back online. Last i read is set it up on both sides with only one port connected and then plug in the 2nd UTP. Something to try tonight. Secondly, i having issues accessing the SMB share with my Macbook Air which is quite important as i move alot of my media via that device. I have no issues accessing the share via my Win10 officeserver. I did also encounter share access issues between the Macbook and the ReadyNAS in the past even to the point i had also set up an AFP share which worked fine up until a certain point where i was able to access the SMB share once again. I am going to set up an AFP share on the FreeNAS tonight and see how the MbAir digests that. I also had some issues accessing the Synology SMB through the MbAir but they were resolved quickly.

 

[degrub]

Are the versions of SMB being used the same across machines or at least compatible ?

 

[ddaenen1]

Are the versions of SMB being used the same across machines or at least compatible ?

I would assume so. A Windows 10 machine, a server with the latest FreeNAS version and a 3 year old Macbook Air.

 

[ddaenen1]

I would assume so. A Windows 10 machine, a server with the latest FreeNAS version and a 3 year old Macbook Air.

Found it! Apparently i had added the IP of my officeserver to the list of allowed machines when i had difficulties with reaching the share when i was configuring plex so i assume that automatically, all other machines were not accepted. Once i remove that IP, miraculously it worked instantly.

 

[ddaenen1]

Making progress: i also got LACP to work. A router reboot and configuration of the network interfaces via the console menu did the trick. Now moved the IP to a DHCP reserved IP and the server is running really well now and monitoring syslog entries daily.

 

[ddaenen1]

Running for a whole week now and i must say, not a single glitch

Access from all devices works flawless and fast and also PMS has no issue whatsoever. Time to put my Netgear ReadyNAS up for sale and maybe look into some bigger drives but not that urgent.

I do feel a bit uncomfortable with the OS running from a USB drive. Thinking about replacing that with a 32Gb or 64Gb SataDOM which the motherboard supports. They are a bit on the expensive side but i am not sure that i have space the squeeze in an SSD and even if i could, i can't secure it. I have been looking into a chassis that has hotswap bays and a bit more internal space to add another internal SSD to house the FreeNAS OS but with the rack depth i have, it is a bit of a challenge. I was also thinking of maybe moving the drives externally into a rack enclosure that connects that with the server using eSata but not sure if this is a good idea in terms of performance. It would allow me to use the diskspace in the server for the SSD with the OS on it and would give more flexibility in terms of adding diskspace but it seems i would need an eSata PCIe that supports port multiplying and makes me think that maybe, i am overcomplicating things a bit.

We will see...

 

[dosborne]

makes me think that maybe, i am overcomplicating things a bit

That's what makes it fun.

 

[ddaenen1]

That's what makes it fun.

And it's about to get more complicated. I think i have a potential solution. I found a Dell Powervault MD1000 with 15 3.5" hot-swap SAS/SATA bays with interposers and RAID Controller Card PCIe 8x and cables included. I can envision hooking this up to the supermicro server, moving all storage space to the powervault and only house an SSD with the FreeNAS OS in the server. This should allow me for enough storage flexibility for a long long time.

 

[ddaenen1]

And it's about to get more complicated. I think i have a potential solution. I found a Dell Powervault MD1000 with 15 3.5" hot-swap SAS/SATA bays with interposers and RAID Controller Card PCIe 8x and cables included. I can envision hooking this up to the supermicro server, moving all storage space to the powervault and only house an SSD with the FreeNAS OS in the server. This should allow me for enough storage flexibility for a long long time.

And so it happened

I bought the MD1000 but instead of the RAID controller, the seller provided me with a 6Gbe HBA SAS controller so FreeNAS can see all disks separately and manage them, which is the recommended setup. Hooked everything up yesterday and to my surprise, it booted at the first attempt. Installed FreeNAS which went flawless and configured it up to the point where i could. I did not set up the vdev pool yet as i don't have all the drives to full up the 15 bays and whilst i don't need all of them filled, i do want enough capacity to define my Plex pool. So i am in search for 10 x 2Tb SAS drives now

 

[ddaenen1]

Update: i found 15 used 2Tb 7200 RPM SAS drives pulled from a server system due to capacity upgrade. According to the seller, fulled tested and no bad tracks/sectors. I Am unsure what the lifetime for an HDD is but since they won't be under load constantly, i guess it will do for my Plex server application. I will get them tomorrow so we can continue the setup.

 

[ddaenen1]

Update: i found 15 used 2Tb 7200 RPM SAS drives pulled from a server system due to capacity upgrade. According to the seller, fulled tested and no bad tracks/sectors. I Am unsure what the lifetime for an HDD is but since they won't be under load constantly, i guess it will do for my Plex server application. I will get them tomorrow so we can continue the setup.

Got my SAS Drives only today due to some logistics troubles getting there. 15 x 2Tb SAS 7200rpm Hitachi (HGST) drives. As the Powervault is now in unifiedcan operate in 7/8 split-mode and be hooked up to a 2nd server via the 2nd EMM, i want to be smart about this as i do not need all the storage space for my Plex as it stands. Fact is that once a ZFS pool is set up, you cannot expand or shrink it removing or adding drives. You can expand by replacing the drives by bigger ones but that is it. So my plan is to make one pool from the 8 drives in raidz2 for Plex giving me about 10Tb space, and a 2nd pool from the 7 other drives that gives me another 8Tb for future use. Not sure what i am going to do with the 2nd pool but at current, thinking about backup or data. I have set up the media pool now which went flawless, not a single glitch. I did struggle a bit with getting the Plex app installed but that was resolved quickly. Now copying my media content over the the pool and then set up LACP which gave me a bit of a headache last time.

 

[ddaenen1]

FreeNAS now running a week and not a glitch. I did reboot over the weekend to configure LACP. I have done exactly what i described above and created a pool for plex with the 8 2Tb drives from the MD1000, copied all my stuff over and it works like a charm.

The SAS drives are doing their job.

I do have the impression though that buffering when i do a fast-forward takes slightly longer in some instances than with the readynas but i am unsure if this is the plex webinterface via the browser or in general. Need to test it on the TV first.

I am a bit puzzled about my LAN speed. Is it normal to have such fluctuations during a test with iperf? If not, where do i need to start investigating this?

 

[ddaenen1]

Finally found me some affordable 4Rx8 8GB RDIMM sticks so i can upgrade the server to 32GB which came in this morning. It seems that FreeNAS is quite RAM-hungry and 32GB is the most this motherboard can take but it is double of my current 16GB.

After installing the new sticks, BIOS reports the correct value and i notice that FreeNAS appears to be booting faster. GUI reports the correct amount of memory and also a quick check on the ARC reports everything to be fine.

I have maxed out the hardware expansion options on the Supermicro server now. Let's run this over time and see how it goes. One thing i can say, to date FreeNAS appears to be extremely stable once configured.

 

[System Error Message]

not bad, i wish i could afford some of the things that you have but i dont even have enough to DIY my own APC with a salt water battery as i have half the materials from recycling and tap water works fine with it. One of the bad things about being in a 3rd world country is that we just cant afford anything and lithium here is a big no since they dont last long over here, even my 3 year old laptop battery has protruded the case.

When i had money i built 2 NAS, one with 6x 3TB drives in raid 5 and an older phenom ii x3 unlocked to 4 cores and overclocked to 4.2Ghz in a cheap 2U enclosure with 2x8GB ram, so while lacking hot swap capability it has space for 9 drives and takes a regular ATX PSU, with a 2nd hand SFP+ NIC and other expansion cards which used PCIe cables inside allowing me to have GPU acceleration too (plex premium has this feature), but the CPU worked fine for transcoding bigger than 4k-> 4k in real time for one of my high quality timelapses. I can reupload the original video if you are interested in testing your hardware.

My 2nd NAS is a salvaged fx 8 core piledriver from a friend has 32GB ECC ram, 2nd hand SFP+ NIC, 5x8TB seagate NAS drives.

Both NAS will be running nextcloud and plex (use apache for nextcloud, not nginx, but you can speed it up with varnish, redis, opcache/accelerator, nginx proxy which can also be your network cache as well).

I do know plex uses nginx but it is better to switch to apache because for php you get better security with apache and make sure to always use the latest php version. you can use a utility like webmin or ispconfig if you want to freely and easily manage your webserver.

The switch from nginx to apache is important so if both plex and gitlab can do it then do it.

Currently my NAS are turned off because of electricity isnt cheap otherwise i would be talking about them and how to set up the way i have. I strongly suggest not running ubuntu due to their arrogance (if you have a problem, their solution is to follow their hardware config, which doesnt work for everyone), stick to debian, opensuse or centos. debian with lean only uses 60MB of ram after it starts up (based on my VPS) before installing any non core software.

 

[ddaenen1]

not bad, i wish i could afford some of the things that you have but i dont even have enough to DIY my own APC with a salt water battery as i have half the materials from recycling and tap water works fine with it. One of the bad things about being in a 3rd world country is that we just cant afford anything and lithium here is a big no since they dont last long over here, even my 3 year old laptop battery has protruded the case.

All my equipment is also 2nd hand or maybe even 3rd hand in some cases. The thing is that there is huge offer on 2nd hand IT material over here and some stuff is really cheap. UPS's are dead cheap as most sell them off when the batteries die as the OEM replacements are almost as expensive as a new one but with a bit of work, you can replace the batteries with same the same brand for less than half the price. Same for HDD's, especially SAS ones as they are not really suitable for most domestic systems and mostly they are replaced when the factory warranty expires so you can pick up 2TB SAS drives, fully tested and error free for as little as 15 USD per piece. IT just takes some time to search and find the right stuff for your application.

Both my servers are running FreeNAS now, one with Plex and the other one with Nextcould and so far without a glitch. I ditched Ubuntu some time ago as i found it was heavy, and some things are just difficult to get done.

The only thing still left to do is figuring out what i will do with the other 7 2TB drives in the Powervault enclosure. One scenario is to hook them up to the Nextcloud server via the 2nd controller in the enclosure and create more Nextcloud space but there is no demand for that right now or alternatively, create a data-pool and move some backup stuff such as Timemachine from the old ever so slow Synology NAS as its only reason for existance is that is has a great photo app in DSM, something that FreeNAS does not have right now.

 

[System Error Message]

All my equipment is also 2nd hand or maybe even 3rd hand in some cases. The thing is that there is huge offer on 2nd hand IT material over here and some stuff is really cheap. UPS's are dead cheap as most sell them off when the batteries die as the OEM replacements are almost as expensive as a new one but with a bit of work, you can replace the batteries with same the same brand for less than half the price. Same for HDD's, especially SAS ones as they are not really suitable for most domestic systems and mostly they are replaced when the factory warranty expires so you can pick up 2TB SAS drives, fully tested and error free for as little as 15 USD per piece. IT just takes some time to search and find the right stuff for your application.

Both my servers are running FreeNAS now, one with Plex and the other one with Nextcould and so far without a glitch. I ditched Ubuntu some time ago as i found it was heavy, and some things are just difficult to get done.

The only thing still left to do is figuring out what i will do with the other 7 2TB drives in the Powervault enclosure. One scenario is to hook them up to the Nextcloud server via the 2nd controller in the enclosure and create more Nextcloud space but there is no demand for that right now or alternatively, create a data-pool and move some backup stuff such as Timemachine from the old ever so slow Synology NAS as its only reason for existance is that is has a great photo app in DSM, something that FreeNAS does not have right now.

most UPSes use lithiums or lead acid. A couple of old UPSes i have use lithium ions, and in this very humid climate >80%, all my lithium batteries started to expand after a year or degrade significantly. Since most UPS use some sort of 12V battery, the charging is specific so i would have to either create some sort of adapter, or replace the charging circuit with my own for it to work with a saltwater battery since i dont think the charge profile is the same as lead acid despite similar voltages.

Over here 2nd hand stuff is very very pricey, combined with our 4x lower salaries, even far less affordable.

If i was living and working in the 1st world, i would buy my own cable spool and wire my own place neatly with 4 socket rj45 faceplates, and other nicer things that are inexpensive. Over here a contractor once quoted a company i worked for a price of 5x higher as i researched the prices before they came for things that i could buy and do myself, and tried to sell me on a regular 13A cable to handle 30A.

 

[ddaenen1]

The only thing still left to do is figuring out what i will do with the other 7 2TB drives in the Powervault enclosure. One scenario is to hook them up to the Nextcloud server via the 2nd controller in the enclosure and create more Nextcloud space but there is no demand for that right now or alternatively, create a data-pool and move some backup stuff such as Timemachine from the old ever so slow Synology NAS as its only reason for existance is that is has a great photo app in DSM, something that FreeNAS does not have right now.

Eventually i went for a data-pool on the remaining 7 drives. Gradually, i will move over stuff from the Synology and also set up a backup for the pictures between the Synology and the FreeNAS.

 

[ddaenen1]

Eventually i went for a data-pool on the remaining 7 drives. Gradually, i will move over stuff from the Synology and also set up a backup for the pictures between the Synology and the FreeNAS.

I reverted back from the decision to move stuff over from the Synology after an event earlier this week where i needed some data in the office of which i knew i had it on my Synology. I logged into synology quick connect and had what i needed. Currently i cannot access my FreeNAS from outside of the LAN and i am not security/firewall-savvy enough to get this to work in a secure way. I tried it with Nextcloud but to date, not successful yet as Nextcloud doesn't support https out of the box and again, too much DIY to mke this work imho. I now turned the 7 drives into a a fileshare and backup pool, with rsync from the synology to the FreeNAS and a time-machine for the MAC.