QNAP TS-x53Pro - follow up to the Primary Site Review

[sfx2000]

And one more... admittedly, this is a different box all together - Core i7-4790 (Haswell) on VMWare Workstation 11 with Win7Pro - so there's a lot more resources to run this benchmark -

It does say that within limitation, the TS-453pro on J1900 can do a decent job with KVM/QEMU (that is the VM engine used with QTS 4.1.3)

Hardware:
Processor: 2 x Intel Core i7-4790 @ 3.59GHz (2 Cores), Motherboard: Intel 440BX, Chipset: Intel 440BX/ZX/DX, Memory: 2048MB, Disk: 21GB VMware Virtual S, Graphics: VMware SVGA II, Audio: Ensoniq ES1371 / Creative, Network: Intel 82545EM Gigabit

Software:
OS: Ubuntu 14.04, Kernel: 3.13.0-51-generic (x86_64), Desktop: GNOME 3.9.90, Display Server: X Server 1.15.1, Display Driver: vmware 13.0.2, Compiler: GCC 4.8.2, File-System: ext4, Screen Resolution: 1076x872, System Layer: VMware

Timed Linux Kernel Compilation 3.18-rc6:
pts/build-linux-kernel-1.5.3
Test 1 of 1
Estimated Trial Run Count: 3
Estimated Time To Completion: 23 Minutes
Running Pre-Test Script @ 15:00:44
Started Run 1 @ 15:01:09
Running Interim Test Script @ 15:05:11
Started Run 2 @ 15:05:15
Running Interim Test Script @ 15:09:11
Started Run 3 @ 15:09:16 [Std. Dev: 0.50%]
Running Post-Test Script @ 15:13:12

Test Results:
238.11216497421
236.14400196075
235.97284603119

Average: 236.74 Seconds

 

[sfx2000]

So, now... let's take a look under the hood - I'll start with SW...

Before we get too deep - check out the QTS 4.1 overview along with the Virtualization Station article...

I'm not going to go too deep here, as those aspects have been well covered...

1) Linux - it's a fairly modern kernel, 3.12.6... but here's where things are a bit scary, and suggests the build-environment/toolchain used...

Linux version 3.12.6 (root@BuildServer48-1) (gcc version 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)) #1 SMP Wed Apr 8 04:32:08 CST 2015
​

Yep, it's the Devil's Compiler... Not a bad thing perhaps, but it's pretty lenient on a lot of things that older/newer versions wouldn't let thru - which build-runners like as builds generally work on the nightly build, but it's really, really dependent on developers not making unintentional mistakes... those compiler warnings are there for a reason. The other concern, it's an old compiler... yeah, it builds ARMv5/v7 and LPIA, but 2007 called, and they want their compiler back... I'm assuming that QNAP has back-ported some of the arch changes and bug-fixes, but perhaps it's time to move forward.

2) Jumping into the shell, it's a weird space - not quite embedded linux, and not quite a full-on distro that many linux folks on x86 are used to - so if one is used to the RedHat or Debian ways, things are strange here... some of the goodies are there, some ain't, but the env parameters for the default shell make it interesting...

[~] # php -v
-sh: php: command not found
[~] # perl -v
-sh: perl: command not found
[~] # gcc -v
-sh: gcc: command not found​

see what I mean? It's like being in busybox to some degree...

Basically, the ssh login is for debug/troubleshooting only, which is fair - just don't count on this distro to be a leading edge or even basic development environment...

Actually, to guys like me, it's an environment to just stay away from... it's a tell-tale that "danger, there be dragons and things you might not know how we did things, so next firmware release, things could, and likely will, go ker-blammo"...

3) Server Daemons - they're all pretty current, recent enough to work for what one expects from a microserver with this feature set- leave things alone, and they'll likely run forever...

Again - resist the urge to change things on the shell, the risk just ain't worth it...

4) That being said - what is the power of the QTS 4.1.3 platform on the TS-x53 (and perhaps the x51 and x63) - KVM/QEMU... sorry ARM guys, this is an intel club...

In the FOSS community, there's a couple of ways to do VM's - one is Xen, the other common way is KVM, the difference is how they're implemented, and compatibility with the guest OS's... they're both good, and there's quite a bit of overlap, but... QNAP, I think, made the right choice going with KVM/QEMU - Xen can host Linux machines, just like KVM, but KVM can do Windows hosts much better, and less headaches... I've done both in different environments, along with VMWare ESX and desktops. IMHO, the approach taken here is the right one... KVM isn't VMWare, but it's pretty damn good...

Since it is KVM, most newer Linux installs know it right from the get-go as a paravirtualized guest, and install the virtio drivers without having to pull them in for IO/Memory management...

Some KVM thoughts - remember - these are virtual CPU's, QNAP has wisely not pinned a CPU slice to a physical core... which might take some explaining...

Let's take an Intel Core i7-4790 - it has four physical core, 8 logical cores, and with KVM and enough memory, I can host 32 or more virtual CPU's... IBM has a few great articles on KVM and resource management, it's more than I can summarize here...

For my case - I'm running QTS on base metal, and an Ubutnu 14.04LTS running on two "cores" in QNAP speak on the KVM - which means that in actuality my Ubuntu VM instance is limited half of the total resources in the physical CPU resource pool... pretty cool, as my VM instance is basically hosting an internal Web Server running a DokuWiki instance, along with an internal Wordpress install... as a LAMP installation.. all contained within a container... and rather than dealing with the risk and weird stuff inside QTS, I've got a full-blown linux that I'm comfortable with.

And doing some profiling, QNAP ensures that Bare Metal for their apps takes precedence, the VM's get what's left over, which in many cases, is quite a bit...

5) Not sure what else to add here - but I've said this a couple of times earlier, and I'll say it again, just because QNAP exposes the shell to QTS, it doesn't mean one should get into the gubbins and start doing stuff - the QTS platform and all the servers/services it offers, it's not very forgiving, and it doesn't take much to blow things up. If one is intent on playing in that sandbox, do it on a spare box, not on your primary NAS if you want to protect your files...

6) one more thing... I wouldn't expose any NAS box to the public internet - not to share files, host a VPN client/server, not even for SSH - the risk is just too much - with the KVM/QEMU capabilities of the platform that QNAP offers, it's another story, as the VM's use a dedicated ethernet port, and there's clear separation between the host and the guest VM's...

 

[sfx2000]

Spent some more time spelunking around in the terminal, and I'm even more convinced that doing any modifications, installing third party package managers, etc... is a quick way to a certain and eventual doom...

This is not a desktop or service type of distribution - it's very much an embedded distro - not a bad thing, as it has great stability and performance - it's just that there are a huge number of symlinks back and forth, and... boom if something changes outside of the QTS environment.

My advice - pretend the shell doesn't exist for day to day stuff - consider it like the fire extinguisher just in case the stove catches on fire, then use it to debug/troubleshoot to save your data if possible...

That being said, the x86 based QTS installs have a great selection of QPKG's that can be installed from the GUI - use those packages with confidence, as they've been vetted by QNAP.

If one really wants to do some low level linux stuff - run a VM on the box - that's fairly safe...

 

[kamina]

Was going to ask you to do some tests with SSD cache, but seems like that won't happen as you already have an array with all drives populated.

 

[sfx2000]

Was going to ask you to do some tests with SSD cache, but seems like that won't happen as you already have an array with all drives populated.

I'm probably too far along with deploying the NAS - I'd have to break the array and rebuild, installing an SSD into one of the bays.. and with three spinning drives, would have to look at a different RAID config all together....

Too bad one could hang something like a Samsung T1 USB3 SSD off one of the ports, but I don't think QNAP would support that without getting down into the console and manually configuring that...

 

[sfx2000]

Was going to ask you to do some tests with SSD cache, but seems like that won't happen as you already have an array with all drives populated.

Was just thinking, if one has enough RAM, instead of sacrificing a slot, just create a small RAM disk, as caching is non-persistent, and while SSD is fast, a RAMdisk is even faster yet...

 

[kamina]

Don't know how qnap is configured but generally Linux does cache files in memory automatically.

 

[sfx2000]

Don't know how qnap is configured but generally Linux does cache files in memory automatically.

Many linux distro's will create a small tmpfs mount for /run, but that is typically retained for the OS itself...

$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/vda1 77G 5.1G 68G 8% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
udev 991M 4.0K 991M 1% /dev
tmpfs 201M 888K 200M 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 1002M 144K 1001M 1% /run/shm
none 100M 32K 100M 1% /run/user​

QNAP QTS 4.1.3 configures two tmpfs partitions, at least on the TS-453Pro - one for /tmp, and the other for /dev/shm - again, this is for system use.

#df
Filesystem Size Used Available Use% Mounted on
none 200.0M 145.4M 54.6M 73% /
devtmpfs 3.8G 4.0k 3.8G 0% /dev
tmpfs 64.0M 2.6M 61.4M 4% /tmp
tmpfs 3.9G 0 3.9G 0% /dev/shm
/dev/md9 509.5M 118.1M 391.3M 23% /mnt/HDA_ROOT
/dev/mapper/cachedev1 5.4T 822.7G 4.6T 15% /share/CACHEDEV1_DATA
/dev/md13 364.2M 284.9M 79.3M 78% /mnt/ext​

If they were to allocate a tmpfs mount, perhaps around 2GB, and make this available for the same purpose as their "SSD Cache" - in a 4 bay NAS, this could be a good thing.

FWIW - RAM disks are about 10 times faster than even the fastest SSD...

One of my app servers does just this - it's a dual socket Xeon with 64GB RAM, and we build a 32GB ram disk and run the store files for a database - none of the data is persistent, so that's not an issue...

 

[sfx2000]

Just a quick follow up...

I've had this now for over two years, and it's been rock solid.

The updates and features added since QTS 4.1 to the current QTS 4.3 have extended the value and usability of the NAS over time.

I don't use the media sharing (DLNA and/or Plex), mostly for SMB/CIFS and NFS shares, along with hosting a local git instance to host various things - I do use virtualization station hosting an ubuntu 16.04LTS VM instance, mostly to tinker about with docker (yes, QTS also has docker, but it's older, and it's better to run Docker on ubuntu directly as one can stay current with things there).

The 3TB seagates have been fine (knock on wood) - I have them still configured as RAID10, mostly for reliability and speed (writes are faster with RAID10 vs. RAID5/6). I am using an external Seagate Backup Plus USB3 drive to back things up on a schedule.

Yes, it was spendy to fully kit it out (NAS plus RAM and drives, total about $1200), but so far it has been a good investment.

 

[sfx2000]

Picked up a QNAP TS-453Pro 8GB RAM the other day - exploring the options - primary decision was based on the feature set and the strong review on SNB...

Believe it or not, it's still running here at the house - been on constantly, I don't spin the drives down, and it's been close to 6 and half years...

I am contemplating a replacement, as the TS-453Pros are starting to age out and more reports of Intel Atom LPC clock out bug (errata VLI89) - a known issue with the Silvermont cores - seems that QNAP is not replacing these under any kind of extended warranty as the devices are aging out anyways...

That, and 6.5 years on a set of spinning rust drives is asking for trouble (SMART looks good, but it usually does until it doesn't).

 

[sfx2000]

Update Mid-2023...

I've had this box now for 8 plus years...

What's happened?

1. HW replacement - did get into a place where it would not start up - apparently it's an Intel thing with the chipset itself
1a. QNAP didn't ack the issue, but I did get a refurb box at a decent price, and my data was on the drives...

2. Seagate 3TB NAS drives - well, they're still spinning...

3. QTS - this is a bit of a mess - basically, QTS5 update/upgrade means backup/reformat as the inline update breaks a lot of things

It's still a good NAS box, older drives, older HW, so keeping an active backup is a good thing...

My primary NAS at the moment is not QNAP - not that I have ongoing issues with them, just wanted to check out alternatives...

That alt was Synology with the DS-723+ that I bought online earlier this year - mostly to see what is different, but also that my current NAS was 8 years old...

Upside I suppose - it supports HW transcoding for certain codecs via QuickSync Video - so depending on the content, it's a plus - that's what the TS453+ is doing these days other than being a backup source for the Syno box...

 

[sfx2000]

PIC's...

 

[sfx2000]

2. Seagate 3TB NAS drives - well, they're still spinning...

They're still spinning - and I'm getting write errors on one of the 4, and it's pretty consistent - since the drive set is 8 years old, if one is starting to die, the rest likely will attend the funeral - so do I do the drive replacement on a 8 year old NAS that has a CPU that will eventually die (again) anyways?

It's been a good run, but it's probably time to retire the box and move on...

It's pretty cool that QNAP has provided at least software support over time - it's still supported for now, and that's been thru a lot of changes since first installing it...

 

[coxhaus]

Yea I have a bunch of old equipment laying around. It will run but how long is the question.

I had to use one to save my new NAS which was showing drive errors.