NAS Hardware Selection (2012)

From Exterior Memory
Revision as of 21:36, 11 September 2012 by MacFreek (Talk | contribs)

Jump to: navigation, search
This article was written in August 2012. This article is no longer updated, and may be outdated.

Requirements

In 2012, my current storage server (NAS) was running full, so I wanted to buy a replacement server. I now have 4(!) external disk drives connected to my main desktop, and wanted to reduce that number. Also, I now often decommission smaller size disks which would still be useful in a server. So my first requirement was a system with at least four drive bays, but preferably more.

I first considered (in order from low price to high quality) a LaCie 5big network 2, Synology DS412+, Drobo FS, or IX Systems' FreeNAS mini and QNAP TS-559 Pro II. All these systems have four or five 3.5" disk bays, and range from €300 to €750 without disks. Add about €100 for each Terabyte disk.

Two requirements prompted my to build my own solution:

Services
The server should run the SMB and AFP protocols, as well as allow direct streaming of media, all over IPv6. A custom server gives full control over the software and is more flexible in the long run (though streaming media is still hard due to the DRM protections nowadays)
Price
Building your own solution costs about the same for the bare system, but is cheaper per terabyte.

So I decided to build a custom FreeBSD (or FreeNAS) system. Not only would that give flexibility, it also allows me to play with ZFS, my favourite file system.

Choice of chassis

I mostly considered Fractal Design and Li Lian chassis, which offer a good wide choice in products. My first decision is what size I wanted to case to be. There are a few cases of less than 10 litre which allow 4 disks, like the [Lian Li PC-Q12]. However, that would limit myself to 2.5" disk bays. I wanted at least six 3.5" bays, and it was possible to achieve this in a mini-tower chassis of 20-25 litre. For comparison, a regular tower PC typically is 30-35 litre in size. Good choices in this size where the Fractal Design Array R2 NAS Chassis (with six 3.5" and one 2.5" disk bays) and the Lian Li PC-Q25 (with seven 3.5" and one 2.5" disk bays, where five of the 3.5" bays are hot swappable). Both these cases support mini-ITX and mini-DTX motherboard (the difference is that a mini-ITX has one expansion slot, and the mini-DTX has two expansion slots).

I picked the Lian Li PC-Q25 after reading a few favourable reviews. For those interested: the specification of the disk bays is somewhat confusing. The PC-Q25 can support eight disks at most. Five of these are in a hot swappable disk bay, the other three on a fixed metal plated. The middle location on the metal plate must be a 2.5" disk; the other locations can be either 2.5" or 3.5" disks.

Choice of CPU

My biggest hurdle was the choice between a low power CPU soldered on the motherboard or a regular pluggable CPU with some more power. CPU World certainly helped me compare the different CPUs, especially in combination with the PassMark Benchmark. I naïvely thought that "a Atom CPU would do", not realising that there are many types of Atom CPUs, AMD alternatives, and also other juicy CPUs that only consume 30-35 Watt of power.

For starters, I looked up the benchmark of CPUs that I used previously. My 3-year old MacBook Pro benchmarks at 1501, while a consumer laptop my wife just bought now benchmarks at 2668. Those are the high-ends for me. The low end is my Soekris net6501 router which also runs FreeBSD. While it's load is near-zero for the switching routing and VPN it does now, it is sluggish when I want to compile a new tool. So that's my low-end. A colleague with his own NAS had a Sempron 140 at first, but recently upgrade to a Athlon II x2 250e so he could use it as a more spicy server.

CPU Frequency #Core (Threads) L2+L3 cache TPD socket GPU Benchmark
Intel Core 2 Duo E6600 2.4 GHz 2 4.0 MiB 65 W LGA 775 no 1501
AMD A8-3510MX 1.8 GHz 4 4.0 MiB 45 W FS1 yes 2668
Intel Atom E640 1.0 GHz 1 (2) 0.5 MiB 3 W BGA 1466 (onboard) yes 250
AMD Sempron 140 2.7 GHz 1 1.0 MiB 45 W AM3 no 752
AMD Athlon II x2 250e 3.0 GHz 2 2.0 MiB 45 W AM3 no 1680
CPU Frequency #Core (Threads) L2+L3 cache TPD socket GPU Benchmark
AMD Fusion C-60 1.0 GHz 2 1.0 MiB 9 W BGA 413 (onboard) yes 563
AMD G-T56N 1.6 GHz 2 1.0 MiB 18 W BGA 413 (onboard) yes 721
AMD E350 1.6 GHz 2 1.0 MiB 18 W BGA 413 (onboard) yes 726
AMD E450 APU 1.7 GHz 2 1.0 MiB 18 W BGA 413 (onboard) yes 740
Intel Atom D525 1.8 GHz 2 (4) 1.0 MiB 13 W BGA 559 (onboard) yes 714
Intel Atom N550 1.5 GHz 2 (4) 1.0 MiB 9 W BGA 559 (onboard) yes 568
Intel Atom N2800 1.9 GHz 2 (4) 1.0 MiB 7 W BGA 559 (onboard) yes 723
Intel Atom D2700 2.1 GHz 2 (4) 1.0 MiB 10 W BGA 559 (onboard) yes 818
Intel Celeron G530T 2.0 GHz 2 2.5 MiB 35 W LGA 1155 yes 1800
Intel Pentium G620T 2.2 GHz 2 3.5 MiB 35 W LGA 1155 yes 2261
Intel Pentium G630T 2.3 GHz 2 3.5 MiB 35 W LGA 1155 yes 2344
Intel Pentium G640T 2.4 GHz 2 3.5 MiB 35 W LGA 1155 yes 2400
Intel i3-2100T 2.5 GHz 2 (4) 3.5 MiB 35 W LGA 1155 yes 3290
Intel i3-2120T 2.6 GHz 2 (4) 3.5 MiB 35 W LGA 1155 yes 3088
Intel i5-2390T 2.7 GHz 2 (4) 3.5 MiB 35 W LGA 1155 yes 4004

I was extremely impressed with the benchmark of the Intel Atom D2700 compared to its power usage. Since Atoms, like AMD E350s are only used as soldered onboard a motherboard, its availability is limited by the motherboards available. While making my purchase, no suitable motherboard was available with a Atom N2800 or D2700. I briefly considered a low power Intel core i3 variant, like the i3-2100T. After some consideration (including the $200 price increase for motherboard, cooling and separate CPU), I decided that my NAS wouldn't need that power, and decided that any low-power CPU with benchmark over 700 would do. That's comparable to an AMD Sempron, and good enough for basic file I/O and an occasional compile of a new kernel.

Choice of Motherboard

Mini-ITX has one expansion slot; mini-DTX has two expansion slots. So it's important to determine how many slots are needed. The main requirement for my motherboard was clearly the number of SATA ports. I needed as much as eight(!) SATA ports, and no motherboard had this many available. So I needed one expansion slot for a SATA card. Some (but not all) Jetway motherboards could use a SATA daughter board, so that would free the regular expansion slot. Unfortunately, the Jetways that support the SATA daughter board all have an old PCI instead of a PCI-express expansion slot, so that turned out to be a pig in a poke. I decided to rather buy a SATA adapter for a PCI-express slot. All modern CPUs (all AMD, Intel Atom, Pentium and Core listed above) have an internal graphics chips (GPU) on-board, so I didn't need a separate graphics card (it's a server after all). So my requirement was just one PCI-express slot, so I could use either mini-ITX or mini-DTX motherboard. In practice, a mini-DTX is harder to get by, so I ended up with mini-ITX.

Here are the issues I paid attentions at:

Brand
I was surprised to see that the better brands (like [Intel], [Supermicro] and to some extend [Jetway]) where not the best choices. Not only where they more expansive, they often didn't have a low-power product. So I mostly looked at [Asus], [ASRock], and to lesser extend [Gigabyte], [Jetway], [Zotac] and [Sapphire].
SATA II or SATA III
At the time of purchase, SATA III (aka SATA-600) was getting more common. SATA-300 is certainly good enough for a regular hard disk drive, though SATA-600 might be useful for a solid state drive.
Expansion slots
Nearly all motherboards has a PCI-expression version 2.0 x16 (sixteen lane) slot, but only x4 (four lanes) are wired. Despite the difference in advertisement as either "PCI-e x4" or "PCI-e 2.0 x16", there was no difference in practice. Attention was necessary: some motherboards still featured old PCI slots instead of PCI-express, while others supported the newer PCI-e 3.0 standard.
USB 2 or USB 3
At the time of purchase, USB 3 was getting more common.
Memory
All models support DDR3, although some use regular 240-pin DIMM slots and others the "laptop" 204-pin SO-DIMM slots. The amount of memory that can be installed, and the speed of the DIMMs did differ considerably,
Network
All models provided one gigabit Ethernet slot. Some motherboards has on-board wifi, but the later was a non-issue for me: wifi is way too slow (both in bandwidth and in latency) for a NAS.
mini PCI-e/mSATA
A few motherboards had a mini PCI-e slot. Such a slot has the same form factor as a mSATA slot, but the latter is not standardised and in practice most mini PCI-e can only be used to connect a wifi circuit, but not a mini SATA disk. If you want to connect a disk, check for mSATA support.
RAID
Since I want to connect multiple disk with ZFS (sometime referred to as RaidZ), I don't want hardware RAID. From experiments in 2010 on a very high end system, we found that software RAID was as fast as hardware RAID. All models support both (legacy) IDE and AHCI modes. AHCI is slightly faster than IDE, but IDE is still the default setting in most BIOS, so be sure to turn it on before installing the Operating System.
Video connector
I have seen any combination of VGA, DVI and HDMI on the motherboard. Most build-in graphics processors even support two (but not three) concurrent displays.
Serial port
I would liked a serial port. Most motherboard still offer an on-board serial connector, but few still provide an external RS-232 port. Those that do tend to be the more expensive non low-power motherboards.
FreeBSD support
It may be worth while to check support for certain hardware by the OS before purchase. This is particular true for expansion cards which usually feature a specific chipset.

The table below lists some of the better options. Despite that Intel has some pretty good south bridges, I wasn't able to find some low-power motherboards with enough SATA-3 ports. On the other hand, despite that the AMD A50M Hudson controller does not support USB3, the vendors en-masse added another chip to include USB3 on the motherboards. Note that the list does not contain maximum memory size and bus speeds; finding that information proved to be too unreliable.

My first choice was the [Asus E35M1-I], but I learned that it was no longer available just when I wanted to make the purchase. The [E35M1-I deluxe] and [E45M1-I] were a lot more expensive, and had one fewer SATA-600 port (it was replaced with eSATA). My second choice was the [Sapphire PURE Mini E350], but I decided against by lack of proper online documentation. The next choice was [ASRock E350M1/USB3], which I bought.

Motherboard CPU Southbridge SATA2 (3Gb/s) SATA3 (6Gbps) eSATA Memory Graphics RAID Expansion slot USB Video Extra
Asus E35M1-I AMD E350 AMD A50M 0 6 0 2× DDR3 DIMM Radeon HD 6310 no (IDE / AHCI) PCI-E 2.0 x4 6 USB2 DVI + VGA
Asus E35M1-I DELUXE AMD E350 AMD A50M 0 5 1 2× DDR3 DIMM Radeon HD 6310 no (IDE / AHCI) PCI-E 2.0 x4 2 USB3 + 4 USB2 HDMI + VGA
Gigabyte GA-E350N-USB3 AMD E350 AMD A50M 0 4 0 2× DDR3 DIMM Radeon HD 6310 no (IDE / AHCI) PCI-E 2.0 x4 2 USB3 + 4 USB2 HDMI + DVI + VGA
Sapphire PURE Mini E350 AMD E350 AMD A50M 0 5 1 2× DDR3 SODIMM Radeon HD 6310 no (IDE / AHCI) PCI-E 2.0 x4 2 USB3 + 4 USB2 HDMI + DVI + VGA mini PCIe (no mSATA)
ASRock E350M1 AMD E350 AMD A50M 0 4 1 2× DDR3 DIMM Radeon HD 6310 no (IDE / AHCI) PCI-E 2.0 x4 6 USB2 HDMI + DVI + VGA
ASRock E350M1/USB3 AMD E350 AMD A50M 0 4 1 2× DDR3 DIMM Radeon HD 6310 no (IDE / AHCI) PCI-E 2.0 x4 2 USB3 + 4 USB2 HDMI + DVI + VGA
Asus E45M1-I DELUXE AMD E450 APU AMD A50M 0 5 1 2× DDR3 DIMM Radeon HD 6320 no (IDE / AHCI) PCI-E 2.0 x4 2 USB3 + 4 USB2 HDMI + VGA wifi
Asus AT5NM10T-I Intel Atom D525 Intel NM10 4 0 0 2× DDR3 SODIMM Intel GMA3150 no (IDE / AHCI) PCI-E 1.0 x4 6 USB2 VGA
Supermicro X7SPA-H-D525 Intel Atom D525 Intel ICH9R 6 0 0 2× DDR3 SODIMM Intel GMA3150 yes (optional) PCI-E 2.0 x4 2 USB2 VGA 2x Ethernet port
Asus P8H67-I Intel Core (separate) Intel H67 4 2 0 2× DDR3 DIMM on Intel core CPU yes (optional) PCI-E 2.0 x16 2 USB3 + 6 USB2 HDMI + DVI + VGA
Asus P8H67-I DELUXE Intel Core (separate) Intel H67 2 2 1 2× DDR3 SODIMM on Intel core CPU yes (optional) PCI-E 2.0 x16 2 USB3 + 4 USB2 HDMI + DVI + VGA
Asus P8H67-I PRO Intel Core (separate) Intel H67 2 2 1 2× DDR3 SODIMM on Intel core CPU yes (optional) PCI-E 2.0 x16 2 USB3 + 4 USB2 HDMI + DVI + VGA wifi
Gigabyte GA-H67N-USB3-B3 Intel Core (separate) Intel H67 2 2 1 2× DDR3 DIMM on Intel core CPU yes (optional) PCI-E 2.0 x16 2 USB3 + 4 USB2 2xDVI + VGA
Zotac H67-ITX-C-E Intel Core (separate) Intel H67 4 2 1 2× DDR3 SODIMM on Intel core CPU yes (optional) PCI-E 2.0 x16 2 USB3 + 4 USB2 HDMI + DVI wifi
Asus P8H77-I Intel Core (separate) Intel H77 4 2 0 2× DDR3 DIMM on Intel core CPU yes (optional) PCI-E 2.0 x16 2 USB3 + 6 USB2 HDMI + DVI + VGA
Intel DQ67EP Intel Core (separate) Intel Q67 2 2 2 2× DDR3 DIMM on Intel core CPU yes (optional) PCI-E 2.0 x16 2 USB3 + 4 USB2 2x DVI mini PCIe (no mSATA)
Jetway (J)NF9E Q77 Intel Core (separate) Intel Q77 4 2 0 2× DDR3 SODIMM on Intel core CPU yes (optional) PCI-E 2.0 x16 2 USB3 + 3 USB2 HDMI + DVI + VGA RS232 port; 2x Ethernet port
Asus P8Z77-I DELUXE Intel Core (separate) Intel Z77 2 2 2 2× DDR3 DIMM on Intel core CPU yes (optional) PCI-E 2.0 x16 4 USB3 + 4 USB2 HDMI + DVI wifi

Choice of SSD and HDD

Choice of Other Hardware

Additional SATA Controller

Replacement Fans

The chassis comes with two fans: a 14cm for the air inlet at the front and a 12cm fan for the air outlet at the top. I was not very happy with the fans, especially because they would both run at full speed all the time: neither has a 4-pin PWM (pulse with modulation) connector, meaning that the motherboard does not control their speed.

I decided to buy two replacement fans, hoping to reduce the noise. The noise of a fan is proportional to the fifth power of the rotational velocity. So a fan at 2000 rpm generates 2⁵ = 32 times as much noise as the same fan at 1000 rpm. Sounds is usually measured in decibel (dB), which is a log-scale of the energy. So a factor of 32 in energy equals to a offset of (10×¹⁰log 32 =) 15 dB. That's a big difference. Here is some stats for noise:

15 dB

rather than in energy, wh speed of a fan has a huge impact on the noise. 

Since the chassis does not have a 5.25" bay, I could not install The 12cm outlet would be controlled with a PWM connector, while the 14cm inlet would run at full speed.

Configuration Loudness Level
Server off (background noise) 30 dB(A)
no fans, 0 HDD (mostly the PSU fan) 31 dB(A)
regular fans, 3 HDD 35 dB(A)
regular CPU fan, silent chassis fans, 3 HDD 33 dB(A)
silent fans, 3 HDD 31 dB(A)


Power Supply

Four factors influenced my choice for the power supply unit (PSU): physical size, power output, available connectors, and efficiency.

The Lian Li PC-Q25 only supports a small power supply unit (PSU). In fact, the company changed the specs from "maximum 180mm" to "maximum 140mm" after reviewers complained about it.

I expected the PC to run at 100 Watt power even with all disks installed, and most PSU are well above 300 Watt. With such a margin, this was of no criterium to me.

I only needed a 24-pin power socket for the motherboard, three 4-pin molex connectors for the five disk bays (some bays share the connector), and three SATA power cables. I didn't expect to use any more connectors for fans (the motherboard has two pins) or PCI-express cards. All PSU that I considered offered this, so this was also not a big criterium to me.

Power efficiency was important to me. Most PSU have an efficiency of 80%, thus 20 Watt of each 100 Watts input would be spoiled as heat in the PSU. I instead opted for a Chieftec A-85 Series CTB-350S, which has a 85% efficiency.

The Result

50 Watt continuous

Noise (use iPhone, not accurate, but good enough)

Check PCI spec

Do IOPS test for SSD and HDD