Another X10SDV-4C-TLN2F-O Build Log

[seltz]

Hi, Figured I'd log my build here to add to the group of folks building with the Supermicro MBD-X10SDV-4C-TLN2F-O (Intel D-1520 45W TDP) board.

I'm migrating an older freenas done ~5+ years ago using a really old Intel SS4200 setup (single-core 35W TDP celeron, 2gb ram upgrade). The biggest mistake I made then was getting something a bit too old, and honestly the machine has been turned off for the past year. The other mistake was dealing with the headless serial i/o - it worked but was definately frustrating at times, and look forward to playing with supermicro's kvm-over-ip feature.

Core build features:
- Have freenas running with the two
- Setup crashplan on it
- Setup some other fun things on it (rss readers, sandstorm.io, etc.)
- Get this done on a budget. My uses are fairly light for now, but may add drives

Bonus features:
- Virtualize freenas and run it with pfsense. Considering the D-1520 cpu has VT-D I think this is possible (see freenas's official blog post about this: http://64.62.136.60/whats-new/2015/05/yes-you-can-virtualize-freenas.html (cache: http://webcache.googleusercontent.c...enas.html/trackback+&cd=2&hl=en&ct=clnk&gl=us ) I still need to do some more research into this.

Hardware purchased so far:
- The motherboard & cpu - MBD-X10SDV-4C-TLN2F-O - ~$416 used
- RAM - 16gb of used Hynix dell server pull stuff - $85 - I may increase this depending on how feasible the pfsense combo turns out.

Hardware I'll be reusing:
- 2 2tb drives, currently setup with ZFS in a raid1 configuration. Plan to back everything up to a separate 2tb drive beforehand just in case something terrible happens.
- possibly my ss4200 case again, if i can do a mitx retrofit similar to https://sites.google.com/site/digitalnotsense/home/intel-ss4200 - I really really like this case, but will replace it if needed.

Still need to purchase:
- a new PSU? to be determined - the original SS4200 psu is ~230W.
- a new UPS.
- ssd for hosting os's - going to do initial experiments just on a usb flash drive, but am considering buying a small ssd for the 'production' setup.
- better fans? Both the original celeron cpu and the d-1520 are setup for passive cooling, though there's a ~10W tdp increase in the upgrade. I may have to have some plastic guide for the airflow, considering the original cpu had this too.



Initial Questions:

Anything fundamentally seem wrong with my build?

 

[miraculix]

- Virtualize freenas and run it with pfsense

Which hypervisor?

- RAM - 16gb of used Hynix dell server pull stuff - $85 - I may increase this depending on how feasible the pfsense combo turns out.

- 2 2tb drives, currently setup with ZFS in a raid1 configuration. Plan to back everything up to a separate 2tb drive beforehand just in case something terrible happens.

16GB RAM total? Should be fine for a pool with just one 2x2TB vdev... but since this is virtualized, 32GB or even 64GB is safer taking into account the hypervisor, FreeNAS guest (ZFS pool and OS) and pfsense guest. Try to stick to memory SKUs from SM's test memory list on the product page for this board.

- ssd for hosting os's - going to do initial experiments just on a usb flash drive, but am considering buying a small ssd for the 'production' setup.

If this was FreeNAS directly on metal I'd say USB is fine since FreeNAS from RAM (tmpfs), but would also recommend x2 mirrored since it's just $10-15USD more and could save a headache. If you're using ESXI, USB may still be OK, otherwise SSD is likely better choice (or SATA DOM)

- better fans? Both the original celeron cpu and the d-1520 are setup for passive cooling, though there's a ~10W tdp increase in the upgrade. I may have to have some plastic guide for the airflow, considering the original cpu had this too.

I'm partial to Noctua but that's just me. Xeon D runs very cool and only needs very modest airflow, but with no airflow heat *will* build up and shut it down within minutes. You need either 60mm directly attached to the heatsink, or case fan air diverted to the heatsink adequately (1U case, or shroud like you mention).

Enjoy the build!

 

[seltz]

Thanks for the advice! Answers to the questions below

Which hypervisor?

No idea yet - I was looking into either bhyve or ESXi - I'm kindof a newbie to hypervisors, so I still have some reading up to do on this. FreeNAS's blog post on virtualization recommends ESXi.

16GB RAM total? Should be fine for a pool with just one 2x2TB vdev... but since this is virtualized, 32GB or even 64GB is safer taking into account the hypervisor, FreeNAS guest (ZFS pool and OS) and pfsense guest. Try to stick to memory SKUs from SM's test memory list on the product page for this board.

Thanks for the tip. Not afraid of buying more RAM, will do so after initial testing. I didn't get the exact sku from SM's test memory list, but looking at the datasheets I couldn't find much difference.

For Hynix, SM recommends (module / chips):
HMA82GU7MFR8N-TF / H5AN8G8NMFR
HMA41GU7AFR8N-TF / H5AN4G8NAFR

I'm using:
HMA42GR7MFR4N-TF/ H5AN4G4NMFR

If this was FreeNAS directly on metal I'd say USB is fine since FreeNAS from RAM (tmpfs), but would also recommend x2 mirrored since it's just $10-15USD more and could save a headache. If you're using ESXI, USB may still be OK, otherwise SSD is likely better choice (or SATA DOM)

Cool, will start looking out for SSDs soon.

I'm partial to Noctua but that's just me. Xeon D runs very cool and only needs very modest airflow, but with no airflow heat *will* build up and shut it down within minutes. You need either 60mm directly attached to the heatsink, or case fan air diverted to the heatsink adequately (1U case, or shroud like you mention).

Yeah, I'm leaning towards case fan air since it looks like things will fit just right with the SS4200 case, which is really made for case fan air. The board came in today and it looks like there's juust enough room to wedge the one of the case's 60mm fan right in-between the lan port and the usb/ipmi port. The case has an additional 60mm fan in the lower end of the case.


Did some basic powerup today booting into the ipmi console, was having issues getting into the bios setup screen but it might've been due to the fact that I haven't installed any RAM yet. Should have another update in a week or so.

 

[seltz]

After some random weekends of finally getting things together, this board is finally fitted and running in the old SS4200 case - there was much dremeling and drilling to make it work, but it's doable -- can share pictures to those interested.. I'm ignoring the virtualization ideas for now and just running baremetal FreeNAS 9.10.1.

So far things seem okay, thought one concern is the cpu temperature - currently cpu temps without much load are reporting 63 degC - is this normal for an Intel Xeon D-1520? I don't have a direct cpu fan on yet (and wasn't planning to since there's already two case fans near it), but I'm wondering if I should. Plan for this to eventually be tucked away somewhere so I don't hear the fan noise so much.

Heat worries aside, next steps are to:

- get a proper ssd for FreeNAS to live on (currently using a usb drive)
- setup crashplan / crashplan central
- learn how to optimize things for power - most of the time the drives don't need to be on
- finish the case and get the power switch to work - currently using ipmi to start it.

 

[seltz]

CPU Fan sorta installed - ended up with a Noctua (NF-A4x10 FLX). Observed temperature go from the insaner 63 degC to ~49 degC. Case fans seem to be spinning at a lower speed too.

 

[droeders]

CPU Fan sorta installed - ended up with a Noctua (NF-A4x10 FLX). Observed temperature go from the insaner 63 degC to ~49 degC. Case fans seem to be spinning at a lower speed too.

I tried running a passively cooled Xeon D-1521 and could not keep it from overheating. Mine would shutdown within a few minutes of starting memtest. This was in a 3U chassis with 4 mid-plane fans. Supermicro told me these motherboards were meant for a short depth 1U chassis with high speed fans blowing directly over the CPU.

That was a no-go for me, so I had them replace it with a cooler with fan and it's been great ever since. I rarely get above 42C.

I love these boards, but they only have a few that ship with the active cooler (the models with 'C+' in them). I wish SM would offer more like this.

 

[JJT211]

Why dont you just migrate everything from your old freenas system to your new one then install pfSense on your old system? Pfsense runs pretty well on just about anything.

Unless you got some serious experience with virtualization, I would think running a firewall on the same machine as your Nas is a bad idea

 

[seltz]

Why dont you just migrate everything from your old freenas system to your new one then install pfSense on your old system? Pfsense runs pretty well on just about anything.

Unless you got some serious experience with virtualization, I would think running a firewall on the same machine as your Nas is a bad idea

I should give an update on that, I was being pretty aspirational but out of laziness more than anything else gave up on that idea and am just running baremetal freenas for now. For the router portion I caved and upgraded to some ubiquiti networking hardware. Also worth noting that this motherboard needs a gigabit-compatible port on the other end for it's non-ipmi ethernet nics. This made me finally get rid of a couple ancient 10/100 routers I was still using.

 

[seltz]

I tried running a passively cooled Xeon D-1521 and could not keep it from overheating. Mine would shutdown within a few minutes of starting memtest. This was in a 3U chassis with 4 mid-plane fans. Supermicro told me these motherboards were meant for a short depth 1U chassis with high speed fans blowing directly over the CPU.

That was a no-go for me, so I had them replace it with a cooler with fan and it's been great ever since. I rarely get above 42C.

I love these boards, but they only have a few that ship with the active cooler (the models with 'C+' in them). I wish SM would offer more like this.

Interesting!

This may explain why I got this board discounted (and saw this board returned/open-box on newegg a couple other times).

Surprisingly didn't have the machine lock up when running for a few days without the cpu fan, though the ss4200 case has pretty low height clearance with hard drives installed (the noctua cpu fan barely fits, and may need to be screwed in at an offset to allow for a tiny bit of room for HDD cables unless I flip the drive over or get cables with a opposing 90deg connections.

Just for anyone curious about seeing pics, here's some in-progress shots:

Basically to get it all to fit in the ss4200 case, I had to do the following:

- dremel out holes for usb / vga / ethernet ports
- cut corners in the case fans to stand over motherboard components
- drilled out new motherboard mount holes / got some high standoffs (stood higher by some nuts after noticing the standard 6mm ones weren't high enough)
- bought an atx power cable extender

Still working on mounting the ssd and the cpu fan, but it surprisingly works even with the original 250W power supply. Haven't futzed with the power-button wiring yet, just doing ipmi for now. Overall happy though, I have an unhealthy obsession with this case's slim form factor paired with being able to hold 4 drives. Power-restrictions aside one might be able to internally mount 1-2 more drives with the extra space from the motherboard size reduction (left is the old ss4200 mobo, right is the x10sdv-4c-tln2f:

 

[JJT211]

Haha yea

I tried running a passively cooled Xeon D-1521 and could not keep it from overheating. Mine would shutdown within a few minutes of starting memtest. This was in a 3U chassis with 4 mid-plane fans. Supermicro told me these motherboards were meant for a short depth 1U chassis with high speed fans blowing directly over the CPU.

That was a no-go for me, so I had them replace it with a cooler with fan and it's been great ever since. I rarely get above 42C.

I love these boards, but they only have a few that ship with the active cooler (the models with 'C+' in them). I wish SM would offer more like this.

Yup, I have the 1521 and had to buy a seperate fan as well.