Haswell low power whitebox for ESXi and Hyper-V

I was searching for a new whitebox for my home lab.  I had the following requirements for the new whitebox:

  • Low power consumption for 24×7 running
  • >16 GB memory
  • Expansion slots for PCI(-E) cards
  • Good performance
  • Low noise

A couple of weeks ago Intel released the new 4th generation Haswell CPUs that consumes less power. Seems to be interesting option for building a low power consumption whitebox. So I did some research and ordered the following hardware components:

CPU i5 4570S Boxed foto-7_thumb1
Motherboard Gigabyte GA-Z87-D3HP
Memory Corsair Vengeance 4 x 8 GB DDR3 PC3-12800 (DDR3-1600) 32GB
Power Supply Seasonic G-360 80 Plus Gold

The following components I reused:

  • The case
  • 2 Intel PCI-e NICS
  • A SSD drive

CPU

Haswell is the codename for the 4th generation Intel Core processors. One of the big improvements of the Haswell CPUs is the idle power consumption. The Intel S-version is also a low-power CPU.  It contains 4 cores and has a 65W TDP. The processor supports vPro, VT-x, VT-d, EPT etc.   A CPU cooler is in the box included. For the full specifications look here.

Motherboard

The Gigabyte GA-Z87-D3HP is a socket 1150 motherboard. The board has  4 memory sockets that support  DDR3 memory up to 32 GB.  It has onboard graphics , 6 x SATA 6 Gb/s connectors, a Intel WG-217v LAN adapter and the following expansion slots:

  • 1 x PCI Express x16 slot
  • 1 x PCI Express x16 slot, running at x4
  • 2 x PCI Express x1 slots
  • 2 x PCI slots

Memory

For the memory I choose the Corsair Vengeance CML32GX3M4A1600C10 4 x 8  PC3-12800 (DDR3-1600) CL10 kit. All the four memory sockets on the board are filled with a 8 GB module (total = 32 GB memory).

Power Supply

The Seasonic G-360 power supply has a 80 Plus Gold certification. This is a great power supply with high efficiency and low noise..

 

Hypervisor support

I tested VMware ESXi 5.1 Update 1 and vSphere 5.5 and Microsoft Windows Server 2012 with the Hyper-V role installed.

VMware ESXi

The onboard Intel I217-V NIC is not recognized  by ESXi 5.1 Update 1. To get the I217-V NIC working in ESXi5.x read the blog post found here. I reused 2  Intel PCI-e NICs, one for LAN and the other for iSCSI and NFS traffic. The onboard SATA controller  (Lynx Point AHCI) is recognized. I use an existing SSD for booting ESXi and running some important VMs. But is is possible to boot ESXi from USB stick. The other VMs are on NAS device. Passthrough is supported by the CPU and motherboard.

image image
image image

 

Microsoft Windows Server 2012 with the Hyper-V role

I tested Windows Server 2012 with the Hyper-V role enabled. The The onboard Intel I217-V NIC is not recognized by default. In the “Enable the Intel I217-V NIC in Windows Server 2012” blog post I explain how to enable the I217-V NIC Windows Server 2012.

 

image image
image image

Power consumption

The whitebox consumes a maximum between 40-50 W. When idle (and that is often) it consumes only 28 a 29 W!

foto (8)

The whitebox components cost about €615. and will be used for running VMs 24×7.  The whitebox meets all the requirements I had.

Control (remotely) the power of your Home Lab

I have a lab at home to test for example VMware vSphere and  Microsoft stuff. Running your home lab for 24/7 will result in a high electricity bill. For a couple of months I use the Internet Control Station ICS-1000  (ICS-1000) to power on my home lab when needed from anywhere.  The ICS-1000 controls (left picture) controls the receivers (right picture).  The ICS-1000 is connected to my router. In the receivers are the power cables plugged from the devices you manage.

foto (2) foto (1)

So when needing my lab environment I  open the App on my iPhone and power on the home lab  from everywhere. After a short time I can remotely access the home lab and connect for example to my:

  • NAS
  • VMware vSphere with ESXi servers environment
  • Microsoft Hyper-V environment

Using the web browser or the iPhone App for example you can program the timers to power on/off devices on specific times and dates.   

image image

I use different receivers through  the whole house and control it with the ICS-1000.  For example I control the light outside the house with timers I programmed in the the ICS-1000. The App has still some limitations and bugs.  For example it is not possible to edit timers.  To change the timers you need to delete and recreate them. Probably in March 2013 the App will be updated to solve some bugs and add new functionality.

Intel X79 whitebox for vSphere 5 and Hyper-V 3

Updates:

Update August 8 2012: Added Microsoft Windows 2012 Hyper-V screenshots and link to blog post how to enable the Intel 82579V NIC

Update August 9 2012: Updating to the latest BIOS enables support for DirectPath I/O in VMware vSphere. Screenshot added

In an earlier blog post (found here) I mentioned that it is time for a new homebrew whitebox based on the Intel X79 chipset. With the X79 chipset it is possible to install 64GB of memory (8 x 8 GB). Because the 8 GB DIMMs are expensive on the moment, I decided to use 8 x 4GB DIMMs (total 32GB).

I decided to create one physical host for testing VMware vSphere 5, vCloud Director, VMware SRM, VMware View 5 etc. The possibility to create a physical ESXi5 server, create virtual ESXi  hosts on it and start VM on the virtual hosts is great! This feature is called nesting. How to do this, can be found on William Lam blog found here.

Components used for the VMware ESXi 5 / Microsoft Windows Server 2012 whitebox:

  • Intel i7-3820 CPU 3.60 GHz, 4 cores, with Hyper threading 8 cores
  • Zalman CNPS10X performance cooler
  • Asus P9X79 s2011 motherboard. Some specs:
      • Socket 2011
      • 8 DIMM slots, supports 64GB memory
      • Expansions slots: 2 x PCIe 3.0 (dual x16), 1 x PCIe (x8 mode), 2 x PCIe 2.0 x1, 1 x PCI
      • 2 SATA 6 Gb/s port, 4 x SATA 3 Gb/s
      • LAN: Intel 82579V Gigabit LAN controller
  • 2 x Corsair Vengeance DDR3- 1600 16GB (4 x 4) kit, total 32GB memory (max 64GB)

The case, power supply, graphical card, RAID controller and extra NIC(S) are reused. Here are some photos of the configuration:

image image
image image
image

When the hardware  configuration was done and tried to power on the system, nothing happened (black screen). The appears that BIOS of the motherboard didn’t know the i7-3820 CPU yet. The cool thing is that the motherboard has a function called “USB BIOS Flashback”. It is possible to flash the BIOS without CPU or memory installed. Here are the steps:

  • Download the latest BIOS from the Asus site;
  • Extract the BIOS on a USB stick;
  • Rename the BIOS file, example: rename “P9X79-ASUS-0906.ROM” in “P9X79.ROM” (important);
  • Place the USB stick in the USB port with the WHITE interior on the back;
  • Press the BIOS flashback button for 3 seconds and the light will begin to flash;
  • Don’t turn of the computer during the BIOS flash;
  • When the flashing light stop, the BIOS has been complete;

After the BIOS update was finished, the system boots and I was able to install VMware ESXi and Windows Server 2012 and enable the Hyper-V role.

vSphere 5 / ESXi 5 screenshots:

image image
Hyper threading enables 8 cores 32GB memory
image image
The onboard SATA controller is listed as Patsburg 6 Port SATA AHCI controller.

Software-RAID does not work

The hardware RAID controller is added as extra PCI card

The onboard Intel 82579V NIC is not supported in ESXi5. Use the procedure found here to add the NIC. Use at your own risk!

The Intel 82574L NIC is added as extra PCIe card.

image

The latest firmware includes support for DirectPath I/O

 

Microsoft Windows Server 2012

It is possible to install Microsoft Windows 2012 and enable the Hyper-V role. Here are some screenshots:

image image
image image

The onboard Intel 82579V NIC is not recognized in Windows Server 2012 by default. How-to enable the Intel 82579V NIC is explained in this explained in this blog post.

This whitebox is a great extension to my home lab!

[ad#banner]