hardware

Intel X79 whitebox for vSphere 5 and Hyper-V 3

Updates:

Update August 8 2012: Added Microsoft Windows 2012 Hyper-V screenshots and link to blog post how to enable the Intel 82579V NIC

Update August 9 2012: Updating to the latest BIOS enables support for DirectPath I/O in VMware vSphere. Screenshot added

In an earlier blog post (found here) I mentioned that it is time for a new homebrew whitebox based on the Intel X79 chipset. With the X79 chipset it is possible to install 64GB of memory (8 x 8 GB). Because the 8 GB DIMMs are expensive on the moment, I decided to use 8 x 4GB DIMMs (total 32GB).

I decided to create one physical host for testing VMware vSphere 5, vCloud Director, VMware SRM, VMware View 5 etc. The possibility to create a physical ESXi5 server, create virtual ESXi  hosts on it and start VM on the virtual hosts is great! This feature is called nesting. How to do this, can be found on William Lam blog found here.

Components used for the VMware ESXi 5 / Microsoft Windows Server 2012 whitebox:

  • Intel i7-3820 CPU 3.60 GHz, 4 cores, with Hyper threading 8 cores
  • Zalman CNPS10X performance cooler
  • Asus P9X79 s2011 motherboard. Some specs:
      • Socket 2011
      • 8 DIMM slots, supports 64GB memory
      • Expansions slots: 2 x PCIe 3.0 (dual x16), 1 x PCIe (x8 mode), 2 x PCIe 2.0 x1, 1 x PCI
      • 2 SATA 6 Gb/s port, 4 x SATA 3 Gb/s
      • LAN: Intel 82579V Gigabit LAN controller
  • 2 x Corsair Vengeance DDR3- 1600 16GB (4 x 4) kit, total 32GB memory (max 64GB)

The case, power supply, graphical card, RAID controller and extra NIC(S) are reused. Here are some photos of the configuration:

image image
image image
image

When the hardware  configuration was done and tried to power on the system, nothing happened (black screen). The appears that BIOS of the motherboard didn’t know the i7-3820 CPU yet. The cool thing is that the motherboard has a function called “USB BIOS Flashback”. It is possible to flash the BIOS without CPU or memory installed. Here are the steps:

  • Download the latest BIOS from the Asus site;
  • Extract the BIOS on a USB stick;
  • Rename the BIOS file, example: rename “P9X79-ASUS-0906.ROM” in “P9X79.ROM” (important);
  • Place the USB stick in the USB port with the WHITE interior on the back;
  • Press the BIOS flashback button for 3 seconds and the light will begin to flash;
  • Don’t turn of the computer during the BIOS flash;
  • When the flashing light stop, the BIOS has been complete;

After the BIOS update was finished, the system boots and I was able to install VMware ESXi and Windows Server 2012 and enable the Hyper-V role.

vSphere 5 / ESXi 5 screenshots:

image image
Hyper threading enables 8 cores 32GB memory
image image
The onboard SATA controller is listed as Patsburg 6 Port SATA AHCI controller.

Software-RAID does not work

The hardware RAID controller is added as extra PCI card

 The onboard Intel 82579V NIC is not supported in ESXi5. Use the procedure found here to add the NIC. Use at your own risk!

The Intel 82574L NIC is added as extra PCIe card.
image

The latest firmware includes support for DirectPath I/O

  

Microsoft Windows Server 2012

It is possible to install Microsoft Windows 2012 and enable the Hyper-V role. Here are some screenshots:

image image
image image

The onboard Intel 82579V NIC is not recognized in Windows Server 2012 by default. How-to enable the Intel 82579V NIC is explained in this explained in this blog post.

This whitebox is a great extension to my home lab!

[ad#banner]

What about VMware Virtual Machine hardware versions

I often get the question: “What Virtual Machine hardware version do I need?”.

It depends on the features you need. If you want for example use the “Changed Blocked Tracking (CBT)” feature, you need at least hardware version 7.

In ESX 3.x hardware version 4 is introduced, in vSphere 4.x hardware version 7 is introduced and in vSphere 5 hardware version 8 is introduced. Here is an overview of the hardware version and the features they have:

Hardware version Features Products
8 – Up to 32 vCPUs per VM
– Maximum 1 TB RAM per VM
– 3-D graphics and high-definition audio
– Smart-card reader support
– USB 3.0 devices are supported
– Improved network driver for the E1000e network adapter, provided by VMware tools
– Greater resources are available with vCloud Director 1.5		 Hardware version 8 is the default for new VM in:
– ESX 5.x
– Fusion 4.x
– Workstation 8.x
– Player 4.x
7 – Serial Attached SCSI (SAS) virtual device for Microsoft Cluster Service — Provides support for running Windows Server 2008 in a Microsoft Cluster Service configuration.
– IDE virtual device — Ideal for supporting older operating systems that lack SCSI drivers.
– VMXNET Generation 3. VMXNET is optimized for performance in a virtual machine 
– Virtual Machine Hot Plug Support— Provides support for adding and removing virtual devices, adding virtual CPUs, and adding memory to a virtual machine without having to power off the virtual machine.
– Change Block Tracking (CBT) support. Incease VADP backups		 Hardware version 7 is the default for new VM in:
– ESX 4.x
– Fusion 3.x
– Fusion 2.x
– Workstation 7.x & 6.5
– Player 3.x
– Server 2.x
4 Hardware version 4 is the default for new VM in:
– ESX 3.x
– ACE 2.x
– Fusion 1.x
– Player 2.x
3   Hardware version 3 is the default for new VM in:
– ESX 2.x
– GSX Server 3.x

Considerations before upgrading the hardware version of the VM:

– Important to know is that upgrading the hardware version of the VM requires downtime!

– Virtual machines with hardware version 7 can only run on ESX(i) 4.x and ESXi 5.x. Virtual machines with hardware version 8 can only run on ESXi 5.x

– When you upgrade from virtual hardware version 4 to version 8, the upgrade is reversible if you take a virtual machine backup or snapshot before performing the upgrade.

– To automate this process, consider using Update Manager for virtual machine upgrades

– Update Manager takes automatic snapshots before performing virtual machine upgrades

– Be sure to upgrade first the VMware tools of the VM.  I you upgrade the virtual hardware before you upgrade VMware Tools, the virtual machine might lose its network settings

– Verify that all VMs and .VMDK files are stored on VMFS3, VMFS5 or NFS volumes

 

Steps in the hardware version upgrade process:

– Do an inventory on the current hardware and VMware tools versions. This can be done for example by using the vCenter client, RVtools utility or PowerCLI

– Install or upgrade the VMware tools (reboot required)

– Power on the VM

– Before upgrading create a backup or snapshot of the VM

– Backup the NIC IP settings with the VMUpgradeHelper.exe command. More information can be found here

– Power off the VM

– Upgrade Virtual Hardware

– Start VM  (reboot after the new hardware is discovered)

– Check if all the IP addresses are correct

 

Downgrade methods:

There is no button in vCenter to revert back to an earlier Hardware version. Here are two methods to go back to an earlier version of the hardware version:

– Create before upgrading the hardware version a snapshot when the VM is powered down.

– Using VMware Converter

 

Upgrading issues to know about:

– Upgrading virtual hardware in ESX 4.x may cause Windows 2008 disks to go offline (more information can be found here)

– After a hardware version upgrade the configuration can be messed up on  for example Microsoft ISA, Microsoft NLB clusters and RSA servers

– After upgrading a Windows virtual machine from hardware version 4 to hardware version 7, virtual NIC settings (such as static IP configuration) are lost. Make sure you backup the VM IP settings with the VMUpgradeHelper.exe command. More information can be found here

 

[ad#banner]