When using a whitebox lab environment at home and like to test for example vSphere 5, vCloud Director, VMware View and MS Hyper-V (nested in VMware vSphere ) you need a lot of processor power and memory. In almost all whitebox lab environment the processor power is not the problem but the amount of memory is.
Till now the Sandy Bridge desktop boards support up to 32GB memory with only four DIMM slots on the motherboard. For 32GB you need 4 * 8GB DIMMs, 8GB DIMMs are very expensive on the moment when writing this post.
Intel Introduced the Sandy Bridge-E processors and motherboards with the X79 based chipset that support this processors. This gives new possibilities for building a new whitebox home lab.
Processor
Intel introduced the Sandy Bridge-E or the 2nd generation Core i7 Extreme Processors. On the moment there are two Sandy Bridge-E processors available:
– Intel Core-I7-3960X 3,3GHz,15M L3-cache, list price around €950,00
– Intel Core-I7-3930K 3,2GHz,12M L3-cache, list price around € 550,00
As you can see the processors are pretty expensive. The Sandy Bridge-E has the following features:
– Socket LGA2011;
– 6 cores (12 cores with Hyper-Threading);
– Quad channel DDR3-1600 memory controller;
– Supports Hyper-Threading, Intel VT-x, VT-d;
– PCI-Express 3.0 support;
– 40 PCI-Express lanes;
– Max TDP 130 W;
– Multiplier unlocked.
The 2nd generation Core i7 Extreme Processors can be compared here.
Begin 2012 Intel will release the Core I7 3820 Sandy Bridge-E processor. This processor will support 4 cores (8 with Hyper-Threading) and have 10MB L3-cache. The price is not announced yet but will be much lower as the Intel 3930K and 3960X processors. When you buy a Sandy Bridge-E processor there is no CPU cooler in the box.
Motherboard
The Intel X79 chipset support the Socket LGA2011. To choose a motherboard you can for example check the following things:
– How much DIMM slots it has (some X79 motherboard have 4 DIMM slots);
– How much expansion slots it has;
– Price;
– How many USB ports and what speed they have;
– Type and number of SATA controllers;
– Type of number of NIC(s).
Important for a whitebox lab configuration is that the SATA controller (if you want to use local storage) and NIC(s) are supported by VMware ESXi. When choosing a motherboard with enough expansion slots you can always add extra RAID and NIC cards that are supported.
The most X79 based motherboards have 8 DIMM slots and supports up to 64GB memory. 8GB memory modules are expensive. So you can make a configuration with 8 x 4GB = 32 GB DDR-3 memory which is much cheaper. When 8GB memory modules become cheaper you can upgrade to 64GB memory.
Asus has a nice overview of all the X79 bases series motherboards they have, found here.
Shopping list
I made a shopping without the case, storage and the power supply. The prices are taken from the Tweakers pricewatch (Dutch) an can change every day.
Shopping list:
Component | List price (€ ) |
Intel Core-I7-3930K | 520,00 |
Asus P9X79 | 230,00 |
4 x 4 x 2GB DDR3-1600 memory (total 32GB) | 160,00 |
Simple processor cooler | 30,00 |
Simple graphic card PCI-E | 25,00 |
Total | 965,00 |
Conclusion
With the Sandy Bridge-E processor and X79 based motherboard you can build a monster whitebox lab environment with the best performance on the moment.
The advantage of using one huge whitebox you can use nesting to run your VMware vSphere and MS Hyper-V environment(s) on one box. An other advantage of using a whitebox is that you can make a low noise system. It’s a lot of money for a whitebox home lab environment but if you you wait for the Intel Core I7 3820 Sandy Bridge-E processor (announced in January 2012) can save you around €270,00 (this is an assumption because the list prices are not available yet).
[ad#banner]
Hi,
Is your setup working well?
Are vt-d, x79 raid controller work well with esxi5?
I plan to make one with deluxe version.
The post didn’t stated that i have such configuration, but i like to have one. The Sandy Bridge-E processor is om the moment in the Netherlands not available. It will be soon available i think. If someone has a Sandy Bridge-E processor and a X79 motherboard it would be great if you post the experience on your home lab.
Hello guys! Bought P9X79 DELUXE with Intel Core-I7-3930K processor for a new whitebox and cant make it work as I would like to :). I do manage to install ESXi 5.0 but in RAID mode datastores are not recognized as well as one of the NICs (one from INTEL). Drivers are still missing. Going to wait until someone with more knowledge sort this out… Using Hyper-V until then.
Cheers!
@SWE Software RAID doesn’t work with ESXi5. use a hardware RAID controller. You can make the onboard NIC to work by using this post https://www.ivobeerens.nl/2011/12/13/vmware-esxi-5-whitebox-nic-support/
@Ivo Beerens “Software RAID doesn’t work with ESXi5. use a hardware RAID controller” Can you explain a bit more, plz? Does it mean that I need to buy a separate controller card? In that case, which one you know works with ESXi 5.0? Built in RAID controller on P9X79 DELUXE does not work with ESXi 5.0…
NIC: I can´t make it work even if i follow those steps (link you gave me https://www.ivobeerens.nl/2011/12/13/vmware-esxi-5-whitebox-nic-support/) But that is not a major issue. I do have other NIC:s I can use in this case (PCIe NIC) just if I make RAID work. Even a whitebox without RAID is unnecessary waste of time in case harddrive broke…
Hyper-V works fine thoe. But the problem is that pfSense I installed worked much better on ESXi (Low troughput on Hyper-V)
So, I would love to make ESXi 5.0 work on my new whitebox but if it means that I need to spend more money to make it work, I´ll just leave Hyper-V on 🙂
@SWE The SATA controller is detected in ESXi (See screen shots, so you attach hard disks and use them in ESXi as separate disks. When you want to use a RAID set, you need a separate supported hardware RAID controller. Software RAID is not recognized in ESXi.
If you cannot make the onboard NIC to to work (follow the procedure as described), then add an extra PCIe NIC that is supported by ESXi.
So spending extra money depends if you want to use a RAID set in your config.
I have an ASUS P9X79 Deluxe with an Intel i3970x running ESXi 5.1, and VT-d isn’t working, despite that the motherboard, chipset and CPU all supposedly support VT-d!
John, re-check VT-d is enabled via BIOS.
Best,
Roberto