For a couple of months I’m searching for an extra whitebox host for extending my home lab environment. My current lab whitebox is a Haswell based whitebox (see: link). Here is an overview of the new lab environment:
For the new whitebox I had the following requirements:
- Hardware such as NICs must be recognized by VMware ESXi
- Use 32 GB memory or more
- Low power consumption
- Expandable
- Small formfactor
- Quiet
- Possibility to run nested hypervisors such as VMware ESXi and Hyper-V
- Remote Management
- Possibility to create a VMware Cluster and use vMotion, HA, DRS and DPM with the existing Haswell host
I reviewed the following popular home lab systems:
- Intel NUC
- Apple Mini
- Gigabyte BRIX
The main reason to NOT choose for one of the above systems is the only support 16 GB of memory. In November 2014 I found a motherboard that passes all the requirements, after reading a review on a Dutch hardware website. The review was about the ASRock C2750 motherboard. After some additional research I ordered the following parts to build this whitebox:
- ASRock C2750 motherboard
- Kingston 4 x8GB, DDR3, PC12800, CL11, Total 32 GB memory
- be quiet System Power supply 7 300W
- Cooler Master Midi Tower N300 ATX
VMware ESXi boots from an USB stick and the VM’s are placed on a iSCSI target so no extra storage is needed. The above parts cost me around € 735,00.
The ASRock C2750D4I motherboard has the following specifications:
- Mini ITX motherboard
- CPU: Intel Avoton C2750 64-bit 8 cores processor (passive cooled)
- Graphics: ASPEED AST2300 16 MB
- Memory: 4 x DDR3 DIMM slots, max: 64 GB memory
- Controller: Intel C2750: 2 SATA3, 4 x SATA2 Marvell SE9172 2 x SATA 3, Marvell SE9230 4 x SATA3. Total of 12 SATA ports.
- NIC: Dual Intel i210 Gigabit LAN adapter
- 1 x PCIe 2.0 x8 slot
- Remote Management: BMC Controller with IPMI dedicated LAN adapter
- TPD of 20 watt
 |
 |
CPU
The Intel Avoton C2750 is an atom based processor and contains 8 cores. It is passive cooled and quiet. The Avoton processor is 64-bit and supports Intel VT-x with Extended Page tables (EPT), so it is possible to nest hypervisors such as ESXi and Hyper-V. The Atom processor with 8 cores gives enough CPU performance for my lab environment.
Memory
The motherboard contains 4 memory banks with a maximum of 64 GB DDR3 memory (4 x 16 GB). I choose to use 4 x 8 GB Kingston DDR3, PC12800, CL11 dimms because of the price. 16 GB modules are to expensive on the moment. The motherboard has 32 GB of memory.
NICs
The ASRock C2750D4I system contains a dual Intel i210 Gigabit LAN adapter. The Intel i210 adapters are standard recognized by ESXi 5.5 and Windows Server 2012 R2. No additional modifications or drivers are needed.
Power consumption
The 300 W Power Supply is more than enough. The processor has a TPD of 20 watt. This whitebox consumes around 35 W with a couple of VMware VMs on it.
 |
The ASRock C2750D4I system is part a VMware cluster with Distributed Power Management (DPM) enabled. When DPM kicks in only 4 watt is used.
Remote Management
Management and remote control is possible because of the BMC (Baseboard Management Controllers) and IPMI (Intelligence Platform Management Interface).
 |
 |
VMware ESXi support
On the ASRock C2750D4I system, VMware ESXi 5.5 Update 2 with the latest updates is installed.
 |
 |
The Intel i210 Gigabit NICs and the Avoton AHCI controllers are recognized out of the box. So VMware VSAN (unsupported) could be an option to use.
 |
 |
The SATA 6 Gb/s controller is not recognized by default. Follow the instruction explained in the following blog to enable the controller:
- How to make your unsupported SATA AHCI Controller work with ESXi 5.5 and 6.0, link.
Update March 23, 2015: Today I installed VMware ESXi 6.0 on the C2750. Everything seems to work.
Windows Hyper-V support
As test I installed vNext Server Technical Preview on the ASRock C2750D4I system (with SSD as local storage) with the Hyper-V role enabled. The two Intel i210 Gigabit NICs are recognized out of the box. It has great performance.
 |
 |
Conclusion
The ASRock C2750D4I motherboard is a great system for building or extending a home lab environment based on VMware or Hyper-V. This board gives enough performance for a home lab and meets all the requirements I had for an additional whitebox host. I use it mainly for nesting VMware ESXi and Hyper-V hypervisors.
Hey,
I must say nice work. I wonder if you can help me here.
I am trying to build a Esxi white box for my lab use. I am a network security person and i need to do build labs for firewalls like Paloalto, checkpoint, cisco and running GNS3 on esxi. Features i am looking must for my lab whitebox are as follow:
it must support all visualization technology with 64 bit and 32 bit support.
at least 64GB of ram,
Remote management is important.
I am not too hardware savvy person, so any advise or help really highly appreciated.
Looking forward to hearing from you.
Thanks.
Meraj
I went for the Supermicro edition of same MB.
Thanks for sharing your experience, I have been following a couple of blogs on home labs utilizing the ASRock C2750D4I and a similar offering from Supermicro.
At present I am using a i7-2600 Shuttle based system with 32GB RAM which has been highly capable when it comes to running quite a large number of virtual machines that I do all at the same time. I use this system mainly as a nested ESXi lab for training and testing new products.
I am very curious to find out how this Intel Avoton C2750 low power processor is capable of supporting 10-12 Windows virtual machines of different memory and cpu requirements all powered up at the same time, mainly in terms of its responsiveness when it usability for lab use?
Ivan:
How do you find your Super Micro based system with regards to the above and which MB do you have?
Thanks,
M
Hello Ivan,
Thanks for the information on this post. How has the build been working for you? I want to build a box for ESXi 5.5 U2 and wanted to know if the CPU on this board has worked out OK for you. Have you run into any performance issues yet?
Have you test the Mombo with esxi6 Baremetal installation?
Could you provide me a full part list of your additional whitebox?
is the PCI Express port usable i.E. for a quad Port Intel Nic?
Thanks for your help.
Hi,
Thanks for your helpful blog.
any ideas: does the BIOS running on ASRock C2750D4I support SR IOV ?
Regards,
Kevin
Ivo and Ivan,
Any light you can shed on how you are finding the performance of running multiple Windows/Linux and also Nested ESXi or Hyper-V VM’s on this mobo with say using a SSD for storage?
I really want to go for this mobo to replace my Shuttle based i7-2600 system, which I have to say has been very impressive at doing the above. I want something quite, low power, light and portable to carry around with a power brick.
Would be nice to be able to see some screenshots of CPU utilization and heat dissipation/cooling problems?
Cheers!
Have you noticed the very high CPU latency under moderate load?
Is the lack of VT-d support in the CPU a problem at all?
Hello guys,
i would like to build a whitebox for support vsphere 6 essentials kit. What do you think about this:
– Cooler Master Cosmos 2 atx case
– Corsair Psu AX1200i modular 80 plus platinum certified
– Cooler Master Nepton 240m for cpu liquid cooler
– Intel Xeon 2011 E5-2620v3 2,4Ghz
– AsRock motherboard X99 WS-E for socket 2011-v3
– LSI MegarRaid Sas 9240-4i SGL 4-PORT 6GB/s sata+sas
– Corsair Dominator Platinum Kit memory ddr4 2666mhz 32gb (4×8)
– Evga Video Card GeForce Gtx970 4gb ram
– 4 Samsung SSD 850 EVO 500gb mz-75e500b/eu
– Intel PRO/1000 PT dual port server adapter 10/100/1000base-tx
for storage i’ve alread a NAS Qnap TS-451 (with 2 gb port interfaces) about 4 tb based on raid5 (here i have 4 hd wd caviar for a total of 8tb)
with this configuration (hoping that it’s good), do you suggest me to install directly the vcenter or before windows 2012 (for example) and after vcenter etc…?
in any case i would like to have a your general opinion to use at the best this hardware at least in a lab test. Thank you for all your suggestions.
I purchased this MB and stuck it in a larger, quiet, case so that I can host all my drives as well.
It installed ESX 6.0.0 (3073146) well, but it takes quite a while to boot and it seems to take for ever at the starting vxpa screen at the end.
I’m a network engineer and I too need it for nested virtualization and cisco stuff.
I’m looking around and no one seems to say one way or the other what ESX version works best, so hopefully i can get that started.
As of this writing, i’m not recommending 6 because it seems unstable and i have to reboot it a bit. Ohh and you only get to use one side of the SATA ports. ESX hates that Marvel stuff, so I’m stuck with 2 SATA3 ports and 4 SATA2 usable. which is enough.
Another gotcha. is that it only supports 3 total USB ports and no USB3.. so yeah… thats 2 in the back and one up front, or 2 upfront, and 1 in the back via jumper setting. Still not a show stopper as all I use USB for is for slow drives to backup to.
I’m hoping all my issues are related to VMWARE, and i originally tried hard to run nested virt under KVM with Fedora server and Centos on the hardware but its a hack because qemu needs to be modified for ESX to run under KVM. theres tons of stuff out there on it.
Hopefully this will get things going and i’ll respond if people have questions or comments. I use this for work to test/validate/stage configs so i’m actively using it.
-dave
Ivo, Are the SATA 6 Gb/s controller as you describe recognized on VMware ESXi 6.0.
-Borg
Yes the SATA 6 Gb/s controllers are recognized
Ivo, Thank you for the responce / bedankt voor je reactie.
Another question. Like to build a home developmentlab. Can you run 3 Esxi nested hosts on your config with a 4 ore 5 Vm’s running Windows server. Domain Controller, Citrix, Sharepoint and 3 VM’s workstations Windows 7 / 8 and 10. Glad to hear your experiance with your config.
-Borg
Is it still working? There are issues with Intel C2xxx CPU family. After 1-2 years some of the boards don’t boot…
I’m also keen to know if anyone has been affected by Intel’s bug in C2xxx family.