Virtual SAN requires minimal 3 ESXi hosts. With version 6.1 of Virtual SAN, Remote Office/Branch Office (ROBO) and small SMB customer environments are supported with Virtual SAN on 2 ESXi nodes. With a 2 node Virtual SAN cluster options such as HA, DRS and vMotion are fully supported.
In a ROBO configuration you have two Virtual SAN data nodes and one witness node. The Virtual SAN data nodes can be in one location. The witness node can reside in the same or another location (not on the Virtual SAN).
A virtual witness appliance is needed when a split brain occurs or performing maintenance to figure out what VMs does have quorum (more than 50% VMs objects needs to be available). This can can be 1 ESXi host with Virtual SAN and the witness or 2 ESXi hosts with Virtual SAN.
A Virtual SAN ROBO environment example looks like this:
- 2 VMware ESXI with Virtual SAN enabled
- A witness appliance is running on a ESXi server in the same or other site.
Here are some considerations for using Virtual SAN ROBO:
Witness
- With Virtual SAN ROBO, a witness appliance is needed. The witness appliance is placed on a third ESXi server. This hosts does not need a Virtual SAN license and SSD disk.
- The witness appliance is a nested ESXi host (ESXi running in a VM).
- It is not supported to run the witness on Virtual SAN.
- The witness hosts stores only VM witness components (metadata).
- The VMs are only protected by a single failure (FTT=1).
- The virtual witness appliance can be configured in the following flavors (depending on the supported VMs):
| Tiny
<=10 VMs) |
Normal
<=500 VMs |
Large
> 500 VMs |
|
| vCPUs | 2 | 2 | 2 |
| RAM | 8 | 16 | 32 |
| Virtual disks (*1) | 8 GB boot
10 GB SSD 15 GB HDD |
8 GB boot
10 GB SSD 350 GB HDD |
8 GB boot
10 GB SSD 350 GB HDD |
| MAX witness components | 750 | 22000 | 45000 |
(*1) The SSD and HDD are virtual disks. There is no need to have a physical SSD disksin the ESXi host were the witness appliance resides.
Hardware
- Deploy Virtual SAN on certified hardware. Check the Virtual SAN HCL!
- For a Virtual SAN disk configuration a minimal of 1 SSD and 1 Magnetic disk is needed. These disk cannot be used for booting ESXi
- For booting ESXi use a USB, SD or SATADOM device
- A small ESXi host can be used for the witness appliance. The witness appliance has no data, only metadata.
Networking
- Cross connecting 2 Virtual SAN ESXi nodes is NOT supported
- For 10 or less VMs a 1 Gbps network connection can be used. For >10 VMs use 10 Gbps
- Network bandwith to the witness: 1.5 Mbps
- Latency to the witness: 500 Milliseconds RTT
- Latency between the data nodes: 5 Milliseconds RTT
Licensing
- Virtual SAN is licensed separately.
- Virtual SAN for ROBO is a license that includes a 25 VM pack license. This license does not include the stretched cluster and All-flash options.
- A maximum of 1 Virtual SAN for ROBO license may be used per site.
- When running less than 25 VMs consider a VSAN standard of advanced license. The standard and advanced licenses are licensed per CPU socket.
- Consider single socket CPU servers to decrease the licensing costs.
- Consider vSphere Essentials (plus) for licensing the vSphere environment to reduce licensing costs.
- Consider ESXi Hypervisor (free) for placing the witness appliance. ESXi Hypervisor cannot be managed by a vCenter Server!
- For each ROBO Virtual SAN you need a dedicated witness appliance.
vCenter Server
- When running the vCenter Server on top of Virtual SAN, powering down the Virtual SAN cluster involves a special procedure (link). Consider placing the vCenter Server on the witness host for simplicity.
Good write up Ivo.
Just a little clarification.
Virtual SAN ROBO Licensing & a ROBO “config” are different, and there has been some confusion.
1) Virtual SAN ROBO Licensing is specific to a 25 VM-Pack, which can be run at a single site, or across multiple sites, with any version of vSphere. Virtual SAN ROBO Licensing is not limited to 2 hosts, but can use more if necessary. Virtual SAN ROBO Licensing has the same features at Virtual SAN Standard licensing in 6.1.
2) 2 Node configurations with Virtual SAN 6.1 can be run using any Virtual SAN license (Standard, Advanced, or ROBO). As 2 Node configurations are mechanically the same as Stretched Clusters, a Virtual SAN Advanced license is required in Virtual SAN 6.1, when more than 2 hosts are used.
More information about Virtual SAN 6.1 Stretched Clusters and 2 Node configurations can be found here:
https://www.vmware.com/files/pdf/products/vsan/VMware-Virtual-SAN-6.1-Stretched-Cluster-Guide.pdf
A short, and to the point guide specific to the bandwidth requirements can be found here:
https://www.vmware.com/files/pdf/products/vsan/vmware-virtual-san-6.1-stretched-cluster-bandwidth-sizing.pdf
Thanks,
Jase
Hello Ivo. Another point to keep well in mind is the need to have a route between the subnet used for VSAN traffic and the witness appliance. This is mandatory to have the metadata replicated between the 3 nodes.
@Jase Thanks for the clarification!
@Fabio If a static route is needed depends on the network configuration. The management VMkernel for the witness VM, in the central datacenter, can
easily reside on the same management VLAN in the central datacenter, not
requiring any static routing.
based on the recommendation of put vCenter into another host, I found that 6.5 also supports vCenter HA, which also makes use of a witness host, can a witness VM handle both vCenter HA and VSAN or I will need one witness for each component?
Hi,
Please can you guide me regarding VSAN ESXI licensing. It’s a Robo setup I am considering here. So, considering if before configuring Robo setup & policies we need to setup 2 ESXi hosts but how this ESXI are going to be licensed! Do we need general ESXi licenses to be implemented first & then after configuring the Robo setup we need to apply Robo license on top of the infrastructure. Please need your urgent suggestion here. My email – pritam81@gmail.com.
Thanks in advance – Pritam.