Slow network throughput between Ubiquiti EdgeRouter VLANs

After putting my NAS in a separate storage VLAN I noticed that my network throughput was slow to and from the NAS. The throughput of a simple file copy from a windows laptop (wired at 1 GbE) to a SMB share on the NAS was only 17,7 MB. The Windows laptop was on a different VLAN then the NAS.

After some troubleshooting I noticed that the throughput within the VLAN was okay  (120 MB/s on a 1 GbE network). The routing between the VLANs is done by Ubiquiti EdgeRouter Lite. I investigated the Ubiquiti EdgeRouter settings by using SSH and noticed that VLAN forwarding for IPv4 was disabled by using the following command:

ubnt@ubnt:~$ show ubnt offload

IP offload module : loaded
IPv4
forwarding: enabled
vlan : disabled
pppoe : disabled
gre : disabled
IPv6
forwarding: disabled
vlan : disabled
pppoe : disabled

IPSec offload module: loaded

Traffic Analysis :
export : disabled
dpi : disabled
version : 1.354

The following commands enable VLAN forwarding for IPv4 and save the setting:

ubnt@ubnt:~$ configure
[edit]
ubnt@ubnt:# set system offload ipv4 vlan enable
[edit]
ubnt@ubnt: commit
[edit]
ubnt@ubnt:# save
Saving configuration to '/config/config.boot' ...
Done
[edit]

This config change can be done without a reboot of the router. After making this change the network throughput was as expected. So make sure to do some tests and check VLAN offloading in the Ubiquiti EdgeRouter (it’s default disabled).

Configure VM autostart in the ESXi Embedded Host Client

For a standalone ESXi host I manage the host with the ESXi Embedded Host Client (HTML client). So no vCenter Server is used to manage this host. The standalone ESXi host is 24×7 up and running and has some critical infrastructure VMs for my lab and home environment. The critical VMs are automatically powered-on when when the ESXi host is booted with the autostart option in the host client.

In the latest versions of the Host Client (In vSphere 6.7 version 1.25 is included that already contains the autostart improvements) the autostart configuration is greatly improved what result in an easier configuration of autostart. if you are on vSphere 6.0 or 6.5 I suggest to upgrade to the latest ESXi Embedded Host Client before configuring autostart.

The upgrade of the Host client is easy, no maintenance mode and reboot of the ESXi host is needed. The upgrade can be done by following these steps:

  • Download the latest VIB for here, link
  • Upload the VIB on a datastore on your ESXi host
  • SSH to the ESXi host
  • Enter the following command to update the host client
esxcli software vib update -v /vmfs/volumes/<datastore>/<vibname.vib>

  • Refesh the host client webpage (https://<esxihostname>/ui/)
  • Check the version information in the host client (Help -> About)

Configure autostart in the ESXi Embedded Host Client

  • Open the ESXi host client by using the following URL:  https://<esxihostname>/ui/
  • Go to: Manage -> System -> Autostart->Edit Settings
    • Enable: Yes
    • Start delay: 60 seconds
    • Stop delay: 60 seconds
    • Stop action: shut down
    • Wait for heartbeat: yes
    • Save

  • Below the screen are the VMs listed. First enable autostart per VM by using the “Enable  autostart for this VM” button.
  • Once the autostart is enabled per VM, the order can be configured by increase or decrease the start order. The autostart order is displayed in the “autostart order” field.
  • Configure the autostart and order for the VMs you want to automatically start when the ESXi server is booted.
  • Reboot the ESXi host to test the autostart

With older versions of the ESXi Embedded Host Client it is “more complicated” to set the start order per VM. To set the autostart order with older versions:

  • Enable autostart as described above

  • In the Virtual Machines section, right click on the  field row and “Select columns”. Select the following columns:
    • Autostart order
    • Start delay
    • Stop delay

  • Right click on the VM with the “Autostart order” status on Unset and select “increase” to enable autostart and set the order

  • Configure the autostart and order for the VMs you want to automatically start when the ESXi server is booted.
  • Reboot the ESXi host to test the autostart

Upgrading a vCenter Server Appliance (VCSA) to version 6.7

Last week VMware launched vSphere 6.7. In this blog post I show how easy it is to upgrade a vCenter Server 6.x appliance to a new vCenter Server 6.7 appliance using the graphical interface (GUI) upgrade. The GUI upgrade uses a two stage process:

  • Stage 1: Deploy a new vCenter Server 6.7 appliance
  • Stage 2: Transfer the services and configuration data from the old to the new appliance

Upgrading the vCenter Server Appliance includes deploying a new appliance (version 6.7). The configuration and data is transferred from old (6.0 or 6.5)  appliance to the new vCenter Server 6.7 Appliance.  The old appliance is still available in a powered down state in the vCenter Server inventory after the upgrade.

vSphere 6.7 is the last release to include vCenter Server for Windows. After this release, vCenter Server for Windows will not be available! So make sure that all new deployments and upgrades are using the the vCenter Server Appliance (VCSA)!

New enhancements

Some cool enhancements of the vCenter Server 6.7 appliance are:

  • The vCenter Server with Embedded PSC supports Enhanced Linked Mode. This gives the following benefits:
    • No load balancer required for high availability and fully supports native vCenter Server High Availability.
    • SSO Site boundary removal provides flexibility of placement.
    • Supports vSphere scale maximums.
    • Allows for 15 deployments in a vSphere Single Sign-On Domain.
    • Reduces the number of nodes to manage and maintain.
  • vSphere 6.7 supports repointing a vCenter Server to another external Platform Services Controller in the same SSO site and different SSO site within the same SSO domain
  • vSphere 6.7 supports repointing a vCenter Server (Appliance only) to another external Platform Services Controller in a different SSO domain.
  • The vSphere Appliance Management Interface (VAMI) on port 5480 has some great new enhancements:
    • Upgraded Clarity interface
    • Dedicated monitor tab
    • Services tab. See the status of the VCSA services and the ability to: stop, start and restart services. So no CLI is needed for that anymore!
    • Backup scheduler. The backup scheduler let you schedule a backup of the VCSA and select how many backups are retained. The supported protocols for backup locations are: FTP, FTPS, HTTP, HTTPS and SCP.
  • The vSphere Client (HTML5) has updated and includes new workflows on Update Manager and vSAN for example.

Before upgrading

Before upgrading make sure to check this:

  • Check the compatibility of the VMware and third party products you are using. When writing this blog the following VMware products are not compatible (yet) with vSphere 6.7:
    • NSX
    • Horizon. Horizon 7.4 is not compatible with the Instant Clone API used in vSphere 6.7. Instant Clone support for vSphere 6.7 will be available in an upcoming Horizon release.
    • VMware Integrated OpenStack (VIO)
    • VMware vSphere Integrated Containers (VIC)
    •  vCloud Director
  • For the upgrade order of multiple VMware products see the “Update sequence for vSphere 6.7 and its compatible VMware products (53710)” KB, link
  • It’s only possible to upgrade the vCenter Server Appliance version 6.0 or 6.5 to 6.7.
  • For vSphere 5.5 you must first upgrade to vSphere 6 or vSphere 6.5 before upgrading to vSphere 6.7
  • Make sure you have enough capacity in the cluster to add an extra vCenter Server Appliance (VCSA). The old appliance can be removed when the upgrade is successful. Here’s an overview of the hardware specifications needed.
  • In vSphere 6.7, only TLS 1.2 is enabled by default. vSphere 6.7 disables TLS 1.0 and TLS 1.1 protocols for improved security. Some applications might support only the older protocols. To revert TLS 1.0 and TLS 1.1 protocols use the TLS reconfigurator tool. The tool can be found in the appliance under: /usr/lib/vmware-TlsReconfigurator/VcTisReconfigurator.
  • Windows 2003 and XP are no longer supported.

Platform Services Controller (PSC) hardware sizing

OptionEnvironmentvCPUMemory (GB)Default Storage (GB)
Platform Services Controller2460

vCenter Server Appliance (VCSA) hardware sizing

OptionEnvironmentvCPUMemory (GB)Default Storage (GB)
TinyUp to 10 hosts or 100 VMs210250
SmallUp to 100 hosts or 1000 VMs416290
MediumUp to 400 hosts or 4000 VMs824425
LargeUp to 1000 hosts or 10000 VMs1632640
X-LargeUp to 2000 hosts or 35000 VMs2448980
  • Use a temporary fixed IP address
  • Make sure that you have the SSO administrator and root account information of the existing VCSA
  • Have a backup of the VCSA
  • Disable Fully Automated DRS during the upgrade

The upgrade steps

In the following steps a single vCenter Server  Appliance with an embedded PSC and vCenter Server role will be upgraded to version 6.7.

  • Mount the VCSA ISO (VMware-VCSA-all-6.7.0-8217866.iso)
  • Navigate to the <drive letter>:\vcsa-ui-installer\win32\ folder and open the installer.exe
  • Choose for the upgrade option. With the option you can upgrade a PSC and vCenter Server appliance

  • The upgrading process will enter “stage 1”, deploy the appliance

  • Connect to the source vCenter Server 6.x appliance

  • Enter the SSO and root username of the VCSA and the ESXi server that manages the source appliance

  • Set up the target appliance VM name and root password. The upgrade will maintain the original FQDN name of the VCSA. This name will be used as VM name in the VCSA inventory and can be changed later!

  • Select the (new) size of the new appliance.

  • Configure the network settings. Make sure to use an new temporarily IP address for the upgrade. After the upgrade the new appliance will use the original IP address!

  • Click on finish to start stage 1

 

 

 

 

 

  • After a while the following message appears and you’re ready to continue to stage 2

  • Stage 2 will copy data from the source vCenter Server Appliance to the new deployed appliance.
  • A pre-upgrade check will be run with informational messages will be shown such as:
    • Disable Fully Automated DRS during the upgrade
    • An NSX extension has been found that may not work after the upgrade

  • The data that needs to migrated can be selected. A new cool thing is that the amount of time that’s involved is displayed.

  • Configure the VMware Customer Experience Improvement Program (CEIP)

  • Ready to start and complete fase 2

  • The source VCSA will be shut down.

  • The data transfer and appliance setup is running

  • A couple of messages will be displayed about for example Auto Deploy and that TLS 1.0 and TLS 1.1 are disabled in vSphere 6.7.

  • Stage 2 is completed and the vCenter Server Appliance is deployed.

  • Now you can access the vCenter Server by using vSphere Client (HTML5), the vSphere Web Client or VMware Appliance Management Interface by using the original FQDN of the vCenter Server Appliance.

After the upgrade the VCSA 6.7 version is: 6.7.10000 build 8217866