HP AMS driver problem in ESXi 5.x

The last weeks several customers contacted me that one or more ESXi hosts are not manageable anymore. The ESXi host has the following symptoms:

  • Unable to manage using the vSphere Client, Web Client or SSH.
  • Unable to perform a vMotion
  • On the ESXi console there is a message: can’t fork
  • Enable services from the console does not work

The VMs are still running on the ESXi host. This is a known issue and can occur when running the following versions of the HP AMS driver:

  • hp-ams 500.9.6.0-12.434156
  • hp-ams-550.9.6.0-12.1198610
  • hp-ams 500.10.0.0-18.434156 (Added 13-11-2014)
  • hp-ams-550.10.0.0-18.1198610  (Added 13-11-2014)

In some situations the ESXi servers is still manageable in vCenter, so vMotion can be used . In all the other situations shutdown the VMs by using RDP or SSH and reboot or reset the ESXi server. The HP AMS driver must be upgraded to version 10.0.1 or can be disabled.  The VMware KB can be found here.

I developed a simple PowerCLI script to list the HP-AMS driver version of each ESXi host.

Update 17-12-2014: I developed a new PowerShell tool called VIBSearch. This this graphical tool it is easier to search a specific VIB. More information can be found here: Link


## Problem with HP AMS hp-ams 500.9.6.0-12.434156 and hp-ams-550.9.6.0-12.1198610
## http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2085618

# Add PowerCLI module
Add-PSSnapin vmware.VimAutomation.core -ErrorAction SilentlyContinue

# VMware VirtualCenter server name
$VCserver = Read-Host "Enter your vCenter server"
$Username= Read-Host "Enter the Username"
$Password = Read-Host "Enter Password"

# Connect to the vCenter Server
Connect-VIServer -Server $VCserver -User $Username -Password $Password -port 443

# Clear screen

$GetHosts = Get-VMHost | Where {$_.ConnectionState -eq “Connected”}
ForEach ($VMHost in $GetHosts) {
$ESXCLI = Get-EsxCli -VMHost $VMHost
Write-Host $VMHost -ForegroundColor Green
$ESXCLI.software.vib.list() | Where {$_.Name -eq “hp-ams”} | Select Name,Vendor,Version | FT

# Disconnect session vCenter
Disconnect-VIserver -Confirm:$false

Example output:


Add HP or Dell drivers to VMware Update Manager (VUM)

HP has created a vSphere Installation Bundle (VIB) depot for all sort of important HP driver bundles for VMware vSphere 5 and higher. The VIP depot contains:

    • HP ESXi 5.0 Management Providers bundle – includes the latest HP Common Information Model (CIM) Providers, HP Integrated Lights-Out (iLO) driver, HP Compaq ROM Utility (CRU) driver, and the new HP Agentless Management Service (AMS).
    • HP ESXi 5.0 Utilities bundle – ESXCLI utilities such as HPBOOTCFG (boot order configuration),HPONCFG (remote iLO configuration) and HPACUCLI (Smart Array reporting and configuration)
    • HP ESXi 5.0 NMI bundle – Non Maskable Interrupt (NMI) driver used to write VMware® errors to the Insight Management Log (IML)
    • HP Agentless Management Service Offline Bundle – a service that provides support for Agentless Management and Active Health. Agentless Management Service provides a wider range of server information (e.g. OS type and version, installed applications, IP addresses) allowing customers to complement hardware management with OS information and alerting. Agentless Management provides Integrated Lights Out (iLO) based robust management without the complexity of OS-based agents. Active Health provides 24×7 mission control for servers, delivering maximum uptime through automated monitoring, diagnostics and alerting.
    • Device Drivers as used in the HP Customized VMware images
    • Latest ProLiant Server and option firmware and driver version recipe

The VIB depot can be used by the following tools:

  • VMware Update Manager (VUM)
  • ImageBuilder

With the HP VIB depot integration for example in VUM there is no need to download and install the bundles manually. In this blog post I explain how to add the HP VIB depot to VMware Update Manager (VUM) by using the following steps:

Add the VIB Depot to VUM

  • Open the vCenter Client
  • Select under Solution and Application in the main screen of the vCenter client – Update Manager
  • Select configuration
  • In the Download sources select – Add Download Source
  • In Source URL use:
    • For HP Updates: http://vibsdepot.hpe.com/index.xml
    • For Dell Updates: http://vmwaredepot.dell.com/index.xml
  • Press OK
  • Select – Download Now to update the baseline


Create a new Baseline

  • Select the Baselines and Groups tab
  • Create
  • Baseline Name  – HP Updates
  • Baseline Type – Host Extension
  • Extensions to Add – Select the extensions you need. In this example I selected the following extensions:
    • hpnmi for ESXi 5.0 v1.3
    • HP ESXi 5.0 Complete Bundle Update 1.3
    • HP ESXi 5.0 Management Bundle 1.2-26
    • HP Utility Bundle ESXi 5.0 v 1.2


  • Finish

Attach the baseline and remediate

  • Attach the baseline to the cluster


  • Select the baseline and scan for patches and extensions
  • The HP bundles that are missing are listed


  • Use Remediate to install the HP bundles

When the hosts are rebooted and the remediation is finished, the Host Compliance overview is 100%.



Using the  HP VIB depot is great way of keep your HP server up-to-date with the latest HP bundles.

More information can be found HP VIB bundle site found here.

Update July 20 2012 : Viktor van den Berg tested the above configuration and found out that this only works with new  installations. For existing installation you need to create an additional Host Patch Baseline. Viktor explains this on his cool blog found here here.

Display HP P4000 SAN/iQ version, installed patches and up-time

During a health check I needed to know the SAN/iQ version, installed patches and the up-time of the HP P4000 storage nodes because of a nasty bug in SAN/iQ 9.0 or 9.0.01 that hang or reboot nodes after 208.5 days (More information about the bug can be found in the HP Customer Advisory found here)

To see the SAN/iQ version, installed patches and the up-time (thanks to Calvin Zito @HPStorageGuy for guiding me to the up-time counter in CMC)


Here are the steps:

1. Open the Centralized Management Console (CMC)

2. Login

3. Expand the cluster

4. Expand the Storage Systems

5.  Select Diagnostics for the first node (1)

6. Select “Hardware information” (2)

7. Select “Click to refresh” (3)


8. The Storage System Software displays the SAN/iQ version and Software Patches installed:


9. Scroll down to the Stat section and look at the Machine Uptime. The up-time is in hours, 2138/24= 89 days up-time for one node.


10. Repeat step 5 till 9 for all storage nodes