Missing datastore (LUN), EnableResignature

 

Yesterday a customer called me that one of the VMware ESX hosts is missing 1 datastore (LUN). The other VMware ESX hosts in the cluster had no problems with the datastore. Looked in the NetApp filer to see the presentation of the LUN, everything was okay.

After investing the log files on the VMware ESX host i found the following message:

Jan 22 12:39:34 boz101 vmkernel: 152:21:01:58.702 cpu4:1040)ALERT: LVM: 4476: vml.020003000060a980004335434b464a4b77554e59744c554e202020:1 may be snapshot: disabling access. See resignaturing section in SAN config guide.

It seems that the VMFS volume has been detected with a different LUN ID. The LUN ID is stored in the LVM header of the volume. When the LUN ID changes you need to enable resignature option to see the datastore again.

In the evening we powered all VMs on that LUN down.  On the host that was missing the datastore, I enabled the resignaturing option by:

– Go to the configuration tab of the VMware ESX host and select Advanced settings

– Click on LVM, and changed  the EnableResignature  0 (off) in 1 (on).

Did a rescan on the VMware ESX host and the LUN appeared as “SNAP-17EABCCC-VMWARE-LUN-03”. Tried the change the “SNAP-17EABCCC-VMWARE-LUN-03” LUN back to the old name “VMWARE-LUN-03”,  the following message appeared:

The name “VMWARE-LUN-03” already exists.

The solution was to switch the view to DATASTORES in vCenter and unregister all VMs on the “SNAP-17EABCCC-VMWARE-LUN-03” LUN. After all VMs were unregistered, i was able to rename it back to  the old “VMWARE-LUN-03” name. Then i register all the VMs on the “VMWARE-LUN-03” LUN and start them.

I checked the datastores on the other VMware ESX hosts,  they were fine. As last step i changed the EnableResignature option back to 0 (off) on the VMware ESX host.

 [ad#verticaal]

 

Move the VirtualCenter SQL database or migrate the VirtualCenter SQL database to different server.

VMware released two new handy KB articles about how to move or migrating  the SQL VirtualCenter database to different server.

To move the database

When relocating your SQL database:

  1. To move SQL Server databases to a new location by using Detach and Attach functions in SQL Server, see Microsoft KB article http://support.microsoft.com/kb/224071/en-us
  2. If you are using the Copy Database Wizard in SQL Server 2000 see, Microsoft KB article http://support.microsoft.com/kb/274463
  3. Update the Datasource (DSN) in the OBDC Administrator to reflect any changes made.
  4. If either the database login or password changed during the relocation, VirtualCenter must be updated to with the new credentials. For more information, see the Modifying the username and password VirtualCenter uses to connect to the database server section of Troubleshooting the database data source used by VirtualCenter Server (1003928).

Read the KB article here.

Migrate the database to a new server:

 
  1. Shutdown the VirtualCenter Server service. For more information, see Stopping, starting, or restarting the VirtualCenter Server service (1003895).
  2. Take a backup of the SQL database.
  3. If the SQL database is also being moved, create a second instance of your database and use the vendor’s tools to migrate the data.Note: For SQL Server, use Microsoft’s Copy Database Wizard. For more information, see http://support.microsoft.com/?kbid=274463

    .

     

     

     

  4. Create the appropriate System DSN connections on the new VirtualCenter Server host.
  5. Begin the installation of the VirtualCenter software on the new server. If you are installing VirtualCenter in a virtual machine, guidelines for deploying VirtualCenter in a virtual machine, including sizing, installation, functionality, and configuration of VMware HA can be found at http://www.vmware.com/vmtn/resources/798.
  6. When prompted, select Use existing database, and provide the correct credentials to that database.
  7. When prompted, select to not re-initialize the data, as this erases all your inventory data.
  8. Reboot the virtual machine when the installation is finished.
  9. When you first start the VirtualCenter Client, it may ask for licenses. Configure the licenses as you had previously in your environment. For more information about licensing for ESX Server 3 hosts, see the Installation Guide. For more information about licensing for ESX Server 3i hosts, see the Setup Guide. You are now able to see the same settings and configuration details.Note: If the IP address of the new VirtualCenter Server has changed your ESX Servers must be made aware of that change, otherwise the ESX Servers continue to send their heartbeats to the original IP address of VirtualCenter and appear as Not Responding.To correct this situation:

     

     

    1. Log in as root with an SSH client to each ESX host.
    2. Use a text editor to change the IP address inside the <serverIp>xxx.xxx.xxx.xxx</serverIp> tags the following file:/etc/opt/vmware/vpxa/vpxa.cfg (VirtualCenter 2.5.x)
      /etc/vmware/vpxa.cfg (VirtualCenter 2.0.x). 

       

    3. Save your changes and exit.
    4. Restart the management agents. For more information, see Restarting the Management agents on an ESX Server (1003490).
    5. Repeat for all ESX hosts connected to the VirtualCenter Server.

 

Read the KB article here.

[ad#verticaal]

The VMware VirtualCenter Server service terminated unexpectedly

After rebooting the VMware VirtualCenter (VC) server, i was unable to logon using the VI-client in VC. In the Windows event log the following  Error is listed:

Event Type: Error
Event Source: Service Control Manager
Event Category: None
Event ID: 7031
Date:  25-9-2008
Time:  13:39:58
User:  N/A
Computer: VC1
Description:
The VMware VirtualCenter Server service terminated unexpectedly.  It has done this 1 time(s).  The following corrective action will be taken in 300000 milliseconds: Restart the service.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

It seems that the VC server service is started before the SQL 2005 service is started. The solution is the delay the VirtualCenter service from starting till the Microsoft SQL 2005 service is started. Microsoft has a KB article “How to delay loading of specific services”.

I did the following:

Open the regedit.exe and browse to the HKLM\SYSTEM\CurrentControlSet\Services\MSSQLSERVER (for MS SQL 2005 standard). So you know the name of the service.

image

Browse to HKLM\SYSTEM\CurrentControlSet\Services\vpxd and open the DependOnService property

image

Add the name MSSQLSERVER to this multi-string. Ensure that the last line is empty.

image

Close the Registry Editor and restart the VirtualCenter server and test if it VMware VirtualCenter service starts.

[ad#verticaal]