POSTS

Recovering from lockdown mode, a corrupt vCenter, and no DCUI

One of the worst case scenarios when securing an ESXi host is disabling the DCUI, enabling lockdown mode, and then losing vCenter for some reason. If your vCenter database is corrupt then you have lost the ability to manage the host. The official answer at this point is to rebuild the host. While I hope you have an automated build process that would make this easy, there is at least one other option to recover your system.

DISCLAIMER: This is not supported or endorsed by VMware. The steps below assume that you have experience with Linux system administration. The official solution is to rebuild your host.

A quick refresher on lockdown mode: When you enable lockdown mode the system removes the permissions for all of the standard users except the vpxuser account which is what vCenter uses to manage the system.

Here are the steps to be able to manage your system again:

  1. Shut the host down. Yes, that means a hard crash for the host and any running VMs.
  2. Reset the password for the vpxuser account to a known value. Here is  an article from Bernhard Bock on doing it for root. The details in the instructions might vary slightly from your environment, but should be enough to get someone experienced with *nix pointed in the right direction. Use this process to reset the vpxuser account instead of the root account.
  3. Add the host into your vCenter inventory using the vpxuser account.
  4. The following steps may not be necessary, but if you are going to run with lockdown mode disabled from now on I would do them just in case the system does not clean up everything properly on a host add.
    1. Enable lockdown mode
    2. Disable lockdown mode

If you see issues with this process or have other ideas on how to recover the host in this situation please add a comment or send me an email so I can update the post.

comments powered by Disqus