HP SPP upgrade failing with unexpected errors

This week while upgrading server Firmware/Drivers on HP ProLiant G9 servers using latest HP SPP, on one server SPP online deployment return with following,

And when we checked install logs found all updates failed. As lately I had faced an issue where HPSUM inventory was failing due to missing HP management tools and as per my past experience I was aware about the fact, VMware update installations or individual vib install may fail if server has no esxi image profile attached/image profile got corrupted hence thought of cross checking the same.

On checking found this server had no host image profile defined,

When checked the esxupdate.log file found entries of the name of HPE custom esxi 6.0 u2 install file that mean the install was done using the correct ISO image and somehow it got corrupted later.

Then checked for the available image profiles by searching for Imgdb.tgz file (this is the image profile backup and by looking at its size we can get an idea if the image profile got corrupted), this will list the two Imgdb.tgz files as follows,

By looking at the size of these files, size of one imgdb.tgz file is comparatively very less and seems got corrupted.

We can fix this broken image profile issue by either doing an upgrade or fresh install. There is one more way to fix the above issue and that is by copying/replacing the host image profile of this host with an image profile from a working host.

We can do that as follows,

Use winscp or any other ftp client to connect to any healthy Esxi host and browse to /bootbank dir and copy the Imgdb.tgz file to your system and now connect to the host that is missing the image profile and paste the earlier copied Imgdb.tgz file to /tmp dir.

Now

Remove the corrupted imgdb.tar file,

rm /bootbank/Imgdb.tgz

Extract the Imgdb.tgz dir copied from healthy host,

tar -xzf /tmp/Imgdb.tgz

then copy the working image profile and related vibs to /var/db/esximg/profiles and /var/db/esximg/vibs directories,

cp /tmp/var/db/esximg/profiles/* /var/db/esximg/profiles/

cp /tmp/var/db/esximg/vibs/* /var/db/esximg/vibs/

After this run following cmd to create backup of image,

/sbin/auto-backup.sh

Now when you check for the installed profile, it will show you one. It wouldn’t reflect the host profile name in host summary until you re-start the host mgmt agents or reboot the host.

After doing above, when I re-initiated the SPP online update, this time host firmware/driver got updated without any further issue.

Related blog post: http://www.provirtualzone.com/esxi-6-0-fix-corrupted-host-imageprofile/

That’s it… J

HP SPP online update: HPSUM host inventory failed with an error like re-enter the credentials

Lately, I was upgrading HP servers firmware/drivers using HP SPP online upgrade method and for one HP ProLiant server after adding the node to HP SUM inventory when I try to inventory my box, got an inventory error and it asks me to re-enter my credentials while the entered credentials were correct.

Usually when a node is added you must inventory it against the SPP repository so HPSUM could compare and determine what is needed for your box.
When googled about the issue, most of the results pointed out this host seems missing some HPE VIBs that must be on the box prior to attempting to use HP SUM, this was strange as this host was installed using the HPE provided custom ESXi 6.0 U2 ISO image.

Here in order to fix the issue, downloaded and installed the HPE custom ESXi 6.0 U2 image bundle and after that there was no issue with SPP update (as this also installed all missing HP management components).

To upgrade the esxi install using VMware offline update bundle zip use,

 esxcli software vib update -d /vmfs/volumes/…./vmware_update_offline_bundle.zip

There are other ways too to upgrade the esxi host, like installing one of the available profiles in offline update bundle.

To list the installed image profile we can use following cmd,

esxcli software profile get

Now list the available image profile in download VMware update offline bundle,

 esxcli software profile list -d /vmfs/volumes/…./vmware_update_offline_bundle.zip

The output of above cmd would list the available image profiles from the downloaded vmware_update_offline_bundle.zip, note down the name of image profile you want to install, now

 esxcli software profile install -d /vmfs/volumes/…./vmware_update_offline_bundle.zip -p vmware_inage-profile name

This will install/update the desired esxi image profile.

That’s it… J