HP SPP upgrade failing with unexpected errors

This week while upgrading server Firmware/Drivers on HP ProLiant G9 servers using latest HP SPP, on one server SPP online deployment return with following,

And when we checked install logs found all updates failed. As lately I had faced an issue where HPSUM inventory was failing due to missing HP management tools and as per my past experience I was aware about the fact, VMware update installations or individual vib install may fail if server has no esxi image profile attached/image profile got corrupted hence thought of cross checking the same.

On checking found this server had no host image profile defined,

When checked the esxupdate.log file found entries of the name of HPE custom esxi 6.0 u2 install file that mean the install was done using the correct ISO image and somehow it got corrupted later.

Then checked for the available image profiles by searching for Imgdb.tgz file (this is the image profile backup and by looking at its size we can get an idea if the image profile got corrupted), this will list the two Imgdb.tgz files as follows,

By looking at the size of these files, size of one imgdb.tgz file is comparatively very less and seems got corrupted.

We can fix this broken image profile issue by either doing an upgrade or fresh install. There is one more way to fix the above issue and that is by copying/replacing the host image profile of this host with an image profile from a working host.

We can do that as follows,

Use winscp or any other ftp client to connect to any healthy Esxi host and browse to /bootbank dir and copy the Imgdb.tgz file to your system and now connect to the host that is missing the image profile and paste the earlier copied Imgdb.tgz file to /tmp dir.

Now

Remove the corrupted imgdb.tar file,

rm /bootbank/Imgdb.tgz

Extract the Imgdb.tgz dir copied from healthy host,

tar -xzf /tmp/Imgdb.tgz

then copy the working image profile and related vibs to /var/db/esximg/profiles and /var/db/esximg/vibs directories,

cp /tmp/var/db/esximg/profiles/* /var/db/esximg/profiles/

cp /tmp/var/db/esximg/vibs/* /var/db/esximg/vibs/

After this run following cmd to create backup of image,

/sbin/auto-backup.sh

Now when you check for the installed profile, it will show you one. It wouldn’t reflect the host profile name in host summary until you re-start the host mgmt agents or reboot the host.

After doing above, when I re-initiated the SPP online update, this time host firmware/driver got updated without any further issue.

Related blog post: http://www.provirtualzone.com/esxi-6-0-fix-corrupted-host-imageprofile/

That’s it… J