Today, I encountered with a weird problem where someone discussed that he was using vCenter a day before and it was working all fine and the very next day, he is not seeing the inventory at all. I asked if there is any permission issue , but he said he logged in with administrator credentials and with complete privilege. He rebooted the appliance just in case and what he found vCenter is not starting and end up error like "failed to start file system"
This was strange thing for me when he said he has done nothing with vCenter and is working all fine and all of a sudden this issue. I inquired more on this and he said "he has no idea what changed" and you know what this is the most dangerous statement for me that nothing changed but software behaved like a ghost. Just kidding.
After some head breaking with logs and few articles from internet I got to know this could have been caused due to unplanned power outages. So need to fix the problem coz this is vCenter and people get panic when their management tool does not work or does not appear to work . I told him with a smile, you messed something but I will help to fix it
so here are some tips to fix the photon OS File system. We made sure, we have snapshot before we proceed with the steps.
Boot the OS again (power on vCenter appliance) and press "e" button that will take you to GRUB edit screen
Add systemd.unit=emergency.target at end of line which starts from Linux
Continue booting by pressing F10
Run below commands to find filesystem
/bin/sh
/bin/mount
/blkid
e2fsck -y /dev/sda3
Power off the vCenter appliance
Power it ON and the vCenter is back with its inventory and permissions as it was there
So this was learning for me and I thought of adding it to my "How-To" list
Thanks for reading.
