We are a small school with a small tech staff and no VMware experts, so I really hope someone can help us out.
Our VCSA 6.0 was doing fine but recently rebooted after a power failure. I realized something was wrong when I got the following when trying to connect with my browser:
503 Service Unavailable (Failed to connect to endpoint: [N7Vmacore4Http16LocalServiceSpecE:0x7fa482aaf3a0] _serverNamespace = /vsphere-client _isRedirect = false _port = 9090). Our backups are also failing with a 503 error.
Here are some notes of interest that I typed while watching the server reboot:
Init SMTP port (sendmail) failed
Waiting for VMware Service Control AGent... OK
Starting VMware vPostgres: Starting VMware Inventory Service...
Waiting for VMware inventory service...
Warning: VMware inventory service may have failed to start
sed: couldn't flush /etc/vmware-vpx//sedGKgW32: No space left on device
sed: couldn't flush /etc/vmware-vpx//sedFjrhvy: No space left on device
vmware-vpxd: VC SSL Certificate does not exist, it will be generated by vpxd
Waiting for the embedded database to start up...........failed
Executing pre-startup scripts...
vmware-vpxd: Starting vpxd by administrative request.
success
vmware-vpxd: Waiting for vpxd to start listening for requests on 8089
Waiting for vpxd to initialize....
VPXD is not running. Cannot collect live core dump....
Warning: VMware ESX Agent manager may have failed to start
Warning: Vmware vsphere profile-driven storage service MHFTS
I've dealt with issues of /storage/log running out of space in the past, and I suspect that some element of the server's storage is either full or needs fsck. However I can't get to bash to check these things. Typing "shell.set --enabled True" from the appliance shell results in "Unknown command: 'shell.set'". Trying to enable bash from the server console results in "DCUI aborted by exception". I also tried booting directly to bash from GRUB, but when I tried to fsck /dev/mapper/log_vg-log, which I'd read about somewhere, I was told that checking a mounted volume could cause severe damage. I umounted it, but then I couldn't figure out how to to fsck it.
I've reached the limit of my knowledge and would appreciate any assistance. Thank you!