Issue
When trying to access Vcenter, the user gets errors and cannot log in. Below is one example. "Unhealty upstream". Also, 503 errors are sometimes seen. Unable to access the web GUI of Vcenter is the common issue.
Environment
1. Product: PremierOne
2. Category: CAD/Mobile
Resolution
How to use the vCert tool to check and renew vCenter and ESXI certificates
PART 1 - OPENING VCERT
*Make sure SSH is enabled for the vCenter virtual machine PINFVCSA or SINFVSCA -
https://blog.ukotic.net/2020/08/06/enable-ssh-on-vcenter-server-7/
1: Copy the vCert tool to the Vcenter appliance virtual machine. Normal - 10.192.3.33 for PR and 10.192.13.33 for DR - see keepass for credentials
Vcert tool here. This is a program that runs on the VCSA virtual machine which is Linux based. The vCert is NOT a certificate. It renews and checks certificates that are already on the machine.
2: Open WinSCP
Add this string to the Protocol options in Winscp
“shell /usr/libexec/sftp-server”
3: Once connected, copy the vCert tool to root directory of the vCenter appliance vm
4: Connect to vCenter via Putty
Login as Root
Password: See keepass
5: Once logged in, type shell
6: Type chmod 777 vCert then press enter to change the vCert tool permissions
7: Once that is done, type ./vCert to launch the vCert program. **case sensitive - please use a lower case v and a capital C.
8: Type y and then press enter to acknowledge the risks
You then get to the menu
PART 2 USING VCERT
Checking for expired certificates
Enter the adminstrator@vsphere.local for user, and the corresponding password
You will then get a list of what certificates are expired or not
Resetting certificates
If certs are expired and Vcenter is having trouble loading or causing other errors like “unhealthy upstream” or 503 error, use option 6 to generate and apply new certificates
Example error that expired certs cause:
After selecting option 6:
You can enter the proper country code, state, locality and organization name such as the site acronym if desired. It is not necessary to fix the issues, but it is nice to do.
After you fill in all the options, vCert will go through and renew all the certs.
At the end, it will ask to restart services. Type yes, then press enter. This is only downtime for vCenter. The P1 Application is not affected by this.
Once the services are finished restarting, it will bring you back to the menu
You can then choose option 1 again to view the certificate status.
To renew ESXI certificates, follow the menu items under option 7 - ESXi certificate operations
PART 3 - 3 Ways to check the dates of expiring certificates
1: Via vSphere Administration (can only be done if vCenter is accessible)
Enter the vSphere login credentials
View the expiry
2nd way to check
In Putty, run this command when connected to the vSphere VM (If you are still in the vCert tool, type “exit” to exit the tool)
for store in $(/usr/lib/vmware-vmafd/bin/vecs-cli store list | grep -v TRUSTED_ROOT_CRLS); do echo "[*] Store :" $store; /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store $store --text | grep -ie "Alias" -ie "Not After";done;
Note all the expiry dates listed
3rd way to check
Run a certificate report in vCert
You’ll be prompted to enter user and password credentials for vCenter to run this.
Once it is done, the saved report is available via Winscp at this location:
Save this file and open it in Notepad++
Once it is opened, use ctrl F to open find, and type “not after” and click find all in current document. Look at the expiry dates and note any that are soon or are already expired
Now you’re a vCert Pro! Have fun and don’t break anything. Reach out to T3 for additional support.