Monthly Archives: October 2017

HTTP Error 500 Post Upgrade to vCloud Director 9.0

This week I decided to jump on the upgrade bandwagon along with a number of other excited people in the vExpert Slack group.  While most, if not all, had success stories I unfortunately ran into some post upgrade portal issues.

The upgrade process to version 9.0 was no different from previous releases.  I followed my regular upgrade process which went off without issue.  When I went to log into the Administrator Portal I was faced with an HTTP Error 500 page.  Argh!

HTTP ERROR 500

Problem accessing /cloud/saml/login/alias/vcd. Reason:

Server Error

Caused by:

javax.servlet.ServletException: org.opensaml.saml2.metadata.provider.MetadataProviderException: No IDP was configured, please update included metadata with at least one IDP at org.springframework.security.saml.SAMLEntryPoint.commence(SAMLEntryPoint.java:161) at org.springframework.security.saml.SAMLEntryPoint.doFilter(SAMLEntryPoint.java:107) at com.vmware.vcloud.web.NestedFilterChain.doFilter(NestedFilterChain.java:45) at com.vmware.vcloud.web.UnfirewalledFilterChainProxy.doFilter(UnfirewalledFilterChainProxy.java:62) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)

To my surprise tenant Portals were fine and able to log in.  This was Admin Portal specific.

Checking the release notes I knew there was a breaking change with Federation and SAML which required you re-register your organization with your SAML IDP.  That’s fine I thought, were not using SAML.  And besides the notes seem to indicate you make the change post upgrade.

System administrators cannot use an existing vSphere SSO configuration to authenticate to vCloud Director.

Federation for the System organization has changed in this release. The System organization can now use any SAML IDP, not just the vSphere Single Sign-On service. Existing federation settings for the System organization are no longer valid and are deleted during the upgrade.

Workaround: Re-register your organization with your SAML IDP. See “Enable Your Organization to Use a SAML Identity Provider” in the vCloud Director Administrator’s Guide

Turns out, though, we were in fact using SAML, or at least had it enabled in a non functioning state.  So despite the release notes stating that it would be deleted, it appeared to remain in a broken state post upgrade and now was preventing the Portal from loading at all.

The solution turned out to be relatively easy with VMware GSS help.  Login to the Admin Portal specifying the full URL to the login.jsp file with your standard System Administrator account.

https://portal.mydomain.local/cloud/login.jsp

Navigate to the Administration Page and then to Federation.  Untick Use SAML Identity Provider and Apply.

The change should take effect immediately.  Logout and back in as you normally would to the portal without the trailing /cloud/login.jsp.

While I’m sure this was a corner case please take note of your SAML settings.  If you don’t use it, make sure you don’t have it enabled.

VMware Update Manager (VUM) Fails To Load Within vSphere Web Client

I recently upgraded my lab VCSA from version 6.5 (Build Number 5705665) to version 6.5 U1 (Build Number 6671409).  After the upgrade I noticed that VMware Update Manager was no longer working correctly.  Navigating around the various VUM pages I received the same consistent error message.

interface com.vmware.vim.binding.integrity.VcIntegrity is not visible from class loader

VUM management page

VUM Tab within an ESXi host

Checking the vCenter services within Administration > System Configuration they all appeared Up and Running.  Though all services were running I never the less restarted the VMware Update Manager service which unfortunately didn’t help.  I also tried restarting a few other services without much success.  So rather than just continuing to randomly restart services I decided to take a tougher approach and restart all services from the CLI.

After the stopping and starting of all vCenter services, which took a few minutes, VUM was back up and running again within the vSphere Web Client.  While this was a fairly drastic step to take, so would have been rebooting the vCenter server, which I’m glad I managed to avoid.

I’ve previous written about restarting vCenter services.  The process is quite simple.   First connect up to the CLI of the VCSA box.  Then run the below two commands.  Both the stopping and starting of services will take a few minutes each.  Once the services are restart the Web Client will take a further few minutes to fully start up and be accessible.  If all is successful Update Manager should be accessible once again.

Command> service-control --stop --all

Command> service-control --start --all

Restarting all the vCenter services like this is obviously a disruptive action.  Connectivity to vCenter will be dropped while the services restart.  Usually restarting all services on vCenter via the CLI is my last ditch attempt to resolve an issue before I attempt a reboot of the appliance.  While restarting the VCSA might have been the easiest thing to do to resolve this issue it’s not always necessary.