During the installation of the vSphere 5.1 Single Sign On component you will be prompted for the Fully Qualified Domain Name or IP of the server you are installing on. If the server you are installing to does not have a DNS record the installation will fail and rollback.
More often than not you’ll have a forward lookup record for your host. The SSO installation, however, will also perform a reverse lookup query on your IP. If you don’t have a reserve record you will be prompted that the installation could not fully resolve the host and the installation may fail (I can tell you it WILL fail).
It’s important to check that the host you are using has both a forward and reverse record.
The vSphere SSO installation seems to take it one step further and query every interface on the server. If you have multiple interfaces to different networks the installation will query the forward and reserve on all interfaces. If records don’t exist for all these interfaces the installation will fail. For the life of my I can’t figure out why this is import for all interfaces.
In my situation I had numerous interfaces on my blade server. Particularly two iSCSI interfaces with no DNS records. The installation would not let me proceed with a successful installation till they were fully resolvable or disabled. It’s also worth noting that if you have link local addresses (i.e. 169.254.x.x) that they will be queried too and fail the installation.
You can confirm if the installation of SSO is failing by looking at the vm_ssoreg.log logfile under your user profile folder.
{system drive}users{username}AppDataLocalTempvm_ssoreg.log