Nested ESXi servers are essentially virtualised ESXi servers. They can be quiet fun to play with. Usually the only reason you’d do this is for a test lab. You don’t need a physical ESXi server to nest virtual ESXi hypervisors. You can choose your product of choice.
In my case I’m running VMware Workstation 9 and virtualised ESXi 5.1 as the second level and then running virtual guests as a third level within the virtual ESXi. In Workstation 9 you have the ability to connect to ESXi hosts and view guest VMs --as below.
You don’t need to do anything special to virtualize ESXi. In fact there’s an option to select ESX as the guest OS in VMware Workstation 9.
When you turn on a virtual ESXi guest (or is it a host???) a message appears in the bottom right corner of the console. It’s pretty self-explanatory. You can create a 32-bit guest VM only on this ESXi server.
The cool cats that VMware are, allow you to run 64-bit VM guests too with a small config change. The feature is only available in ESXi 5 and above.
On the virtualised ESXi guest enable console access (or SSH) and login as root. The file ‘config’ in /etc/vmware needs to be modified and an additional line add, ‘vhv.allow = “TRUE”’.
If you’re a VI fan go ahead and use that. If you’re lazy like me you can just append it to the file with echo.
echo 'vhv.allow = "TRUE"' >> /etc/vmware/config
Once the change is made no reboot is necassary. You will now be able to run 64-bit guest VMs within your already virtualised ESXi host. In an extreme example I ran another ESXi guest at the third level to prove a 64-bit OS would run. On a second virtualised ESXi guest I’m just running a 32-bit Linux VM at that same third level.
For my test lab three levels is as deep as I need to go. I’ve read that it’s also as deep as you can go as guest VMs will fail to run at a fourth level. I don’t have the resources to test this out in my lab. I heard that there was a VMworld presentation on multiple level nesting ESXi servers. Would certainly be worth finding.
Appendix
As with many things VMware, nested ESXi is not officially supported.
VMware Link: Support for running ESX/ESXi as a nested virtualization solution
I had trouble with the vmxnet3 network drivers with the nested ESXi. Instead I chose to go with E1000 particular for that third level.