Install a VMWare ESXi 6.0 Hypervisor in a Hyper-V VM

Recently I’ve been playing around with the new Hyper-V Nested Virtualization feature within Windows 10 (build 10565 and greater) and Windows Server 2016. It is pretty cool to be able to create virtualized lab environments running that contain Hyper-V clusters. But what if we want a lab that contains VMWare ESXi Hypervisors running on Hyper-V host. I couldn’t find the process documented anywhere and I couldn’t even confirm if it should be possible. But after lots of asking a lot of annoying questions – thanks Adam Burns – Googling and hair pulling I managed to get it going:

ss_vmwareinhv_proof

So this seems like a good topic for a blog post.

What You’ll Need

You are going to need a few things to get this working:

  • A Hyper-V host running on Windows 10 (built 10565 or greater) or Windows Server 2016 TP4.
  • Enable-NestedVM.ps1 – A PowerShell script for enabling Nested Virtualization in a Hyper-V VM. Click here to get the file from the Microsoft team on GitHub.
  • A VMWare account – just sign up for one here if you don’t already have one.
  • VMWare PowerShell CLI installed – I used 6.3 release 1 that I downloaded from here.
  • ESXi-Customizer-PS.ps1 – A PowerShell script for injecting network drivers into an ESXi 5.x/6.x ISO. I downloaded it from here.

I suggest you download all of the above items to a working folder – I called mine d:\ESX-In-Hyper-V, so these instructions will reflect that, but you can call your folder what ever you like.

You should end up with a folder containing these files:

ss_vmwareinhv_neededfiles

And before you ask: No, you don’t need an VMWare ESXi 6.0 ISO – this will get downloaded and produced for us.

The Process

Part 1 – Prepare an ESXi 6.0 ISO with Network Drivers

The biggest problem I ran into when trying to install ESXi onto Hyper-V was that the ESXi kernel doesn’t come with drivers for the Microsoft Virtual Network Adapter or the Microsoft Legacy Network Adapter (emulates a DECchip 21140). So you’ll need to inject these drivers into the VMWare ESXi 6.0 ISO. Luckily there is a script available and the appropriate drivers DECchip 21140 (called “net-tulip” for some reason) that makes this process a breeze:

  1. Install WMWare PowerCLI.
  2. Open a PowerShell console.
  3. Enter the following commands:
    CD D:\ESX-In-Hyper-V\
    .\ESXi-Customizer-PS-v2.4.ps1 -v60 -vft -load net-tulip
    
  4. After a few minutes the VMWare ESXi 6.0 ISO will be downloaded and the “net-tulip” drivers merged with it:

ss_vmwareinhv_createesxiiso

The ISO will now be available in the D:\ESX-In-Hyper-V folder:

ss_vmwareinhv_neededfilesandiso

Part 2 – Create the Hyper-V VM

  1. In Hyper-V Manager create a new Virtual Machine:ss_vmwareinhv_newvmpath
  2. Click Next.
  3. Select Generation 1 and click Next.
  4. Set the Startup Memory to at least 4096MB.
  5. Uncheck Use Dynamic Memory for this Virtual Machine:ss_vmwareinhv_newvmmemory
  6. Click Next.
  7. Don’t bother to Configure Networking on the next step – just click Next.
  8. Select Create a new virtual hard disk and set the Size to 10GB (this is just going to be the boot disk for the ESXi Hypervisor):ss_vmwareinhv_newvmdisk
  9. Click Next.
  10. Select Install an operating system from a bootable CD/DVD-ROM.
  11. Select Image file (.iso) and browse to the ISO created in Part 1.ss_vmwareinhv_newvminstallation
  12. Click Next then click Finish to create the Virtual Machine:ss_vmwareinhv_newvm
  13. Right click the new Virtual Machine and select Settings.
  14. Select the Processor node and increase the Number of Virtual Processors to at least 2:ss_vmwareinhv_vmsettings_processor
  15. Select the existing Network Adapter node and click Remove:ss_vmwareinhv_vmsettings_removenetwork
  16. Select the Add Hardware node and select Legacy Network Adapter:ss_vmwareinhv_vmsettings_addnetwork
  17. Click Add.ss_vmwareinhv_vmsettings_addlegacy
  18. Select a Virtual Switch to connect the ESXi Host to.
  19. Click OK.

The Virtual Machine is almost ready to start up, but there is one more thing to do.

Part 3 – Enable Nested Virtualization

Before the starting up the Virtual Machine we need to enable Nested Virtualization Extensions on it. This is done by running a PowerShell script.

  1. Open a PowerShell console.
  2. Enter the following commands (adjusting the vmName to match the name of your Virtual Machine):
    CD D:\ESX-In-Hyper-V\
    .\Enable-NestedVm.ps1 -vmName 'VMWARE ESXi Host 1'
    
  3. Enter Y when asked to confirm any of the changes:ss_vmwareinhv_enablenestedvirtualization
  4. The Virtual Machine is now ready to have ESXi installed into it.

If you run into any problems with enabling nested virtualization, I’d recommend reviewing the documentation. Covering all the possible ways Nested Virtualization might not be configured correctly is beyond the scope of this post. Also, this is still a preview feature and so may still have issues.

Part 4 – Boot ESXi Virtual Machine

  1. Start up the ESXi Virtual Machine and make sure you’re connected to it so you can see the ESXi boot screen:ss_vmwareinhv_bootfirst
  2. Quickly press Tab.
  3. Add the ignoreHeadless=TRUE to the Boot Options: ss_vmwareinhv_bootoptions
  4. Press Enter.
  5. The ESXi Installation system will start up.ss_vmwareinhv_bootscreenfirst
  6. After a couple of minutes the VMWare ESXi 6.0.0 Installer will start up:ss_vmwareinhv_esxiinstaller
  7. You can now go through the ESXi installation process.
  8. You will receive this warning during the installation process but you can ignore it:ss_vmwareinhv_esxiinstallerwaring
  9. The installation process will begin:ss_vmwareinhv_esxiinstallerinstall
  10. Once the ESXi installation has completed you will see this message:ss_vmwareinhv_esxiinstallercomplete
  11. Eject the ESXi Installation ISO before rebooting the Virtual Machine:ss_vmwareinhv_ejectiso
  12. Press Enter to reboot the VM.

Part 5 – Configure the ESXi Boot Options

The final thing we have to do is permanently set the boot options for the ESXi host so that the ignoreHeadless setting is always set to TRUE.

  1. When the ESXi machine reboots, quickly press SHIFT-O to set the boot options.
  2. Add the ignoreHeadless=TRUE to the Boot Options:ss_vmwareinhv_bootsecondoptions
  3. Press Enter to boot up the ESXi host:ss_vmwareinhv_bootsecond_started
  4. Once the ESXi has booted up, press F2.
  5. Enter the root login credentials that were set during the ESXi installation process.
  6. Select Troubleshooting Options and press Enter.
  7. Select Enable ESXi Shell and press Enter:ss_vmwareinhv_bootsecond_enableshell
  8. Press ALT+F1 to bring up the console:ss_vmwareinhv_bootsecond_console
  9. Enter your root credentials.
  10. Enter the following command:
    esxcfg-advcfg --set-kernel "TRUE" ignoreHeadless

    ss_vmwareinhv_bootsecond_command

  11. Press ALT+F2 to return to the main ESXi screen.

The ESXi host can now be restarted without having to worry about the ignoreHeadless=TRUE setting.

You now have a fully running ESXi Host running inside a Hyper-V Virtual Machine. I shouldn’t have to point out that this is a completely unsupported way of installing an ESXi Host and should never be used for production workloads. But at least we now have a way of running ESXi Hosts in a Hyper-V Lab environment.

Here’s hoping that someone finds this useful!

Advertisements

156 thoughts on “Install a VMWare ESXi 6.0 Hypervisor in a Hyper-V VM

  1. RichMD says:

    I tried this with ESX 6.5 before seeing the blacklisted nic issue. However, since Windows 2016 supports Discrete device assignment, I would think that could resolve the issue if you have a dedicated NIC that can be assigned to the VM for ESX 6.5. Has anyone tried this?

    Like

  2. Maarten van der Slik says:

    ESXi 6.5 doesnt install, but ESXi 6.0 does work. But when I try to start a VM I get the message:

    •VMware ESX and Hyper-V are not compatible. Remove the Hyper-V role from the system before running VMware ESX.
    •Failed to start the virtual machine.

    I will try it again with ESXi 5.5.

    Like

  3. aman says:

    I have got the ESXi 6.0 and ESXi 5.1 up and running and it even picks up an IP from DHCP, however I am not able to connect to it with putty , no ping to and fro . it does not work with static IP as well. I have a very basic setup with no VLANS or anything. Would you know what could be the issue ?
    Host – 2016 Server with hyper-v role
    guest 1 – ESXi 6.0
    guest 2 – ESXi 5.1
    Legacy nics and nesting virtualization on guests enabled.

    Like

  4. someone else here mentioned they are hitting the same issue as me.
    The install goes fine, configuration fine, set up my storage etc, and when I come to create a VM:

    •VMware ESX and Hyper-V are not compatible. Remove the Hyper-V role from the system before running VMware ESX.
    •Failed to start the virtual machine.

    Nesting is enabled, running the script again tells me no action required as nesting is enabled. Any ideas?

    Like

    • Hi There,

      I think the switch type to use in this case is an Internal switch. Although, I think from memory you can’t ping the ESXi from the host. You can only ping it from another VM attached to the Internal Switch also using a Legacy Network Adapter. I’m not sure if this can be fixed.

      HTH

      Like

      • Sure you can, Create an External vSwitch, this create a vSwitch on the Hyper-V host (Win 8 or 10 or Server 2016) which ever you have Hyper-V installed on, this will bind the vSwitch to a physical adapter on the host. In this case most people will have a flat /24 subnet with their routers acting as the DHCP server. Any VM’s booted up connected to the external vSwitch will get an IP from the router’s DHCP server. Find this IP and ping away. In my case I setup a Windows 7 virtual machine and connected to the external vSwitch. I then added a secondary nic on this VM connected to my internal vSwitch. This VM was then one I could RDP into and connect to hosts within the internal private vSwitch.

        Like

  5. Ken says:

    Hi there thanks heaps – managed to get all working – now how do i go about adding the storage? test environment – i am using external HDDs as storage – dunno if possible or not.
    Hope to hear back from you

    Thanks
    Kp

    Like

    • Hi Ken,

      You should be able to create additional storage VHD’s on your External HDD’s and attach them to the ESXi Virtual Machine. Although you’ll want to make sure the External HDD’s are attached when ESXi is booted. I’d also suggest attaching the new VHD’s as SCSI devices (rather than IDE). Does that help?

      THanks
      Dan

      Like

  6. JYM972 says:

    Found an easy solution for the host to communicate with outside:
    – Create vSwitch Internal
    – Used my ‘usual’ pfSense router, added a Legacy NIC connected to vSwitch Internal
    For better performance, forced the NIC to full duplex

    Like

  7. Thanks for this great post! I managed to get this solution working today on a Windows Server 2016 box – just with VMware 6.0, due to the blacklisted NIC driver, but still – great for a little demo environment which I’m putting together.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s