Using a Windows Virtual NAT with a Hyper-V Lab

One of the new features introduced into Windows in build 10586 and above was the new NAT Virtual Switch. This feature was primarily introduced to ease the introduction of the Windows Containers in the upcoming release of Windows Server 2016.

In more recent builds of Windows (build 14295 and above) the NAT Virtual Switch has been removed in favor of a new Virtual NAT Device that exists separate from the Hyper-V Virtual Switch.

This new Virtual NAT Device is more inline with Microsoft’s Software Defined Networking approach. It also allows us to create multiple Hyper-V Lab environments where each Lab is completely isolated from any others but still be connected to the Internet by way of the Virtual NAT Device.

Previously, to give all the machines in a Lab internet access we would have had to use:

  • An External Switch – Connect all machines to an External Virtual Switch that was connected to the internet via one of the Hyper-V Host’s network adapters.
  • A Guest NAT – Install a NAT onto one of the Guest Virtual Machines in the Lab. For example, install Windows Server 2012 R2 with the Remote Access role and configure a NAT. This would still require at least this node in the Lab to be connected to the internet via an External Virtual Switch.

Each of these approaches had some drawbacks:

  1. Each Lab was not completely isolated from the other labs.
  2. An entire guest might need to be provisioned to provide internet access to the other machines in the Lab.

But using the Virtual NAT device allows us to configure Labs with complete network isolation but still being connected to the internet without the use of a guest NAT.


So, to configure a pair of Labs like in the diagram above all we need is to execute a few PowerShell Cmdlets.

Note: Make sure your Hyper-V host is at least build 14295 (Windows 10 build 14295 or Windows Server 2016 TP5). Otherwise these cmdlets will fail.

If you want some more detail on setting up a Virtual NAT, see Set up a NAT Network.

Configure Hyper-V Lab with NAT

To configure a Hyper-V Lab with NAT, perform the following steps, executing any PowerShell cmdlets in an Administrator PowerShell console.

  1. Create a Hyper-V Internal Virtual Switch on your Host:
    New-VMSwitch -Name Lab1 -SwitchType Internal

    This will also create a Virtual Network Adapter connected to the host.

  2. Assign the gateway IP address of the NAT to the Virtual Network Adapter:
    # Get the MAC Address of the VM Adapter bound to the virtual switch
    $MacAddress = (Get-VMNetworkAdapter -ManagementOS -SwitchName Lab1).MacAddress
    # Use the MAC Address of the Virtual Adapter to look up the Adapter in the Net Adapter list
    $Adapter = Get-NetAdapter | Where-Object { (($_.MacAddress -replace '-','') -eq $MacAddress) }
    New-NetIPAddress –IPAddress -PrefixLength 24 -InterfaceIndex $Adapter.ifIndex
  3. Create the Virtual NAT device:
    New-NetNat –Name Lab1NAT –InternalIPInterfaceAddressPrefix
  4. Configure the network settings on each guest virtual network adapter assigned to the virtual switch in the subnet and configure the default gateway to be

That’s it – all machines in the Lab should have access to the internet and be completely isolated as well. Naturally I have updated the LabBuilder system to support this new functionality as well.

I hope this was useful and happy NATing.


Install a VMWare ESXi 6.0 Hypervisor in a Hyper-V VM

Update 19th February 2018:

This article has had a lot more attention than I ever expected! This has bought to light several issues with the process as well as changes made to Hyper-V and ESXi that break the process. You could wade through all 256 comments and assemble the corrections yourself, but user @Gambit has helpfully done this for me! So I’ve included the summary here:

@Tom Watson: Adding a line to /etc/vmware/config with vmx.allowNested = “TRUE”
Since all VM’s will be running nested (whole point of article) this is a must if you want them to start! The alternative is adding this line to every VMX file for each guest, Toms solution was much more elegant.

@burnett437: Configure ESXi Management vSwitch “accept” “promiscuous mode”. Apparently this was a known issue even in nested ESXi 5.5. While the host network will operate for a while without problem, the “promiscuous mode” policy will eventually be tripped and you wont be able to talk to the host at random times (I originally thought it was stability issues with the Tulip driver). Now I think this has to due with the nature of nested virtual switches (a VMWare vSwitch insude of a Hyper-V Virtual Switch). When this happens I found you could “down/up” the management vnicX to get it back, but just set the setting not worry about it.

@me: Network threat detection doesn’t like nested virtual switches either. My Symantec Endpoint Client would occasionally block traffic coming from my local NIC (vSphere Client/vSphere Converter) to this nested Host (Windows Firewall did NOT seem to care), but SEP occasionally tripped cause all “suspicious” addressing to the nested host.

@me: Use vSphere converter for guest images coming from other VMWare products….duh. ESXi doesn’t like split vhdk’s, guest won’t boot. This is a beginner mistake, but this article is for Hyper-V admins that may not know the subtle nuances between VMWare products.

@RichMD: I like the idea of hardware pass-through on the NIC. I may tinker with this because it may resolve several issues, not just ESXi v6.5 NIC blacklist issue.
Theoretically, passing the physical NIC directly to the nested ESXi Host could/should resolve the “promiscuous mode”, “Network Threat Protection” and even the “Half Duplex Legacy Adapter Requirement” (very slow network performance) problems. I may revisit this…I think my server board has a 2nd NIC I could try this with if I get the time.

The original article starts here:

Recently I’ve been playing around with the new Hyper-V Nested Virtualization feature within Windows 10 (build 10565 and greater) and Windows Server 2016. It is pretty cool to be able to create virtualized lab environments running that contain Hyper-V clusters. But what if we want a lab that contains VMWare ESXi Hypervisors running on Hyper-V host. I couldn’t find the process documented anywhere and I couldn’t even confirm if it should be possible. But after lots of asking a lot of annoying questions – thanks Adam Burns – Googling and hair pulling I managed to get it going:


So this seems like a good topic for a blog post.

What You’ll Need

You are going to need a few things to get this working:

  • A Hyper-V host running on Windows 10 (built 10565 or greater) or Windows Server 2016 TP4.
  • Enable-NestedVM.ps1 – A PowerShell script for enabling Nested Virtualization in a Hyper-V VM. Click here to get the file from the Microsoft team on GitHub.
  • A VMWare account – just sign up for one here if you don’t already have one.
  • VMWare PowerShell CLI installed – I used 6.3 release 1 that I downloaded from here.
  • ESXi-Customizer-PS.ps1 – A PowerShell script for injecting network drivers into an ESXi 5.x/6.x ISO. I downloaded it from here.

I suggest you download all of the above items to a working folder – I called mine d:\ESX-In-Hyper-V, so these instructions will reflect that, but you can call your folder what ever you like.

You should end up with a folder containing these files:


And before you ask: No, you don’t need an VMWare ESXi 6.0 ISO – this will get downloaded and produced for us.

The Process

Part 1 – Prepare an ESXi 6.0 ISO with Network Drivers

The biggest problem I ran into when trying to install ESXi onto Hyper-V was that the ESXi kernel doesn’t come with drivers for the Microsoft Virtual Network Adapter or the Microsoft Legacy Network Adapter (emulates a DECchip 21140). So you’ll need to inject these drivers into the VMWare ESXi 6.0 ISO. Luckily there is a script available and the appropriate drivers DECchip 21140 (called “net-tulip” for some reason) that makes this process a breeze:

  1. Install WMWare PowerCLI.
  2. Open a PowerShell console.
  3. Enter the following commands:
    CD D:\ESX-In-Hyper-V\
    .\ESXi-Customizer-PS-v2.4.ps1 -v60 -vft -load net-tulip
  4. After a few minutes the VMWare ESXi 6.0 ISO will be downloaded and the “net-tulip” drivers merged with it:


The ISO will now be available in the D:\ESX-In-Hyper-V folder:


Part 2 – Create the Hyper-V VM

  1. In Hyper-V Manager create a new Virtual Machine:ss_vmwareinhv_newvmpath
  2. Click Next.
  3. Select Generation 1 and click Next.
  4. Set the Startup Memory to at least 4096MB.
  5. Uncheck Use Dynamic Memory for this Virtual Machine:ss_vmwareinhv_newvmmemory
  6. Click Next.
  7. Don’t bother to Configure Networking on the next step – just click Next.
  8. Select Create a new virtual hard disk and set the Size to 10GB (this is just going to be the boot disk for the ESXi Hypervisor):ss_vmwareinhv_newvmdisk
  9. Click Next.
  10. Select Install an operating system from a bootable CD/DVD-ROM.
  11. Select Image file (.iso) and browse to the ISO created in Part 1.ss_vmwareinhv_newvminstallation
  12. Click Next then click Finish to create the Virtual Machine:ss_vmwareinhv_newvm
  13. Right click the new Virtual Machine and select Settings.
  14. Select the Processor node and increase the Number of Virtual Processors to at least 2:ss_vmwareinhv_vmsettings_processor
  15. Select the existing Network Adapter node and click Remove:ss_vmwareinhv_vmsettings_removenetwork
  16. Select the Add Hardware node and select Legacy Network Adapter:ss_vmwareinhv_vmsettings_addnetwork
  17. Click Add.ss_vmwareinhv_vmsettings_addlegacy
  18. Select a Virtual Switch to connect the ESXi Host to.
  19. Click OK.

The Virtual Machine is almost ready to start up, but there is one more thing to do.

Part 3 – Enable Nested Virtualization

Before the starting up the Virtual Machine we need to enable Nested Virtualization Extensions on it. This is done by running a PowerShell script.

  1. Open a PowerShell console.
  2. Enter the following commands (adjusting the vmName to match the name of your Virtual Machine):
    CD D:\ESX-In-Hyper-V\
    .\Enable-NestedVm.ps1 -vmName 'VMWARE ESXi Host 1'
  3. Enter Y when asked to confirm any of the changes:ss_vmwareinhv_enablenestedvirtualization
  4. The Virtual Machine is now ready to have ESXi installed into it.

If you run into any problems with enabling nested virtualization, I’d recommend reviewing the documentation. Covering all the possible ways Nested Virtualization might not be configured correctly is beyond the scope of this post. Also, this is still a preview feature and so may still have issues.

Part 4 – Boot ESXi Virtual Machine

  1. Start up the ESXi Virtual Machine and make sure you’re connected to it so you can see the ESXi boot screen:ss_vmwareinhv_bootfirst
  2. Quickly press Tab.
  3. Add the ignoreHeadless=TRUE to the Boot Options: ss_vmwareinhv_bootoptions
  4. Press Enter.
  5. The ESXi Installation system will start up.ss_vmwareinhv_bootscreenfirst
  6. After a couple of minutes the VMWare ESXi 6.0.0 Installer will start up:ss_vmwareinhv_esxiinstaller
  7. You can now go through the ESXi installation process.
  8. You will receive this warning during the installation process but you can ignore it:ss_vmwareinhv_esxiinstallerwaring
  9. The installation process will begin:ss_vmwareinhv_esxiinstallerinstall
  10. Once the ESXi installation has completed you will see this message:ss_vmwareinhv_esxiinstallercomplete
  11. Eject the ESXi Installation ISO before rebooting the Virtual Machine:ss_vmwareinhv_ejectiso
  12. Press Enter to reboot the VM.

Part 5 – Configure the ESXi Boot Options

The final thing we have to do is permanently set the boot options for the ESXi host so that the ignoreHeadless setting is always set to TRUE.

  1. When the ESXi machine reboots, quickly press SHIFT-O to set the boot options.
  2. Add the ignoreHeadless=TRUE to the Boot Options:ss_vmwareinhv_bootsecondoptions
  3. Press Enter to boot up the ESXi host:ss_vmwareinhv_bootsecond_started
  4. Once the ESXi has booted up, press F2.
  5. Enter the root login credentials that were set during the ESXi installation process.
  6. Select Troubleshooting Options and press Enter.
  7. Select Enable ESXi Shell and press Enter:ss_vmwareinhv_bootsecond_enableshell
  8. Press ALT+F1 to bring up the console:ss_vmwareinhv_bootsecond_console
  9. Enter your root credentials.
  10. Enter the following command:
    esxcfg-advcfg --set-kernel "TRUE" ignoreHeadless


  11. Press ALT+F2 to return to the main ESXi screen.

The ESXi host can now be restarted without having to worry about the ignoreHeadless=TRUE setting.

You now have a fully running ESXi Host running inside a Hyper-V Virtual Machine. I shouldn’t have to point out that this is a completely unsupported way of installing an ESXi Host and should never be used for production workloads. But at least we now have a way of running ESXi Hosts in a Hyper-V Lab environment.

Here’s hoping that someone finds this useful!

Get an Array of Localized Hyper-V Integration Service Names

Today’s PowerShell snippet is used to get a list of Localized captions for the available Integration Services available on a Hyper-V host. I needed this because LabBuilder allows the individual Integration Services to be enabled or disabled per Lab Virtual Machine.

It does this using the Integration Service names configured in the configuration XML file. The problem of course is localization – something I often overlook. If you need to enable/disable an Integration Service on a VM, you need to know the name of it. The name of course is a localized string, so you need to know what the possible values are on the current machine culture.

So, after a lot of digging around in the WMI/CIM I managed to locate the various classes I need and converted them into a simple function:

function GetIntegrationServiceNames {
$Captions = @()
$Classes = @(
foreach ($Class in $Classes)
$Captions += (Get-CimInstance `
-Class $Class `
-Namespace Root\Virtualization\V2 `
-Property Caption | Select-Object -First 1).Caption
} # foreach
# This Integration Service is registered in CIM but are not exposed in Hyper-V
# 'Msvm_RdvComponentSettingData'
return $Captions
} # GetIntegrationServiceNames

The output of the function looks like this for English US:

Time Synchronization
Guest Service Interface
Key-Value Pair Exchange

Hopefully someone will find it handy.

Which Physical Network Adapters are bound to Virtual Switches?

Today’s post has quite a long title for what is going to be a fairly short post. While making some improvements to LabBuilder, I had a need to find out which physical network adapters on a host are bound to Hyper-V Virtual Switches. This is because a single physical adapter can only be bound to a single External Virtual Switch.

So I wrote a few lines of PowerShell that would do the trick:

$MacAddress = `
(Get-VMNetworkAdapter `
-ManagementOS `
-Name (Get-VMSwitch | ? {
$_.SwitchType -eq 'External'
Get-NetAdapter -Physical | ? {
($_.MacAddress -replace '-','') -in $MacAddress

The first piece gets a list of MAC addresses for all Virtual Network Adapters that are configured for use by the host OS (managementOS) on External Switches.

The second piece then gets the list of Physical network adapters that match the MAC addresses from the first line. I had to use a –Replace to get rid of the dashes in the Physical network adapter MAC address so that I could compare it with the MAC Address in the Virtual Network Adapters. It would be nice if the MAC address format was standard across all modules, but it is a pretty minor complaint.

So as you can see, PowerShell makes this unbelievably easy. This piece of code allows me to ensure that when LabBuilder is creating a new External Switch it doesn’t use a physical adapter that has already been used.

Get the BIOS GUID of a Hyper-V VM

I’ve just spent the last few hours looking into how I can get the BIOS GUID from a Hyper-V VM from inside the Host OS. I needed this so I could use it to pre-stage devices in Windows Deployment Services. I could have used the MAC address of course, but I decided I wanted to use the BIOS GUID instead.

So after a fair bit of hunting all I could turn up was an older VBS script. I decided this wasn’t ideal and so went about investigating how I might do this in PowerShell (this is a PowerShell blog mainly after all). Well after a few minutes I came up with this (rather long) command:

$VMName = 'My VM'
(Get-CimInstance -Namespace Root\Virtualization\V2 -ClassName Msvm_VirtualSystemSettingData -Filter "ElementName = '$VMName'").BiosGUID

It uses WMI/CIM, but does seem to work nicely (don’t forget to set the name of the VM):


Good night!


Prevent Template VIrtual Machines from Accidentally being Booted

Here’s a quick tip for Wednesday night:

If you have a VM you have syspreped so that you can use it as a template to create other new VM’s, set the VHD/VHDx file(s) for the VM read-only so that you won’t unsysprep (is that a word?) it by accident. I have spent many wasted minutes re-sysprepping VM’s because I accidentally booted up a template VM.

Enable Device Naming on all Virtual Net Adapters on a VM Host

After a couple of bumps upgrading my development laptop to Windows 10, I finally got to update all my Hyper-V lab VMs to the new version of Hyper-V. This included updating the Virtual Machine Configuration version and enabling virtual network adapter Device Naming – see What’s new in Hyper-V in Technical Preview for more information.

But having a number of VM’s running on this Hyper-V host I couldn’t be bothered updating them all by hand. So as usual, PowerShell to the rescue.

First up, to upgrade the configuration of all the VMs on this host so the new features can be used I ran the following command:

Get-VM  | Update-VMVersion

Once that was completed (which took about 10 seconds) I could then enable the Device Naming feature of all the Virtual Network Adapters on all Generation 2 VM’s(this feature isn’t supported on Generation 1 VM’s). The Device Naming feature will label the Network Adapter in the guest OS (for supported operating systems) with the name of the Virtual Network Adapter set in the host.

To enable Device Naming on all Generation 2 Network Adapters on all VM’s on the host:

Get-VM | Where-Object -Property VirtualMachineSubType -eq 'Generation2' | Get-VMNetworkAdapter | Set-VMNetworkAdapter -DeviceNaming On

All in all this was much easier than the eternal clicking I would have to have used in the UI. I could have even combined the two steps into one command:

Get-VM | Update-VMVersion -Passthru | Where-Object -Property VirtualMachineSubType -eq 'Generation2' |  Get-VMNetworkAdapter | Set-VMNetworkAdapte
r -DeviceNaming On

 So now it’s off to try some of the other new Hyper-V features.

Multiple VHD/VHDx Optimization using PowerShell Workflows

Like most tech people I have lots of Hyper-V VM’s scattered across various computers at home. Some of these VM’s are running on Server OS hosts (Server 2012 R2) and some running on client OS hosts (Windows 8.1) on my desktop or laptop. These VMs also get varying amount of use – lab and dev machines getting used most of the time while “experimentation” machines getting booted only rarely.

I also like to run my heavily used VMs on fast SSD drives to keep them “snappy”. But like most people I have only a limited amount of SSD space. I’m also quite an obsessive neat freak. These two things combined means I like to keep the VHD/VHDx files used by my VMs as small as possible.

Keeping VHD/VHDx files trim is can easily be performed inside the Hyper-V management tool by clicking the Edit Disk… button and using the Edit Virtual Hard Disk Wizard to Compact the desired VHD/VHDx file:

Compact a VHD using the Edit Virtual Hard Disk wizard

This of course is all a bit manual and can be really time-consuming when performing this on lots of virtual hard disks. This sounds like it could be performed by a PowerShell command.

After about 5 minutes of investigation I came up with this simple PowerShell command:

Get-VM | Where { $_.State -eq 'Off' } | Get-VMHardDiskDrive | Optimize-VHD -Mode Full

It basically performs a full optimization on all VHD/VHDx files attached to all Virtual Machines that are in the Off state on the host the command is run on. This does the job quite well but has a few annoyances:

  • The optimization is performed in series.
  • Running guests won’t be optimized.
  • The command only works on VMs on the host the command is run on.

Looking at the list of annoyances it seems a PowerShell workflow might be a good solution. So, after a few hours coding and testing (I can’t tell the number of times my VMs were rebooted and optimized over the day) I managed to complete a module containing a PS Workflow that was up to the task (documentation contained in the link):

Optimize Hyper-V VHDs using PowerShell Workflow

The module can be installed into a PowerShell modules folder (or imported from any location) and the workflow called like in the same way a normal cmdlet would be called:

Optimize-VHDsWorkflow -ComputerName HV-01,HV-02 -VMName NTB01,NTB02,NTB03 -Mode Quick

The above command would optimize all VHD/VHDx files attached to VMs called NTB01, NTB02 or NTB03 on hosts HV-01 and HV-02. It would perform this optimization in parallel meaning all VHDs would be optimized at the same time. Care obviously needs to be taken here, because optimizing too many VHDs running off the same data store could saturate the IO leading to performance of the data store being crippled.

When this workflow is run, any VMs that are running will not have their VHDs optimized. This is probably a good thing for production guests, but for my test lab guest, I want them to be shutdown automatically, have their VHDs optimized and have them automatically started back up. So I implemented a switch called AllowRestart:

Optimize-VHDsWorkflow -AllowRestart -Mode Full -Verbose

The AllowRestart switch will allow a guests that are in a running state to be shut down (not turned off or forced), an optimization performed and then started back up. If a guest is not in a running state it will just be optimized. If the guest can’t be shut down using a normal shut down (because the guest doesn’t have Hyper-V tools running or installed or isn’t running a compatible OS) then it won’t be optimized.

You can also use the Verbose switch to show more information about the workflow process:

The Optimize Verbose Process

There really isn’t much to the process and it could even be scheduled via Task Manager to be performed automatically. If anyone has any comments or feature requests, please let me know – I’m always enjoy a challenge!

\m/ \m/