!!! WORK IN PROGRESS !!!
In this blogpost I’m trying to get the latest Nutanix Community Edition (CE 2.1) installed nested in Microsoft Hyper-V (Windows 2019 version).
Before we continue there are some requirements:
- Make sure you have a list of IP-Addresses available on your own LAN which are not in use;
- For each node 2 ips (AHV and CVM);
- These addresses should be accessible from your management machine;
- 1 ip for the cluster virtual ip; (not going to use it in this blogpost, but you can add them later when the cluster is running)
- 1 ip for each Prism Central virtual machine; (not going to use it in this blogpost, but you can install Prism Central later on your own when the cluster is running)
- 1 ip for Prism Central virtual ip; (You can use this when you have Prism Central installed)
- Have enough memory available on the Proxmox environment.
The requirements for Nutanix CE are: (Link)
- 32GB memory minimum ;
- I recommend at least 64GB as 32GB will give you no space to run virtual machines;
- 4 CPU Cores minimum;
- 64GB Boot Disk minimum;
- 200GB Hot Tier Disk minimum;
- 500GB Cold Tier Disk minimum.
There are some “rules”:
- Your Hyper-V host must have at least 2 network interfaces;
- This because we need to do a network passthrough as the drivers for the virtual interfaces (Legacy and Synthetic) are not present in CE;
- This also means you can only run 1 node per host (or have more free interfaces in the host);
- You can’t do live migrations;
- There are no disk serials in Hyper-V, so you can’t create a 3 node cluster;
- We are stuck at a 1 node cluster;
- We need a Gen1 virtual machine;
- You can’t boot from SCSI in GEN1 so the AHV disk and CDROM must be IDE;
Before we continue I assume Hyper-V is already installed on the host. I have 1 free network interface which is not part of any Hyper-V virtual switch. First let us check if we can use that free interface for passthrough (Discrete Device Assignment). Open an elevated powershell and run:
curl -o SurveyDDA.ps1 https://raw.githubusercontent.com/MicrosoftDocs/Virtualization-Documentation/live/hyperv-tools/DiscreteDeviceAssignment/SurveyDDA.ps1
.\SurveyDDA.ps1
Search your free network interface and look if it can be used. My output is (so mine can be used):
If you don’t have an interface free you can’t run CE and you can stop reading 😉 (Or continue, just for fun).
Create a virtual machine in Hyper-V with the following specs:
- Name: CE21
- Generation 1. (Generation 2 will not work as we need IDE disks to get it working)
- Memory: Minimum 32GB I’m doing 64GB;
- Dont connect it to a network. We will do this later;
- Create a 64GB disk. This is the AHV boot disk;
- Mount the CE installer ISO file to the CDROM.
Edit the created virtual machine and make the following changes:
- CPU Cores: Set it to minimum 4 cores. I’m doing 16 cores;
- Add SCSI Disk: Minimum 200GB, this is for the CVM;
- Add SCSI Disk: Minimum 500GB, this is for DATA;
- Remove the “not connected” network adapter;
- Set: Automatic Stop Action to “Turn off the virtual machine”
Your config should look like this:
No we need to run some powershell command. Start an elevated powershell command line and do the following:
Set-VMProcessor -vmname <Your-VM-Name> -ExposeVirtualizationExtensions $True
This will make sure we can run a hypervisor in the virtual machine.
Now we need to passthrough our network interface directly to the created virtual machine. Find the “device instance path” from the interface you are going to use and copy it.
Open PowerShell ISE (as admin) and paste the following script:
$VMName = "CE21" # Your VM Name
$InstanceID = "PCI\VEN_8086&DEV_10FB&SUBSYS_000C8086&REV_01
$VMName = "CE21" # Your VM Name
$InstanceID = "PCI\VEN_8086&DEV_10FB&SUBSYS_000C8086&REV_01\00E0EDFFFF7D39A500" # Your Copied interface path
$VM = Get-VM -Name $VMName
$DEV = (Get-PnpDevice -PresentOnly).Where{ $_.InstanceId -like $InstanceID }
Disable-PnpDevice -InstanceId $dev.InstanceId -Confirm:$false
$LocationPath = (Get-PnpDeviceProperty -KeyName DEVPKEY_Device_LocationPaths -InstanceId $DEV.InstanceId).Data[0]
Dismount-VmHostAssignableDevice -LocationPath $LocationPath -Force -Verbose
Add-VMAssignableDevice -VM $VM -LocationPath $LocationPath -Verbose
E0EDFFFF7D39A500" # Your Copied interface path
$VM = Get-VM -Name $VMName
$DEV = (Get-PnpDevice -PresentOnly).Where{ $_.InstanceId -like $InstanceID }
Disable-PnpDevice -InstanceId $dev.InstanceId -Confirm:$false
$LocationPath = (Get-PnpDeviceProperty -KeyName DEVPKEY_Device_LocationPaths -InstanceId $DEV.InstanceId).Data[0]
Dismount-VmHostAssignableDevice -LocationPath $LocationPath -Force -Verbose
Add-VMAssignableDevice -VM $VM -LocationPath $LocationPath -Verbose
When all successful start the Virtual Machine and let it boot until it crashed 😉
This is because the installer cant read the mac address for some reason. But the network is working (do a ping 1.1.1.1 and you should have a reply, if DHCP is enabled on the subnet).
Now type the following: vi /root/phoenix/sysUtil.py
When the VI editor is opened type “:” and then 1076 and press enter. This will jump you to line 1076.
Change the cluster_id to some numbers. Like the example below:
Save the file by typing: “Escape” “:wq”
Restart the installer by typing: ./ce_installer && screen -r
The installer will boot and the wizard is presented. Notice that there are no disk serials. So we cant create a 3 node cluster and are stuck with a 1 node cluster.
If someone knows how to enable disk serials in Hyper-V virtual machines, let me know..
After you choose the correct ip’s and make changes to the disk assignments you can start the installer.
When the installer is done. Unmount the ISO and reboot:
After the reboot I didn’t got any reply on my pings. So I started looking and noticed that my network interface is not found anymore.
This is pretty strange as the hardware is an official Nutanix node, so the network interface is on the HCL. The installer wasn’ t having issues either. So work in progress but I just wanted to post this blog post so others can help as well 😉
