Re: ESXi under KVM

[Bandan Das <bsd@redhat.com>]

Oleg Ilyin <on.ilyin@gmail.com> writes:

> Hi,
>  could you help me please to solve my issue with run VM on ESXi which
> running on KVM?
>
>
> I have   Ubuntu 14.04.4 LTS
>              ESX  6.0   vmware-201601001-3380124-iso
>              Running hypervisor: QEMU 2.4.0
>
> I try to run VM on EXSi  which running under KVM.
> So, ESXi was successfull installed,  the command in ps :
>
> qemu-system-x86_64 -enable-kvm -name esx -S -machine
> pc-i440fx-2.4,accel=kvm,usb=off -cpu Nehalem,+invpcid,+vmx -m 8096
> -realtime mlock=off -smp 4,sockets=4,cores=1,threads=1 -uuid
> b2c3b37b-9596-98ab-8e0d-f05df89d2920 -no-user-config -nodefaults
> -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/esx.monitor,server,nowait
> -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc
> -no-shutdown -boot strict=on -device
> piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive
> file=/var/lib/libvirt/images/esx-20.img,if=none,id=drive-ide0-0-0,format=raw
> -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1
> -drive if=none,id=drive-ide0-1-0,readonly=on,format=raw -device
> ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev
> tap,fd=26,id=hostnet0 -device
> e1000,netdev=hostnet0,id=net0,mac=52:54:00:96:74:1c,bus=pci.0,addr=0x3
> -chardev pty,id=charserial0 -device
> isa-serial,chardev=charserial0,id=serial0 -vnc 0.0.0.0:0 -device
> cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device
> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4
>
> But I always get error like :
> Virtualized INtel VT-x/EPT is not supported on this platform.
> Continure without virtualized msg.intel.hvhwmmu? during starting VM on
> ESXi

I haven't tried to figure out that virtual hardware mode ESX has but with testing help
from infernix on #qemu, I know these are the things ESX looks for:

invept global

invept single context (easy to advertise a dummy, kvm actually used to but we removed
it sometime in the past)

ept execute only bit (on my todo list)

vpid:
global/single context/single context retaining globals
I first two are advertised, the last one isn't and just advertising this bit seems to work
atleast for the boot part.

Also, you probably need kvm=off.

If you get to the log when esx tries to launch the guest, we will
have a clearer picture of what's actually failing but I suspect what
I mentioned above is what you are hitting.

> cat /proc/cpuinfo  on 0 host machine
> flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
> mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe
> syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts
> rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq
> dtes64 monitor ds_cpl vmx smx est tm2 ssse3 fma cx16 xtpr pdcm pcid
> dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer xsave avx
> f16c rdrand lahf_lm abm ida arat epb xsaveopt pln pts dtherm
> tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2
> smep bmi2 erms invpcid
>
> I added these options in  /etc/vmware/config
>
> vhv.allow = "TRUE"
> hv.assumeEnabled="TRUE"
> vhv.enable = "TRUE"

As far as I know, this won't work. You have to trick esx into thinking
it's running on real hardware. vhv has esx-isms we probably don't know
of yet.

Bandan

> vmx.allowNested = "TRUE"
>
> But, it didn't help me.  ( I rebooted ESXi also)
>
> It affected only for 32-bit VM, which failed with new error .
>
>
> cat /sys/module/kvm_intel/parameters/nested
> Y
> cat /sys/module/kvm/parameters/ignore_msrs
> Y
> cat /sys/module/kvm_intel/parameters/ept
> Y
>
> lsmod |grep kvm
> kvm_intel             143148  6
> kvm                   451729  1 kvm_intel
>
>
> my xml:
>
> <domain type='kvm' id='41'>
>   <name>esx</name>
>   <uuid>b2c3b37b-9596-98ab-8e0d-f05df89d2920</uuid>
>   <memory unit='KiB'>8290304</memory>
>   <currentMemory unit='KiB'>8290304</currentMemory>
>   <vcpu placement='static'>4</vcpu>
>   <resource>
>     <partition>/machine</partition>
>   </resource>
>   <os>
>     <type arch='x86_64' machine='pc-i440fx-2.4'>hvm</type>
>     <boot dev='hd'/>
>   </os>
>   <features>
>     <acpi/>
>     <apic/>
>     <pae/>
>   </features>
>   <cpu mode='custom' match='exact'>
>     <model fallback='allow'>Nehalem</model>
>     <vendor>Intel</vendor>
>     <feature policy='require' name='vmx'/>
>     <feature policy='require' name='lm'/>
>     <feature policy='require' name='lahf_lm'/>
>     <feature policy='require' name='nx'/>
>     <feature policy='require' name='fxsr'/>
>     <feature policy='require' name='invpcid'/>
>   </cpu>
>   <clock offset='utc'/>
>   <on_poweroff>destroy</on_poweroff>
>   <on_reboot>restart</on_reboot>
>   <on_crash>restart</on_crash>
>   <devices>
>     <emulator>/usr/bin/kvm-spice</emulator>
>     <disk type='file' device='disk'>
>       <driver name='qemu' type='raw'/>
>       <source file='/var/lib/libvirt/images/esx-20.img'/>
>       <target dev='hda' bus='ide'/>
>       <alias name='ide0-0-0'/>
>       <address type='drive' controller='0' bus='0' target='0' unit='0'/>
>     </disk>
>     <disk type='block' device='cdrom'>
>       <driver name='qemu' type='raw'/>
>       <target dev='hdc' bus='ide'/>
>       <readonly/>
>       <alias name='ide0-1-0'/>
>       <address type='drive' controller='0' bus='1' target='0' unit='0'/>
>     </disk>
>     <controller type='usb' index='0'>
>       <alias name='usb0'/>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x01'
> function='0x2'/>
>     </controller>
>     <controller type='pci' index='0' model='pci-root'>
>       <alias name='pci.0'/>
>     </controller>
>     <controller type='ide' index='0'>
>       <alias name='ide0'/>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x01'
> function='0x1'/>
>     </controller>
>     <interface type='bridge'>
>       <mac address='52:54:00:96:74:1c'/>
>       <source bridge='virbr0'/>
>       <target dev='vnet0'/>
>       <model type='e1000'/>
>       <alias name='net0'/>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x03'
> function='0x0'/>
>     </interface>
>     <serial type='pty'>
>       <source path='/dev/pts/2'/>
>       <target port='0'/>
>       <alias name='serial0'/>
>     </serial>
>     <console type='pty' tty='/dev/pts/2'>
>       <source path='/dev/pts/2'/>
>       <target type='serial' port='0'/>
>       <alias name='serial0'/>
>     </console>
>     <input type='mouse' bus='ps2'/>
>     <input type='keyboard' bus='ps2'/>
>     <graphics type='vnc' port='5900' autoport='yes' listen='0.0.0.0'>
>       <listen type='address' address='0.0.0.0'/>
>     </graphics>
>     <video>
>       <model type='cirrus' vram='9216' heads='1'/>
>       <alias name='video0'/>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
> function='0x0'/>
>     </video>
>     <memballoon model='virtio'>
>       <alias name='balloon0'/>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x04'
> function='0x0'/>
>     </memballoon>
>   </devices>
>   <seclabel type='none'/>
> </domain>
>
> I also try to run it on 2.5.1 and 2.6, but it didn't help me.
>
>
> please, what is wrong ?  how to be able to run VM on ESXi which runs on the KVM?
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html