[Qemu-devel] [Help] Windows2012 as Guest 64+cores on KVM Halts

[Qemu-devel] [Help] Windows2012 as Guest 64+cores on KVM Halts

[hangaohuai]

Dear all:
I try to boot windows2012R2 (more than 64 cores)on kvm platform with hyper-v on*<relaxed state='on'/>*.
the guest halts on the starting time.
Also tested in these cases:
<topology sockets='64' cores='1' threads='1'/>  - boot sucess
<topology sockets='1' cores='64' threads='1'/>  - boot sucess
<topology sockets='1' cores='1' threads='64'/>  - boot sucess
<topology sockets='1' cores='68' threads='1'/>  - boot failed
<topology sockets='68' cores='1' threads='1'/>  - boot sucess(in Guest we can found 64)
<topology sockets='1' cores='1' threads='68'/>  - boot failed

Environment:
GUEST:
  windows2012R2
HOST:
    kernel version:4.10.0-rc4
    QEMU version:QEMU emulator version 2.8.50 (v2.8.0-664-gd1c82f7-dirty)
part of GuestXML:
  <vcpu placement='static'>68</vcpu>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-i440fx-2.3'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
    <hyperv>
      <relaxed state='on'/>
    </hyperv>
  </features>
  <cpu>
    <topology sockets='1' cores='68' threads='1'/>
  </cpu>

//////////////////////////////////
debug info:
virsh qemu-monitor-command win2012 --hmp "info cpus"
  CPU #61: pc=0xfffff8031296a21c thread_id=21410
  CPU #62: pc=0xfffff8031296a21c thread_id=21411
  CPU #63: pc=0xfffff8031296a21c thread_id=21414
  CPU #64: pc=0x00000000000fd31c (halted) thread_id=21415
  CPU #65: pc=0x00000000000fd31c (halted) thread_id=21416
  CPU #66: pc=0x00000000000fd31c (halted) thread_id=21417
  CPU #67: pc=0x00000000000fd31c (halted) thread_id=21418


Any suggestions?

Thanks a lot.

Gaohuai Han

Re: [Qemu-devel] [Help] Windows2012 as Guest 64+cores on KVM Halts

[Yan Vugenfirer]

Hi,

First off all to use more than 64 CPUs on Windows you should use CPU grouping. Each group can contain maximum 64 CPUs. Here is a good article explaining possible configurations:
https://msdn.microsoft.com/en-us/windows/hardware/drivers/devtest/boot-parameters-to-test-drivers-for-multiple-processor-group-support <https://msdn.microsoft.com/en-us/windows/hardware/drivers/devtest/boot-parameters-to-test-drivers-for-multiple-processor-group-support> and another one on the grouping subject: https://blogs.technet.microsoft.com/mlucas/2012/03/15/windows-server-and-processor-cores/ <https://blogs.technet.microsoft.com/mlucas/2012/03/15/windows-server-and-processor-cores/>

Regarding the boot failure - it is quite possible that Windows cannot handle more that 64 logical CPUs on one physical CPU. 

Best regards,
Yan.
> On 9 Feb 2017, at 08:18, hangaohuai <hangaohuai@huawei.com> wrote:
> 
> Dear all:
> I try to boot windows2012R2 (more than 64 cores)on kvm platform with hyper-v on*<relaxed state='on'/>*.
> the guest halts on the starting time.
> Also tested in these cases:
> <topology sockets='64' cores='1' threads='1'/>  - boot sucess
> <topology sockets='1' cores='64' threads='1'/>  - boot sucess
> <topology sockets='1' cores='1' threads='64'/>  - boot sucess
> <topology sockets='1' cores='68' threads='1'/>  - boot failed
> <topology sockets='68' cores='1' threads='1'/>  - boot sucess(in Guest we can found 64)
> <topology sockets='1' cores='1' threads='68'/>  - boot failed
> 
> Environment:
> GUEST:
>  windows2012R2
> HOST:
>    kernel version:4.10.0-rc4
>    QEMU version:QEMU emulator version 2.8.50 (v2.8.0-664-gd1c82f7-dirty)
> part of GuestXML:
>  <vcpu placement='static'>68</vcpu>
>  <resource>
>    <partition>/machine</partition>
>  </resource>
>  <os>
>    <type arch='x86_64' machine='pc-i440fx-2.3'>hvm</type>
>    <boot dev='hd'/>
>  </os>
>  <features>
>    <acpi/>
>    <apic/>
>    <pae/>
>    <hyperv>
>      <relaxed state='on'/>
>    </hyperv>
>  </features>
>  <cpu>
>    <topology sockets='1' cores='68' threads='1'/>
>  </cpu>
> 
> //////////////////////////////////
> debug info:
> virsh qemu-monitor-command win2012 --hmp "info cpus"
>  CPU #61: pc=0xfffff8031296a21c thread_id=21410
>  CPU #62: pc=0xfffff8031296a21c thread_id=21411
>  CPU #63: pc=0xfffff8031296a21c thread_id=21414
>  CPU #64: pc=0x00000000000fd31c (halted) thread_id=21415
>  CPU #65: pc=0x00000000000fd31c (halted) thread_id=21416
>  CPU #66: pc=0x00000000000fd31c (halted) thread_id=21417
>  CPU #67: pc=0x00000000000fd31c (halted) thread_id=21418
> 
> 
> Any suggestions?
> 
> Thanks a lot.
> 
> Gaohuai Han
> 
> 

Re: [Qemu-devel] [Help] Windows2012 as Guest 64+cores on KVM Halts

[Gonglei (Arei)]

Hi,

> Behalf Of Yan Vugenfirer
> 
> Hi,
> 
> First off all to use more than 64 CPUs on Windows you should use CPU grouping.
> Each group can contain maximum 64 CPUs. Here is a good article explaining
> possible configurations:
> https://msdn.microsoft.com/en-us/windows/hardware/drivers/devtest/boot-p
> arameters-to-test-drivers-for-multiple-processor-group-support
> <https://msdn.microsoft.com/en-us/windows/hardware/drivers/devtest/boot-
> parameters-to-test-drivers-for-multiple-processor-group-support> and another
> one on the grouping subject:
> https://blogs.technet.microsoft.com/mlucas/2012/03/15/windows-server-and-
> processor-cores/
> <https://blogs.technet.microsoft.com/mlucas/2012/03/15/windows-server-an
> d-processor-cores/>
> 
> Regarding the boot failure - it is quite possible that Windows cannot handle
> more that 64 logical CPUs on one physical CPU.
> 

Thanks for your reply. :)

But We tested the same cases on Xen platform and VMware, and
the guest booted successfully.

Today I also tested on tcg, the guest also boots successfully.

BTW, if we disable hyper-v, there isn't problem on boot for the case.

QEMU command line (enable hyper-v with hv_relaxed):

# ./qemu-system-x86_64 -name win2012 -machine accel=kvm,usb=off -cpu qemu64,hv_relaxed -m 4001 -realtime mlock=off -smp 68,sockets=1,cores=68,threads=1 \
-drive file=/mnt/sdb/gonglei/win2012_r2_dc_64_raw,if=none,id=drive-ide0-0-0 -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 \
-boot menu=on -device piix3-usb-uhci,id=uhci -chardev file,id=seabios,path=/home/seabios.log -device isa-debugcon,iobase=0x402,chardev=seabios \
-monitor stdio -device cirrus-vga,id=video0,vgamem_mb=16 -vnc 0.0.0.0:10

So, I think it's possible a KVM bug on Hyper-V enlightenments support. 

Any help will be appreciated!

CC'ing Paolo, Radim and Vadim.

Thanks,
-Gonglei

> Best regards,
> Yan.
> > On 9 Feb 2017, at 08:18, hangaohuai <hangaohuai@huawei.com> wrote:
> >
> > Dear all:
> > I try to boot windows2012R2 (more than 64 cores)on kvm platform with
> hyper-v on*<relaxed state='on'/>*.
> > the guest halts on the starting time.
> > Also tested in these cases:
> > <topology sockets='64' cores='1' threads='1'/>  - boot sucess
> > <topology sockets='1' cores='64' threads='1'/>  - boot sucess
> > <topology sockets='1' cores='1' threads='64'/>  - boot sucess
> > <topology sockets='1' cores='68' threads='1'/>  - boot failed
> > <topology sockets='68' cores='1' threads='1'/>  - boot sucess(in Guest we
> can found 64)
> > <topology sockets='1' cores='1' threads='68'/>  - boot failed
> >
> > Environment:
> > GUEST:
> >  windows2012R2
> > HOST:
> >    kernel version:4.10.0-rc4
> >    QEMU version:QEMU emulator version 2.8.50
> (v2.8.0-664-gd1c82f7-dirty)
> > part of GuestXML:
> >  <vcpu placement='static'>68</vcpu>
> >  <resource>
> >    <partition>/machine</partition>
> >  </resource>
> >  <os>
> >    <type arch='x86_64' machine='pc-i440fx-2.3'>hvm</type>
> >    <boot dev='hd'/>
> >  </os>
> >  <features>
> >    <acpi/>
> >    <apic/>
> >    <pae/>
> >    <hyperv>
> >      <relaxed state='on'/>
> >    </hyperv>
> >  </features>
> >  <cpu>
> >    <topology sockets='1' cores='68' threads='1'/>
> >  </cpu>
> >
> > //////////////////////////////////
> > debug info:
> > virsh qemu-monitor-command win2012 --hmp "info cpus"
> >  CPU #61: pc=0xfffff8031296a21c thread_id=21410
> >  CPU #62: pc=0xfffff8031296a21c thread_id=21411
> >  CPU #63: pc=0xfffff8031296a21c thread_id=21414
> >  CPU #64: pc=0x00000000000fd31c (halted) thread_id=21415
> >  CPU #65: pc=0x00000000000fd31c (halted) thread_id=21416
> >  CPU #66: pc=0x00000000000fd31c (halted) thread_id=21417
> >  CPU #67: pc=0x00000000000fd31c (halted) thread_id=21418
> >
> >
> > Any suggestions?
> >
> > Thanks a lot.
> >
> > Gaohuai Han
> >
> >

Re: [Qemu-devel] [Help] Windows2012 as Guest 64+cores on KVM Halts

[Paolo Bonzini]

On 10/02/2017 10:31, Gonglei (Arei) wrote:
> But We tested the same cases on Xen platform and VMware, and
> the guest booted successfully.

Were these two also tested with enlightenments enabled?  TCG surely isn't.

Paolo

> Today I also tested on tcg, the guest also boots successfully.
> 
> BTW, if we disable hyper-v, there isn't problem on boot for the case.
> 
> QEMU command line (enable hyper-v with hv_relaxed):
> 
> # ./qemu-system-x86_64 -name win2012 -machine accel=kvm,usb=off -cpu qemu64,hv_relaxed -m 4001 -realtime mlock=off -smp 68,sockets=1,cores=68,threads=1 \
> -drive file=/mnt/sdb/gonglei/win2012_r2_dc_64_raw,if=none,id=drive-ide0-0-0 -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 \
> -boot menu=on -device piix3-usb-uhci,id=uhci -chardev file,id=seabios,path=/home/seabios.log -device isa-debugcon,iobase=0x402,chardev=seabios \
> -monitor stdio -device cirrus-vga,id=video0,vgamem_mb=16 -vnc 0.0.0.0:10
> 
> So, I think it's possible a KVM bug on Hyper-V enlightenments support. 
> 
> Any help will be appreciated!
> 
> CC'ing Paolo, Radim and Vadim.

Re: [Qemu-devel] [Help] Windows2012 as Guest 64+cores on KVM Halts

[Gonglei (Arei)]

Hi Paolo,

> 
> 
> On 10/02/2017 10:31, Gonglei (Arei) wrote:
> > But We tested the same cases on Xen platform and VMware, and
> > the guest booted successfully.
> 
> Were these two also tested with enlightenments enabled?  TCG surely isn't.
> 
About TCG, I just remove ' accel=kvm,' and 'hy_releaxed' from the below QEMU
Command line, I thought the hyper-V enabled then. Sorry about that.

But for Xen, we set 'viridian=1' which be thought the Hyper-V is enabled.

For VMWare we also enabled the Hyper-V enlightenments.

Thanks,
-Gonglei

> Paolo
> 
> > Today I also tested on tcg, the guest also boots successfully.
> >
> > BTW, if we disable hyper-v, there isn't problem on boot for the case.
> >
> > QEMU command line (enable hyper-v with hv_relaxed):
> >
> > # ./qemu-system-x86_64 -name win2012 -machine accel=kvm,usb=off -cpu
> qemu64,hv_relaxed -m 4001 -realtime mlock=off -smp
> 68,sockets=1,cores=68,threads=1 \
> > -drive
> file=/mnt/sdb/gonglei/win2012_r2_dc_64_raw,if=none,id=drive-ide0-0-0
> -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 \
> > -boot menu=on -device piix3-usb-uhci,id=uhci -chardev
> file,id=seabios,path=/home/seabios.log -device
> isa-debugcon,iobase=0x402,chardev=seabios \
> > -monitor stdio -device cirrus-vga,id=video0,vgamem_mb=16 -vnc 0.0.0.0:10
> >
> > So, I think it's possible a KVM bug on Hyper-V enlightenments support.
> >
> > Any help will be appreciated!
> >
> > CC'ing Paolo, Radim and Vadim.

Re: [Qemu-devel] [Help] Windows2012 as Guest 64+cores on KVM Halts

[Paolo Bonzini]

> 
> > On 10/02/2017 10:31, Gonglei (Arei) wrote:
> > > But We tested the same cases on Xen platform and VMware, and
> > > the guest booted successfully.
> > 
> > Were these two also tested with enlightenments enabled?  TCG surely isn't.
> 
> About TCG, I just remove ' accel=kvm,' and 'hy_releaxed' from the below QEMU
> Command line, I thought the hyper-V enabled then. Sorry about that.
> 
> But for Xen, we set 'viridian=1' which be thought the Hyper-V is enabled.
> 
> For VMWare we also enabled the Hyper-V enlightenments.

Ok, thanks for the clarifications!

Paolo

Re: [Qemu-devel] [Help] Windows2012 as Guest 64+cores on KVM Halts

[Vadim Rozenfeld]

On Sat, 2017-02-11 at 10:39 -0500, Paolo Bonzini wrote:
> > 
> > 
> > > 
> > > On 10/02/2017 10:31, Gonglei (Arei) wrote:
> > > > 
> > > > But We tested the same cases on Xen platform and VMware, and
> > > > the guest booted successfully.
> > > 
> > > Were these two also tested with enlightenments enabled?  TCG
> > > surely isn't.
> > 
> > About TCG, I just remove ' accel=kvm,' and 'hy_releaxed' from the
> > below QEMU
> > Command line, I thought the hyper-V enabled then. Sorry about that.
> > 
> > But for Xen, we set 'viridian=1' which be thought the Hyper-V is
> > enabled.
> > 
> > For VMWare we also enabled the Hyper-V enlightenments.
If I'm not mistaken, even Hyper-V server doesn't allow specify more
than 64 vCPUs for Generation 1 VMs.
In any case, if you are only interested in hv_relaxed, you can drop it
off for WS2012 as long as you have cpu hypervisor flag
(CPUID.1:ECX [bit 31]=1) turned on.

Best regards,
Vadim.
 
> 
> Ok, thanks for the clarifications!
> 
> Paolo

Re: [Qemu-devel] [Help] Windows2012 as Guest 64+cores on KVM Halts

[Gonglei (Arei)]

Hi,

> 
> On Sat, 2017-02-11 at 10:39 -0500, Paolo Bonzini wrote:
> > >
> > >
> > > >
> > > > On 10/02/2017 10:31, Gonglei (Arei) wrote:
> > > > >
> > > > > But We tested the same cases on Xen platform and VMware, and
> > > > > the guest booted successfully.
> > > >
> > > > Were these two also tested with enlightenments enabled?  TCG
> > > > surely isn't.
> > >
> > > About TCG, I just remove ' accel=kvm,' and 'hy_releaxed' from the
> > > below QEMU
> > > Command line, I thought the hyper-V enabled then. Sorry about that.
> > >
> > > But for Xen, we set 'viridian=1' which be thought the Hyper-V is
> > > enabled.
> > >
> > > For VMWare we also enabled the Hyper-V enlightenments.
> If I'm not mistaken, even Hyper-V server doesn't allow specify more
> than 64 vCPUs for Generation 1 VMs.

Normally yes, but I found the explanation from Microsoft document about it:

Maximum Supported Virtual Processors

On Windows operating systems versions through Windows Server 2008 R2, 
reporting the HV#1 hypervisor interface limits the Windows virtual machine 
to a maximum of 64 VPs, regardless of what is reported via CPUID.40000005.EAX.
Starting with Windows Server 2012 and Windows 8, if CPUID.40000005.EAX 
contains a value of -1, Windows assumes that the hypervisor imposes no specific
limit to the number of VPs. In this case, Windows Server 2012 guest VMs may
use more than 64 VPs, up to the maximum supported number of processors 
applicable to the specific Windows version being used.

Link: https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/reference/tlfs

"Requirements for Implementing the Microsoft Hypervisor Interface"

And the below patch works for me, I can support max 255 vcpus for WS2012
with hyper-v enlightenments.

diff --git a/target/i386/kvm.c b/target/i386/kvm.c
index 27fd050..efe3cbc 100644
--- a/target/i386/kvm.c
+++ b/target/i386/kvm.c
@@ -772,7 +772,7 @@ int kvm_arch_init_vcpu(CPUState *cs)

         c = &cpuid_data.entries[cpuid_i++];
         c->function = HYPERV_CPUID_IMPLEMENT_LIMITS;
-        c->eax = 0x40;
+        c->eax = -1;
         c->ebx = 0x40;

         kvm_base = KVM_CPUID_SIGNATURE_NEXT;

> In any case, if you are only interested in hv_relaxed, you can drop it
> off for WS2012 as long as you have cpu hypervisor flag
> (CPUID.1:ECX [bit 31]=1) turned on.
> 
hy_relaxed is just a example of enabling hyperv-v enlightenments.

Thanks,
-Gonglei

Re: [Qemu-devel] [Help] Windows2012 as Guest 64+cores on KVM Halts

[Vadim Rozenfeld]

On Thu, 2017-02-16 at 01:31 +0000, Gonglei (Arei) wrote:
> Hi,
> 
> > 
> > 
> > On Sat, 2017-02-11 at 10:39 -0500, Paolo Bonzini wrote:
> > > 
> > > > 
> > > > 
> > > > 
> > > > > 
> > > > > 
> > > > > On 10/02/2017 10:31, Gonglei (Arei) wrote:
> > > > > > 
> > > > > > 
> > > > > > But We tested the same cases on Xen platform and VMware,
> > > > > > and
> > > > > > the guest booted successfully.
> > > > > 
> > > > > Were these two also tested with enlightenments enabled?  TCG
> > > > > surely isn't.
> > > > 
> > > > About TCG, I just remove ' accel=kvm,' and 'hy_releaxed' from
> > > > the
> > > > below QEMU
> > > > Command line, I thought the hyper-V enabled then. Sorry about
> > > > that.
> > > > 
> > > > But for Xen, we set 'viridian=1' which be thought the Hyper-V
> > > > is
> > > > enabled.
> > > > 
> > > > For VMWare we also enabled the Hyper-V enlightenments.
> > If I'm not mistaken, even Hyper-V server doesn't allow specify more
> > than 64 vCPUs for Generation 1 VMs.
> 
> Normally yes, but I found the explanation from Microsoft document
> about it:
> 
> Maximum Supported Virtual Processors
> 
> On Windows operating systems versions through Windows Server 2008
> R2, 
> reporting the HV#1 hypervisor interface limits the Windows virtual
> machine 
> to a maximum of 64 VPs, regardless of what is reported via
> CPUID.40000005.EAX.
> Starting with Windows Server 2012 and Windows 8, if
> CPUID.40000005.EAX 
> contains a value of -1, Windows assumes that the hypervisor imposes
> no specific
> limit to the number of VPs. In this case, Windows Server 2012 guest
> VMs may
> use more than 64 VPs, up to the maximum supported number of
> processors 
> applicable to the specific Windows version being used.
> 
> Link: https://docs.microsoft.com/en-us/virtualization/hyper-v-on-wind
> ows/reference/tlfs
> 
> "Requirements for Implementing the Microsoft Hypervisor Interface"
> 
> And the below patch works for me, I can support max 255 vcpus for
> WS2012
> with hyper-v enlightenments.
> 
> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
> index 27fd050..efe3cbc 100644
> --- a/target/i386/kvm.c
> +++ b/target/i386/kvm.c
> @@ -772,7 +772,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
> 
>          c = &cpuid_data.entries[cpuid_i++];
>          c->function = HYPERV_CPUID_IMPLEMENT_LIMITS;
> -        c->eax = 0x40;
> +        c->eax = -1;
>          c->ebx = 0x40;
> 
>          kvm_base = KVM_CPUID_SIGNATURE_NEXT;
> 

Nice.
I tried the following patch some time ago. Unfortunately it didn't work
for me for some reason:

@@ -772,8 +773,9 @@ int kvm_arch_init_vcpu(CPUState *cs)
 
         c = &cpuid_data.entries[cpuid_i++];
         c->function = HYPERV_CPUID_IMPLEMENT_LIMITS;
-        c->eax = 0x40;
-        c->ebx = 0x40;
+        c->eax = 0x000000f0;//0x40;
+        c->ebx = 0x00000200;//0x40;
+        c->ecx = 0x00000648;

I used the same numbers as provided by WS2016 for both Gen1 and Gen2
VMs.

> > 
> > > > 
> > In any case, if you are only interested in hv_relaxed, you can drop
> > it
> > off for WS2012 as long as you have cpu hypervisor flag
> > (CPUID.1:ECX [bit 31]=1) turned on.
> > 
> hy_relaxed is just a example of enabling hyperv-v enlightenments.
> 
> Thanks,
> -Gonglei

Re: [Qemu-devel] [Help] Windows2012 as Guest 64+cores on KVM Halts

[Paolo Bonzini]

On 16/02/2017 02:31, Gonglei (Arei) wrote:
> And the below patch works for me, I can support max 255 vcpus for WS2012
> with hyper-v enlightenments.
> 
> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
> index 27fd050..efe3cbc 100644
> --- a/target/i386/kvm.c
> +++ b/target/i386/kvm.c
> @@ -772,7 +772,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
> 
>          c = &cpuid_data.entries[cpuid_i++];
>          c->function = HYPERV_CPUID_IMPLEMENT_LIMITS;
> -        c->eax = 0x40;
> +        c->eax = -1;
>          c->ebx = 0x40;
> 
>          kvm_base = KVM_CPUID_SIGNATURE_NEXT;

This needs to depend on the machine type, but apart from that I think
you should submit the patch for 2.9.

Thanks,

Paolo

Re: [Qemu-devel] [Help] Windows2012 as Guest 64+cores on KVM Halts

[Gonglei (Arei)]

Hi Paolo,

> 
> 
> On 16/02/2017 02:31, Gonglei (Arei) wrote:
> > And the below patch works for me, I can support max 255 vcpus for WS2012
> > with hyper-v enlightenments.
> >
> > diff --git a/target/i386/kvm.c b/target/i386/kvm.c
> > index 27fd050..efe3cbc 100644
> > --- a/target/i386/kvm.c
> > +++ b/target/i386/kvm.c
> > @@ -772,7 +772,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
> >
> >          c = &cpuid_data.entries[cpuid_i++];
> >          c->function = HYPERV_CPUID_IMPLEMENT_LIMITS;
> > -        c->eax = 0x40;
> > +        c->eax = -1;
> >          c->ebx = 0x40;
> >
> >          kvm_base = KVM_CPUID_SIGNATURE_NEXT;
> 
> This needs to depend on the machine type, but apart from that I think

I don't know why. Because the negative effects for this change don't exist
on current QEMU IIUC, and we don't have compatible problems for live migration.

> you should submit the patch for 2.9.
> 

Thanks,
-Gonglei

Re: [Qemu-devel] [Help] Windows2012 as Guest 64+cores on KVM Halts

[Paolo Bonzini]

On 20/02/2017 10:19, Gonglei (Arei) wrote:
> Hi Paolo,
> 
>>
>>
>> On 16/02/2017 02:31, Gonglei (Arei) wrote:
>>> And the below patch works for me, I can support max 255 vcpus for WS2012
>>> with hyper-v enlightenments.
>>>
>>> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
>>> index 27fd050..efe3cbc 100644
>>> --- a/target/i386/kvm.c
>>> +++ b/target/i386/kvm.c
>>> @@ -772,7 +772,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
>>>
>>>          c = &cpuid_data.entries[cpuid_i++];
>>>          c->function = HYPERV_CPUID_IMPLEMENT_LIMITS;
>>> -        c->eax = 0x40;
>>> +        c->eax = -1;
>>>          c->ebx = 0x40;
>>>
>>>          kvm_base = KVM_CPUID_SIGNATURE_NEXT;
>>
>> This needs to depend on the machine type, but apart from that I think
> 
> I don't know why. Because the negative effects for this change don't exist
> on current QEMU IIUC, and we don't have compatible problems for live migration.

CPUID should never change with the same machine type and command line.

Paolo

>> you should submit the patch for 2.9.
>>
> 
> Thanks,
> -Gonglei
> 

Re: [Qemu-devel] [Help] Windows2012 as Guest 64+cores on KVM Halts

[Gonglei (Arei)]

> 
> On 20/02/2017 10:19, Gonglei (Arei) wrote:
> > Hi Paolo,
> >
> >>
> >>
> >> On 16/02/2017 02:31, Gonglei (Arei) wrote:
> >>> And the below patch works for me, I can support max 255 vcpus for
> WS2012
> >>> with hyper-v enlightenments.
> >>>
> >>> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
> >>> index 27fd050..efe3cbc 100644
> >>> --- a/target/i386/kvm.c
> >>> +++ b/target/i386/kvm.c
> >>> @@ -772,7 +772,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
> >>>
> >>>          c = &cpuid_data.entries[cpuid_i++];
> >>>          c->function = HYPERV_CPUID_IMPLEMENT_LIMITS;
> >>> -        c->eax = 0x40;
> >>> +        c->eax = -1;
> >>>          c->ebx = 0x40;
> >>>
> >>>          kvm_base = KVM_CPUID_SIGNATURE_NEXT;
> >>
> >> This needs to depend on the machine type, but apart from that I think
> >
> > I don't know why. Because the negative effects for this change don't exist
> > on current QEMU IIUC, and we don't have compatible problems for live
> migration.
> 
> CPUID should never change with the same machine type and command line.
> 
OK, then how do we add compat code for the CPUID change? It's not a visible property.

Thanks,
-Gonglei

> Paolo
> 
> >> you should submit the patch for 2.9.
> >>
> >
> > Thanks,
> > -Gonglei
> >

Re: [Qemu-devel] [Help] Windows2012 as Guest 64+cores on KVM Halts

[Paolo Bonzini]

On 20/02/2017 12:54, Gonglei (Arei) wrote:
>>
>> On 20/02/2017 10:19, Gonglei (Arei) wrote:
>>> Hi Paolo,
>>>
>>>>
>>>>
>>>> On 16/02/2017 02:31, Gonglei (Arei) wrote:
>>>>> And the below patch works for me, I can support max 255 vcpus for
>> WS2012
>>>>> with hyper-v enlightenments.
>>>>>
>>>>> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
>>>>> index 27fd050..efe3cbc 100644
>>>>> --- a/target/i386/kvm.c
>>>>> +++ b/target/i386/kvm.c
>>>>> @@ -772,7 +772,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
>>>>>
>>>>>          c = &cpuid_data.entries[cpuid_i++];
>>>>>          c->function = HYPERV_CPUID_IMPLEMENT_LIMITS;
>>>>> -        c->eax = 0x40;
>>>>> +        c->eax = -1;
>>>>>          c->ebx = 0x40;
>>>>>
>>>>>          kvm_base = KVM_CPUID_SIGNATURE_NEXT;
>>>>
>>>> This needs to depend on the machine type, but apart from that I think
>>>
>>> I don't know why. Because the negative effects for this change don't exist
>>> on current QEMU IIUC, and we don't have compatible problems for live
>> migration.
>>
>> CPUID should never change with the same machine type and command line.
>>
> OK, then how do we add compat code for the CPUID change? It's not a visible property.

You can add a new property, something like hv-cpuid-limits-eax.

Paolo

Re: [Qemu-devel] [Help] Windows2012 as Guest 64+cores on KVM Halts

[Gonglei (Arei)]

> 
> 
> On 20/02/2017 12:54, Gonglei (Arei) wrote:
> >>
> >> On 20/02/2017 10:19, Gonglei (Arei) wrote:
> >>> Hi Paolo,
> >>>
> >>>>
> >>>>
> >>>> On 16/02/2017 02:31, Gonglei (Arei) wrote:
> >>>>> And the below patch works for me, I can support max 255 vcpus for
> >> WS2012
> >>>>> with hyper-v enlightenments.
> >>>>>
> >>>>> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
> >>>>> index 27fd050..efe3cbc 100644
> >>>>> --- a/target/i386/kvm.c
> >>>>> +++ b/target/i386/kvm.c
> >>>>> @@ -772,7 +772,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
> >>>>>
> >>>>>          c = &cpuid_data.entries[cpuid_i++];
> >>>>>          c->function = HYPERV_CPUID_IMPLEMENT_LIMITS;
> >>>>> -        c->eax = 0x40;
> >>>>> +        c->eax = -1;
> >>>>>          c->ebx = 0x40;
> >>>>>
> >>>>>          kvm_base = KVM_CPUID_SIGNATURE_NEXT;
> >>>>
> >>>> This needs to depend on the machine type, but apart from that I think
> >>>
> >>> I don't know why. Because the negative effects for this change don't exist
> >>> on current QEMU IIUC, and we don't have compatible problems for live
> >> migration.
> >>
> >> CPUID should never change with the same machine type and command line.
> >>
> > OK, then how do we add compat code for the CPUID change? It's not a visible
> property.
> 
> You can add a new property, something like hv-cpuid-limits-eax.
> 
Sounds good. Let me try.  :)

Thanks,
-Gonglei

Re: [Qemu-devel] [Help] Windows2012 as Guest 64+cores on KVM Halts

[Vadim Rozenfeld]

On 21/02/17 00:18, Gonglei (Arei) wrote:
>>
>> On 20/02/2017 12:54, Gonglei (Arei) wrote:
>>>> On 20/02/2017 10:19, Gonglei (Arei) wrote:
>>>>> Hi Paolo,
>>>>>
>>>>>>
>>>>>> On 16/02/2017 02:31, Gonglei (Arei) wrote:
>>>>>>> And the below patch works for me, I can support max 255 vcpus for
>>>> WS2012
>>>>>>> with hyper-v enlightenments.
>>>>>>>
>>>>>>> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
>>>>>>> index 27fd050..efe3cbc 100644
>>>>>>> --- a/target/i386/kvm.c
>>>>>>> +++ b/target/i386/kvm.c
>>>>>>> @@ -772,7 +772,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
>>>>>>>
>>>>>>>           c = &cpuid_data.entries[cpuid_i++];
>>>>>>>           c->function = HYPERV_CPUID_IMPLEMENT_LIMITS;
>>>>>>> -        c->eax = 0x40;
>>>>>>> +        c->eax = -1;
>>>>>>>           c->ebx = 0x40;
>>>>>>>
>>>>>>>           kvm_base = KVM_CPUID_SIGNATURE_NEXT;
>>>>>> This needs to depend on the machine type, but apart from that I think
>>>>> I don't know why. Because the negative effects for this change don't exist
>>>>> on current QEMU IIUC, and we don't have compatible problems for live
>>>> migration.
>>>>
>>>> CPUID should never change with the same machine type and command line.
>>>>
>>> OK, then how do we add compat code for the CPUID change? It's not a visible
>> property.
>>
>> You can add a new property, something like hv-cpuid-limits-eax.
>>
> Sounds good. Let me try.  :)
>
> Thanks,
> -Gonglei
Hi Gonglei,
Any update on this issue?

We've found that SVVP on WS2016  is passing successfully with the above 
change
and more than 64 vCPUs defined.

Vadim.

Re: [Qemu-devel] [Help] Windows2012 as Guest 64+cores on KVM Halts

[Gonglei (Arei)]

> -----Original Message-----
> From: Vadim Rozenfeld [mailto:vrozenfe@redhat.com]
> Sent: Tuesday, September 05, 2017 2:33 PM
> To: Gonglei (Arei); Paolo Bonzini
> Cc: Hangaohuai; Yan Vugenfirer; qemu-devel@nongnu.org;
> kvm@vger.kernel.org; Radim Krcmar
> Subject: Re: [Qemu-devel] [Help] Windows2012 as Guest 64+cores on KVM
> Halts
> 
> 
> 
> On 21/02/17 00:18, Gonglei (Arei) wrote:
> >>
> >> On 20/02/2017 12:54, Gonglei (Arei) wrote:
> >>>> On 20/02/2017 10:19, Gonglei (Arei) wrote:
> >>>>> Hi Paolo,
> >>>>>
> >>>>>>
> >>>>>> On 16/02/2017 02:31, Gonglei (Arei) wrote:
> >>>>>>> And the below patch works for me, I can support max 255 vcpus for
> >>>> WS2012
> >>>>>>> with hyper-v enlightenments.
> >>>>>>>
> >>>>>>> diff --git a/target/i386/kvm.c b/target/i386/kvm.c
> >>>>>>> index 27fd050..efe3cbc 100644
> >>>>>>> --- a/target/i386/kvm.c
> >>>>>>> +++ b/target/i386/kvm.c
> >>>>>>> @@ -772,7 +772,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
> >>>>>>>
> >>>>>>>           c = &cpuid_data.entries[cpuid_i++];
> >>>>>>>           c->function = HYPERV_CPUID_IMPLEMENT_LIMITS;
> >>>>>>> -        c->eax = 0x40;
> >>>>>>> +        c->eax = -1;
> >>>>>>>           c->ebx = 0x40;
> >>>>>>>
> >>>>>>>           kvm_base = KVM_CPUID_SIGNATURE_NEXT;
> >>>>>> This needs to depend on the machine type, but apart from that I think
> >>>>> I don't know why. Because the negative effects for this change don't
> exist
> >>>>> on current QEMU IIUC, and we don't have compatible problems for live
> >>>> migration.
> >>>>
> >>>> CPUID should never change with the same machine type and command
> line.
> >>>>
> >>> OK, then how do we add compat code for the CPUID change? It's not a
> visible
> >> property.
> >>
> >> You can add a new property, something like hv-cpuid-limits-eax.
> >>
> > Sounds good. Let me try.  :)
> >
> > Thanks,
> > -Gonglei
> Hi Gonglei,
> Any update on this issue?
> 
> We've found that SVVP on WS2016  is passing successfully with the above
> change
> and more than 64 vCPUs defined.
> 
> Vadim.

Thanks for reminding me :)

I'll send this patch this evening, and welcome to test it then.


Regards,
-Gonglei