Re: [PULL 15/22] x86: Grant AMX permission for guest

[Yang Zhong <yang.zhong@intel.com>]

On Wed, Mar 16, 2022 at 04:57:39PM +0100, Peter Krempa wrote:
> On Tue, Mar 08, 2022 at 12:34:38 +0100, Paolo Bonzini wrote:
> > From: Yang Zhong <yang.zhong@intel.com>
> > 
> > Kernel allocates 4K xstate buffer by default. For XSAVE features
> > which require large state component (e.g. AMX), Linux kernel
> > dynamically expands the xstate buffer only after the process has
> > acquired the necessary permissions. Those are called dynamically-
> > enabled XSAVE features (or dynamic xfeatures).
> > 
> > There are separate permissions for native tasks and guests.
> > 
> > Qemu should request the guest permissions for dynamic xfeatures
> > which will be exposed to the guest. This only needs to be done
> > once before the first vcpu is created.
> > 
> > KVM implemented one new ARCH_GET_XCOMP_SUPP system attribute API to
> > get host side supported_xcr0 and Qemu can decide if it can request
> > dynamically enabled XSAVE features permission.
> > https://lore.kernel.org/all/20220126152210.3044876-1-pbonzini@redhat.com/
> > 
> > Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
> > Signed-off-by: Yang Zhong <yang.zhong@intel.com>
> > Signed-off-by: Jing Liu <jing2.liu@intel.com>
> > Message-Id: <20220217060434.52460-4-yang.zhong@intel.com>
> > Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> > ---
> >  target/i386/cpu.c          |  7 +++++
> >  target/i386/cpu.h          |  4 +++
> >  target/i386/kvm/kvm-cpu.c  | 12 ++++----
> >  target/i386/kvm/kvm.c      | 57 ++++++++++++++++++++++++++++++++++++++
> >  target/i386/kvm/kvm_i386.h |  1 +
> >  5 files changed, 75 insertions(+), 6 deletions(-)
> 
> With this commit qemu crashes for me when invoking the following
> QMP command:
> 
> $ ~pipo/git/qemu.git/build/qemu-system-x86_64 -S -no-user-config -nodefaults -nographic -machine none,accel=kvm -qmp stdio
> {"QMP": {"version": {"qemu": {"micro": 90, "minor": 2, "major": 6}, "package": "v7.0.0-rc0-8-g1d60bb4b14"}, "capabilities": ["oob"]}}
> {'execute':'qmp_capabilities'}
> {"return": {}}
> {"execute":"qom-list-properties","arguments":{"typename":"max-x86_64-cpu"},"id":"libvirt-41"}
> qemu-system-x86_64: ../target/i386/kvm/kvm-cpu.c:105: kvm_cpu_xsave_init: Assertion `esa->size == eax' failed.
> Aborted (core dumped)
> 
> Note that the above is on a box with an 'AMD Ryzen 9 3900X'.
> 
> Curiously on a laptop with an Intel chip (Intel(R) Core(TM) i7-10610U)
> it seems to work.

  
  Paolo, I debugged this issue and found this issue is caused by xstate feature bit9
  (MPK, which like pku in intel) in the some AMD platforms.

  #AMD Spec, p409
  https://www.amd.com/system/files/TechDocs/24593.pdf

  I checked the cpuid info from AMD EPYC 7402P server and ECX=0x9, the eax is 0x40,
  which is different with eax=0x00000008 in Intel platform. So, the ASSERT is generated
  by AMX changes.

  ##AMD host
  0x0000000d 0x00: eax=0x00000207 ebx=0x00000340 ecx=0x00000380 edx=0x00000000
  0x0000000d 0x01: eax=0x0000000f ebx=0x00000340 ecx=0x00000000 edx=0x00000000
  0x0000000d 0x02: eax=0x00000100 ebx=0x00000240 ecx=0x00000000 edx=0x00000000
  0x0000000d 0x09: eax=0x00000040 ebx=0x00000340 ecx=0x00000000 edx=0x00000000

  ##Intel host
  0x0000000d 0x00: eax=0x000602e7 ebx=0x00002b00 ecx=0x00002b00 edx=0x00000000
  0x0000000d 0x01: eax=0x0000001f ebx=0x00002d00 ecx=0x0000dd00 edx=0x00000000
  0x0000000d 0x02: eax=0x00000100 ebx=0x00000240 ecx=0x00000000 edx=0x00000000
  0x0000000d 0x05: eax=0x00000040 ebx=0x00000440 ecx=0x00000000 edx=0x00000000
  0x0000000d 0x06: eax=0x00000200 ebx=0x00000480 ecx=0x00000000 edx=0x00000000
  0x0000000d 0x07: eax=0x00000400 ebx=0x00000680 ecx=0x00000000 edx=0x00000000
  0x0000000d 0x08: eax=0x00000080 ebx=0x00000000 ecx=0x00000001 edx=0x00000000
  0x0000000d 0x09: eax=0x00000008 ebx=0x00000a80 ecx=0x00000000 edx=0x00000000
  0x0000000d 0x0a: eax=0x00000008 ebx=0x00000000 ecx=0x00000001 edx=0x00000000
  0x0000000d 0x0b: eax=0x00000010 ebx=0x00000000 ecx=0x00000001 edx=0x00000000
  0x0000000d 0x0c: eax=0x00000018 ebx=0x00000000 ecx=0x00000001 edx=0x00000000
  0x0000000d 0x0e: eax=0x00000030 ebx=0x00000000 ecx=0x00000001 edx=0x00000000
  0x0000000d 0x0f: eax=0x00000328 ebx=0x00000000 ecx=0x00000001 edx=0x00000000
  0x0000000d 0x11: eax=0x00000040 ebx=0x00000ac0 ecx=0x00000002 edx=0x00000000
  0x0000000d 0x12: eax=0x00002000 ebx=0x00000b00 ecx=0x00000006 edx=0x00000000

  But I also checked same cpuid info from AMD MILAN server, the eax=0x00000008 in ECX=0x9.
  So, for this ECX=0x9, the eax values in different AMD server are different.

  How can we handle those different value since we have used host_cpuid() to read host's
  registers? thanks!

  Yang