[RFC] KVM / QEMU: Introduce Interface for Querying APICv Info

[RFC] KVM / QEMU: Introduce Interface for Querying APICv Info

[Suthikulpanit, Suravee]

Hi All,

Currently, we don't have a good way to check whether APICV is active on a VM.
Normally, For AMD SVM AVIC, users either have to check for trace point, or using
"perf kvm stat live" to catch AVIC-related #VMEXIT.

For KVM, I would like to propose introducing a new IOCTL interface (i.e. KVM_GET_APICV_INFO),
where user-space tools (e.g. QEMU monitor) can query run-time information of APICv for VM and vCPUs
such as APICv inhibit reason flags.

For QEMU, we can leverage the "info lapic" command, and append the APICV information after
all LAPIC register information:

For example:

----- Begin Snippet -----
(qemu) info lapic 0
dumping local APIC state for CPU 0

LVT0     0x00010700 active-hi edge  masked                      ExtINT (vec 0)
LVT1     0x00000400 active-hi edge                              NMI
LVTPC    0x00010000 active-hi edge  masked                      Fixed  (vec 0)
LVTERR   0x000000fe active-hi edge                              Fixed  (vec 254)
LVTTHMR  0x00010000 active-hi edge  masked                      Fixed  (vec 0)
LVTT     0x000400ee active-hi edge                 tsc-deadline Fixed  (vec 238)
Timer    DCR=0x0 (divide by 2) initial_count = 0 current_count = 0
SPIV     0x000001ff APIC enabled, focus=off, spurious vec 255
ICR      0x000000fd physical edge de-assert no-shorthand
ICR2     0x00000005 cpu 5 (X2APIC ID)
ESR      0x00000000
ISR      (none)
IRR      (none)

APR 0x00 TPR 0x00 DFR 0x0f LDR 0x00PPR 0x00

APICV   vm inhibit: 0x10 <-- HERE
APICV vcpu inhibit: 0 <-- HERE

------ End Snippet ------

Otherwise, we can have APICv-specific info command (e.g. info apicv).

Any suggestions are much appreciated.

Best Regards,
Suravee

Re: [RFC] KVM / QEMU: Introduce Interface for Querying APICv Info

[Chao Gao]

On Fri, May 20, 2022 at 10:30:40AM +0700, Suthikulpanit, Suravee wrote:
>Hi All,
>
>Currently, we don't have a good way to check whether APICV is active on a VM.
>Normally, For AMD SVM AVIC, users either have to check for trace point, or using
>"perf kvm stat live" to catch AVIC-related #VMEXIT.
>
>For KVM, I would like to propose introducing a new IOCTL interface (i.e. KVM_GET_APICV_INFO),
>where user-space tools (e.g. QEMU monitor) can query run-time information of APICv for VM and vCPUs
>such as APICv inhibit reason flags.
>
>For QEMU, we can leverage the "info lapic" command, and append the APICV information after
>all LAPIC register information:
>
>For example:
>
>----- Begin Snippet -----
>(qemu) info lapic 0
>dumping local APIC state for CPU 0
>
>LVT0     0x00010700 active-hi edge  masked                      ExtINT (vec 0)
>LVT1     0x00000400 active-hi edge                              NMI
>LVTPC    0x00010000 active-hi edge  masked                      Fixed  (vec 0)
>LVTERR   0x000000fe active-hi edge                              Fixed  (vec 254)
>LVTTHMR  0x00010000 active-hi edge  masked                      Fixed  (vec 0)
>LVTT     0x000400ee active-hi edge                 tsc-deadline Fixed  (vec 238)
>Timer    DCR=0x0 (divide by 2) initial_count = 0 current_count = 0
>SPIV     0x000001ff APIC enabled, focus=off, spurious vec 255
>ICR      0x000000fd physical edge de-assert no-shorthand
>ICR2     0x00000005 cpu 5 (X2APIC ID)
>ESR      0x00000000
>ISR      (none)
>IRR      (none)
>
>APR 0x00 TPR 0x00 DFR 0x0f LDR 0x00PPR 0x00
>
>APICV   vm inhibit: 0x10 <-- HERE
>APICV vcpu inhibit: 0 <-- HERE
>
>------ End Snippet ------
>
>Otherwise, we can have APICv-specific info command (e.g. info apicv).

I think this information can be added to kvm per-vm/vcpu debugfs. Then no
qemu change is needed.

Re: [RFC] KVM / QEMU: Introduce Interface for Querying APICv Info

[Suravee Suthikulpanit]

On 5/20/22 12:26 PM, Chao Gao wrote:
> On Fri, May 20, 2022 at 10:30:40AM +0700, Suthikulpanit, Suravee wrote:
>> Hi All,
>>
>> Currently, we don't have a good way to check whether APICV is active on a VM.
>> Normally, For AMD SVM AVIC, users either have to check for trace point, or using
>> "perf kvm stat live" to catch AVIC-related #VMEXIT.
>>
>> For KVM, I would like to propose introducing a new IOCTL interface (i.e. KVM_GET_APICV_INFO),
>> where user-space tools (e.g. QEMU monitor) can query run-time information of APICv for VM and vCPUs
>> such as APICv inhibit reason flags.
>>
>> For QEMU, we can leverage the "info lapic" command, and append the APICV information after
>> all LAPIC register information:
>>
>> For example:
>>
>> ----- Begin Snippet -----
>> (qemu) info lapic 0
>> dumping local APIC state for CPU 0
>>
>> LVT0     0x00010700 active-hi edge  masked                      ExtINT (vec 0)
>> LVT1     0x00000400 active-hi edge                              NMI
>> LVTPC    0x00010000 active-hi edge  masked                      Fixed  (vec 0)
>> LVTERR   0x000000fe active-hi edge                              Fixed  (vec 254)
>> LVTTHMR  0x00010000 active-hi edge  masked                      Fixed  (vec 0)
>> LVTT     0x000400ee active-hi edge                 tsc-deadline Fixed  (vec 238)
>> Timer    DCR=0x0 (divide by 2) initial_count = 0 current_count = 0
>> SPIV     0x000001ff APIC enabled, focus=off, spurious vec 255
>> ICR      0x000000fd physical edge de-assert no-shorthand
>> ICR2     0x00000005 cpu 5 (X2APIC ID)
>> ESR      0x00000000
>> ISR      (none)
>> IRR      (none)
>>
>> APR 0x00 TPR 0x00 DFR 0x0f LDR 0x00PPR 0x00
>>
>> APICV   vm inhibit: 0x10 <-- HERE
>> APICV vcpu inhibit: 0 <-- HERE
>>
>> ------ End Snippet ------
>>
>> Otherwise, we can have APICv-specific info command (e.g. info apicv).
> 
> I think this information can be added to kvm per-vm/vcpu debugfs. Then no
> qemu change is needed.

I used to suggest the KVM debugfs approach in the past, but someone has suggested that it might be better to have a 
proper interface and leverage QEMU monitor. The debugfs would be difficult to use if we have large number of VMs, where 
we need to locate qemu PID and search in the /sys/kernel/debug/kvm/xxxx. Although, it would be easy to write a shell 
script to read the information from these files.

With IOCTL interface, other user-space tools/libraries can also query this information.

We can also have both :)

Best Regards,
Suravee