Re: How does kvm support x2apic?

["Zhang Haoyu" <zhanghy@sangfor.com>]

>>>>>> Hi,
>>>>>>
>>>>>> According to this:
>>>>>>
>>>>>> https://github.com/torvalds/linux/commit/0d1de2d901f4ba0972a3886496a44fb1d3300dbd
>>>>>>
>>>>>> It looks like kvm have been supporting x2apic since kernel 2.6.32, or
>>>>>> even earlier.
>>>>>>
>>>>> This patch is to emulate x2apic for guest, then guest can benefit from some advantages of x2apic, like using RDMSR/WRMSR instead of mmio.
>>>>>
>>>>>> However, this following patch:
>>>>>>
>>>>>> https://github.com/torvalds/linux/commit/8d14695f9542e9e0195d6e41ddaa52c32322adf5
>>>>>>
>>>>>> Also claims that adding support for x2apic. But this later patch was
>>>>>> for kernel 3.9.
>>>>> This patch is to support virtual x2apic mode, which can virtualizing MSR-based APIC accesses by configuring MSR bitmaps,
>>>>> you can specify some MSR-based accesses without VM exit, the other MSR-based accesses with VM exit,
>>>>> which belongs to APIC virtualization from certain angle.
>>>> Thanks Haoyu, I am reading the Intel x2apic manual.
>>>>
>>>> http://www.intel.com/content/dam/doc/specification-update/64-architecture-x2apic-specification.pdf
>>>>
>>>> but I don't see the so called "virtual x2apic mode", it seems that the
>>>> manual does not mention anything about that. Do you mean that when the
>>>> Guest OS sets bit 10 (x2apic mode enable bit) of the it virtualized
>>>> IA32_APIC_BASE MSR, it is entering a virtual x2apic mode? But isn't
>>>> this the same as the aforementioned "emulate x2apic for guest"? Or,
>>>> "emulate x2apic for guest" is the foundation of "support virtual
>>>> x2apic mode"?
>>>>
>>>> -Jidong
>>>>
>>>Oh, I found the answer, the virtualized x2apic mode is not defined in
>>>the x2apic manual, but is defined in the intel SDM manual, chapter 29
>>>- APIC virtualization and virtual interrupts. Ignore my questions
>>>please, but thank you anyway.
>>>
>> Yes, virtual x2apic mode is part of APICv.
>> Guest has no idea about it is running in a virtual machine,
>> VMM also prevent guest from enabling/disabling virtual x2apic mode or some other virtualization configurations.
>>
>
>Regarding this "VMM also prevent guest from enabling/disabling virtual
>x2apic mode", do you mean that guest is not allowed to do something
>like:
>
>Assuming we are in the guest, and we see the original value of the
>IA32_APICBASE MSR is 0xfee00900, which means EN bit and BSP bit was
>enabled. And now, we try to set bit 10 (EXTD bit):
>
>wrmsr 0x1b 0xfee00d00.
>
>Do you mean this wrmsr operation is not allowed? If fact, I tried
>this, and this simply crashes the guest kernel. (I am using Linux
>2.6.34 as the guest kernel, and Linux 3.14 as the host kernel.) And
>this is the exact reason that I initialized this whole discussion. My
>feeling is this might be a kvm bug, so I wish to figure out and try to
>fix it.
>
The MSR address range 800H through BFFH is architecturally reserved and dedicated for accessing APIC registers in x2APIC mode,
you are writing the wrong MSR address.
Please see SDN 3A 10.12.1.2 x2APIC Register Address Space, you will get the answer.

>>>>>>
>>>>>> So I am very confused:
>>>>>>
>>>>>> First, what's the difference between these two patches? Or say, does
>>>>>> kvm support x2apic since kernel 2.6.32, or since kernel 3.9?
>>>>>>
>>>>> kvm support x2apic since kernel 2.6.32, but not support virtual x2apic mode until kernel 3.9.
>>>>>
>>>>>> Second, can guest use x2apic even if the host does not? (Assuming qemu
>>>>>>has exposed this feature to guest.) The word "use" means something
>>>>>> like, accessing x2apic registers, setting the x2apic enable bit in
>>>>>> IA32_APICBASE MSR (i.e. bit 10).
>>>>>>
>>>>> Yes, you can use x2apic even if the PCPU does not support, and benefit from it, like performance bonus from MSR accesses instead of MMIO.
>>>>> But, if I remember correctly, if your guest does not support interrupt-remmping, you only can use physical destination mode when using x2apic,
>>>>> please see enable_IR_x2apic().
>>>>>
>>>>>> -Jidong