[PATCH] KVM: x86: fix APICv/x2AVIC disabled when vm reboot by itself

[PATCH] KVM: x86: fix APICv/x2AVIC disabled when vm reboot by itself

[Yuan ZhaoXiong]

This patch fixes that VM rebooting itself will cause APICv
disabled when VM is started with APICv/x2AVIC enabled.

When a VM reboot itself, The Qemu whill reset LAPIC by invoking
ioctl(KVM_SET_LAPIC, ...) to disable x2APIC mode and set APIC_ID
to its vcpuid in xAPIC mode.

That will be handled in KVM as follows:

     kvm_vcpu_ioctl_set_lapic
       kvm_apic_set_state
	  kvm_lapic_set_base  =>  disable X2APIC mode
	    kvm_apic_state_fixup
	      kvm_lapic_xapic_id_updated
	        kvm_xapic_id(apic) != apic->vcpu->vcpu_id
		kvm_set_apicv_inhibit(APICV_INHIBIT_REASON_APIC_ID_MODIFIED)
	   memcpy(vcpu->arch.apic->regs, s->regs, sizeof(*s))  => update APIC_ID

kvm_apic_set_state invokes kvm_lapic_set_base to disable x2APIC mode
firstly, but don't change APIC_ID, APIC_ID is 32 bits in x2APIC mode
and 8 bist(bit 24 ~ bit 31) in xAPIC mode. So kvm_lapic_xapic_id_updated
will set APICV_INHIBIT_REASON_APIC_ID_MODIFIED bit inhibit and disable
APICv/x2AVIC.

kvm_lapic_xapic_id_updated must be called after APIC_ID is changed.

Fixes: 3743c2f02517 ("KVM: x86: inhibit APICv/AVIC on changes to APIC ID or APIC base")

Signed-off-by: Yuan ZhaoXiong <yuanzhaoxiong@baidu.com>
---
 arch/x86/kvm/lapic.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index d7639d1..bf5ce86 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -2722,8 +2722,6 @@ static int kvm_apic_state_fixup(struct kvm_vcpu *vcpu,
 			icr = __kvm_lapic_get_reg64(s->regs, APIC_ICR);
 			__kvm_lapic_set_reg(s->regs, APIC_ICR2, icr >> 32);
 		}
-	} else {
-		kvm_lapic_xapic_id_updated(vcpu->arch.apic);
 	}
 
 	return 0;
@@ -2759,6 +2757,9 @@ int kvm_apic_set_state(struct kvm_vcpu *vcpu, struct kvm_lapic_state *s)
 	}
 	memcpy(vcpu->arch.apic->regs, s->regs, sizeof(*s));
 
+	if (!apic_x2apic_mode(apic))
+		kvm_lapic_xapic_id_updated(apic);
+
 	atomic_set_release(&apic->vcpu->kvm->arch.apic_map_dirty, DIRTY);
 	kvm_recalculate_apic_map(vcpu->kvm);
 	kvm_apic_set_version(vcpu);
-- 
1.8.3.1

Re: [PATCH] KVM: x86: fix APICv/x2AVIC disabled when vm reboot by itself

[Sean Christopherson]

On Fri, Dec 02, 2022, Yuan ZhaoXiong wrote:
> This patch fixes that VM rebooting itself will cause APICv
> disabled when VM is started with APICv/x2AVIC enabled.
> 
> When a VM reboot itself, The Qemu whill reset LAPIC by invoking
> ioctl(KVM_SET_LAPIC, ...) to disable x2APIC mode and set APIC_ID
> to its vcpuid in xAPIC mode.
> 
> That will be handled in KVM as follows:
> 
>      kvm_vcpu_ioctl_set_lapic
>        kvm_apic_set_state
> 	  kvm_lapic_set_base  =>  disable X2APIC mode
> 	    kvm_apic_state_fixup
> 	      kvm_lapic_xapic_id_updated
> 	        kvm_xapic_id(apic) != apic->vcpu->vcpu_id
> 		kvm_set_apicv_inhibit(APICV_INHIBIT_REASON_APIC_ID_MODIFIED)
> 	   memcpy(vcpu->arch.apic->regs, s->regs, sizeof(*s))  => update APIC_ID
> 
> kvm_apic_set_state invokes kvm_lapic_set_base to disable x2APIC mode
> firstly, but don't change APIC_ID, APIC_ID is 32 bits in x2APIC mode
> and 8 bist(bit 24 ~ bit 31) in xAPIC mode. So kvm_lapic_xapic_id_updated
> will set APICV_INHIBIT_REASON_APIC_ID_MODIFIED bit inhibit and disable
> APICv/x2AVIC.
> 
> kvm_lapic_xapic_id_updated must be called after APIC_ID is changed.
> 
> Fixes: 3743c2f02517 ("KVM: x86: inhibit APICv/AVIC on changes to APIC ID or APIC base")
> 
> Signed-off-by: Yuan ZhaoXiong <yuanzhaoxiong@baidu.com>
> ---
>  arch/x86/kvm/lapic.c | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
> index d7639d1..bf5ce86 100644
> --- a/arch/x86/kvm/lapic.c
> +++ b/arch/x86/kvm/lapic.c
> @@ -2722,8 +2722,6 @@ static int kvm_apic_state_fixup(struct kvm_vcpu *vcpu,
>  			icr = __kvm_lapic_get_reg64(s->regs, APIC_ICR);
>  			__kvm_lapic_set_reg(s->regs, APIC_ICR2, icr >> 32);
>  		}
> -	} else {
> -		kvm_lapic_xapic_id_updated(vcpu->arch.apic);
>  	}
>  
>  	return 0;
> @@ -2759,6 +2757,9 @@ int kvm_apic_set_state(struct kvm_vcpu *vcpu, struct kvm_lapic_state *s)
>  	}
>  	memcpy(vcpu->arch.apic->regs, s->regs, sizeof(*s));
>  
> +	if (!apic_x2apic_mode(apic))
> +		kvm_lapic_xapic_id_updated(apic);
> +

Already posted[*], along with a pile of other APIC fixes.  Hopefully it will land
in 6.2.

[*] https://lore.kernel.org/all/20221001005915.2041642-7-seanjc@google.com

Re: [PATCH] KVM: x86: fix APICv/x2AVIC disabled when vm reboot by itself

[Paolo Bonzini]

On 12/2/22 16:49, Sean Christopherson wrote:
> On Fri, Dec 02, 2022, Yuan ZhaoXiong wrote:
>> This patch fixes that VM rebooting itself will cause APICv
>> disabled when VM is started with APICv/x2AVIC enabled.
>>
>> When a VM reboot itself, The Qemu whill reset LAPIC by invoking
>> ioctl(KVM_SET_LAPIC, ...) to disable x2APIC mode and set APIC_ID
>> to its vcpuid in xAPIC mode.
>>
>> That will be handled in KVM as follows:
>>
>>       kvm_vcpu_ioctl_set_lapic
>>         kvm_apic_set_state
>> 	  kvm_lapic_set_base  =>  disable X2APIC mode
>> 	    kvm_apic_state_fixup
>> 	      kvm_lapic_xapic_id_updated
>> 	        kvm_xapic_id(apic) != apic->vcpu->vcpu_id
>> 		kvm_set_apicv_inhibit(APICV_INHIBIT_REASON_APIC_ID_MODIFIED)
>> 	   memcpy(vcpu->arch.apic->regs, s->regs, sizeof(*s))  => update APIC_ID
>>
>> kvm_apic_set_state invokes kvm_lapic_set_base to disable x2APIC mode
>> firstly, but don't change APIC_ID, APIC_ID is 32 bits in x2APIC mode
>> and 8 bist(bit 24 ~ bit 31) in xAPIC mode. So kvm_lapic_xapic_id_updated
>> will set APICV_INHIBIT_REASON_APIC_ID_MODIFIED bit inhibit and disable
>> APICv/x2AVIC.
>>
>> kvm_lapic_xapic_id_updated must be called after APIC_ID is changed.
>>
>> Fixes: 3743c2f02517 ("KVM: x86: inhibit APICv/AVIC on changes to APIC ID or APIC base")
>>
>> Signed-off-by: Yuan ZhaoXiong <yuanzhaoxiong@baidu.com>
>> ---
>>   arch/x86/kvm/lapic.c | 5 +++--
>>   1 file changed, 3 insertions(+), 2 deletions(-)
>>
>> diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
>> index d7639d1..bf5ce86 100644
>> --- a/arch/x86/kvm/lapic.c
>> +++ b/arch/x86/kvm/lapic.c
>> @@ -2722,8 +2722,6 @@ static int kvm_apic_state_fixup(struct kvm_vcpu *vcpu,
>>   			icr = __kvm_lapic_get_reg64(s->regs, APIC_ICR);
>>   			__kvm_lapic_set_reg(s->regs, APIC_ICR2, icr >> 32);
>>   		}
>> -	} else {
>> -		kvm_lapic_xapic_id_updated(vcpu->arch.apic);
>>   	}
>>   
>>   	return 0;
>> @@ -2759,6 +2757,9 @@ int kvm_apic_set_state(struct kvm_vcpu *vcpu, struct kvm_lapic_state *s)
>>   	}
>>   	memcpy(vcpu->arch.apic->regs, s->regs, sizeof(*s));
>>   
>> +	if (!apic_x2apic_mode(apic))
>> +		kvm_lapic_xapic_id_updated(apic);
>> +
> 
> Already posted[*], along with a pile of other APIC fixes.  Hopefully it will land
> in 6.2.

I'll let this patch overtake the others since it is stable@ material.

I'll also volunteer someone to write a testcase (we can add an 
apicv_enabled boolean statistic too for the sake of the test).

Paolo

> [*] https://lore.kernel.org/all/20221001005915.2041642-7-seanjc@google.com