Re: [PATCH] KVM: Avoid atomic operations when kicking the running vCPU

[Paolo Bonzini <pbonzini@redhat.com>]

On 20/10/21 21:34, Sean Christopherson wrote:
>>
>> +	/*
>> +	 * The only state change done outside the vcpu mutex is IN_GUEST_MODE
>> +	 * to EXITING_GUEST_MODE.  Therefore the moderately expensive "should
>> +	 * kick" check does not need atomic operations if kvm_vcpu_kick is used
>> +	 * within the vCPU thread itself.
>> +	 */
>> +	if (vcpu == __this_cpu_read(kvm_running_vcpu)) {
>> +		if (vcpu->mode == IN_GUEST_MODE)
>> +			WRITE_ONCE(vcpu->mode, EXITING_GUEST_MODE);
> 
> Fun.  I had a whole thing typed out about this being unsafe because it implicitly
> relies on a pending request and that there's a kvm_vcpu_exit_request() check _after_
> this kick.  Then I saw your other patches, and then I realized we already have this
> bug in the kvm_arch_vcpu_should_kick() below.

Yeah, the three patches are independent but part of the same rabbit hole.

> Anyways, I also think we should add do:
> 
> 	if (vcpu == __this_cpu_read(kvm_running_vcpu)) {
> 		if (vcpu->mode == IN_GUEST_MODE &&
> 		    !WARN_ON_ONCE(!kvm_request_pending(vcpu)))
> 			WRITE_ONCE(vcpu->mode, EXITING_GUEST_MODE);
> 		goto out;
> 	}
> 
> The idea being that delaying or even missing an event in case of a KVM bug is
> preferable to letting the vCPU state become invalid due to running in the guest
> with EXITING_GUEST_MODE.

On one hand I like the idea of having a safety net; for example a test 
similar to this one would have triggered for the naked 
kvm_vcpu_exiting_guest_mode(vcpu) call in vmx_sync_pir_to_irr.

On the other hand, "request-less VCPU kicks", as 
Documentation/virt/kvm/vcpu-requests.rst calls them, are a thing; PPC 
book3s_hv does not use vcpu->requests at all. For an artificial but more 
relatable example, the ON bit takes the role of vcpu->requests when 
processing PIR.  Hence the code below would be suboptimal but still correct:

         for (;;) {
                 exit_fastpath = static_call(kvm_x86_run)(vcpu);
                 if (likely(exit_fastpath !=
			   EXIT_FASTPATH_REENTER_GUEST))
                         break;

                 if (vcpu->arch.apicv_active && pi_test_on(vcpu))
                         kvm_vcpu_kick(vcpu);

                 if (unlikely(kvm_vcpu_exit_request(vcpu))) {
                         exit_fastpath = EXIT_FASTPATH_EXIT_HANDLED;
                         break;
                 }
         }

All that really matters is that every call to kvm_x86_run is guarded by 
kvm_vcpu_exit_request(vcpu), and indeed that's what is restored by "KVM: 
x86: check for interrupts before deciding whether to exit the fast 
path".  The other architectures also have similar checks, though again 
it's a bit hard to find it for book3s_hv (due to not using 
vcpu->requests) and MIPS (which only uses KVM_REQ_TLB_FLUSH).

Paolo