[PATCH] KVM: X86: always reset st->preempted in record_steal_time()

[PATCH] KVM: X86: always reset st->preempted in record_steal_time()

[Lai Jiangshan]

From: Lai Jiangshan <laijs@linux.alibaba.com>

st->preempted needs to be reset in record_steal_time() to clear the
KVM_VCPU_PREEMPTED bit.

But the commit 66570e966dd9 ("kvm: x86: only provide PV features if
enabled in guest's CPUID") made it cleared conditionally and
KVM_VCPU_PREEMPTED might not be cleared when entering into the guest.

Also make st->preempted be only read once, so that trace_kvm_pv_tlb_flush()
and kvm_vcpu_flush_tlb_guest() is consistent with same value of st->preempted.

Cc: Oliver Upton <oupton@google.com>
Fixes: 66570e966dd9 ("kvm: x86: only provide PV features if enabled in guest's CPUID")
Signed-off-by: Lai Jiangshan <laijs@linux.alibaba.com>
---
 arch/x86/kvm/x86.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index bbc4e04e67ad..b8a7259ebd14 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -3101,10 +3101,14 @@ static void record_steal_time(struct kvm_vcpu *vcpu)
 	 * expensive IPIs.
 	 */
 	if (guest_pv_has(vcpu, KVM_FEATURE_PV_TLB_FLUSH)) {
+		u8 st_preempted = xchg(&st->preempted, 0);
+
 		trace_kvm_pv_tlb_flush(vcpu->vcpu_id,
-				       st->preempted & KVM_VCPU_FLUSH_TLB);
-		if (xchg(&st->preempted, 0) & KVM_VCPU_FLUSH_TLB)
+				       st_preempted & KVM_VCPU_FLUSH_TLB);
+		if (st_preempted & KVM_VCPU_FLUSH_TLB)
 			kvm_vcpu_flush_tlb_guest(vcpu);
+	} else {
+		st->preempted = 0;
 	}
 
 	vcpu->arch.st.preempted = 0;
-- 
2.19.1.6.gb485710b

Re: [PATCH] KVM: X86: always reset st->preempted in record_steal_time()

[Wanpeng Li]

On Fri, 28 May 2021 at 14:11, Lai Jiangshan <jiangshanlai@gmail.com> wrote:
>
> From: Lai Jiangshan <laijs@linux.alibaba.com>
>
> st->preempted needs to be reset in record_steal_time() to clear the
> KVM_VCPU_PREEMPTED bit.
>
> But the commit 66570e966dd9 ("kvm: x86: only provide PV features if
> enabled in guest's CPUID") made it cleared conditionally and
> KVM_VCPU_PREEMPTED might not be cleared when entering into the guest.
>
> Also make st->preempted be only read once, so that trace_kvm_pv_tlb_flush()
> and kvm_vcpu_flush_tlb_guest() is consistent with same value of st->preempted.
>
> Cc: Oliver Upton <oupton@google.com>
> Fixes: 66570e966dd9 ("kvm: x86: only provide PV features if enabled in guest's CPUID")

It has already been fixed by commit 1eff0ada88b48 (KVM: X86: Fix vCPU
preempted state from guest's point of view) in kvm/master.

    Wanpeng

[PATCH] KVM: X86: reset and read st->preempted in atomic way

[Lai Jiangshan]

From: Lai Jiangshan <laijs@linux.alibaba.com>

In record_steal_time(), st->preempted is read twice, and
trace_kvm_pv_tlb_flush() might output result inconsistent if
kvm_vcpu_flush_tlb_guest() see a different st->preempted later.

It is a very trivial problem and hardly has actual harm and can be
avoided by reseting and reading st->preempted in atomic way via xchg().

Signed-off-by: Lai Jiangshan <laijs@linux.alibaba.com>

diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 0087d3532c98..fba39fe162da 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -3117,9 +3117,11 @@ static void record_steal_time(struct kvm_vcpu *vcpu)
 	 * expensive IPIs.
 	 */
 	if (guest_pv_has(vcpu, KVM_FEATURE_PV_TLB_FLUSH)) {
+		u8 st_preempted = xchg(&st->preempted, 0);
+
 		trace_kvm_pv_tlb_flush(vcpu->vcpu_id,
-				       st->preempted & KVM_VCPU_FLUSH_TLB);
-		if (xchg(&st->preempted, 0) & KVM_VCPU_FLUSH_TLB)
+				       st_preempted & KVM_VCPU_FLUSH_TLB);
+		if (st_preempted & KVM_VCPU_FLUSH_TLB)
 			kvm_vcpu_flush_tlb_guest(vcpu);
 	} else {
 		st->preempted = 0;
-- 
2.19.1.6.gb485710b

Re: [PATCH] KVM: X86: reset and read st->preempted in atomic way

[Sean Christopherson]

Nit: the shortlog is somewhat inaccurate now, maybe just:

  KVM: x86: Ensure PV TLB flush tracepoint reflects KVM behavior

or something along those lines.  Not sure what the best wording is :-/

On Tue, Jun 01, 2021, Lai Jiangshan wrote:
> From: Lai Jiangshan <laijs@linux.alibaba.com>
> 
> In record_steal_time(), st->preempted is read twice, and
> trace_kvm_pv_tlb_flush() might output result inconsistent if
> kvm_vcpu_flush_tlb_guest() see a different st->preempted later.
> 
> It is a very trivial problem and hardly has actual harm and can be
> avoided by reseting and reading st->preempted in atomic way via xchg().
> 
> Signed-off-by: Lai Jiangshan <laijs@linux.alibaba.com>

I saw this quirk too, but couldn't quite bring myself to care enought to test a
patch :-)

Reviewed-by: Sean Christopherson <seanjc@google.com>