[PATCH v2 4/4] KVM: Isolate apicv_update_lock and apicv_nr_irq_window_req in a cacheline

[Sean Christopherson <seanjc@google.com>]

Force apicv_update_lock and apicv_nr_irq_window_req to reside in their own
cacheline to avoid generating significant contention due to false sharing
when KVM is contantly creating IRQ windows.  E.g. apicv_inhibit_reasons is
read on every VM-Enter; disabled_exits is read on page faults, on PAUSE
exits, if a vCPU is scheduled out, etc.; kvmclock_offset is read every time
a vCPU needs to refresh kvmclock, and so on and so forth.

Isolating the write-mostly fields from all other (read-mostly) fields
improves performance by 7-8% when running netperf TCP_RR between two guests
on the same physical host when using an in-kernel PIT in re-inject mode.

Reported-by: Naveen N Rao (AMD) <naveen@kernel.org>
Closes: https://lore.kernel.org/all/yrxhngndj37edud6tj5y3vunaf7nirwor4n63yf4275wdocnd3@c77ujgialc6r
Tested-by: Naveen N Rao (AMD) <naveen@kernel.org>
Signed-off-by: Sean Christopherson <seanjc@google.com>
---
 arch/x86/include/asm/kvm_host.h | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index b08baeff98b2..8a9f797b6a68 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1435,13 +1435,23 @@ struct kvm_arch {
 	bool apic_access_memslot_enabled;
 	bool apic_access_memslot_inhibited;
 
+	/*
+	 * Force apicv_update_lock and apicv_nr_irq_window_req to reside in a
+	 * dedicated cacheline.  They are write-mostly, whereas most everything
+	 * else in kvm_arch is read-mostly.  Note that apicv_inhibit_reasons is
+	 * read-mostly: toggling VM-wide inhibits is rare; _checking_ for
+	 * inhibits is common.
+	 */
+	____cacheline_aligned
 	/*
 	 * Protects apicv_inhibit_reasons and apicv_nr_irq_window_req (with an
 	 * asterisk, see kvm_inc_or_dec_irq_window_inhibit() for details).
 	 */
 	struct rw_semaphore apicv_update_lock;
-	unsigned long apicv_inhibit_reasons;
 	atomic_t apicv_nr_irq_window_req;
+	____cacheline_aligned
+
+	unsigned long apicv_inhibit_reasons;
 
 	gpa_t wall_clock;
 
-- 
2.52.0.457.g6b5491de43-goog