[PATCH v3 0/3] LoongArch: KVM: Add separate vmid support

[PATCH v3 0/3] LoongArch: KVM: Add separate vmid support

[Bibo Mao]

LoongArch KVM hypervisor supports two-level MMU, vpid index is used
for stage1 MMU and vmid index is used for stage2 MMU.

On 3A5000, vmid must be the same with vpid. On 3A6000 platform vmid
may separate from vpid. There are such advantages if separated vmid
is supported.
  1. One VM uses one vmid, vCPUs on the same VM can share the same vmid.
  2. If one vCPU switch between different physical CPUs, old vmid can be
     still useful if it is not expired.
  3. For remote tlb flush, it is to flush TLBs which are mapping
     GPA --> HPA, only vmid need update and vpid need not.

Here add separate vmid feature support, vmid feature detecting method
comes from CPUCFG8 capability when LVZ is enabled.
---
v2 ... v3:
  1. Merge patch 1-2 into one and patch 3-4 into one.
  2. Return directly and do not update vmid in function __kvm_check_vmid()
     if feature cpu_has_guestid is disabled by AI agent review.
  3. In function kvm_arch_disable_virtualization_cpu disable feature
     CSR_GTLBC_USES2TGID, similiar with enable cpu virtualization,
     the same from AI agent review.
  4. Rename USEVMID with USES2TGID, VMID is KVM software word and S2TGID
     is hardware word.
  5. Set cpu bit with vcpu->kvm->arch.tlb_flush_pending with
     KVM_REQ_TLB_FLUSH, similiar with request KVM_REQ_TLB_FLUSH_GPA.

v1 ... v2:
  1. Move vmid together with vpid in patch1, rename it with hw_vmid.
  2. Rename kvm_check_vcpuid() with __kvm_check_vpid(), and function
     kvm_check_vmid() with __kvm_check_vmid().
  3. Merge part of patch 2, patch 3/4 together.
  4. Use bit8 of CPUCFG8 as guestid capability rather than LVZ version,
     since it is hard to decide whether this feature exists with high LVZ
     version in future.
  5. Set cpu bit with vcpu->kvm->arch.tlb_flush_pending rather than
     using add request KVM_REQ_TLB_FLUSH_GPA, it needs flush TLB on CPU
     where page fault happens and vCPU maybe migrates to other CPUs.
---
Bibo Mao (3):
  LoongArch: KVM: Add vmid support for stage2 MMU
  LoongArch: KVM: Implement vmid updating logic
  LoongArch: KVM: Enable separate vmid feature

 arch/loongarch/include/asm/kvm_host.h  | 10 ++++
 arch/loongarch/include/asm/loongarch.h |  5 ++
 arch/loongarch/kernel/asm-offsets.c    |  1 +
 arch/loongarch/kernel/cpu-probe.c      |  6 +++
 arch/loongarch/kvm/main.c              | 71 ++++++++++++++++++++++++--
 arch/loongarch/kvm/mmu.c               | 30 +++++++++--
 arch/loongarch/kvm/switch.S            |  5 +-
 arch/loongarch/kvm/tlb.c               | 19 ++++++-
 8 files changed, 135 insertions(+), 12 deletions(-)


base-commit: 73e3f0710014fe6d4ed98cfc02292f6121db7558
-- 
2.39.3

[PATCH v3 1/3] LoongArch: KVM: Add vmid support for stage2 MMU

[Bibo Mao]

LoongArch KVM hypervisor supports two-level MMU, vpid index is used
for stage1 MMU and vmid index is used for stage2 MMU.

On 3A5000, vmid must be the same with vpid. On 3A6000 platform vmid
may separate from vpid. If vCPU migrate to different physical CPUs,
vpid need change however vmid can keep unchanged, so that TLB entries
for stage2 MMU is valid still. Also vmid index of the whole VM machine
is the same on one physical CPU, all vCPUs on the VM can share the
same vmid index on one physical CPU.

Similar with ASID, there is different ID management to support multiple
VMs and vCPUs, the corresponding ID management function is
__kvm_check_vpid() and __kvm_check_vmid().

Signed-off-by: Bibo Mao <maobibo@loongson.cn>
---
 arch/loongarch/include/asm/kvm_host.h |  3 +++
 arch/loongarch/kernel/asm-offsets.c   |  1 +
 arch/loongarch/kvm/main.c             | 20 ++++++++++++++++++--
 arch/loongarch/kvm/switch.S           |  5 ++---
 arch/loongarch/kvm/tlb.c              |  5 ++++-
 5 files changed, 28 insertions(+), 6 deletions(-)

diff --git a/arch/loongarch/include/asm/kvm_host.h b/arch/loongarch/include/asm/kvm_host.h
index 23cfbecebbd7..5c2f76d1e2f8 100644
--- a/arch/loongarch/include/asm/kvm_host.h
+++ b/arch/loongarch/include/asm/kvm_host.h
@@ -232,6 +232,9 @@ struct kvm_vcpu_arch {
 	/* Cache for pages needed inside spinlock regions */
 	struct kvm_mmu_memory_cache mmu_page_cache;
 
+	/* hw vmid info for guest VM */
+	unsigned long tgid;
+
 	/* vcpu's vpid */
 	u64 vpid;
 	gpa_t flush_gpa;
diff --git a/arch/loongarch/kernel/asm-offsets.c b/arch/loongarch/kernel/asm-offsets.c
index 1b861cbc5e10..f1ad3d57eadd 100644
--- a/arch/loongarch/kernel/asm-offsets.c
+++ b/arch/loongarch/kernel/asm-offsets.c
@@ -300,6 +300,7 @@ static void __used output_kvm_defines(void)
 	OFFSET(KVM_ARCH_HSP, kvm_vcpu_arch, host_sp);
 	OFFSET(KVM_ARCH_HTP, kvm_vcpu_arch, host_tp);
 	OFFSET(KVM_ARCH_HPGD, kvm_vcpu_arch, host_pgd);
+	OFFSET(KVM_ARCH_VMID, kvm_vcpu_arch, tgid);
 	OFFSET(KVM_ARCH_KVMPGD, kvm_vcpu_arch, kvm_pgd);
 	OFFSET(KVM_ARCH_HANDLE_EXIT, kvm_vcpu_arch, handle_exit);
 	OFFSET(KVM_ARCH_HEENTRY, kvm_vcpu_arch, host_eentry);
diff --git a/arch/loongarch/kvm/main.c b/arch/loongarch/kvm/main.c
index 3e1005526f4b..a98f0284134a 100644
--- a/arch/loongarch/kvm/main.c
+++ b/arch/loongarch/kvm/main.c
@@ -225,7 +225,7 @@ static void kvm_update_vpid(struct kvm_vcpu *vcpu, int cpu)
 	vcpu->arch.vpid = vpid;
 }
 
-void kvm_check_vpid(struct kvm_vcpu *vcpu)
+static void __kvm_check_vpid(struct kvm_vcpu *vcpu)
 {
 	int cpu;
 	bool migrated;
@@ -253,7 +253,6 @@ void kvm_check_vpid(struct kvm_vcpu *vcpu)
 		kvm_update_vpid(vcpu, cpu);
 		trace_kvm_vpid_change(vcpu, vcpu->arch.vpid);
 		vcpu->cpu = cpu;
-		kvm_clear_request(KVM_REQ_TLB_FLUSH_GPA, vcpu);
 
 		/*
 		 * LLBCTL is a separated guest CSR register from host, a general
@@ -279,6 +278,23 @@ void kvm_check_vpid(struct kvm_vcpu *vcpu)
 	}
 }
 
+static void __kvm_check_vmid(struct kvm_vcpu *vcpu)
+{
+	unsigned long vmid;
+
+	vmid = vcpu->arch.vpid & vpid_mask;
+	if (vcpu->arch.tgid != vmid) {
+		vcpu->arch.tgid = vcpu->arch.vpid & vpid_mask;
+		kvm_clear_request(KVM_REQ_TLB_FLUSH_GPA, vcpu);
+	}
+}
+
+void kvm_check_vpid(struct kvm_vcpu *vcpu)
+{
+	__kvm_check_vpid(vcpu);
+	__kvm_check_vmid(vcpu);
+}
+
 void kvm_init_vmcs(struct kvm *kvm)
 {
 	kvm->arch.vmcs = vmcs;
diff --git a/arch/loongarch/kvm/switch.S b/arch/loongarch/kvm/switch.S
index 936e4ae3e408..af972394fd55 100644
--- a/arch/loongarch/kvm/switch.S
+++ b/arch/loongarch/kvm/switch.S
@@ -66,9 +66,8 @@
 	ld.d	t0, a2, KVM_ARCH_KVMPGD
 	csrwr	t0, LOONGARCH_CSR_PGDL
 
-	/* Mix GID and RID */
-	csrrd		t1, LOONGARCH_CSR_GSTAT
-	bstrpick.w	t1, t1, CSR_GSTAT_GID_SHIFT_END, CSR_GSTAT_GID_SHIFT
+	/* Set VMID for gpa --> hpa mapping */
+	ld.d		t1, a2, KVM_ARCH_VMID
 	csrrd		t0, LOONGARCH_CSR_GTLBC
 	bstrins.w	t0, t1, CSR_GTLBC_TGID_SHIFT_END, CSR_GTLBC_TGID_SHIFT
 	csrwr		t0, LOONGARCH_CSR_GTLBC
diff --git a/arch/loongarch/kvm/tlb.c b/arch/loongarch/kvm/tlb.c
index ebdbe9264e9c..afcacb62988a 100644
--- a/arch/loongarch/kvm/tlb.c
+++ b/arch/loongarch/kvm/tlb.c
@@ -23,7 +23,10 @@ void kvm_flush_tlb_all(void)
 
 void kvm_flush_tlb_gpa(struct kvm_vcpu *vcpu, unsigned long gpa)
 {
+	unsigned int vmid;
+
 	lockdep_assert_irqs_disabled();
 	gpa &= (PAGE_MASK << 1);
-	invtlb(INVTLB_GID_ADDR, read_csr_gstat() & CSR_GSTAT_GID, gpa);
+	vmid = (vcpu->arch.tgid << CSR_GSTAT_GID_SHIFT) & CSR_GSTAT_GID;
+	invtlb(INVTLB_GID_ADDR, vmid, gpa);
 }
-- 
2.39.3

[PATCH v3 2/3] LoongArch: KVM: Implement vmid updating logic

[Bibo Mao]

VMID calculation method is the same with ASID on LoongArch, it is
percpu vmid calculation method. For every physical CPU, VMID of
different VM is different, and it is the same for different vCPUs
of the same VM.

When vCPU is scheduled on the physical CPU, it checked vmid of this
VM and the global cached vmid, and judge whether it is valid or not.

Signed-off-by: Bibo Mao <maobibo@loongson.cn>
---
 arch/loongarch/include/asm/kvm_host.h |  7 ++++
 arch/loongarch/kvm/main.c             | 55 ++++++++++++++++++++++++---
 arch/loongarch/kvm/mmu.c              | 30 +++++++++++++--
 arch/loongarch/kvm/tlb.c              | 14 +++++++
 4 files changed, 97 insertions(+), 9 deletions(-)

diff --git a/arch/loongarch/include/asm/kvm_host.h b/arch/loongarch/include/asm/kvm_host.h
index 5c2f76d1e2f8..8273122604d8 100644
--- a/arch/loongarch/include/asm/kvm_host.h
+++ b/arch/loongarch/include/asm/kvm_host.h
@@ -79,6 +79,7 @@ struct kvm_arch_memory_slot {
 #define HOST_MAX_PMNUM			16
 struct kvm_context {
 	unsigned long vpid_cache;
+	unsigned long vmid_cache;
 	struct kvm_vcpu *last_vcpu;
 	/* Host PMU CSR */
 	u64 perf_ctrl[HOST_MAX_PMNUM];
@@ -132,6 +133,8 @@ struct kvm_arch {
 	unsigned long kvm_features;
 
 	s64 time_offset;
+	cpumask_t tlb_flush_pending;
+	unsigned long vmid[NR_CPUS];
 	struct kvm_context __percpu *vmcs;
 	struct loongarch_ipi *ipi;
 	struct loongarch_dmsintc *dmsintc;
@@ -319,6 +322,8 @@ bool kvm_arch_pmi_in_guest(struct kvm_vcpu *vcpu);
 int kvm_arch_vcpu_dump_regs(struct kvm_vcpu *vcpu);
 
 /* MMU handling */
+void kvm_flush_tlb_all_stage1(void);
+void kvm_flush_tlb_all_stage2(void);
 void kvm_flush_tlb_all(void);
 void kvm_flush_tlb_gpa(struct kvm_vcpu *vcpu, unsigned long gpa);
 int kvm_handle_mm_fault(struct kvm_vcpu *vcpu, unsigned long badv, bool write, int ecode);
@@ -353,6 +358,8 @@ static inline void kvm_arch_vcpu_block_finish(struct kvm_vcpu *vcpu) {}
 static inline void kvm_arch_free_memslot(struct kvm *kvm, struct kvm_memory_slot *slot) {}
 void kvm_check_vpid(struct kvm_vcpu *vcpu);
 enum hrtimer_restart kvm_swtimer_wakeup(struct hrtimer *timer);
+#define __KVM_HAVE_ARCH_FLUSH_REMOTE_TLBS
+int kvm_arch_flush_remote_tlbs(struct kvm *kvm);
 void kvm_arch_flush_remote_tlbs_memslot(struct kvm *kvm, const struct kvm_memory_slot *memslot);
 void kvm_init_vmcs(struct kvm *kvm);
 void kvm_exc_entry(void);
diff --git a/arch/loongarch/kvm/main.c b/arch/loongarch/kvm/main.c
index a98f0284134a..0099d0f0b803 100644
--- a/arch/loongarch/kvm/main.c
+++ b/arch/loongarch/kvm/main.c
@@ -218,7 +218,10 @@ static void kvm_update_vpid(struct kvm_vcpu *vcpu, int cpu)
 		++vpid; /* vpid 0 reserved for root */
 
 		/* start new vpid cycle */
-		kvm_flush_tlb_all();
+		if (!cpu_has_guestid)
+			kvm_flush_tlb_all();
+		else
+			kvm_flush_tlb_all_stage1();
 	}
 
 	context->vpid_cache = vpid;
@@ -278,15 +281,54 @@ static void __kvm_check_vpid(struct kvm_vcpu *vcpu)
 	}
 }
 
-static void __kvm_check_vmid(struct kvm_vcpu *vcpu)
+static void kvm_update_vmid(struct kvm_vcpu *vcpu, int cpu)
 {
 	unsigned long vmid;
+	struct kvm_context *context;
+
+	context = per_cpu_ptr(vcpu->kvm->arch.vmcs, cpu);
+	vmid = context->vmid_cache + 1;
+	if (!(vmid & vpid_mask)) {
+		/* finish round of vmid loop */
+		if (unlikely(!vmid))
+			vmid = vpid_mask + 1;
+
+		++vmid; /* vmid 0 reserved for root */
+
+		/* start new vmid cycle */
+		kvm_flush_tlb_all_stage2();
+	}
+
+	context->vmid_cache = vmid;
+	vcpu->kvm->arch.vmid[cpu] = vmid;
+}
 
-	vmid = vcpu->arch.vpid & vpid_mask;
-	if (vcpu->arch.tgid != vmid) {
-		vcpu->arch.tgid = vcpu->arch.vpid & vpid_mask;
-		kvm_clear_request(KVM_REQ_TLB_FLUSH_GPA, vcpu);
+static void __kvm_check_vmid(struct kvm_vcpu *vcpu)
+{
+	int cpu;
+	unsigned long ver, old, vmid;
+
+	/* On some machines like 3A5000, vmid needs the same with vpid */
+	if (!cpu_has_guestid) {
+		vmid = vcpu->arch.vpid & vpid_mask;
+		if (vcpu->arch.tgid != vmid) {
+			vcpu->arch.tgid = vcpu->arch.vpid & vpid_mask;
+			kvm_clear_request(KVM_REQ_TLB_FLUSH_GPA, vcpu);
+		}
+		return;
 	}
+
+	cpu = smp_processor_id();
+	if (cpumask_test_and_clear_cpu(cpu, &vcpu->kvm->arch.tlb_flush_pending))
+		vcpu->kvm->arch.vmid[cpu] = 0;
+
+	/* Check if the vmid is out of an older version */
+	ver = vcpu->kvm->arch.vmid[cpu] & ~vpid_mask;
+	old = this_cpu_ptr(vcpu->kvm->arch.vmcs)->vmid_cache & ~vpid_mask;
+	if (ver != old)
+		kvm_update_vmid(vcpu, cpu);
+
+	vcpu->arch.tgid = vcpu->kvm->arch.vmid[cpu] & vpid_mask;
 }
 
 void kvm_check_vpid(struct kvm_vcpu *vcpu)
@@ -392,6 +434,7 @@ static int kvm_loongarch_env_init(void)
 	for_each_possible_cpu(cpu) {
 		context = per_cpu_ptr(vmcs, cpu);
 		context->vpid_cache = vpid_mask + 1;
+		context->vmid_cache = vpid_mask + 1;
 		context->last_vcpu = NULL;
 	}
 
diff --git a/arch/loongarch/kvm/mmu.c b/arch/loongarch/kvm/mmu.c
index e104897aa532..e07ba28cc565 100644
--- a/arch/loongarch/kvm/mmu.c
+++ b/arch/loongarch/kvm/mmu.c
@@ -893,7 +893,12 @@ static int kvm_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, bool write)
 		 * there is invalid tlb with small page
 		 * need flush these invalid tlbs for current vcpu
 		 */
-		kvm_make_request(KVM_REQ_TLB_FLUSH, vcpu);
+		if (!cpu_has_ptw) {
+			if (!cpu_has_guestid)
+				kvm_make_request(KVM_REQ_TLB_FLUSH, vcpu);
+			else
+				cpumask_set_cpu(vcpu->cpu, &vcpu->kvm->arch.tlb_flush_pending);
+		}
 		++kvm->stat.hugepages;
 	}  else if (kvm_pte_huge(*ptep) && write)
 		ptep = kvm_split_huge(vcpu, ptep, gfn);
@@ -929,8 +934,11 @@ int kvm_handle_mm_fault(struct kvm_vcpu *vcpu, unsigned long gpa, bool write, in
 		 *
 		 * With SW PTW, invalid TLB is added in TLB refill exception.
 		 */
-		vcpu->arch.flush_gpa = gpa;
-		kvm_make_request(KVM_REQ_TLB_FLUSH_GPA, vcpu);
+		if (!cpu_has_guestid) {
+			vcpu->arch.flush_gpa = gpa;
+			kvm_make_request(KVM_REQ_TLB_FLUSH_GPA, vcpu);
+		} else
+			cpumask_set_cpu(vcpu->cpu, &vcpu->kvm->arch.tlb_flush_pending);
 	}
 
 	return 0;
@@ -940,6 +948,22 @@ void kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot)
 {
 }
 
+int kvm_arch_flush_remote_tlbs(struct kvm *kvm)
+{
+	/*
+	 * Queue a TLB invalidation for each CPU to perform on next
+	 * vcpu loading
+	 */
+	if (cpu_has_guestid) {
+		cpumask_setall(&kvm->arch.tlb_flush_pending);
+		/* Be sure that other CPUS can watch the changes */
+		smp_wmb();
+	}
+
+	/* Return 1 continue to send ipi to running vCPUs */
+	return 1;
+}
+
 void kvm_arch_flush_remote_tlbs_memslot(struct kvm *kvm,
 					const struct kvm_memory_slot *memslot)
 {
diff --git a/arch/loongarch/kvm/tlb.c b/arch/loongarch/kvm/tlb.c
index afcacb62988a..994216c98ea2 100644
--- a/arch/loongarch/kvm/tlb.c
+++ b/arch/loongarch/kvm/tlb.c
@@ -21,6 +21,20 @@ void kvm_flush_tlb_all(void)
 	local_irq_restore(flags);
 }
 
+/* Invalidate all stage1 TLB entries including GVA-->GPA mappings */
+void kvm_flush_tlb_all_stage1(void)
+{
+	lockdep_assert_irqs_disabled();
+	invtlb_all(INVGTLB_ALLGID_GVA_TO_GPA, 0, 0);
+}
+
+/* Invalidate all stage2 TLB entries including GPA-->HPA  mappings */
+void kvm_flush_tlb_all_stage2(void)
+{
+	lockdep_assert_irqs_disabled();
+	invtlb_all(INVTLB_ALLGID_GPA_TO_HPA, 0, 0);
+}
+
 void kvm_flush_tlb_gpa(struct kvm_vcpu *vcpu, unsigned long gpa)
 {
 	unsigned int vmid;
-- 
2.39.3

[PATCH v3 3/3] LoongArch: KVM: Enable separate vmid feature

[Bibo Mao]

With CSR GTLBC shortname for Guest TLB Control Register, separate vmid
feature will be enabled if bit 14 CSR_GTLBC_USES2TGID is set. Enable
this if S2TGID feature is detected from cpucfg8.

Signed-off-by: Bibo Mao <maobibo@loongson.cn>
---
 arch/loongarch/include/asm/loongarch.h | 5 +++++
 arch/loongarch/kernel/cpu-probe.c      | 6 ++++++
 arch/loongarch/kvm/main.c              | 6 ++++--
 3 files changed, 15 insertions(+), 2 deletions(-)

diff --git a/arch/loongarch/include/asm/loongarch.h b/arch/loongarch/include/asm/loongarch.h
index 2a6bc99177d8..88e3bdfd4971 100644
--- a/arch/loongarch/include/asm/loongarch.h
+++ b/arch/loongarch/include/asm/loongarch.h
@@ -138,6 +138,9 @@
 #define  CPUCFG6_PMBITS_SHIFT		8
 #define  CPUCFG6_UPM			BIT(14)
 
+#define LOONGARCH_CPUCFG8		0x8
+#define  CPUCFG8_S2TGID			BIT(8)
+
 #define LOONGARCH_CPUCFG16		0x10
 #define  CPUCFG16_L1_IUPRE		BIT(0)
 #define  CPUCFG16_L1_IUUNIFY		BIT(1)
@@ -350,6 +353,8 @@
 #define  CSR_GTLBC_TGID_WIDTH		8
 #define  CSR_GTLBC_TGID_SHIFT_END	(CSR_GTLBC_TGID_SHIFT + CSR_GTLBC_TGID_WIDTH - 1)
 #define  CSR_GTLBC_TGID			(_ULCAST_(0xff) << CSR_GTLBC_TGID_SHIFT)
+#define  CSR_GTLBC_USES2TGID_SHIFT	14
+#define  CSR_GTLBC_USES2TGID		(_ULCAST_(0x1) << CSR_GTLBC_USES2TGID_SHIFT)
 #define  CSR_GTLBC_TOTI_SHIFT		13
 #define  CSR_GTLBC_TOTI			(_ULCAST_(0x1) << CSR_GTLBC_TOTI_SHIFT)
 #define  CSR_GTLBC_USETGID_SHIFT	12
diff --git a/arch/loongarch/kernel/cpu-probe.c b/arch/loongarch/kernel/cpu-probe.c
index 74d31f260dfd..e849ebacd268 100644
--- a/arch/loongarch/kernel/cpu-probe.c
+++ b/arch/loongarch/kernel/cpu-probe.c
@@ -241,6 +241,12 @@ static void cpu_probe_common(struct cpuinfo_loongarch *c)
 	if (config & CPUCFG6_PMP)
 		c->options |= LOONGARCH_CPU_PMP;
 
+	if (c->options & LOONGARCH_CPU_LVZ) {
+		config = read_cpucfg(LOONGARCH_CPUCFG8);
+		if (config & CPUCFG8_S2TGID)
+			c->options |= LOONGARCH_CPU_GUESTID;
+	}
+
 	config = csr_read32(LOONGARCH_CSR_ASID);
 	config = (config & CSR_ASID_BIT) >> CSR_ASID_BIT_SHIFT;
 	asid_mask = GENMASK(config - 1, 0);
diff --git a/arch/loongarch/kvm/main.c b/arch/loongarch/kvm/main.c
index 0099d0f0b803..e43ed4c11d31 100644
--- a/arch/loongarch/kvm/main.c
+++ b/arch/loongarch/kvm/main.c
@@ -358,7 +358,7 @@ int kvm_arch_enable_virtualization_cpu(void)
 	write_csr_gcfg(0);
 	write_csr_gstat(0);
 	write_csr_gintc(0);
-	clear_csr_gtlbc(CSR_GTLBC_USETGID | CSR_GTLBC_TOTI);
+	clear_csr_gtlbc(CSR_GTLBC_USETGID | CSR_GTLBC_TOTI | CSR_GTLBC_USES2TGID);
 
 	/*
 	 * Enable virtualization features granting guest direct control of
@@ -381,6 +381,8 @@ int kvm_arch_enable_virtualization_cpu(void)
 
 	/* Enable using TGID  */
 	set_csr_gtlbc(CSR_GTLBC_USETGID);
+	if (cpu_has_guestid)
+		set_csr_gtlbc(CSR_GTLBC_USES2TGID);
 	kvm_debug("GCFG:%lx GSTAT:%lx GINTC:%lx GTLBC:%lx",
 		  read_csr_gcfg(), read_csr_gstat(), read_csr_gintc(), read_csr_gtlbc());
 
@@ -399,7 +401,7 @@ void kvm_arch_disable_virtualization_cpu(void)
 	write_csr_gcfg(0);
 	write_csr_gstat(0);
 	write_csr_gintc(0);
-	clear_csr_gtlbc(CSR_GTLBC_USETGID | CSR_GTLBC_TOTI);
+	clear_csr_gtlbc(CSR_GTLBC_USETGID | CSR_GTLBC_TOTI | CSR_GTLBC_USES2TGID);
 
 	/* Flush any remaining guest TLB entries */
 	kvm_flush_tlb_all();
-- 
2.39.3

Re: [PATCH v3 2/3] LoongArch: KVM: Implement vmid updating logic

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] Precise single-GPA TLB flushes are replaced with full VMID reallocations on page faults for CPUs with `cpu_has_guestid`.
- [High] Data race on the `tlb_flush_pending` bitmask between non-atomic and atomic operations.
--

commit 79994ac14e7ca6c42512f0cad89100f577b61b57
Author: Bibo Mao <maobibo@loongson.cn>

LoongArch: KVM: Implement vmid updating logic

This commit implements VMID updating logic for LoongArch, calculating VMIDs
on a per-CPU basis similar to ASIDs. It adds checks during vCPU scheduling to
ensure the VMID is valid against the globally cached VMID.

> diff --git a/arch/loongarch/kvm/main.c b/arch/loongarch/kvm/main.c
> index a98f0284134a0..0099d0f0b8030 100644
> --- a/arch/loongarch/kvm/main.c
> +++ b/arch/loongarch/kvm/main.c
[ ... ]
> @@ -278,15 +281,54 @@ static void __kvm_check_vpid(struct kvm_vcpu *vcpu)
[ ... ]
> +static void __kvm_check_vmid(struct kvm_vcpu *vcpu)
> +{
> +	int cpu;
> +	unsigned long ver, old, vmid;
> +
> +	/* On some machines like 3A5000, vmid needs the same with vpid */
> +	if (!cpu_has_guestid) {
> +		vmid = vcpu->arch.vpid & vpid_mask;
> +		if (vcpu->arch.tgid != vmid) {
> +			vcpu->arch.tgid = vcpu->arch.vpid & vpid_mask;
> +			kvm_clear_request(KVM_REQ_TLB_FLUSH_GPA, vcpu);
> +		}
> +		return;
>  	}
> +
> +	cpu = smp_processor_id();
> +	if (cpumask_test_and_clear_cpu(cpu, &vcpu->kvm->arch.tlb_flush_pending))
> +		vcpu->kvm->arch.vmid[cpu] = 0;

[Severity: High]
Is there a risk of a data race on the tlb_flush_pending bitmask here? 
In this function, we perform an atomic read-modify-write operation using 
cpumask_test_and_clear_cpu(). However, kvm_arch_flush_remote_tlbs() 
concurrently performs a non-atomic memory write using cpumask_setall() 
on the same mask.

Mixing non-atomic stores with atomic operations on the same memory location 
can lead to lost bit updates. If a remote TLB flush request is lost due to 
this race, could a CPU continue executing with stale TLB entries?

> +
> +	/* Check if the vmid is out of an older version */
> +	ver = vcpu->kvm->arch.vmid[cpu] & ~vpid_mask;
> +	old = this_cpu_ptr(vcpu->kvm->arch.vmcs)->vmid_cache & ~vpid_mask;
> +	if (ver != old)
> +		kvm_update_vmid(vcpu, cpu);
> +
> +	vcpu->arch.tgid = vcpu->kvm->arch.vmid[cpu] & vpid_mask;
>  }
>  
>  void kvm_check_vpid(struct kvm_vcpu *vcpu)
[ ... ]
> diff --git a/arch/loongarch/kvm/mmu.c b/arch/loongarch/kvm/mmu.c
> index e104897aa5328..e07ba28cc5655 100644
> --- a/arch/loongarch/kvm/mmu.c
> +++ b/arch/loongarch/kvm/mmu.c
> @@ -893,7 +893,12 @@ static int kvm_map_page(struct kvm_vcpu *vcpu, unsigned long gpa, bool write)
>  		 * there is invalid tlb with small page
>  		 * need flush these invalid tlbs for current vcpu
>  		 */
> -		kvm_make_request(KVM_REQ_TLB_FLUSH, vcpu);
> +		if (!cpu_has_ptw) {
> +			if (!cpu_has_guestid)
> +				kvm_make_request(KVM_REQ_TLB_FLUSH, vcpu);
> +			else
> +				cpumask_set_cpu(vcpu->cpu, &vcpu->kvm->arch.tlb_flush_pending);

[Severity: High]
Could this lead to a severe performance regression due to excessive TLB 
flushes? By setting the tlb_flush_pending bit instead of issuing a precise 
local single-page flush, this forces __kvm_check_vmid() to allocate a 
completely new VMID for the vCPU. 

This new VMID allocation causes the hardware to abandon all cached Stage 2 
TLB entries associated with the old VMID. Doing this on every TLB Modified 
exception (such as the first write to a tracked huge page) appears to 
guarantee a 100% TLB miss rate for existing mappings.

> +		}
>  		++kvm->stat.hugepages;
>  	}  else if (kvm_pte_huge(*ptep) && write)
>  		ptep = kvm_split_huge(vcpu, ptep, gfn);
> @@ -929,8 +934,11 @@ int kvm_handle_mm_fault(struct kvm_vcpu *vcpu, unsigned long gpa, bool write, in
>  		 *
>  		 * With SW PTW, invalid TLB is added in TLB refill exception.
>  		 */
> -		vcpu->arch.flush_gpa = gpa;
> -		kvm_make_request(KVM_REQ_TLB_FLUSH_GPA, vcpu);
> +		if (!cpu_has_guestid) {
> +			vcpu->arch.flush_gpa = gpa;
> +			kvm_make_request(KVM_REQ_TLB_FLUSH_GPA, vcpu);
> +		} else
> +			cpumask_set_cpu(vcpu->cpu, &vcpu->kvm->arch.tlb_flush_pending);

[Severity: High]
Similar to the huge page mapping case above, does replacing the precise 
KVM_REQ_TLB_FLUSH_GPA request with the tlb_flush_pending bit cause a full 
VMID cycle when mapping normal pages? If so, this would flush the entire 
TLB for the VM on this CPU during routine page faults.

>  	}
>  
>  	return 0;
> @@ -940,6 +948,22 @@ void kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot)
>  {
>  }
>  
> +int kvm_arch_flush_remote_tlbs(struct kvm *kvm)
> +{
> +	/*
> +	 * Queue a TLB invalidation for each CPU to perform on next
> +	 * vcpu loading
> +	 */
> +	if (cpu_has_guestid) {
> +		cpumask_setall(&kvm->arch.tlb_flush_pending);

[Severity: High]
Is this non-atomic memory write to tlb_flush_pending safe against concurrent 
atomic operations? As noted earlier, __kvm_check_vmid() uses the atomic 
cpumask_test_and_clear_cpu() on this same mask. Mixing non-atomic stores 
with atomic operations on the same memory location can cause data corruption 
and lost updates.

> +		/* Be sure that other CPUS can watch the changes */
> +		smp_wmb();
> +	}
> +
> +	/* Return 1 continue to send ipi to running vCPUs */
> +	return 1;
> +}

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260831094716.4016652-1-maobibo@loongson.cn?part=2

Re: [PATCH v3 0/3] LoongArch: KVM: Add separate vmid support

[Huacai Chen]

Applied, thanks.


Huacai

On Mon, Aug 31, 2026 at 5:54 PM Bibo Mao <maobibo@loongson.cn> wrote:
>
> LoongArch KVM hypervisor supports two-level MMU, vpid index is used
> for stage1 MMU and vmid index is used for stage2 MMU.
>
> On 3A5000, vmid must be the same with vpid. On 3A6000 platform vmid
> may separate from vpid. There are such advantages if separated vmid
> is supported.
>   1. One VM uses one vmid, vCPUs on the same VM can share the same vmid.
>   2. If one vCPU switch between different physical CPUs, old vmid can be
>      still useful if it is not expired.
>   3. For remote tlb flush, it is to flush TLBs which are mapping
>      GPA --> HPA, only vmid need update and vpid need not.
>
> Here add separate vmid feature support, vmid feature detecting method
> comes from CPUCFG8 capability when LVZ is enabled.
> ---
> v2 ... v3:
>   1. Merge patch 1-2 into one and patch 3-4 into one.
>   2. Return directly and do not update vmid in function __kvm_check_vmid()
>      if feature cpu_has_guestid is disabled by AI agent review.
>   3. In function kvm_arch_disable_virtualization_cpu disable feature
>      CSR_GTLBC_USES2TGID, similiar with enable cpu virtualization,
>      the same from AI agent review.
>   4. Rename USEVMID with USES2TGID, VMID is KVM software word and S2TGID
>      is hardware word.
>   5. Set cpu bit with vcpu->kvm->arch.tlb_flush_pending with
>      KVM_REQ_TLB_FLUSH, similiar with request KVM_REQ_TLB_FLUSH_GPA.
>
> v1 ... v2:
>   1. Move vmid together with vpid in patch1, rename it with hw_vmid.
>   2. Rename kvm_check_vcpuid() with __kvm_check_vpid(), and function
>      kvm_check_vmid() with __kvm_check_vmid().
>   3. Merge part of patch 2, patch 3/4 together.
>   4. Use bit8 of CPUCFG8 as guestid capability rather than LVZ version,
>      since it is hard to decide whether this feature exists with high LVZ
>      version in future.
>   5. Set cpu bit with vcpu->kvm->arch.tlb_flush_pending rather than
>      using add request KVM_REQ_TLB_FLUSH_GPA, it needs flush TLB on CPU
>      where page fault happens and vCPU maybe migrates to other CPUs.
> ---
> Bibo Mao (3):
>   LoongArch: KVM: Add vmid support for stage2 MMU
>   LoongArch: KVM: Implement vmid updating logic
>   LoongArch: KVM: Enable separate vmid feature
>
>  arch/loongarch/include/asm/kvm_host.h  | 10 ++++
>  arch/loongarch/include/asm/loongarch.h |  5 ++
>  arch/loongarch/kernel/asm-offsets.c    |  1 +
>  arch/loongarch/kernel/cpu-probe.c      |  6 +++
>  arch/loongarch/kvm/main.c              | 71 ++++++++++++++++++++++++--
>  arch/loongarch/kvm/mmu.c               | 30 +++++++++--
>  arch/loongarch/kvm/switch.S            |  5 +-
>  arch/loongarch/kvm/tlb.c               | 19 ++++++-
>  8 files changed, 135 insertions(+), 12 deletions(-)
>
>
> base-commit: 73e3f0710014fe6d4ed98cfc02292f6121db7558
> --
> 2.39.3
>