[PATCH 0/6] LoongArch: KVM: Add separate vmid support

[PATCH 0/6] LoongArch: KVM: Add separate vmid support

[Bibo Mao]

LoongArch KVM hypervisor supports two-level MMU, vpid index is used
for stage1 MMU and vmid index is used for stage2 MMU.

On 3A5000, vmid must be the same with vpid. On 3A6000 platform vmid
may separate from vpid. There are such advantages if separate vmid
is supported.
  1. One VM uses one vmid, vCPUs on the same VM can share the same vmid.
  2. If one vCPU switch between different physical CPU, old vmid can be
     still useful if the old vmid is not expired
  3. For remote tlb flush, it is to flush TLBs which are mapping
     GPA --> HPA, only vmid need update and vpid need not.

Here add separate vmid feature support, vmid feature detecting method
comes from LVZ version with 2 and LVZ is enabled.

Bibo Mao (6):
  LoongArch: KVM: Add vmid support for stage2 MMU
  LoongArch: KVM: Add separate vCPU and VM id update function
  LoongArch: KVM: Add separate vmid feature support
  LoongArch: KVM: implement vmid updating logic
  LoongArch: KVM: Add remote tlb flushing support
  LoongArch: KVM: Enable separate vmid feature

 arch/loongarch/include/asm/kvm_host.h  | 10 ++++
 arch/loongarch/include/asm/loongarch.h |  2 +
 arch/loongarch/kernel/asm-offsets.c    |  1 +
 arch/loongarch/kernel/cpu-probe.c      |  7 ++-
 arch/loongarch/kvm/main.c              | 76 ++++++++++++++++++++++++--
 arch/loongarch/kvm/mmu.c               | 16 ++++++
 arch/loongarch/kvm/switch.S            |  5 +-
 arch/loongarch/kvm/tlb.c               | 19 ++++++-
 arch/loongarch/kvm/vcpu.c              |  7 ++-
 9 files changed, 133 insertions(+), 10 deletions(-)


base-commit: f5098b6bae761e346ebcd9da7f95622c04733cff
-- 
2.39.3

[PATCH 1/6] LoongArch: KVM: Add vmid support for stage2 MMU

[Bibo Mao]

LoongArch KVM hypervisor supports two-level MMU, vpid index is used
for stage1 MMU and vmid index is used for stage2 MMU.

On 3A5000, vmid must be the same with vpid. On 3A6000 platform vmid
may separate from vpid. If vcpu migrate to different physical CPUs,
vpid need change however vmid can keep the same with old value. Also
vmid index of the while VM machine on physical CPU the same, all vCPUs
on the VM can share the same vmid index on one physical CPU.

Here vmid index is added and it keeps the same with vpid still.

Signed-off-by: Bibo Mao <maobibo@loongson.cn>
---
 arch/loongarch/include/asm/kvm_host.h | 3 +++
 arch/loongarch/kernel/asm-offsets.c   | 1 +
 arch/loongarch/kvm/main.c             | 1 +
 arch/loongarch/kvm/switch.S           | 5 ++---
 arch/loongarch/kvm/tlb.c              | 5 ++++-
 5 files changed, 11 insertions(+), 4 deletions(-)

diff --git a/arch/loongarch/include/asm/kvm_host.h b/arch/loongarch/include/asm/kvm_host.h
index 23cfbecebbd7..ec7b0c402385 100644
--- a/arch/loongarch/include/asm/kvm_host.h
+++ b/arch/loongarch/include/asm/kvm_host.h
@@ -189,6 +189,9 @@ struct kvm_vcpu_arch {
 	unsigned long host_tp;
 	unsigned long host_pgd;
 
+	/* vmid info for guest VM */
+	unsigned long vmid;
+
 	/* Host CSRs are used when handling exits from guest */
 	unsigned long badi;
 	unsigned long badv;
diff --git a/arch/loongarch/kernel/asm-offsets.c b/arch/loongarch/kernel/asm-offsets.c
index 1b861cbc5e10..32ef6adfc8dd 100644
--- a/arch/loongarch/kernel/asm-offsets.c
+++ b/arch/loongarch/kernel/asm-offsets.c
@@ -300,6 +300,7 @@ static void __used output_kvm_defines(void)
 	OFFSET(KVM_ARCH_HSP, kvm_vcpu_arch, host_sp);
 	OFFSET(KVM_ARCH_HTP, kvm_vcpu_arch, host_tp);
 	OFFSET(KVM_ARCH_HPGD, kvm_vcpu_arch, host_pgd);
+	OFFSET(KVM_ARCH_VMID, kvm_vcpu_arch, vmid);
 	OFFSET(KVM_ARCH_KVMPGD, kvm_vcpu_arch, kvm_pgd);
 	OFFSET(KVM_ARCH_HANDLE_EXIT, kvm_vcpu_arch, handle_exit);
 	OFFSET(KVM_ARCH_HEENTRY, kvm_vcpu_arch, host_eentry);
diff --git a/arch/loongarch/kvm/main.c b/arch/loongarch/kvm/main.c
index 3e1005526f4b..6e3e8efa1dc2 100644
--- a/arch/loongarch/kvm/main.c
+++ b/arch/loongarch/kvm/main.c
@@ -223,6 +223,7 @@ static void kvm_update_vpid(struct kvm_vcpu *vcpu, int cpu)
 
 	context->vpid_cache = vpid;
 	vcpu->arch.vpid = vpid;
+	vcpu->arch.vmid = vcpu->arch.vpid & vpid_mask;
 }
 
 void kvm_check_vpid(struct kvm_vcpu *vcpu)
diff --git a/arch/loongarch/kvm/switch.S b/arch/loongarch/kvm/switch.S
index 936e4ae3e408..af972394fd55 100644
--- a/arch/loongarch/kvm/switch.S
+++ b/arch/loongarch/kvm/switch.S
@@ -66,9 +66,8 @@
 	ld.d	t0, a2, KVM_ARCH_KVMPGD
 	csrwr	t0, LOONGARCH_CSR_PGDL
 
-	/* Mix GID and RID */
-	csrrd		t1, LOONGARCH_CSR_GSTAT
-	bstrpick.w	t1, t1, CSR_GSTAT_GID_SHIFT_END, CSR_GSTAT_GID_SHIFT
+	/* Set VMID for gpa --> hpa mapping */
+	ld.d		t1, a2, KVM_ARCH_VMID
 	csrrd		t0, LOONGARCH_CSR_GTLBC
 	bstrins.w	t0, t1, CSR_GTLBC_TGID_SHIFT_END, CSR_GTLBC_TGID_SHIFT
 	csrwr		t0, LOONGARCH_CSR_GTLBC
diff --git a/arch/loongarch/kvm/tlb.c b/arch/loongarch/kvm/tlb.c
index ebdbe9264e9c..38daf936021d 100644
--- a/arch/loongarch/kvm/tlb.c
+++ b/arch/loongarch/kvm/tlb.c
@@ -23,7 +23,10 @@ void kvm_flush_tlb_all(void)
 
 void kvm_flush_tlb_gpa(struct kvm_vcpu *vcpu, unsigned long gpa)
 {
+	unsigned int vmid;
+
 	lockdep_assert_irqs_disabled();
 	gpa &= (PAGE_MASK << 1);
-	invtlb(INVTLB_GID_ADDR, read_csr_gstat() & CSR_GSTAT_GID, gpa);
+	vmid = (vcpu->arch.vmid << CSR_GSTAT_GID_SHIFT) & CSR_GSTAT_GID;
+	invtlb(INVTLB_GID_ADDR, vmid, gpa);
 }
-- 
2.39.3

[PATCH 2/6] LoongArch: KVM: Add separate vCPU and VM id update function

[Bibo Mao]

There are two-level MMU on LoongArch KVM system, the first level (stage1)
MMU is mapping for GVA --> GPA, the second level (stage2) is mapping
for GPA --> HPA. Similar with ASID, there is different ID management
to support multiple VMs and vCPUs. The stage1 MMU is called vCPUID, and
the second MMU is called VMID, the corresponding ID management function
is kvm_check_vcpuid() and kvm_check_vmid().

Signed-off-by: Bibo Mao <maobibo@loongson.cn>
---
 arch/loongarch/kvm/main.c | 21 ++++++++++++++++++---
 1 file changed, 18 insertions(+), 3 deletions(-)

diff --git a/arch/loongarch/kvm/main.c b/arch/loongarch/kvm/main.c
index 6e3e8efa1dc2..c1e1d8424fa7 100644
--- a/arch/loongarch/kvm/main.c
+++ b/arch/loongarch/kvm/main.c
@@ -223,10 +223,9 @@ static void kvm_update_vpid(struct kvm_vcpu *vcpu, int cpu)
 
 	context->vpid_cache = vpid;
 	vcpu->arch.vpid = vpid;
-	vcpu->arch.vmid = vcpu->arch.vpid & vpid_mask;
 }
 
-void kvm_check_vpid(struct kvm_vcpu *vcpu)
+static void kvm_check_vcpuid(struct kvm_vcpu *vcpu)
 {
 	int cpu;
 	bool migrated;
@@ -254,7 +253,6 @@ void kvm_check_vpid(struct kvm_vcpu *vcpu)
 		kvm_update_vpid(vcpu, cpu);
 		trace_kvm_vpid_change(vcpu, vcpu->arch.vpid);
 		vcpu->cpu = cpu;
-		kvm_clear_request(KVM_REQ_TLB_FLUSH_GPA, vcpu);
 
 		/*
 		 * LLBCTL is a separated guest CSR register from host, a general
@@ -280,6 +278,23 @@ void kvm_check_vpid(struct kvm_vcpu *vcpu)
 	}
 }
 
+static void kvm_check_vmid(struct kvm_vcpu *vcpu)
+{
+	unsigned long vmid;
+
+	vmid = vcpu->arch.vpid & vpid_mask;
+	if (vcpu->arch.vmid != vmid) {
+		vcpu->arch.vmid = vmid;
+		kvm_clear_request(KVM_REQ_TLB_FLUSH_GPA, vcpu);
+	}
+}
+
+void kvm_check_vpid(struct kvm_vcpu *vcpu)
+{
+	kvm_check_vcpuid(vcpu);
+	kvm_check_vmid(vcpu);
+}
+
 void kvm_init_vmcs(struct kvm *kvm)
 {
 	kvm->arch.vmcs = vmcs;
-- 
2.39.3

[PATCH 3/6] LoongArch: KVM: Add separate vmid feature support

[Bibo Mao]

Feature cpu_has_guestid is used to check whether separate vmid/vpid
is supported or not. Also add different vmid updating function.

Signed-off-by: Bibo Mao <maobibo@loongson.cn>
---
 arch/loongarch/include/asm/kvm_host.h |  2 ++
 arch/loongarch/kvm/main.c             | 18 +++++++++++++-----
 arch/loongarch/kvm/tlb.c              | 14 ++++++++++++++
 3 files changed, 29 insertions(+), 5 deletions(-)

diff --git a/arch/loongarch/include/asm/kvm_host.h b/arch/loongarch/include/asm/kvm_host.h
index ec7b0c402385..e93307cbe740 100644
--- a/arch/loongarch/include/asm/kvm_host.h
+++ b/arch/loongarch/include/asm/kvm_host.h
@@ -319,6 +319,8 @@ bool kvm_arch_pmi_in_guest(struct kvm_vcpu *vcpu);
 int kvm_arch_vcpu_dump_regs(struct kvm_vcpu *vcpu);
 
 /* MMU handling */
+void kvm_flush_tlb_all_stage1(void);
+void kvm_flush_tlb_all_stage2(void);
 void kvm_flush_tlb_all(void);
 void kvm_flush_tlb_gpa(struct kvm_vcpu *vcpu, unsigned long gpa);
 int kvm_handle_mm_fault(struct kvm_vcpu *vcpu, unsigned long badv, bool write, int ecode);
diff --git a/arch/loongarch/kvm/main.c b/arch/loongarch/kvm/main.c
index c1e1d8424fa7..3df46e02594f 100644
--- a/arch/loongarch/kvm/main.c
+++ b/arch/loongarch/kvm/main.c
@@ -218,7 +218,10 @@ static void kvm_update_vpid(struct kvm_vcpu *vcpu, int cpu)
 		++vpid; /* vpid 0 reserved for root */
 
 		/* start new vpid cycle */
-		kvm_flush_tlb_all();
+		if (!cpu_has_guestid)
+			kvm_flush_tlb_all();
+		else
+			kvm_flush_tlb_all_stage1();
 	}
 
 	context->vpid_cache = vpid;
@@ -282,10 +285,15 @@ static void kvm_check_vmid(struct kvm_vcpu *vcpu)
 {
 	unsigned long vmid;
 
-	vmid = vcpu->arch.vpid & vpid_mask;
-	if (vcpu->arch.vmid != vmid) {
-		vcpu->arch.vmid = vmid;
-		kvm_clear_request(KVM_REQ_TLB_FLUSH_GPA, vcpu);
+	/* On some machines like 3A5000, vmid needs the same with vpid */
+	if (!cpu_has_guestid) {
+		vmid = vcpu->arch.vpid & vpid_mask;
+		if (vcpu->arch.vmid != vmid) {
+			vcpu->arch.vmid = vmid;
+			kvm_clear_request(KVM_REQ_TLB_FLUSH_GPA, vcpu);
+		}
+
+		return;
 	}
 }
 
diff --git a/arch/loongarch/kvm/tlb.c b/arch/loongarch/kvm/tlb.c
index 38daf936021d..1d95e2208e82 100644
--- a/arch/loongarch/kvm/tlb.c
+++ b/arch/loongarch/kvm/tlb.c
@@ -21,6 +21,20 @@ void kvm_flush_tlb_all(void)
 	local_irq_restore(flags);
 }
 
+/* Invalidate all stage1 TLB entries including GVA-->GPA mappings */
+void kvm_flush_tlb_all_stage1(void)
+{
+	lockdep_assert_irqs_disabled();
+	invtlb_all(INVGTLB_ALLGID_GVA_TO_GPA, 0, 0);
+}
+
+/* Invalidate all stage2 TLB entries including GPA-->HPA  mappings */
+void kvm_flush_tlb_all_stage2(void)
+{
+	lockdep_assert_irqs_disabled();
+	invtlb_all(INVTLB_ALLGID_GPA_TO_HPA, 0, 0);
+}
+
 void kvm_flush_tlb_gpa(struct kvm_vcpu *vcpu, unsigned long gpa)
 {
 	unsigned int vmid;
-- 
2.39.3

[PATCH 4/6] LoongArch: KVM: implement vmid updating logic

[Bibo Mao]

For every physical CPU, there is one vmid calculation method. For
vCPUs on the same VM, vmid is the same. However for vCPUs on
different VM, vmid is different. When vCPU is scheduled on the
physical CPU, it checked vmid of this VM and the global cached
vmid, and judge whether it is valid or not.

Signed-off-by: Bibo Mao <maobibo@loongson.cn>
---
 arch/loongarch/include/asm/kvm_host.h |  2 ++
 arch/loongarch/kvm/main.c             | 42 ++++++++++++++++++++++++++-
 2 files changed, 43 insertions(+), 1 deletion(-)

diff --git a/arch/loongarch/include/asm/kvm_host.h b/arch/loongarch/include/asm/kvm_host.h
index e93307cbe740..a01623c42c7a 100644
--- a/arch/loongarch/include/asm/kvm_host.h
+++ b/arch/loongarch/include/asm/kvm_host.h
@@ -79,6 +79,7 @@ struct kvm_arch_memory_slot {
 #define HOST_MAX_PMNUM			16
 struct kvm_context {
 	unsigned long vpid_cache;
+	unsigned long vmid_cache;
 	struct kvm_vcpu *last_vcpu;
 	/* Host PMU CSR */
 	u64 perf_ctrl[HOST_MAX_PMNUM];
@@ -132,6 +133,7 @@ struct kvm_arch {
 	unsigned long kvm_features;
 
 	s64 time_offset;
+	unsigned long vmid[NR_CPUS];
 	struct kvm_context __percpu *vmcs;
 	struct loongarch_ipi *ipi;
 	struct loongarch_dmsintc *dmsintc;
diff --git a/arch/loongarch/kvm/main.c b/arch/loongarch/kvm/main.c
index 3df46e02594f..7b3661da8958 100644
--- a/arch/loongarch/kvm/main.c
+++ b/arch/loongarch/kvm/main.c
@@ -281,9 +281,33 @@ static void kvm_check_vcpuid(struct kvm_vcpu *vcpu)
 	}
 }
 
-static void kvm_check_vmid(struct kvm_vcpu *vcpu)
+static void kvm_update_vmid(struct kvm_vcpu *vcpu, int cpu)
 {
 	unsigned long vmid;
+	struct kvm_context *context;
+
+	context = per_cpu_ptr(vcpu->kvm->arch.vmcs, cpu);
+	vmid = context->vmid_cache + 1;
+	if (!(vmid & vpid_mask)) {
+		/* finish round of vmid loop */
+		if (unlikely(!vmid))
+			vmid = vpid_mask + 1;
+
+		++vmid; /* vmid 0 reserved for root */
+
+		/* start new vmid cycle */
+		kvm_flush_tlb_all_stage2();
+	}
+
+	context->vmid_cache = vmid;
+	vcpu->kvm->arch.vmid[cpu] = vmid;
+}
+
+static void kvm_check_vmid(struct kvm_vcpu *vcpu)
+{
+	int cpu;
+	unsigned long ver, old, vmid;
+	struct kvm_context *context;
 
 	/* On some machines like 3A5000, vmid needs the same with vpid */
 	if (!cpu_has_guestid) {
@@ -295,6 +319,21 @@ static void kvm_check_vmid(struct kvm_vcpu *vcpu)
 
 		return;
 	}
+
+	cpu = smp_processor_id();
+	context = per_cpu_ptr(vcpu->kvm->arch.vmcs, cpu);
+
+	/*
+	 * Check if our vmid is of an older version
+	 */
+	ver = vcpu->kvm->arch.vmid[cpu] & ~vpid_mask;
+	old = context->vmid_cache & ~vpid_mask;
+	if (ver != old) {
+		kvm_update_vmid(vcpu, cpu);
+		kvm_clear_request(KVM_REQ_TLB_FLUSH_GPA, vcpu);
+	}
+
+	vcpu->arch.vmid = vcpu->kvm->arch.vmid[cpu] & vpid_mask;
 }
 
 void kvm_check_vpid(struct kvm_vcpu *vcpu)
@@ -400,6 +439,7 @@ static int kvm_loongarch_env_init(void)
 	for_each_possible_cpu(cpu) {
 		context = per_cpu_ptr(vmcs, cpu);
 		context->vpid_cache = vpid_mask + 1;
+		context->vmid_cache = vpid_mask + 1;
 		context->last_vcpu = NULL;
 	}
 
-- 
2.39.3

[PATCH 5/6] LoongArch: KVM: Add remote tlb flushing support

[Bibo Mao]

With remote tlb flushing, vpid index stays unchanged and only vmid
index is updated, since remote tlb flushing is to flush TLBs relative
GPA --> HPA.

For flushing method, cpumask tlb_flush_pending is added and set for
all possible CPUs. When vCPUs is sched on the physical CPU, vmid is
updated and cpumask for this physical CPU is cleared.

Signed-off-by: Bibo Mao <maobibo@loongson.cn>
---
 arch/loongarch/include/asm/kvm_host.h |  3 +++
 arch/loongarch/kvm/main.c             |  2 ++
 arch/loongarch/kvm/mmu.c              | 16 ++++++++++++++++
 arch/loongarch/kvm/vcpu.c             |  7 ++++++-
 4 files changed, 27 insertions(+), 1 deletion(-)

diff --git a/arch/loongarch/include/asm/kvm_host.h b/arch/loongarch/include/asm/kvm_host.h
index a01623c42c7a..2c3cb475abf9 100644
--- a/arch/loongarch/include/asm/kvm_host.h
+++ b/arch/loongarch/include/asm/kvm_host.h
@@ -133,6 +133,7 @@ struct kvm_arch {
 	unsigned long kvm_features;
 
 	s64 time_offset;
+	cpumask_t tlb_flush_pending;
 	unsigned long vmid[NR_CPUS];
 	struct kvm_context __percpu *vmcs;
 	struct loongarch_ipi *ipi;
@@ -357,6 +358,8 @@ static inline void kvm_arch_vcpu_block_finish(struct kvm_vcpu *vcpu) {}
 static inline void kvm_arch_free_memslot(struct kvm *kvm, struct kvm_memory_slot *slot) {}
 void kvm_check_vpid(struct kvm_vcpu *vcpu);
 enum hrtimer_restart kvm_swtimer_wakeup(struct hrtimer *timer);
+#define __KVM_HAVE_ARCH_FLUSH_REMOTE_TLBS
+int kvm_arch_flush_remote_tlbs(struct kvm *kvm);
 void kvm_arch_flush_remote_tlbs_memslot(struct kvm *kvm, const struct kvm_memory_slot *memslot);
 void kvm_init_vmcs(struct kvm *kvm);
 void kvm_exc_entry(void);
diff --git a/arch/loongarch/kvm/main.c b/arch/loongarch/kvm/main.c
index 7b3661da8958..86b1af823e2b 100644
--- a/arch/loongarch/kvm/main.c
+++ b/arch/loongarch/kvm/main.c
@@ -322,6 +322,8 @@ static void kvm_check_vmid(struct kvm_vcpu *vcpu)
 
 	cpu = smp_processor_id();
 	context = per_cpu_ptr(vcpu->kvm->arch.vmcs, cpu);
+	if (cpumask_test_and_clear_cpu(cpu, &vcpu->kvm->arch.tlb_flush_pending))
+		vcpu->kvm->arch.vmid[cpu] = 0;
 
 	/*
 	 * Check if our vmid is of an older version
diff --git a/arch/loongarch/kvm/mmu.c b/arch/loongarch/kvm/mmu.c
index e104897aa532..560a2d0f111f 100644
--- a/arch/loongarch/kvm/mmu.c
+++ b/arch/loongarch/kvm/mmu.c
@@ -940,6 +940,22 @@ void kvm_arch_sync_dirty_log(struct kvm *kvm, struct kvm_memory_slot *memslot)
 {
 }
 
+int kvm_arch_flush_remote_tlbs(struct kvm *kvm)
+{
+	/*
+	 * Queue a TLB invalidation for each CPU to perform on next
+	 * vcpu loading
+	 */
+	if (cpu_has_guestid) {
+		cpumask_setall(&kvm->arch.tlb_flush_pending);
+		/* Be sure that other CPUS can watch the changes */
+		smp_wmb();
+	}
+
+	/* Return 1 continue to send ipi to running vCPUs */
+	return 1;
+}
+
 void kvm_arch_flush_remote_tlbs_memslot(struct kvm *kvm,
 					const struct kvm_memory_slot *memslot)
 {
diff --git a/arch/loongarch/kvm/vcpu.c b/arch/loongarch/kvm/vcpu.c
index 20c207d80e31..19c81ae8941a 100644
--- a/arch/loongarch/kvm/vcpu.c
+++ b/arch/loongarch/kvm/vcpu.c
@@ -210,7 +210,12 @@ static int kvm_check_requests(struct kvm_vcpu *vcpu)
 		return RESUME_GUEST;
 
 	if (kvm_check_request(KVM_REQ_TLB_FLUSH, vcpu))
-		vcpu->arch.vpid = 0;  /* Drop vpid for this vCPU */
+		/*
+		 * vpid need the same with vmid if vpid is not separated
+		 * with vmid
+		 */
+		if (!cpu_has_guestid)
+			vcpu->arch.vpid = 0;  /* Drop vpid for this vCPU */
 
 	if (kvm_dirty_ring_check_request(vcpu))
 		return RESUME_HOST;
-- 
2.39.3

[PATCH 6/6] LoongArch: KVM: Enable separate vmid feature

[Bibo Mao]

With CSR GTLBC shortname for Guest TLB Control Register, separate vmid
feature will be enabled if bit 14 CSR_GTLBC_USEVMID is set. Enable
this feature if cpu_has_guestid is true when LVZ is enabled and the LVZ
version is 2.

Signed-off-by: Bibo Mao <maobibo@loongson.cn>
---
 arch/loongarch/include/asm/loongarch.h | 2 ++
 arch/loongarch/kernel/cpu-probe.c      | 7 ++++++-
 arch/loongarch/kvm/main.c              | 4 +++-
 3 files changed, 11 insertions(+), 2 deletions(-)

diff --git a/arch/loongarch/include/asm/loongarch.h b/arch/loongarch/include/asm/loongarch.h
index 2a6bc99177d8..4c4249446a61 100644
--- a/arch/loongarch/include/asm/loongarch.h
+++ b/arch/loongarch/include/asm/loongarch.h
@@ -350,6 +350,8 @@
 #define  CSR_GTLBC_TGID_WIDTH		8
 #define  CSR_GTLBC_TGID_SHIFT_END	(CSR_GTLBC_TGID_SHIFT + CSR_GTLBC_TGID_WIDTH - 1)
 #define  CSR_GTLBC_TGID			(_ULCAST_(0xff) << CSR_GTLBC_TGID_SHIFT)
+#define  CSR_GTLBC_USEVMID_SHIFT	14
+#define  CSR_GTLBC_USEVMID		(_ULCAST_(0x1) << CSR_GTLBC_USEVMID_SHIFT)
 #define  CSR_GTLBC_TOTI_SHIFT		13
 #define  CSR_GTLBC_TOTI			(_ULCAST_(0x1) << CSR_GTLBC_TOTI_SHIFT)
 #define  CSR_GTLBC_USETGID_SHIFT	12
diff --git a/arch/loongarch/kernel/cpu-probe.c b/arch/loongarch/kernel/cpu-probe.c
index 74d31f260dfd..cbf1dafac0a0 100644
--- a/arch/loongarch/kernel/cpu-probe.c
+++ b/arch/loongarch/kernel/cpu-probe.c
@@ -135,7 +135,7 @@ static void set_isa(struct cpuinfo_loongarch *c, unsigned int isa)
 
 static void cpu_probe_common(struct cpuinfo_loongarch *c)
 {
-	unsigned int config;
+	unsigned int config, version;
 	unsigned long asid_mask;
 
 	c->options = LOONGARCH_CPU_CPUCFG | LOONGARCH_CPU_CSR | LOONGARCH_CPU_VINT;
@@ -221,6 +221,11 @@ static void cpu_probe_common(struct cpuinfo_loongarch *c)
 	if (config & CPUCFG2_LVZP) {
 		c->options |= LOONGARCH_CPU_LVZ;
 		elf_hwcap |= HWCAP_LOONGARCH_LVZ;
+
+		/* Separate VMID and VPID with LVZ version == 2 */
+		version = (config & CPUCFG2_LVZVER) >> 11;
+		if (version == 2)
+			c->options |= LOONGARCH_CPU_GUESTID;
 	}
 #ifdef CONFIG_CPU_HAS_LBT
 	if (config & CPUCFG2_X86BT) {
diff --git a/arch/loongarch/kvm/main.c b/arch/loongarch/kvm/main.c
index 86b1af823e2b..43f5b40786f2 100644
--- a/arch/loongarch/kvm/main.c
+++ b/arch/loongarch/kvm/main.c
@@ -365,7 +365,7 @@ int kvm_arch_enable_virtualization_cpu(void)
 	write_csr_gcfg(0);
 	write_csr_gstat(0);
 	write_csr_gintc(0);
-	clear_csr_gtlbc(CSR_GTLBC_USETGID | CSR_GTLBC_TOTI);
+	clear_csr_gtlbc(CSR_GTLBC_USETGID | CSR_GTLBC_TOTI | CSR_GTLBC_USEVMID);
 
 	/*
 	 * Enable virtualization features granting guest direct control of
@@ -388,6 +388,8 @@ int kvm_arch_enable_virtualization_cpu(void)
 
 	/* Enable using TGID  */
 	set_csr_gtlbc(CSR_GTLBC_USETGID);
+	if (cpu_has_guestid)
+		set_csr_gtlbc(CSR_GTLBC_USEVMID);
 	kvm_debug("GCFG:%lx GSTAT:%lx GINTC:%lx GTLBC:%lx",
 		  read_csr_gcfg(), read_csr_gstat(), read_csr_gintc(), read_csr_gtlbc());
 
-- 
2.39.3

Re: [PATCH 6/6] LoongArch: KVM: Enable separate vmid feature

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] CSR_GTLBC_USEVMID is not cleared when virtualization is disabled, leaving the CPU in an inconsistent state and leaking hypervisor state to the underlying hardware.
--

--- Patch [6]: [PATCH 6/6] LoongArch: KVM: Enable separate vmid feature ---
commit 255d457e2558ae0a9e092e8d465e179d08390542
Author: Bibo Mao <maobibo@loongson.cn>

LoongArch: KVM: Enable separate vmid feature

With CSR GTLBC shortname for Guest TLB Control Register, separate vmid
feature will be enabled if bit 14 CSR_GTLBC_USEVMID is set. Enable
this feature if cpu_has_guestid is true when LVZ is enabled and the LVZ
version is 2.

> diff --git a/arch/loongarch/kvm/main.c b/arch/loongarch/kvm/main.c
> index 86b1af823e2b4..43f5b40786f24 100644
> --- a/arch/loongarch/kvm/main.c
> +++ b/arch/loongarch/kvm/main.c
> @@ -365,7 +365,7 @@ int kvm_arch_enable_virtualization_cpu(void)
>  	write_csr_gcfg(0);
>  	write_csr_gstat(0);
>  	write_csr_gintc(0);
> -	clear_csr_gtlbc(CSR_GTLBC_USETGID | CSR_GTLBC_TOTI);
> +	clear_csr_gtlbc(CSR_GTLBC_USETGID | CSR_GTLBC_TOTI | CSR_GTLBC_USEVMID);

[Severity: Medium]
Since this clears the newly added CSR_GTLBC_USEVMID bit during initialization,
should it also be cleared in kvm_arch_disable_virtualization_cpu() when
hardware virtualization is disabled?

Currently, kvm_arch_disable_virtualization_cpu() only clears the prior bits:

arch/loongarch/kvm/main.c:kvm_arch_disable_virtualization_cpu() {
	write_csr_gcfg(0);
	write_csr_gstat(0);
	write_csr_gintc(0);
	clear_csr_gtlbc(CSR_GTLBC_USETGID | CSR_GTLBC_TOTI);
	...
}

Does missing this bit during teardown leave the CPU in an inconsistent state
by retaining KVM-specific control bits in the Guest TLB Control register?

This might cause issues during CPU offline/online cycles, kexec into a
different kernel, or when another hypervisor initializes the CPU.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260727072109.2151675-1-maobibo@loongson.cn?part=6

Re: [PATCH 3/6] LoongArch: KVM: Add separate vmid feature support

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Critical] Missing VMID updating logic when `cpu_has_guestid` is true breaks stage-2 MMU isolation.
--

--- Patch [3]: [PATCH 3/6] LoongArch: KVM: Add separate vmid feature support ---
commit a0b8bed9027bd243aca0ea026c90d7c9e7dd4d26
Author: Bibo Mao <maobibo@loongson.cn>

LoongArch: KVM: Add separate vmid feature support

This commit introduces support for a separate VMID feature on LoongArch
by checking the cpu_has_guestid flag. It adds different VMID update logic
and new stage-1 and stage-2 TLB flush functions.

> diff --git a/arch/loongarch/kvm/main.c b/arch/loongarch/kvm/main.c
> index c1e1d8424fa7e..3df46e02594f4 100644
> --- a/arch/loongarch/kvm/main.c
> +++ b/arch/loongarch/kvm/main.c

[ ... ]

> @@ -282,10 +285,15 @@ static void kvm_check_vmid(struct kvm_vcpu *vcpu)
>  {
>  	unsigned long vmid;
>  
> -	vmid = vcpu->arch.vpid & vpid_mask;
> -	if (vcpu->arch.vmid != vmid) {
> -		vcpu->arch.vmid = vmid;
> -		kvm_clear_request(KVM_REQ_TLB_FLUSH_GPA, vcpu);
> +	/* On some machines like 3A5000, vmid needs the same with vpid */
> +	if (!cpu_has_guestid) {
> +		vmid = vcpu->arch.vpid & vpid_mask;
> +		if (vcpu->arch.vmid != vmid) {
> +			vcpu->arch.vmid = vmid;
> +			kvm_clear_request(KVM_REQ_TLB_FLUSH_GPA, vcpu);
> +		}
> +
> +		return;
>  	}
>  }

[Severity: Critical]
Does this bypass VMID assignment when cpu_has_guestid is true?

If cpu_has_guestid is true, the function falls through and returns without
assigning vcpu->arch.vmid. Since vcpu->arch.vmid is zero-initialized, doesn't
it remain 0 (which is reserved for the root/host environment)?

When kvm_switch_to_guest() in arch/loongarch/kvm/switch.S loads
vcpu->arch.vmid and writes it to hardware GTLBC.TGID:

	/* Set VMID for gpa --> hpa mapping */
	ld.d		t1, a2, KVM_ARCH_VMID
	csrrd		t0, LOONGARCH_CSR_GTLBC
	bstrins.w	t0, t1, CSR_GTLBC_TGID_SHIFT_END, CSR_GTLBC_TGID_SHIFT
	csrwr		t0, LOONGARCH_CSR_GTLBC

Will this cause guests to run with VMID 0 and break stage-2 MMU isolation?

> diff --git a/arch/loongarch/kvm/tlb.c b/arch/loongarch/kvm/tlb.c
> index 38daf936021d1..1d95e2208e82e 100644
> --- a/arch/loongarch/kvm/tlb.c
> +++ b/arch/loongarch/kvm/tlb.c

[ ... ]

> +/* Invalidate all stage2 TLB entries including GPA-->HPA  mappings */
> +void kvm_flush_tlb_all_stage2(void)
> +{
> +	lockdep_assert_irqs_disabled();
> +	invtlb_all(INVTLB_ALLGID_GPA_TO_HPA, 0, 0);
> +}

[Severity: Critical]
Is this function reachable?

Because the VMID updating logic appears to be omitted from kvm_check_vmid()
when cpu_has_guestid is true, is this stage-2 TLB flush function ever called
during VMID wraparound?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260727072109.2151675-1-maobibo@loongson.cn?part=3

Re: [PATCH 5/6] LoongArch: KVM: Add remote tlb flushing support

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Local TLB flush requests via KVM_REQ_TLB_FLUSH are silently ignored when cpu_has_guestid is true.
--

--- Patch [5]: [PATCH 5/6] LoongArch: KVM: Add remote tlb flushing support ---
commit 8224c08441c8eee94923d3771b8015f38a7278ab
Author: Bibo Mao <maobibo@loongson.cn>

LoongArch: KVM: Add remote tlb flushing support

This commit introduces remote TLB flushing support by queueing a TLB
invalidation for each CPU to perform on the next vCPU load. It adds a
tlb_flush_pending cpumask and updates the vmid index when scheduled on
a physical CPU, while keeping the vpid index unchanged.

[ ... ]

> diff --git a/arch/loongarch/kvm/vcpu.c b/arch/loongarch/kvm/vcpu.c
> index 20c207d80e31a..19c81ae8941a9 100644
> --- a/arch/loongarch/kvm/vcpu.c
> +++ b/arch/loongarch/kvm/vcpu.c
> @@ -210,7 +210,12 @@ static int kvm_check_requests(struct kvm_vcpu *vcpu)
>  		return RESUME_GUEST;
>  
>  	if (kvm_check_request(KVM_REQ_TLB_FLUSH, vcpu))
> -		vcpu->arch.vpid = 0;  /* Drop vpid for this vCPU */
> +		/*
> +		 * vpid need the same with vmid if vpid is not separated
> +		 * with vmid
> +		 */
> +		if (!cpu_has_guestid)
> +			vcpu->arch.vpid = 0;  /* Drop vpid for this vCPU */

[Severity: High]
Does this silently ignore local TLB flush requests when cpu_has_guestid is
true?

If a local KVM_REQ_TLB_FLUSH is made, for example by kvm_map_page() in
arch/loongarch/kvm/mmu.c when upgrading to a huge page, kvm_check_request()
will clear the request bit here.

However, if cpu_has_guestid is true, the VPID is not dropped. Because this
is a local flush, the tlb_flush_pending cpumask is not set globally, meaning
kvm_check_vmid() will not drop the VMID either.

Will this cause the local TLB flush to be completely lost, potentially
leading to an infinite page fault loop if the guest hits a stale invalid
TLB entry on hardware with a page table walker?

[ ... ]

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260727072109.2151675-1-maobibo@loongson.cn?part=5

Re: [PATCH 4/6] LoongArch: KVM: implement vmid updating logic

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:

New issues:
- [Critical] The separation of VMID from VPID for hardware with `cpu_has_guestid` breaks the handling of `KVM_REQ_TLB_FLUSH` for Stage-2 TLBs.
- [High] `kvm_check_vmid` prematurely clears the deferred `KVM_REQ_TLB_FLUSH_GPA` request when a vCPU allocates a new VMID on a single physical CPU, leading to stale TLB entries if the vCPU migrates.

Pre-existing issues:
- [High] Error paths in `kvm_loongarch_env_init()` and `kvm_loongarch_init()` fail to clean up dynamically allocated memory and unregister callbacks/devices.
--

--- Patch [4]: [PATCH 4/6] LoongArch: KVM: implement vmid updating logic ---
commit e654984b1077924ffa92adca677721884f2ab85a
Author: Bibo Mao <maobibo@loongson.cn>

LoongArch: KVM: implement vmid updating logic
    
This commit separates the VMID calculation from the VPID logic for LoongArch KVM.
It implements a per-physical-CPU VMID array for VMs to handle hardware with
cpu_has_guestid, and introduces functions to manage VMID allocation and wrapping.

> diff --git a/arch/loongarch/kvm/main.c b/arch/loongarch/kvm/main.c
> index 3df46e02594f4..7b3661da89587 100644
> --- a/arch/loongarch/kvm/main.c
> +++ b/arch/loongarch/kvm/main.c
[ ... ]
> @@ -281,9 +281,33 @@ static void kvm_check_vcpuid(struct kvm_vcpu *vcpu)
>  	}
>  }
>  
> -static void kvm_check_vmid(struct kvm_vcpu *vcpu)
> +static void kvm_update_vmid(struct kvm_vcpu *vcpu, int cpu)
[ ... ]
> +static void kvm_check_vmid(struct kvm_vcpu *vcpu)
> +{
> +	int cpu;
> +	unsigned long ver, old, vmid;
> +	struct kvm_context *context;
[ ... ]
> +	cpu = smp_processor_id();
> +	context = per_cpu_ptr(vcpu->kvm->arch.vmcs, cpu);
> +
> +	/*
> +	 * Check if our vmid is of an older version
> +	 */
> +	ver = vcpu->kvm->arch.vmid[cpu] & ~vpid_mask;
> +	old = context->vmid_cache & ~vpid_mask;
> +	if (ver != old) {
> +		kvm_update_vmid(vcpu, cpu);
> +		kvm_clear_request(KVM_REQ_TLB_FLUSH_GPA, vcpu);
> +	}

[Severity: Critical]
Does separating VMID from VPID break the KVM_REQ_TLB_FLUSH handling?

When the KVM core issues a remote TLB flush via kvm_flush_remote_tlbs(),
kvm_check_requests() handles this by setting vcpu->arch.vpid = 0 to force a
new VPID. 

However, this new logic only evaluates the VM's per-physical-CPU VMID array
(vcpu->kvm->arch.vmid[cpu]). Without checking if the VPID was cleared or
resetting the per-cpu VMID state, does this mean KVM will fail to refresh
the VMID and miss flushing the Stage-2 TLB when requested by the core?
This could allow guests to retain access to unmapped or repurposed host
memory during reclaim.

[Severity: High]
Is it safe to clear KVM_REQ_TLB_FLUSH_GPA based purely on a local
per-physical-CPU VMID update?

If a vCPU sets KVM_REQ_TLB_FLUSH_GPA on CPU A and gets preempted, it could
migrate to CPU B. This block would detect an older version, allocate a new
VMID on CPU B, and blindly clear the global per-vCPU KVM_REQ_TLB_FLUSH_GPA
request.

If the vCPU later migrates back to CPU A and reuses its valid old VMID
(since the VMID on CPU A hasn't wrapped), wouldn't the required GPA flush
on CPU A be missed because the request flag was already prematurely cleared
on CPU B?

> +
> +	vcpu->arch.vmid = vcpu->kvm->arch.vmid[cpu] & vpid_mask;
>  }
>  
>  void kvm_check_vpid(struct kvm_vcpu *vcpu)
> @@ -400,6 +439,7 @@ static int kvm_loongarch_env_init(void)
>  	for_each_possible_cpu(cpu) {
>  		context = per_cpu_ptr(vmcs, cpu);
>  		context->vpid_cache = vpid_mask + 1;
> +		context->vmid_cache = vpid_mask + 1;
>  		context->last_vcpu = NULL;
>  	}

[Severity: High]
This is a pre-existing issue, but I noticed that error paths during
initialization do not seem to clean up allocated structures or unregister
callbacks.

arch/loongarch/kvm/main.c:kvm_loongarch_env_init() {
    ...
    ret = kvm_loongarch_register_ipi_device();
    if (ret)
        return ret;
    ...
}

If a function like kvm_loongarch_register_ipi_device() returns an error,
the function returns directly without freeing vmcs or kvm_loongarch_ops,
and without calling kvm_unregister_perf_callbacks().

If the module load is aborted, does this leave perf_guest_cbs pointing to
unmapped memory, potentially causing a use-after-free kernel panic on the
next perf interrupt?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260727072109.2151675-1-maobibo@loongson.cn?part=4