* [PATCH v7 0/3] Add extioi virt extension support
@ 2024-08-23 6:39 Bibo Mao
2024-08-23 6:39 ` [PATCH v7 1/3] LoongArch: KVM: Enable paravirt feature control from VMM Bibo Mao
` (2 more replies)
0 siblings, 3 replies; 8+ messages in thread
From: Bibo Mao @ 2024-08-23 6:39 UTC (permalink / raw)
To: Tianrui Zhao, Huacai Chen, Thomas Gleixner
Cc: WANG Xuerui, kvm, loongarch, linux-kernel, virtualization, x86,
Song Gao
KVM_FEATURE_VIRT_EXTIOI is paravirt feature defined with EXTIOI
interrupt controller, it can route interrupt to 256 vCPUs and CPU
interrupt pin IP0-IP7. Now EXTIOI irqchip is emulated in user space
rather than kernel space, here interface is provided for VMM to pass
this feature to KVM hypervisor.
Also interface is provided for user-mode VMM to detect and enable/disable
paravirt features from KVM hypervisor. And api kvm_para_has_feature() is
available on LoongArch for device driver to detect paravirt features and
do some optimization.
---
v6 ... v7:
1. Replase function name guest_pv_has() to check whether paravirt feature
is supported with kvm_guest_has_pv_feature(), since there is
similiar function kvm_guest_has_lsx/lasx
2. Keep notation about CPUCFG area 0x40000000 -- 0x400000ff in header
file arch/loongarch/include/asm/loongarch.h
3. Remove function kvm_eiointc_init() and inline it with caller
function.
v5 ... v6:
1. Put KVM hypervisor type checking function kvm_para_available()
inside function kvm_arch_para_features(), so that upper caller
is easy to use.
2. Add inline function guest_pv_has() in KVM module to judge whether
the specific paravirt feature is supported or not. And do valid
checking at hypercall and user space ioctl entrance with it.
3. Fix some coding style issue such as variable declarations and spell
checking.
v4 ... v5:
1. Refresh annotation "WITH Linux-syscall-note" about uapi header file
arch/loongarch/include/uapi/asm/kvm_para.h
v3 ... v4:
1. Implement function kvm_para_has_feature() on LoongArch platform,
and redefine feature with normal number rather than bitmap number,
since function kvm_para_has_feature() requires this.
2. Add extioi virt extension support in this patch set.
3. Update extioi virt extension support patch with review comments,
including documentation, using kvm_para_has_feature() to detect
features etc.
v2 ... v3:
1. Add interface to detect and enable/disable paravirt features in
KVM hypervisor.
2. Implement function kvm_arch_para_features() for device driver in
VM side to detected supported paravirt features.
v1 ... v2:
1. Update changelog suggested by WangXuerui.
2. Fix typo issue in function kvm_loongarch_cpucfg_set_attr(),
usr_features should be assigned directly, also suggested by
WangXueRui.
---
Bibo Mao (3):
LoongArch: KVM: Enable paravirt feature control from VMM
LoongArch: KVM: Implement function kvm_para_has_feature
irqchip/loongson-eiointc: Add extioi virt extension support
.../arch/loongarch/irq-chip-model.rst | 64 +++++++++++
.../zh_CN/arch/loongarch/irq-chip-model.rst | 55 +++++++++
arch/loongarch/include/asm/irq.h | 1 +
arch/loongarch/include/asm/kvm_host.h | 7 ++
arch/loongarch/include/asm/kvm_para.h | 11 ++
arch/loongarch/include/asm/kvm_vcpu.h | 4 +
arch/loongarch/include/asm/loongarch.h | 11 +-
arch/loongarch/include/uapi/asm/Kbuild | 2 -
arch/loongarch/include/uapi/asm/kvm.h | 5 +
arch/loongarch/include/uapi/asm/kvm_para.h | 24 ++++
arch/loongarch/kernel/paravirt.c | 35 +++---
arch/loongarch/kvm/exit.c | 19 ++--
arch/loongarch/kvm/vcpu.c | 47 ++++++--
arch/loongarch/kvm/vm.c | 43 ++++++-
drivers/irqchip/irq-loongson-eiointc.c | 106 ++++++++++++++----
15 files changed, 368 insertions(+), 66 deletions(-)
create mode 100644 arch/loongarch/include/uapi/asm/kvm_para.h
base-commit: aa0743a229366e8c1963f1b72a1c974a9d15f08f
--
2.39.3
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v7 1/3] LoongArch: KVM: Enable paravirt feature control from VMM
2024-08-23 6:39 [PATCH v7 0/3] Add extioi virt extension support Bibo Mao
@ 2024-08-23 6:39 ` Bibo Mao
2024-08-23 6:39 ` [PATCH v7 2/3] LoongArch: KVM: Implement function kvm_para_has_feature Bibo Mao
2024-08-23 6:39 ` [PATCH v7 3/3] irqchip/loongson-eiointc: Add extioi virt extension support Bibo Mao
2 siblings, 0 replies; 8+ messages in thread
From: Bibo Mao @ 2024-08-23 6:39 UTC (permalink / raw)
To: Tianrui Zhao, Huacai Chen, Thomas Gleixner
Cc: WANG Xuerui, kvm, loongarch, linux-kernel, virtualization, x86,
Song Gao
Export kernel paravirt features to user space, so that VMM can control
the single paravirt feature. By default paravirt features will be the same
with kvm supported features if VMM does not set it.
Also a new feature KVM_FEATURE_VIRT_EXTIOI is added which can be set from
user space. This feature indicates that the virt EXTIOI can route
interrupts to 256 vCPUs, rather than 4 vCPUs like with real HW.
Signed-off-by: Bibo Mao <maobibo@loongson.cn>
---
arch/loongarch/include/asm/kvm_host.h | 7 ++++
arch/loongarch/include/asm/kvm_para.h | 1 +
arch/loongarch/include/asm/kvm_vcpu.h | 4 ++
arch/loongarch/include/asm/loongarch.h | 11 +----
arch/loongarch/include/uapi/asm/Kbuild | 2 -
arch/loongarch/include/uapi/asm/kvm.h | 5 +++
arch/loongarch/include/uapi/asm/kvm_para.h | 24 +++++++++++
arch/loongarch/kernel/paravirt.c | 8 ++--
arch/loongarch/kvm/exit.c | 19 ++++-----
arch/loongarch/kvm/vcpu.c | 47 ++++++++++++++++++----
arch/loongarch/kvm/vm.c | 43 +++++++++++++++++++-
11 files changed, 139 insertions(+), 32 deletions(-)
create mode 100644 arch/loongarch/include/uapi/asm/kvm_para.h
diff --git a/arch/loongarch/include/asm/kvm_host.h b/arch/loongarch/include/asm/kvm_host.h
index 5f0677e03817..b73f6678e38a 100644
--- a/arch/loongarch/include/asm/kvm_host.h
+++ b/arch/loongarch/include/asm/kvm_host.h
@@ -107,6 +107,8 @@ struct kvm_arch {
unsigned int root_level;
spinlock_t phyid_map_lock;
struct kvm_phyid_map *phyid_map;
+ /* Enabled PV features */
+ unsigned long pv_features;
s64 time_offset;
struct kvm_context __percpu *vmcs;
@@ -136,6 +138,11 @@ enum emulation_result {
#define KVM_LARCH_SWCSR_LATEST (0x1 << 3)
#define KVM_LARCH_HWCSR_USABLE (0x1 << 4)
+#define LOONGARCH_PV_FEAT_UPDATED BIT_ULL(63)
+#define LOONGARCH_PV_FEAT_MASK \
+ (BIT(KVM_FEATURE_IPI) | BIT(KVM_FEATURE_STEAL_TIME) | \
+ BIT(KVM_FEATURE_VIRT_EXTIOI))
+
struct kvm_vcpu_arch {
/*
* Switch pointer-to-function type to unsigned long
diff --git a/arch/loongarch/include/asm/kvm_para.h b/arch/loongarch/include/asm/kvm_para.h
index 43ec61589e6c..39d7483ab8fd 100644
--- a/arch/loongarch/include/asm/kvm_para.h
+++ b/arch/loongarch/include/asm/kvm_para.h
@@ -2,6 +2,7 @@
#ifndef _ASM_LOONGARCH_KVM_PARA_H
#define _ASM_LOONGARCH_KVM_PARA_H
+#include <uapi/asm/kvm_para.h>
/*
* Hypercall code field
*/
diff --git a/arch/loongarch/include/asm/kvm_vcpu.h b/arch/loongarch/include/asm/kvm_vcpu.h
index c416cb7125c0..050ce49ec744 100644
--- a/arch/loongarch/include/asm/kvm_vcpu.h
+++ b/arch/loongarch/include/asm/kvm_vcpu.h
@@ -125,4 +125,8 @@ static inline bool kvm_pvtime_supported(void)
return !!sched_info_on();
}
+static __always_inline bool kvm_guest_has_pv_feature(struct kvm_vcpu *vcpu, unsigned int feature)
+{
+ return vcpu->kvm->arch.pv_features & BIT(feature);
+}
#endif /* __ASM_LOONGARCH_KVM_VCPU_H__ */
diff --git a/arch/loongarch/include/asm/loongarch.h b/arch/loongarch/include/asm/loongarch.h
index 04a78010fc72..54059bead7e6 100644
--- a/arch/loongarch/include/asm/loongarch.h
+++ b/arch/loongarch/include/asm/loongarch.h
@@ -160,16 +160,9 @@
/*
* CPUCFG index area: 0x40000000 -- 0x400000ff
- * SW emulation for KVM hypervirsor
+ * SW emulation for KVM hypervirsor, see
+ * arch/loongarch/include/uapi/asm/kvm_para.h
*/
-#define CPUCFG_KVM_BASE 0x40000000
-#define CPUCFG_KVM_SIZE 0x100
-
-#define CPUCFG_KVM_SIG (CPUCFG_KVM_BASE + 0)
-#define KVM_SIGNATURE "KVM\0"
-#define CPUCFG_KVM_FEATURE (CPUCFG_KVM_BASE + 4)
-#define KVM_FEATURE_IPI BIT(1)
-#define KVM_FEATURE_STEAL_TIME BIT(2)
#ifndef __ASSEMBLY__
diff --git a/arch/loongarch/include/uapi/asm/Kbuild b/arch/loongarch/include/uapi/asm/Kbuild
index c6d141d7b7d7..517761419999 100644
--- a/arch/loongarch/include/uapi/asm/Kbuild
+++ b/arch/loongarch/include/uapi/asm/Kbuild
@@ -1,4 +1,2 @@
# SPDX-License-Identifier: GPL-2.0
syscall-y += unistd_64.h
-
-generic-y += kvm_para.h
diff --git a/arch/loongarch/include/uapi/asm/kvm.h b/arch/loongarch/include/uapi/asm/kvm.h
index ddc5cab0ffd0..719490e64e1c 100644
--- a/arch/loongarch/include/uapi/asm/kvm.h
+++ b/arch/loongarch/include/uapi/asm/kvm.h
@@ -82,6 +82,11 @@ struct kvm_fpu {
#define KVM_IOC_CSRID(REG) LOONGARCH_REG_64(KVM_REG_LOONGARCH_CSR, REG)
#define KVM_IOC_CPUCFG(REG) LOONGARCH_REG_64(KVM_REG_LOONGARCH_CPUCFG, REG)
+/* Device Control API on vm fd */
+#define KVM_LOONGARCH_VM_FEAT_CTRL 0
+#define KVM_LOONGARCH_VM_FEAT_PV_IPI 5
+#define KVM_LOONGARCH_VM_FEAT_PV_STEALTIME 6
+
/* Device Control API on vcpu fd */
#define KVM_LOONGARCH_VCPU_CPUCFG 0
#define KVM_LOONGARCH_VCPU_PVTIME_CTRL 1
diff --git a/arch/loongarch/include/uapi/asm/kvm_para.h b/arch/loongarch/include/uapi/asm/kvm_para.h
new file mode 100644
index 000000000000..5dfe675709ab
--- /dev/null
+++ b/arch/loongarch/include/uapi/asm/kvm_para.h
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+#ifndef _UAPI_ASM_KVM_PARA_H
+#define _UAPI_ASM_KVM_PARA_H
+
+#include <linux/types.h>
+
+/*
+ * CPUCFG index area: 0x40000000 -- 0x400000ff
+ * SW emulation for KVM hypervirsor
+ */
+#define CPUCFG_KVM_BASE 0x40000000
+#define CPUCFG_KVM_SIZE 0x100
+#define CPUCFG_KVM_SIG (CPUCFG_KVM_BASE + 0)
+#define KVM_SIGNATURE "KVM\0"
+#define CPUCFG_KVM_FEATURE (CPUCFG_KVM_BASE + 4)
+#define KVM_FEATURE_IPI 1
+#define KVM_FEATURE_STEAL_TIME 2
+/*
+ * BIT 24 - 31 is features configurable by user space vmm
+ * With VIRT_EXTIOI feature, interrupt can route to 256 VCPUs
+ */
+#define KVM_FEATURE_VIRT_EXTIOI 24
+
+#endif /* _UAPI_ASM_KVM_PARA_H */
diff --git a/arch/loongarch/kernel/paravirt.c b/arch/loongarch/kernel/paravirt.c
index 9c9b75b76f62..cc6bf096cb88 100644
--- a/arch/loongarch/kernel/paravirt.c
+++ b/arch/loongarch/kernel/paravirt.c
@@ -175,7 +175,7 @@ int __init pv_ipi_init(void)
return 0;
feature = read_cpucfg(CPUCFG_KVM_FEATURE);
- if (!(feature & KVM_FEATURE_IPI))
+ if (!(feature & BIT(KVM_FEATURE_IPI)))
return 0;
#ifdef CONFIG_SMP
@@ -206,7 +206,7 @@ static int pv_enable_steal_time(void)
}
addr |= KVM_STEAL_PHYS_VALID;
- kvm_hypercall2(KVM_HCALL_FUNC_NOTIFY, KVM_FEATURE_STEAL_TIME, addr);
+ kvm_hypercall2(KVM_HCALL_FUNC_NOTIFY, BIT(KVM_FEATURE_STEAL_TIME), addr);
return 0;
}
@@ -214,7 +214,7 @@ static int pv_enable_steal_time(void)
static void pv_disable_steal_time(void)
{
if (has_steal_clock)
- kvm_hypercall2(KVM_HCALL_FUNC_NOTIFY, KVM_FEATURE_STEAL_TIME, 0);
+ kvm_hypercall2(KVM_HCALL_FUNC_NOTIFY, BIT(KVM_FEATURE_STEAL_TIME), 0);
}
#ifdef CONFIG_SMP
@@ -266,7 +266,7 @@ int __init pv_time_init(void)
return 0;
feature = read_cpucfg(CPUCFG_KVM_FEATURE);
- if (!(feature & KVM_FEATURE_STEAL_TIME))
+ if (!(feature & BIT(KVM_FEATURE_STEAL_TIME)))
return 0;
has_steal_clock = 1;
diff --git a/arch/loongarch/kvm/exit.c b/arch/loongarch/kvm/exit.c
index ea73f9dc2cc6..8ae72f35fba9 100644
--- a/arch/loongarch/kvm/exit.c
+++ b/arch/loongarch/kvm/exit.c
@@ -50,9 +50,7 @@ static int kvm_emu_cpucfg(struct kvm_vcpu *vcpu, larch_inst inst)
vcpu->arch.gprs[rd] = *(unsigned int *)KVM_SIGNATURE;
break;
case CPUCFG_KVM_FEATURE:
- ret = KVM_FEATURE_IPI;
- if (kvm_pvtime_supported())
- ret |= KVM_FEATURE_STEAL_TIME;
+ ret = vcpu->kvm->arch.pv_features & LOONGARCH_PV_FEAT_MASK;
vcpu->arch.gprs[rd] = ret;
break;
default:
@@ -697,8 +695,8 @@ static long kvm_save_notify(struct kvm_vcpu *vcpu)
id = kvm_read_reg(vcpu, LOONGARCH_GPR_A1);
data = kvm_read_reg(vcpu, LOONGARCH_GPR_A2);
switch (id) {
- case KVM_FEATURE_STEAL_TIME:
- if (!kvm_pvtime_supported())
+ case BIT(KVM_FEATURE_STEAL_TIME):
+ if (!kvm_guest_has_pv_feature(vcpu, KVM_FEATURE_STEAL_TIME))
return KVM_HCALL_INVALID_CODE;
if (data & ~(KVM_STEAL_PHYS_MASK | KVM_STEAL_PHYS_VALID))
@@ -712,10 +710,10 @@ static long kvm_save_notify(struct kvm_vcpu *vcpu)
kvm_make_request(KVM_REQ_STEAL_UPDATE, vcpu);
break;
default:
- break;
+ return KVM_HCALL_INVALID_CODE;
};
- return 0;
+ return KVM_HCALL_INVALID_CODE;
};
/*
@@ -786,8 +784,11 @@ static void kvm_handle_service(struct kvm_vcpu *vcpu)
switch (func) {
case KVM_HCALL_FUNC_IPI:
- kvm_send_pv_ipi(vcpu);
- ret = KVM_HCALL_SUCCESS;
+ if (kvm_guest_has_pv_feature(vcpu, KVM_FEATURE_IPI)) {
+ kvm_send_pv_ipi(vcpu);
+ ret = KVM_HCALL_SUCCESS;
+ } else
+ ret = KVM_HCALL_INVALID_CODE;
break;
case KVM_HCALL_FUNC_NOTIFY:
ret = kvm_save_notify(vcpu);
diff --git a/arch/loongarch/kvm/vcpu.c b/arch/loongarch/kvm/vcpu.c
index 16756ffb55e8..00852515962c 100644
--- a/arch/loongarch/kvm/vcpu.c
+++ b/arch/loongarch/kvm/vcpu.c
@@ -730,6 +730,8 @@ static int kvm_loongarch_cpucfg_has_attr(struct kvm_vcpu *vcpu,
switch (attr->attr) {
case 2:
return 0;
+ case CPUCFG_KVM_FEATURE:
+ return 0;
default:
return -ENXIO;
}
@@ -740,7 +742,7 @@ static int kvm_loongarch_cpucfg_has_attr(struct kvm_vcpu *vcpu,
static int kvm_loongarch_pvtime_has_attr(struct kvm_vcpu *vcpu,
struct kvm_device_attr *attr)
{
- if (!kvm_pvtime_supported() ||
+ if (!kvm_guest_has_pv_feature(vcpu, KVM_FEATURE_STEAL_TIME) ||
attr->attr != KVM_LOONGARCH_VCPU_PVTIME_GPA)
return -ENXIO;
@@ -773,9 +775,18 @@ static int kvm_loongarch_cpucfg_get_attr(struct kvm_vcpu *vcpu,
uint64_t val;
uint64_t __user *uaddr = (uint64_t __user *)attr->addr;
- ret = _kvm_get_cpucfg_mask(attr->attr, &val);
- if (ret)
- return ret;
+ switch (attr->attr) {
+ case 0 ... (KVM_MAX_CPUCFG_REGS - 1):
+ ret = _kvm_get_cpucfg_mask(attr->attr, &val);
+ if (ret)
+ return ret;
+ break;
+ case CPUCFG_KVM_FEATURE:
+ val = vcpu->kvm->arch.pv_features & LOONGARCH_PV_FEAT_MASK;
+ break;
+ default:
+ return -ENXIO;
+ }
put_user(val, uaddr);
@@ -788,7 +799,7 @@ static int kvm_loongarch_pvtime_get_attr(struct kvm_vcpu *vcpu,
u64 gpa;
u64 __user *user = (u64 __user *)attr->addr;
- if (!kvm_pvtime_supported() ||
+ if (!kvm_guest_has_pv_feature(vcpu, KVM_FEATURE_STEAL_TIME) ||
attr->attr != KVM_LOONGARCH_VCPU_PVTIME_GPA)
return -ENXIO;
@@ -821,7 +832,29 @@ static int kvm_loongarch_vcpu_get_attr(struct kvm_vcpu *vcpu,
static int kvm_loongarch_cpucfg_set_attr(struct kvm_vcpu *vcpu,
struct kvm_device_attr *attr)
{
- return -ENXIO;
+ u64 __user *user = (u64 __user *)attr->addr;
+ u64 val, valid;
+ struct kvm *kvm = vcpu->kvm;
+
+ switch (attr->attr) {
+ case CPUCFG_KVM_FEATURE:
+ if (get_user(val, user))
+ return -EFAULT;
+
+ valid = LOONGARCH_PV_FEAT_MASK;
+ if (val & ~valid)
+ return -EINVAL;
+
+ /* All vCPUs need set the same pv features */
+ if ((kvm->arch.pv_features & LOONGARCH_PV_FEAT_UPDATED) &&
+ ((kvm->arch.pv_features & valid) != val))
+ return -EINVAL;
+ kvm->arch.pv_features = val | LOONGARCH_PV_FEAT_UPDATED;
+ return 0;
+
+ default:
+ return -ENXIO;
+ }
}
static int kvm_loongarch_pvtime_set_attr(struct kvm_vcpu *vcpu,
@@ -831,7 +864,7 @@ static int kvm_loongarch_pvtime_set_attr(struct kvm_vcpu *vcpu,
u64 gpa, __user *user = (u64 __user *)attr->addr;
struct kvm *kvm = vcpu->kvm;
- if (!kvm_pvtime_supported() ||
+ if (!kvm_guest_has_pv_feature(vcpu, KVM_FEATURE_STEAL_TIME) ||
attr->attr != KVM_LOONGARCH_VCPU_PVTIME_GPA)
return -ENXIO;
diff --git a/arch/loongarch/kvm/vm.c b/arch/loongarch/kvm/vm.c
index 6b2e4f66ad26..3234f3e85dc0 100644
--- a/arch/loongarch/kvm/vm.c
+++ b/arch/loongarch/kvm/vm.c
@@ -5,6 +5,7 @@
#include <linux/kvm_host.h>
#include <asm/kvm_mmu.h>
+#include <asm/kvm_vcpu.h>
const struct _kvm_stats_desc kvm_vm_stats_desc[] = {
KVM_GENERIC_VM_STATS(),
@@ -39,6 +40,10 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
spin_lock_init(&kvm->arch.phyid_map_lock);
kvm_init_vmcs(kvm);
+ /* Enable all pv features by default */
+ kvm->arch.pv_features = BIT(KVM_FEATURE_IPI);
+ if (kvm_pvtime_supported())
+ kvm->arch.pv_features |= BIT(KVM_FEATURE_STEAL_TIME);
kvm->arch.gpa_size = BIT(cpu_vabits - 1);
kvm->arch.root_level = CONFIG_PGTABLE_LEVELS - 1;
kvm->arch.invalid_ptes[0] = 0;
@@ -99,7 +104,43 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
return r;
}
+static int kvm_vm_feature_has_attr(struct kvm *kvm, struct kvm_device_attr *attr)
+{
+ switch (attr->attr) {
+ case KVM_LOONGARCH_VM_FEAT_PV_IPI:
+ return 0;
+ case KVM_LOONGARCH_VM_FEAT_PV_STEALTIME:
+ if (kvm_pvtime_supported())
+ return 0;
+ return -ENXIO;
+ default:
+ return -ENXIO;
+ }
+}
+
+static int kvm_vm_has_attr(struct kvm *kvm, struct kvm_device_attr *attr)
+{
+ switch (attr->group) {
+ case KVM_LOONGARCH_VM_FEAT_CTRL:
+ return kvm_vm_feature_has_attr(kvm, attr);
+ default:
+ return -ENXIO;
+ }
+}
+
int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg)
{
- return -ENOIOCTLCMD;
+ struct kvm *kvm = filp->private_data;
+ void __user *argp = (void __user *)arg;
+ struct kvm_device_attr attr;
+
+ switch (ioctl) {
+ case KVM_HAS_DEVICE_ATTR:
+ if (copy_from_user(&attr, argp, sizeof(attr)))
+ return -EFAULT;
+
+ return kvm_vm_has_attr(kvm, &attr);
+ default:
+ return -EINVAL;
+ }
}
--
2.39.3
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH v7 2/3] LoongArch: KVM: Implement function kvm_para_has_feature
2024-08-23 6:39 [PATCH v7 0/3] Add extioi virt extension support Bibo Mao
2024-08-23 6:39 ` [PATCH v7 1/3] LoongArch: KVM: Enable paravirt feature control from VMM Bibo Mao
@ 2024-08-23 6:39 ` Bibo Mao
2024-08-23 6:39 ` [PATCH v7 3/3] irqchip/loongson-eiointc: Add extioi virt extension support Bibo Mao
2 siblings, 0 replies; 8+ messages in thread
From: Bibo Mao @ 2024-08-23 6:39 UTC (permalink / raw)
To: Tianrui Zhao, Huacai Chen, Thomas Gleixner
Cc: WANG Xuerui, kvm, loongarch, linux-kernel, virtualization, x86,
Song Gao
Function kvm_para_has_feature() is to detect supported paravirt features,
it can be used by device driver to detect and enable paravirt features,
such as extioi irqchip driver can detect KVM_FEATURE_VIRT_EXTIOI and do
some optimization.
Signed-off-by: Bibo Mao <maobibo@loongson.cn>
---
arch/loongarch/include/asm/kvm_para.h | 10 +++++++++
arch/loongarch/kernel/paravirt.c | 31 ++++++++++++++-------------
2 files changed, 26 insertions(+), 15 deletions(-)
diff --git a/arch/loongarch/include/asm/kvm_para.h b/arch/loongarch/include/asm/kvm_para.h
index 39d7483ab8fd..2174fb53c2e7 100644
--- a/arch/loongarch/include/asm/kvm_para.h
+++ b/arch/loongarch/include/asm/kvm_para.h
@@ -155,10 +155,20 @@ static __always_inline long kvm_hypercall5(u64 fid,
return ret;
}
+#ifdef CONFIG_PARAVIRT
+bool kvm_para_available(void);
+unsigned int kvm_arch_para_features(void);
+#else
+static inline bool kvm_para_available(void)
+{
+ return false;
+}
+
static inline unsigned int kvm_arch_para_features(void)
{
return 0;
}
+#endif
static inline unsigned int kvm_arch_para_hints(void)
{
diff --git a/arch/loongarch/kernel/paravirt.c b/arch/loongarch/kernel/paravirt.c
index cc6bf096cb88..dc3cebcb550a 100644
--- a/arch/loongarch/kernel/paravirt.c
+++ b/arch/loongarch/kernel/paravirt.c
@@ -151,11 +151,14 @@ static void pv_init_ipi(void)
}
#endif
-static bool kvm_para_available(void)
+bool kvm_para_available(void)
{
int config;
static int hypervisor_type;
+ if (!cpu_has_hypervisor)
+ return false;
+
if (!hypervisor_type) {
config = read_cpucfg(CPUCFG_KVM_SIG);
if (!memcmp(&config, KVM_SIGNATURE, 4))
@@ -165,17 +168,21 @@ static bool kvm_para_available(void)
return hypervisor_type == HYPERVISOR_KVM;
}
-int __init pv_ipi_init(void)
+unsigned int kvm_arch_para_features(void)
{
- int feature;
+ static int feature;
- if (!cpu_has_hypervisor)
- return 0;
if (!kvm_para_available())
return 0;
- feature = read_cpucfg(CPUCFG_KVM_FEATURE);
- if (!(feature & BIT(KVM_FEATURE_IPI)))
+ if (!feature)
+ feature = read_cpucfg(CPUCFG_KVM_FEATURE);
+ return feature;
+}
+
+int __init pv_ipi_init(void)
+{
+ if (!kvm_para_has_feature(KVM_FEATURE_IPI))
return 0;
#ifdef CONFIG_SMP
@@ -258,15 +265,9 @@ static struct notifier_block pv_reboot_nb = {
int __init pv_time_init(void)
{
- int r, feature;
-
- if (!cpu_has_hypervisor)
- return 0;
- if (!kvm_para_available())
- return 0;
+ int r;
- feature = read_cpucfg(CPUCFG_KVM_FEATURE);
- if (!(feature & BIT(KVM_FEATURE_STEAL_TIME)))
+ if (!kvm_para_has_feature(KVM_FEATURE_STEAL_TIME))
return 0;
has_steal_clock = 1;
--
2.39.3
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH v7 3/3] irqchip/loongson-eiointc: Add extioi virt extension support
2024-08-23 6:39 [PATCH v7 0/3] Add extioi virt extension support Bibo Mao
2024-08-23 6:39 ` [PATCH v7 1/3] LoongArch: KVM: Enable paravirt feature control from VMM Bibo Mao
2024-08-23 6:39 ` [PATCH v7 2/3] LoongArch: KVM: Implement function kvm_para_has_feature Bibo Mao
@ 2024-08-23 6:39 ` Bibo Mao
2024-08-28 14:27 ` Huacai Chen
2 siblings, 1 reply; 8+ messages in thread
From: Bibo Mao @ 2024-08-23 6:39 UTC (permalink / raw)
To: Tianrui Zhao, Huacai Chen, Thomas Gleixner
Cc: WANG Xuerui, kvm, loongarch, linux-kernel, virtualization, x86,
Song Gao
Interrupts can be routed to maximal four virtual CPUs with one HW
EIOINTC interrupt controller model, since interrupt routing is encoded with
CPU bitmap and EIOINTC node combined method. Here add the EIOINTC virt
extension support so that interrupts can be routed to 256 vCPUs on
hypervisor mode. CPU bitmap is replaced with normal encoding and EIOINTC
node type is removed, so there are 8 bits for cpu selection, at most 256
vCPUs are supported for interrupt routing.
Co-developed-by: Song Gao <gaosong@loongson.cn>
Signed-off-by: Song Gao <gaosong@loongson.cn>
Signed-off-by: Bibo Mao <maobibo@loongson.cn>
---
.../arch/loongarch/irq-chip-model.rst | 64 +++++++++++
.../zh_CN/arch/loongarch/irq-chip-model.rst | 55 +++++++++
arch/loongarch/include/asm/irq.h | 1 +
drivers/irqchip/irq-loongson-eiointc.c | 106 ++++++++++++++----
4 files changed, 205 insertions(+), 21 deletions(-)
diff --git a/Documentation/arch/loongarch/irq-chip-model.rst b/Documentation/arch/loongarch/irq-chip-model.rst
index 7988f4192363..d2350780ad1d 100644
--- a/Documentation/arch/loongarch/irq-chip-model.rst
+++ b/Documentation/arch/loongarch/irq-chip-model.rst
@@ -85,6 +85,70 @@ to CPUINTC directly::
| Devices |
+---------+
+Virtual extended IRQ model
+==========================
+
+In this model, IPI (Inter-Processor Interrupt) and CPU Local Timer interrupt
+go to CPUINTC directly, CPU UARTS interrupts go to PCH-PIC, while all other
+devices interrupts go to PCH-PIC/PCH-MSI and gathered by V-EIOINTC (Virtual
+Extended I/O Interrupt Controller), and then go to CPUINTC directly::
+
+ +-----+ +-------------------+ +-------+
+ | IPI |--> | CPUINTC(0-255vcpu)| <-- | Timer |
+ +-----+ +-------------------+ +-------+
+ ^
+ |
+ +-----------+
+ | V-EIOINTC |
+ +-----------+
+ ^ ^
+ | |
+ +---------+ +---------+
+ | PCH-PIC | | PCH-MSI |
+ +---------+ +---------+
+ ^ ^ ^
+ | | |
+ +--------+ +---------+ +---------+
+ | UARTs | | Devices | | Devices |
+ +--------+ +---------+ +---------+
+
+
+Description
+-----------
+V-EIOINTC (Virtual Extended I/O Interrupt Controller) is an extension of
+EIOINTC, it only works in VM mode which runs in KVM hypervisor. Interrupts can
+be routed to up to four vCPUs via standard EIOINTC, however with V-EIOINTC
+interrupts can be routed to up to 256 virtual cpus.
+
+With standard EIOINTC, interrupt routing setting includes two parts: eight
+bits for CPU selection and four bits for CPU IP (Interrupt Pin) selection.
+For CPU selection there is four bits for EIOINTC node selection, four bits
+for EIOINTC CPU selection. Bitmap method is used for CPU selection and
+CPU IP selection, so interrupt can only route to CPU0 - CPU3 and IP0-IP3 in
+one EIOINTC node.
+
+With V-EIOINTC it supports to route more CPUs and CPU IP (Interrupt Pin),
+there are two newly added registers with V-EIOINTC.
+
+EXTIOI_VIRT_FEATURES
+--------------------
+This register is read-only register, which indicates supported features with
+V-EIOINTC. Feature EXTIOI_HAS_INT_ENCODE and EXTIOI_HAS_CPU_ENCODE is added.
+
+Feature EXTIOI_HAS_INT_ENCODE is part of standard EIOINTC. If it is 1, it
+indicates that CPU Interrupt Pin selection can be normal method rather than
+bitmap method, so interrupt can be routed to IP0 - IP15.
+
+Feature EXTIOI_HAS_CPU_ENCODE is entension of V-EIOINTC. If it is 1, it
+indicates that CPU selection can be normal method rather than bitmap method,
+so interrupt can be routed to CPU0 - CPU255.
+
+EXTIOI_VIRT_CONFIG
+------------------
+This register is read-write register, for compatibility intterupt routed uses
+the default method which is the same with standard EIOINTC. If the bit is set
+with 1, it indicated HW to use normal method rather than bitmap method.
+
ACPI-related definitions
========================
diff --git a/Documentation/translations/zh_CN/arch/loongarch/irq-chip-model.rst b/Documentation/translations/zh_CN/arch/loongarch/irq-chip-model.rst
index f1e9ab18206c..d696bd394c02 100644
--- a/Documentation/translations/zh_CN/arch/loongarch/irq-chip-model.rst
+++ b/Documentation/translations/zh_CN/arch/loongarch/irq-chip-model.rst
@@ -87,6 +87,61 @@ PCH-LPC/PCH-MSI，然后被EIOINTC统一收集，再直接到达CPUINTC::
| Devices |
+---------+
+虚拟扩展IRQ模型
+===============
+
+在这种模型里面, IPI(Inter-Processor Interrupt) 和CPU本地时钟中断直接发送到CPUINTC,
+CPU串口 (UARTs) 中断发送到PCH-PIC, 而其他所有设备的中断则分别发送到所连接的PCH_PIC/
+PCH-MSI, 然后V-EIOINTC统一收集，再直接到达CPUINTC::
+
+ +-----+ +-------------------+ +-------+
+ | IPI |--> | CPUINTC(0-255vcpu)| <-- | Timer |
+ +-----+ +-------------------+ +-------+
+ ^
+ |
+ +-----------+
+ | V-EIOINTC |
+ +-----------+
+ ^ ^
+ | |
+ +---------+ +---------+
+ | PCH-PIC | | PCH-MSI |
+ +---------+ +---------+
+ ^ ^ ^
+ | | |
+ +--------+ +---------+ +---------+
+ | UARTs | | Devices | | Devices |
+ +--------+ +---------+ +---------+
+
+V-EIOINTC 是EIOINTC的扩展, 仅工作在hyperisor模式下, 中断经EIOINTC最多可个路由到４个
+虚拟cpu. 但中断经V-EIOINTC最多可个路由到256个虚拟cpu.
+
+传统的EIOINTC中断控制器，中断路由分为两个部分：8比特用于控制路由到哪个CPU，
+4比特用于控制路由到特定CPU的哪个中断管脚.控制CPU路由的8比特前4比特用于控制
+路由到哪个EIOINTC节点，后4比特用于控制此节点哪个CPU。中断路由在选择CPU路由
+和CPU中断管脚路由时，使用bitmap编码方式而不是正常编码方式，所以对于一个
+EIOINTC中断控制器节点，中断只能路由到CPU0 - CPU3，中断管教IP0-IP3。
+
+V-EIOINTC新增了两个寄存器，支持中断路由到更多CPU个和中断管脚。
+
+V-EIOINTC功能寄存器
+-------------------
+功能寄存器是只读寄存器，用于显示V-EIOINTC支持的特性，目前两个支持两个特性
+EXTIOI_HAS_INT_ENCODE 和 EXTIOI_HAS_CPU_ENCODE。
+
+特性EXTIOI_HAS_INT_ENCODE是传统EIOINTC中断控制器的一个特性，如果此比特为1，
+显示CPU中断管脚路由方式支持正常编码，而不是bitmap编码，所以中断可以路由到
+管脚IP0 - IP15。
+
+特性EXTIOI_HAS_CPU_ENCODE是V-EIOINTC新增特性，如果此比特为1，表示CPU路由
+方式支持正常编码，而不是bitmap编码，所以中断可以路由到CPU0 - CPU255。
+
+V-EIOINTC配置寄存器
+-------------------
+配置寄存器是可读写寄存器，为了兼容性考虑，如果不写此寄存器，中断路由采用
+和传统EIOINTC相同的路由设置。如果对应比特设置为1，表示采用正常路由方式而
+不是bitmap编码的路由方式。
+
ACPI相关的定义
==============
diff --git a/arch/loongarch/include/asm/irq.h b/arch/loongarch/include/asm/irq.h
index 480418bc5071..ce85d4c7d225 100644
--- a/arch/loongarch/include/asm/irq.h
+++ b/arch/loongarch/include/asm/irq.h
@@ -54,6 +54,7 @@ extern struct acpi_vector_group pch_group[MAX_IO_PICS];
extern struct acpi_vector_group msi_group[MAX_IO_PICS];
#define CORES_PER_EIO_NODE 4
+#define CORES_PER_VEIO_NODE 256
#define LOONGSON_CPU_UART0_VEC 10 /* CPU UART0 */
#define LOONGSON_CPU_THSENS_VEC 14 /* CPU Thsens */
diff --git a/drivers/irqchip/irq-loongson-eiointc.c b/drivers/irqchip/irq-loongson-eiointc.c
index b1f2080be2be..bc54f81ec129 100644
--- a/drivers/irqchip/irq-loongson-eiointc.c
+++ b/drivers/irqchip/irq-loongson-eiointc.c
@@ -14,6 +14,7 @@
#include <linux/irqdomain.h>
#include <linux/irqchip/chained_irq.h>
#include <linux/kernel.h>
+#include <linux/kvm_para.h>
#include <linux/syscore_ops.h>
#include <asm/numa.h>
@@ -24,15 +25,37 @@
#define EIOINTC_REG_ISR 0x1800
#define EIOINTC_REG_ROUTE 0x1c00
+#define EXTIOI_VIRT_FEATURES 0x40000000
+#define EXTIOI_HAS_VIRT_EXTENSION BIT(0)
+#define EXTIOI_HAS_ENABLE_OPTION BIT(1)
+#define EXTIOI_HAS_INT_ENCODE BIT(2)
+#define EXTIOI_HAS_CPU_ENCODE BIT(3)
+#define EXTIOI_VIRT_CONFIG 0x40000004
+#define EXTIOI_ENABLE BIT(1)
+#define EXTIOI_ENABLE_INT_ENCODE BIT(2)
+#define EXTIOI_ENABLE_CPU_ENCODE BIT(3)
+
#define VEC_REG_COUNT 4
#define VEC_COUNT_PER_REG 64
#define VEC_COUNT (VEC_REG_COUNT * VEC_COUNT_PER_REG)
#define VEC_REG_IDX(irq_id) ((irq_id) / VEC_COUNT_PER_REG)
#define VEC_REG_BIT(irq_id) ((irq_id) % VEC_COUNT_PER_REG)
#define EIOINTC_ALL_ENABLE 0xffffffff
+#define EIOINTC_ALL_ENABLE_VEC_MASK(vector) (EIOINTC_ALL_ENABLE & ~BIT(vector & 0x1F))
+#define EIOINTC_REG_ENABLE_VEC(vector) (EIOINTC_REG_ENABLE + ((vector >> 5) << 2))
+#define EIOINTC_USE_CPU_ENCODE BIT(0)
#define MAX_EIO_NODES (NR_CPUS / CORES_PER_EIO_NODE)
+/*
+ * Routing registers are 32bit, and there is 8-bit route setting for every
+ * interrupt vector. So one Route register contains four vectors routing
+ * information.
+ */
+#define EIOINTC_REG_ROUTE_VEC(vector) (EIOINTC_REG_ROUTE + (vector & ~0x03))
+#define EIOINTC_REG_ROUTE_VEC_SHIFT(vector) ((vector & 0x03) << 3)
+#define EIOINTC_REG_ROUTE_VEC_MASK(vector) (0xff << EIOINTC_REG_ROUTE_VEC_SHIFT(vector))
+
static int nr_pics;
struct eiointc_priv {
@@ -42,6 +65,7 @@ struct eiointc_priv {
cpumask_t cpuspan_map;
struct fwnode_handle *domain_handle;
struct irq_domain *eiointc_domain;
+ int flags;
};
static struct eiointc_priv *eiointc_priv[MAX_IO_PICS];
@@ -57,7 +81,13 @@ static void eiointc_enable(void)
static int cpu_to_eio_node(int cpu)
{
- return cpu_logical_map(cpu) / CORES_PER_EIO_NODE;
+ int cores;
+
+ if (kvm_para_has_feature(KVM_FEATURE_VIRT_EXTIOI))
+ cores = CORES_PER_VEIO_NODE;
+ else
+ cores = CORES_PER_EIO_NODE;
+ return cpu_logical_map(cpu) / cores;
}
#ifdef CONFIG_SMP
@@ -89,6 +119,16 @@ static void eiointc_set_irq_route(int pos, unsigned int cpu, unsigned int mnode,
static DEFINE_RAW_SPINLOCK(affinity_lock);
+static void virt_extioi_set_irq_route(unsigned int vector, unsigned int cpu)
+{
+ unsigned long reg = EIOINTC_REG_ROUTE_VEC(vector);
+ u32 data = iocsr_read32(reg);
+
+ data &= ~EIOINTC_REG_ROUTE_VEC_MASK(vector);
+ data |= cpu_logical_map(cpu) << EIOINTC_REG_ROUTE_VEC_SHIFT(vector);
+ iocsr_write32(data, reg);
+}
+
static int eiointc_set_irq_affinity(struct irq_data *d, const struct cpumask *affinity, bool force)
{
unsigned int cpu;
@@ -105,18 +145,24 @@ static int eiointc_set_irq_affinity(struct irq_data *d, const struct cpumask *af
}
vector = d->hwirq;
- regaddr = EIOINTC_REG_ENABLE + ((vector >> 5) << 2);
-
- /* Mask target vector */
- csr_any_send(regaddr, EIOINTC_ALL_ENABLE & (~BIT(vector & 0x1F)),
- 0x0, priv->node * CORES_PER_EIO_NODE);
-
- /* Set route for target vector */
- eiointc_set_irq_route(vector, cpu, priv->node, &priv->node_map);
-
- /* Unmask target vector */
- csr_any_send(regaddr, EIOINTC_ALL_ENABLE,
- 0x0, priv->node * CORES_PER_EIO_NODE);
+ regaddr = EIOINTC_REG_ENABLE_VEC(vector);
+
+ if (priv->flags & EIOINTC_USE_CPU_ENCODE) {
+ iocsr_write32(EIOINTC_ALL_ENABLE_VEC_MASK(vector), regaddr);
+ virt_extioi_set_irq_route(vector, cpu);
+ iocsr_write32(EIOINTC_ALL_ENABLE, regaddr);
+ } else {
+ /* Mask target vector */
+ csr_any_send(regaddr, EIOINTC_ALL_ENABLE_VEC_MASK(vector),
+ 0x0, priv->node * CORES_PER_EIO_NODE);
+
+ /* Set route for target vector */
+ eiointc_set_irq_route(vector, cpu, priv->node, &priv->node_map);
+
+ /* Unmask target vector */
+ csr_any_send(regaddr, EIOINTC_ALL_ENABLE,
+ 0x0, priv->node * CORES_PER_EIO_NODE);
+ }
irq_data_update_effective_affinity(d, cpumask_of(cpu));
@@ -140,17 +186,23 @@ static int eiointc_index(int node)
static int eiointc_router_init(unsigned int cpu)
{
- int i, bit;
- uint32_t data;
- uint32_t node = cpu_to_eio_node(cpu);
- int index = eiointc_index(node);
+ int i, bit, cores, index, node;
+ unsigned int data;
+
+ node = cpu_to_eio_node(cpu);
+ index = eiointc_index(node);
if (index < 0) {
pr_err("Error: invalid nodemap!\n");
- return -1;
+ return -EINVAL;
}
- if ((cpu_logical_map(cpu) % CORES_PER_EIO_NODE) == 0) {
+ if (eiointc_priv[index]->flags & EIOINTC_USE_CPU_ENCODE)
+ cores = CORES_PER_VEIO_NODE;
+ else
+ cores = CORES_PER_EIO_NODE;
+
+ if ((cpu_logical_map(cpu) % cores) == 0) {
eiointc_enable();
for (i = 0; i < eiointc_priv[0]->vec_count / 32; i++) {
@@ -166,7 +218,9 @@ static int eiointc_router_init(unsigned int cpu)
for (i = 0; i < eiointc_priv[0]->vec_count / 4; i++) {
/* Route to Node-0 Core-0 */
- if (index == 0)
+ if (eiointc_priv[index]->flags & EIOINTC_USE_CPU_ENCODE)
+ bit = cpu_logical_map(0);
+ else if (index == 0)
bit = BIT(cpu_logical_map(0));
else
bit = (eiointc_priv[index]->node << 4) | 1;
@@ -370,7 +424,7 @@ static int __init acpi_cascade_irqdomain_init(void)
static int __init eiointc_init(struct eiointc_priv *priv, int parent_irq,
u64 node_map)
{
- int i;
+ int i, val;
node_map = node_map ? node_map : -1ULL;
for_each_possible_cpu(i) {
@@ -390,6 +444,16 @@ static int __init eiointc_init(struct eiointc_priv *priv, int parent_irq,
return -ENOMEM;
}
+ if (kvm_para_has_feature(KVM_FEATURE_VIRT_EXTIOI)) {
+ val = iocsr_read32(EXTIOI_VIRT_FEATURES);
+ if (val & EXTIOI_HAS_CPU_ENCODE) {
+ val = iocsr_read32(EXTIOI_VIRT_CONFIG);
+ val |= EXTIOI_ENABLE_CPU_ENCODE;
+ iocsr_write32(val, EXTIOI_VIRT_CONFIG);
+ priv->flags = EIOINTC_USE_CPU_ENCODE;
+ }
+ }
+
eiointc_priv[nr_pics++] = priv;
eiointc_router_init(0);
irq_set_chained_handler_and_data(parent_irq, eiointc_irq_dispatch, priv);
--
2.39.3
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH v7 3/3] irqchip/loongson-eiointc: Add extioi virt extension support
2024-08-23 6:39 ` [PATCH v7 3/3] irqchip/loongson-eiointc: Add extioi virt extension support Bibo Mao
@ 2024-08-28 14:27 ` Huacai Chen
2024-08-29 1:46 ` maobibo
0 siblings, 1 reply; 8+ messages in thread
From: Huacai Chen @ 2024-08-28 14:27 UTC (permalink / raw)
To: Bibo Mao
Cc: Tianrui Zhao, Thomas Gleixner, WANG Xuerui, kvm, loongarch,
linux-kernel, virtualization, x86, Song Gao
Hi, Bibo,
On Fri, Aug 23, 2024 at 2:39 PM Bibo Mao <maobibo@loongson.cn> wrote:
>
> Interrupts can be routed to maximal four virtual CPUs with one HW
> EIOINTC interrupt controller model, since interrupt routing is encoded with
> CPU bitmap and EIOINTC node combined method. Here add the EIOINTC virt
> extension support so that interrupts can be routed to 256 vCPUs on
> hypervisor mode. CPU bitmap is replaced with normal encoding and EIOINTC
> node type is removed, so there are 8 bits for cpu selection, at most 256
> vCPUs are supported for interrupt routing.
>
> Co-developed-by: Song Gao <gaosong@loongson.cn>
> Signed-off-by: Song Gao <gaosong@loongson.cn>
> Signed-off-by: Bibo Mao <maobibo@loongson.cn>
> ---
> .../arch/loongarch/irq-chip-model.rst | 64 +++++++++++
> .../zh_CN/arch/loongarch/irq-chip-model.rst | 55 +++++++++
> arch/loongarch/include/asm/irq.h | 1 +
> drivers/irqchip/irq-loongson-eiointc.c | 106 ++++++++++++++----
> 4 files changed, 205 insertions(+), 21 deletions(-)
>
> diff --git a/Documentation/arch/loongarch/irq-chip-model.rst b/Documentation/arch/loongarch/irq-chip-model.rst
> index 7988f4192363..d2350780ad1d 100644
> --- a/Documentation/arch/loongarch/irq-chip-model.rst
> +++ b/Documentation/arch/loongarch/irq-chip-model.rst
> @@ -85,6 +85,70 @@ to CPUINTC directly::
> | Devices |
> +---------+
>
> +Virtual extended IRQ model
> +==========================
> +
> +In this model, IPI (Inter-Processor Interrupt) and CPU Local Timer interrupt
> +go to CPUINTC directly, CPU UARTS interrupts go to PCH-PIC, while all other
> +devices interrupts go to PCH-PIC/PCH-MSI and gathered by V-EIOINTC (Virtual
> +Extended I/O Interrupt Controller), and then go to CPUINTC directly::
> +
> + +-----+ +-------------------+ +-------+
> + | IPI |--> | CPUINTC(0-255vcpu)| <-- | Timer |
> + +-----+ +-------------------+ +-------+
> + ^
> + |
> + +-----------+
> + | V-EIOINTC |
> + +-----------+
> + ^ ^
> + | |
> + +---------+ +---------+
> + | PCH-PIC | | PCH-MSI |
> + +---------+ +---------+
> + ^ ^ ^
> + | | |
> + +--------+ +---------+ +---------+
> + | UARTs | | Devices | | Devices |
> + +--------+ +---------+ +---------+
> +
> +
> +Description
> +-----------
> +V-EIOINTC (Virtual Extended I/O Interrupt Controller) is an extension of
> +EIOINTC, it only works in VM mode which runs in KVM hypervisor. Interrupts can
> +be routed to up to four vCPUs via standard EIOINTC, however with V-EIOINTC
> +interrupts can be routed to up to 256 virtual cpus.
> +
> +With standard EIOINTC, interrupt routing setting includes two parts: eight
> +bits for CPU selection and four bits for CPU IP (Interrupt Pin) selection.
> +For CPU selection there is four bits for EIOINTC node selection, four bits
> +for EIOINTC CPU selection. Bitmap method is used for CPU selection and
> +CPU IP selection, so interrupt can only route to CPU0 - CPU3 and IP0-IP3 in
> +one EIOINTC node.
> +
> +With V-EIOINTC it supports to route more CPUs and CPU IP (Interrupt Pin),
> +there are two newly added registers with V-EIOINTC.
> +
> +EXTIOI_VIRT_FEATURES
> +--------------------
> +This register is read-only register, which indicates supported features with
> +V-EIOINTC. Feature EXTIOI_HAS_INT_ENCODE and EXTIOI_HAS_CPU_ENCODE is added.
> +
> +Feature EXTIOI_HAS_INT_ENCODE is part of standard EIOINTC. If it is 1, it
> +indicates that CPU Interrupt Pin selection can be normal method rather than
> +bitmap method, so interrupt can be routed to IP0 - IP15.
> +
> +Feature EXTIOI_HAS_CPU_ENCODE is entension of V-EIOINTC. If it is 1, it
> +indicates that CPU selection can be normal method rather than bitmap method,
> +so interrupt can be routed to CPU0 - CPU255.
> +
> +EXTIOI_VIRT_CONFIG
> +------------------
> +This register is read-write register, for compatibility intterupt routed uses
> +the default method which is the same with standard EIOINTC. If the bit is set
> +with 1, it indicated HW to use normal method rather than bitmap method.
> +
> ACPI-related definitions
> ========================
>
> diff --git a/Documentation/translations/zh_CN/arch/loongarch/irq-chip-model.rst b/Documentation/translations/zh_CN/arch/loongarch/irq-chip-model.rst
> index f1e9ab18206c..d696bd394c02 100644
> --- a/Documentation/translations/zh_CN/arch/loongarch/irq-chip-model.rst
> +++ b/Documentation/translations/zh_CN/arch/loongarch/irq-chip-model.rst
> @@ -87,6 +87,61 @@ PCH-LPC/PCH-MSI，然后被EIOINTC统一收集，再直接到达CPUINTC::
> | Devices |
> +---------+
>
> +虚拟扩展IRQ模型
> +===============
> +
> +在这种模型里面, IPI(Inter-Processor Interrupt) 和CPU本地时钟中断直接发送到CPUINTC,
> +CPU串口 (UARTs) 中断发送到PCH-PIC, 而其他所有设备的中断则分别发送到所连接的PCH_PIC/
> +PCH-MSI, 然后V-EIOINTC统一收集，再直接到达CPUINTC::
> +
> + +-----+ +-------------------+ +-------+
> + | IPI |--> | CPUINTC(0-255vcpu)| <-- | Timer |
> + +-----+ +-------------------+ +-------+
> + ^
> + |
> + +-----------+
> + | V-EIOINTC |
> + +-----------+
> + ^ ^
> + | |
> + +---------+ +---------+
> + | PCH-PIC | | PCH-MSI |
> + +---------+ +---------+
> + ^ ^ ^
> + | | |
> + +--------+ +---------+ +---------+
> + | UARTs | | Devices | | Devices |
> + +--------+ +---------+ +---------+
> +
> +V-EIOINTC 是EIOINTC的扩展, 仅工作在hyperisor模式下, 中断经EIOINTC最多可个路由到４个
> +虚拟cpu. 但中断经V-EIOINTC最多可个路由到256个虚拟cpu.
> +
> +传统的EIOINTC中断控制器，中断路由分为两个部分：8比特用于控制路由到哪个CPU，
> +4比特用于控制路由到特定CPU的哪个中断管脚.控制CPU路由的8比特前4比特用于控制
> +路由到哪个EIOINTC节点，后4比特用于控制此节点哪个CPU。中断路由在选择CPU路由
> +和CPU中断管脚路由时，使用bitmap编码方式而不是正常编码方式，所以对于一个
> +EIOINTC中断控制器节点，中断只能路由到CPU0 - CPU3，中断管教IP0-IP3。
> +
> +V-EIOINTC新增了两个寄存器，支持中断路由到更多CPU个和中断管脚。
> +
> +V-EIOINTC功能寄存器
> +-------------------
> +功能寄存器是只读寄存器，用于显示V-EIOINTC支持的特性，目前两个支持两个特性
> +EXTIOI_HAS_INT_ENCODE 和 EXTIOI_HAS_CPU_ENCODE。
> +
> +特性EXTIOI_HAS_INT_ENCODE是传统EIOINTC中断控制器的一个特性，如果此比特为1，
> +显示CPU中断管脚路由方式支持正常编码，而不是bitmap编码，所以中断可以路由到
> +管脚IP0 - IP15。
> +
> +特性EXTIOI_HAS_CPU_ENCODE是V-EIOINTC新增特性，如果此比特为1，表示CPU路由
> +方式支持正常编码，而不是bitmap编码，所以中断可以路由到CPU0 - CPU255。
> +
> +V-EIOINTC配置寄存器
> +-------------------
> +配置寄存器是可读写寄存器，为了兼容性考虑，如果不写此寄存器，中断路由采用
> +和传统EIOINTC相同的路由设置。如果对应比特设置为1，表示采用正常路由方式而
> +不是bitmap编码的路由方式。
> +
> ACPI相关的定义
> ==============
>
> diff --git a/arch/loongarch/include/asm/irq.h b/arch/loongarch/include/asm/irq.h
> index 480418bc5071..ce85d4c7d225 100644
> --- a/arch/loongarch/include/asm/irq.h
> +++ b/arch/loongarch/include/asm/irq.h
> @@ -54,6 +54,7 @@ extern struct acpi_vector_group pch_group[MAX_IO_PICS];
> extern struct acpi_vector_group msi_group[MAX_IO_PICS];
>
> #define CORES_PER_EIO_NODE 4
> +#define CORES_PER_VEIO_NODE 256
>
> #define LOONGSON_CPU_UART0_VEC 10 /* CPU UART0 */
> #define LOONGSON_CPU_THSENS_VEC 14 /* CPU Thsens */
> diff --git a/drivers/irqchip/irq-loongson-eiointc.c b/drivers/irqchip/irq-loongson-eiointc.c
> index b1f2080be2be..bc54f81ec129 100644
> --- a/drivers/irqchip/irq-loongson-eiointc.c
> +++ b/drivers/irqchip/irq-loongson-eiointc.c
> @@ -14,6 +14,7 @@
> #include <linux/irqdomain.h>
> #include <linux/irqchip/chained_irq.h>
> #include <linux/kernel.h>
> +#include <linux/kvm_para.h>
> #include <linux/syscore_ops.h>
> #include <asm/numa.h>
>
> @@ -24,15 +25,37 @@
> #define EIOINTC_REG_ISR 0x1800
> #define EIOINTC_REG_ROUTE 0x1c00
>
> +#define EXTIOI_VIRT_FEATURES 0x40000000
> +#define EXTIOI_HAS_VIRT_EXTENSION BIT(0)
> +#define EXTIOI_HAS_ENABLE_OPTION BIT(1)
> +#define EXTIOI_HAS_INT_ENCODE BIT(2)
> +#define EXTIOI_HAS_CPU_ENCODE BIT(3)
> +#define EXTIOI_VIRT_CONFIG 0x40000004
> +#define EXTIOI_ENABLE BIT(1)
> +#define EXTIOI_ENABLE_INT_ENCODE BIT(2)
> +#define EXTIOI_ENABLE_CPU_ENCODE BIT(3)
I think qemu hasn't release with v-eiointc? So we still have a chance
to modify qemu and this driver to simplify registers:
+#define EXTIOI_VIRT_FEATS 0x40000000
+#define EXTIOI_HAS_INT_ENCODE BIT(0)
+#define EXTIOI_HAS_CPU_ENCODE BIT(1)
+#define EXTIOI_VIRT_CONFIG 0x40000004
+#define EXTIOI_ENABLE_INT_ENCODE BIT(0)
+#define EXTIOI_ENABLE_CPU_ENCODE BIT(1)
> +
> #define VEC_REG_COUNT 4
> #define VEC_COUNT_PER_REG 64
> #define VEC_COUNT (VEC_REG_COUNT * VEC_COUNT_PER_REG)
> #define VEC_REG_IDX(irq_id) ((irq_id) / VEC_COUNT_PER_REG)
> #define VEC_REG_BIT(irq_id) ((irq_id) % VEC_COUNT_PER_REG)
> #define EIOINTC_ALL_ENABLE 0xffffffff
> +#define EIOINTC_ALL_ENABLE_VEC_MASK(vector) (EIOINTC_ALL_ENABLE & ~BIT(vector & 0x1F))
> +#define EIOINTC_REG_ENABLE_VEC(vector) (EIOINTC_REG_ENABLE + ((vector >> 5) << 2))
> +#define EIOINTC_USE_CPU_ENCODE BIT(0)
>
> #define MAX_EIO_NODES (NR_CPUS / CORES_PER_EIO_NODE)
>
> +/*
> + * Routing registers are 32bit, and there is 8-bit route setting for every
> + * interrupt vector. So one Route register contains four vectors routing
> + * information.
> + */
> +#define EIOINTC_REG_ROUTE_VEC(vector) (EIOINTC_REG_ROUTE + (vector & ~0x03))
> +#define EIOINTC_REG_ROUTE_VEC_SHIFT(vector) ((vector & 0x03) << 3)
> +#define EIOINTC_REG_ROUTE_VEC_MASK(vector) (0xff << EIOINTC_REG_ROUTE_VEC_SHIFT(vector))
> +
> static int nr_pics;
>
> struct eiointc_priv {
> @@ -42,6 +65,7 @@ struct eiointc_priv {
> cpumask_t cpuspan_map;
> struct fwnode_handle *domain_handle;
> struct irq_domain *eiointc_domain;
> + int flags;
> };
>
> static struct eiointc_priv *eiointc_priv[MAX_IO_PICS];
> @@ -57,7 +81,13 @@ static void eiointc_enable(void)
>
> static int cpu_to_eio_node(int cpu)
> {
> - return cpu_logical_map(cpu) / CORES_PER_EIO_NODE;
> + int cores;
> +
> + if (kvm_para_has_feature(KVM_FEATURE_VIRT_EXTIOI))
> + cores = CORES_PER_VEIO_NODE;
> + else
> + cores = CORES_PER_EIO_NODE;
> + return cpu_logical_map(cpu) / cores;
> }
>
> #ifdef CONFIG_SMP
> @@ -89,6 +119,16 @@ static void eiointc_set_irq_route(int pos, unsigned int cpu, unsigned int mnode,
>
> static DEFINE_RAW_SPINLOCK(affinity_lock);
>
> +static void virt_extioi_set_irq_route(unsigned int vector, unsigned int cpu)
Use veiointc_set_irq_route() can keep consistency with real machines.
> +{
> + unsigned long reg = EIOINTC_REG_ROUTE_VEC(vector);
> + u32 data = iocsr_read32(reg);
> +
> + data &= ~EIOINTC_REG_ROUTE_VEC_MASK(vector);
> + data |= cpu_logical_map(cpu) << EIOINTC_REG_ROUTE_VEC_SHIFT(vector);
> + iocsr_write32(data, reg);
> +}
> +
> static int eiointc_set_irq_affinity(struct irq_data *d, const struct cpumask *affinity, bool force)
> {
> unsigned int cpu;
> @@ -105,18 +145,24 @@ static int eiointc_set_irq_affinity(struct irq_data *d, const struct cpumask *af
> }
>
> vector = d->hwirq;
> - regaddr = EIOINTC_REG_ENABLE + ((vector >> 5) << 2);
> -
> - /* Mask target vector */
> - csr_any_send(regaddr, EIOINTC_ALL_ENABLE & (~BIT(vector & 0x1F)),
> - 0x0, priv->node * CORES_PER_EIO_NODE);
> -
> - /* Set route for target vector */
> - eiointc_set_irq_route(vector, cpu, priv->node, &priv->node_map);
> -
> - /* Unmask target vector */
> - csr_any_send(regaddr, EIOINTC_ALL_ENABLE,
> - 0x0, priv->node * CORES_PER_EIO_NODE);
> + regaddr = EIOINTC_REG_ENABLE_VEC(vector);
> +
> + if (priv->flags & EIOINTC_USE_CPU_ENCODE) {
> + iocsr_write32(EIOINTC_ALL_ENABLE_VEC_MASK(vector), regaddr);
> + virt_extioi_set_irq_route(vector, cpu);
> + iocsr_write32(EIOINTC_ALL_ENABLE, regaddr);
We can add the same comments as real machines here.
Huacai
> + } else {
> + /* Mask target vector */
> + csr_any_send(regaddr, EIOINTC_ALL_ENABLE_VEC_MASK(vector),
> + 0x0, priv->node * CORES_PER_EIO_NODE);
> +
> + /* Set route for target vector */
> + eiointc_set_irq_route(vector, cpu, priv->node, &priv->node_map);
> +
> + /* Unmask target vector */
> + csr_any_send(regaddr, EIOINTC_ALL_ENABLE,
> + 0x0, priv->node * CORES_PER_EIO_NODE);
> + }
>
> irq_data_update_effective_affinity(d, cpumask_of(cpu));
>
> @@ -140,17 +186,23 @@ static int eiointc_index(int node)
>
> static int eiointc_router_init(unsigned int cpu)
> {
> - int i, bit;
> - uint32_t data;
> - uint32_t node = cpu_to_eio_node(cpu);
> - int index = eiointc_index(node);
> + int i, bit, cores, index, node;
> + unsigned int data;
> +
> + node = cpu_to_eio_node(cpu);
> + index = eiointc_index(node);
>
> if (index < 0) {
> pr_err("Error: invalid nodemap!\n");
> - return -1;
> + return -EINVAL;
> }
>
> - if ((cpu_logical_map(cpu) % CORES_PER_EIO_NODE) == 0) {
> + if (eiointc_priv[index]->flags & EIOINTC_USE_CPU_ENCODE)
> + cores = CORES_PER_VEIO_NODE;
> + else
> + cores = CORES_PER_EIO_NODE;
> +
> + if ((cpu_logical_map(cpu) % cores) == 0) {
> eiointc_enable();
>
> for (i = 0; i < eiointc_priv[0]->vec_count / 32; i++) {
> @@ -166,7 +218,9 @@ static int eiointc_router_init(unsigned int cpu)
>
> for (i = 0; i < eiointc_priv[0]->vec_count / 4; i++) {
> /* Route to Node-0 Core-0 */
> - if (index == 0)
> + if (eiointc_priv[index]->flags & EIOINTC_USE_CPU_ENCODE)
> + bit = cpu_logical_map(0);
> + else if (index == 0)
> bit = BIT(cpu_logical_map(0));
> else
> bit = (eiointc_priv[index]->node << 4) | 1;
> @@ -370,7 +424,7 @@ static int __init acpi_cascade_irqdomain_init(void)
> static int __init eiointc_init(struct eiointc_priv *priv, int parent_irq,
> u64 node_map)
> {
> - int i;
> + int i, val;
>
> node_map = node_map ? node_map : -1ULL;
> for_each_possible_cpu(i) {
> @@ -390,6 +444,16 @@ static int __init eiointc_init(struct eiointc_priv *priv, int parent_irq,
> return -ENOMEM;
> }
>
> + if (kvm_para_has_feature(KVM_FEATURE_VIRT_EXTIOI)) {
> + val = iocsr_read32(EXTIOI_VIRT_FEATURES);
> + if (val & EXTIOI_HAS_CPU_ENCODE) {
> + val = iocsr_read32(EXTIOI_VIRT_CONFIG);
> + val |= EXTIOI_ENABLE_CPU_ENCODE;
> + iocsr_write32(val, EXTIOI_VIRT_CONFIG);
> + priv->flags = EIOINTC_USE_CPU_ENCODE;
> + }
> + }
> +
> eiointc_priv[nr_pics++] = priv;
> eiointc_router_init(0);
> irq_set_chained_handler_and_data(parent_irq, eiointc_irq_dispatch, priv);
> --
> 2.39.3
>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v7 3/3] irqchip/loongson-eiointc: Add extioi virt extension support
2024-08-28 14:27 ` Huacai Chen
@ 2024-08-29 1:46 ` maobibo
2024-08-29 12:29 ` Huacai Chen
0 siblings, 1 reply; 8+ messages in thread
From: maobibo @ 2024-08-29 1:46 UTC (permalink / raw)
To: Huacai Chen
Cc: Tianrui Zhao, Thomas Gleixner, WANG Xuerui, kvm, loongarch,
linux-kernel, virtualization, x86, Song Gao
Huacai,
On 2024/8/28 下午10:27, Huacai Chen wrote:
> Hi, Bibo,
>
> On Fri, Aug 23, 2024 at 2:39 PM Bibo Mao <maobibo@loongson.cn> wrote:
>>
>> Interrupts can be routed to maximal four virtual CPUs with one HW
>> EIOINTC interrupt controller model, since interrupt routing is encoded with
>> CPU bitmap and EIOINTC node combined method. Here add the EIOINTC virt
>> extension support so that interrupts can be routed to 256 vCPUs on
>> hypervisor mode. CPU bitmap is replaced with normal encoding and EIOINTC
>> node type is removed, so there are 8 bits for cpu selection, at most 256
>> vCPUs are supported for interrupt routing.
>>
>> Co-developed-by: Song Gao <gaosong@loongson.cn>
>> Signed-off-by: Song Gao <gaosong@loongson.cn>
>> Signed-off-by: Bibo Mao <maobibo@loongson.cn>
>> ---
>> .../arch/loongarch/irq-chip-model.rst | 64 +++++++++++
>> .../zh_CN/arch/loongarch/irq-chip-model.rst | 55 +++++++++
>> arch/loongarch/include/asm/irq.h | 1 +
>> drivers/irqchip/irq-loongson-eiointc.c | 106 ++++++++++++++----
>> 4 files changed, 205 insertions(+), 21 deletions(-)
>>
>> diff --git a/Documentation/arch/loongarch/irq-chip-model.rst b/Documentation/arch/loongarch/irq-chip-model.rst
>> index 7988f4192363..d2350780ad1d 100644
>> --- a/Documentation/arch/loongarch/irq-chip-model.rst
>> +++ b/Documentation/arch/loongarch/irq-chip-model.rst
>> @@ -85,6 +85,70 @@ to CPUINTC directly::
>> | Devices |
>> +---------+
>>
>> +Virtual extended IRQ model
>> +==========================
>> +
>> +In this model, IPI (Inter-Processor Interrupt) and CPU Local Timer interrupt
>> +go to CPUINTC directly, CPU UARTS interrupts go to PCH-PIC, while all other
>> +devices interrupts go to PCH-PIC/PCH-MSI and gathered by V-EIOINTC (Virtual
>> +Extended I/O Interrupt Controller), and then go to CPUINTC directly::
>> +
>> + +-----+ +-------------------+ +-------+
>> + | IPI |--> | CPUINTC(0-255vcpu)| <-- | Timer |
>> + +-----+ +-------------------+ +-------+
>> + ^
>> + |
>> + +-----------+
>> + | V-EIOINTC |
>> + +-----------+
>> + ^ ^
>> + | |
>> + +---------+ +---------+
>> + | PCH-PIC | | PCH-MSI |
>> + +---------+ +---------+
>> + ^ ^ ^
>> + | | |
>> + +--------+ +---------+ +---------+
>> + | UARTs | | Devices | | Devices |
>> + +--------+ +---------+ +---------+
>> +
>> +
>> +Description
>> +-----------
>> +V-EIOINTC (Virtual Extended I/O Interrupt Controller) is an extension of
>> +EIOINTC, it only works in VM mode which runs in KVM hypervisor. Interrupts can
>> +be routed to up to four vCPUs via standard EIOINTC, however with V-EIOINTC
>> +interrupts can be routed to up to 256 virtual cpus.
>> +
>> +With standard EIOINTC, interrupt routing setting includes two parts: eight
>> +bits for CPU selection and four bits for CPU IP (Interrupt Pin) selection.
>> +For CPU selection there is four bits for EIOINTC node selection, four bits
>> +for EIOINTC CPU selection. Bitmap method is used for CPU selection and
>> +CPU IP selection, so interrupt can only route to CPU0 - CPU3 and IP0-IP3 in
>> +one EIOINTC node.
>> +
>> +With V-EIOINTC it supports to route more CPUs and CPU IP (Interrupt Pin),
>> +there are two newly added registers with V-EIOINTC.
>> +
>> +EXTIOI_VIRT_FEATURES
>> +--------------------
>> +This register is read-only register, which indicates supported features with
>> +V-EIOINTC. Feature EXTIOI_HAS_INT_ENCODE and EXTIOI_HAS_CPU_ENCODE is added.
>> +
>> +Feature EXTIOI_HAS_INT_ENCODE is part of standard EIOINTC. If it is 1, it
>> +indicates that CPU Interrupt Pin selection can be normal method rather than
>> +bitmap method, so interrupt can be routed to IP0 - IP15.
>> +
>> +Feature EXTIOI_HAS_CPU_ENCODE is entension of V-EIOINTC. If it is 1, it
>> +indicates that CPU selection can be normal method rather than bitmap method,
>> +so interrupt can be routed to CPU0 - CPU255.
>> +
>> +EXTIOI_VIRT_CONFIG
>> +------------------
>> +This register is read-write register, for compatibility intterupt routed uses
>> +the default method which is the same with standard EIOINTC. If the bit is set
>> +with 1, it indicated HW to use normal method rather than bitmap method.
>> +
>> ACPI-related definitions
>> ========================
>>
>> diff --git a/Documentation/translations/zh_CN/arch/loongarch/irq-chip-model.rst b/Documentation/translations/zh_CN/arch/loongarch/irq-chip-model.rst
>> index f1e9ab18206c..d696bd394c02 100644
>> --- a/Documentation/translations/zh_CN/arch/loongarch/irq-chip-model.rst
>> +++ b/Documentation/translations/zh_CN/arch/loongarch/irq-chip-model.rst
>> @@ -87,6 +87,61 @@ PCH-LPC/PCH-MSI，然后被EIOINTC统一收集，再直接到达CPUINTC::
>> | Devices |
>> +---------+
>>
>> +虚拟扩展IRQ模型
>> +===============
>> +
>> +在这种模型里面, IPI(Inter-Processor Interrupt) 和CPU本地时钟中断直接发送到CPUINTC,
>> +CPU串口 (UARTs) 中断发送到PCH-PIC, 而其他所有设备的中断则分别发送到所连接的PCH_PIC/
>> +PCH-MSI, 然后V-EIOINTC统一收集，再直接到达CPUINTC::
>> +
>> + +-----+ +-------------------+ +-------+
>> + | IPI |--> | CPUINTC(0-255vcpu)| <-- | Timer |
>> + +-----+ +-------------------+ +-------+
>> + ^
>> + |
>> + +-----------+
>> + | V-EIOINTC |
>> + +-----------+
>> + ^ ^
>> + | |
>> + +---------+ +---------+
>> + | PCH-PIC | | PCH-MSI |
>> + +---------+ +---------+
>> + ^ ^ ^
>> + | | |
>> + +--------+ +---------+ +---------+
>> + | UARTs | | Devices | | Devices |
>> + +--------+ +---------+ +---------+
>> +
>> +V-EIOINTC 是EIOINTC的扩展, 仅工作在hyperisor模式下, 中断经EIOINTC最多可个路由到４个
>> +虚拟cpu. 但中断经V-EIOINTC最多可个路由到256个虚拟cpu.
>> +
>> +传统的EIOINTC中断控制器，中断路由分为两个部分：8比特用于控制路由到哪个CPU，
>> +4比特用于控制路由到特定CPU的哪个中断管脚.控制CPU路由的8比特前4比特用于控制
>> +路由到哪个EIOINTC节点，后4比特用于控制此节点哪个CPU。中断路由在选择CPU路由
>> +和CPU中断管脚路由时，使用bitmap编码方式而不是正常编码方式，所以对于一个
>> +EIOINTC中断控制器节点，中断只能路由到CPU0 - CPU3，中断管教IP0-IP3。
>> +
>> +V-EIOINTC新增了两个寄存器，支持中断路由到更多CPU个和中断管脚。
>> +
>> +V-EIOINTC功能寄存器
>> +-------------------
>> +功能寄存器是只读寄存器，用于显示V-EIOINTC支持的特性，目前两个支持两个特性
>> +EXTIOI_HAS_INT_ENCODE 和 EXTIOI_HAS_CPU_ENCODE。
>> +
>> +特性EXTIOI_HAS_INT_ENCODE是传统EIOINTC中断控制器的一个特性，如果此比特为1，
>> +显示CPU中断管脚路由方式支持正常编码，而不是bitmap编码，所以中断可以路由到
>> +管脚IP0 - IP15。
>> +
>> +特性EXTIOI_HAS_CPU_ENCODE是V-EIOINTC新增特性，如果此比特为1，表示CPU路由
>> +方式支持正常编码，而不是bitmap编码，所以中断可以路由到CPU0 - CPU255。
>> +
>> +V-EIOINTC配置寄存器
>> +-------------------
>> +配置寄存器是可读写寄存器，为了兼容性考虑，如果不写此寄存器，中断路由采用
>> +和传统EIOINTC相同的路由设置。如果对应比特设置为1，表示采用正常路由方式而
>> +不是bitmap编码的路由方式。
>> +
>> ACPI相关的定义
>> ==============
>>
>> diff --git a/arch/loongarch/include/asm/irq.h b/arch/loongarch/include/asm/irq.h
>> index 480418bc5071..ce85d4c7d225 100644
>> --- a/arch/loongarch/include/asm/irq.h
>> +++ b/arch/loongarch/include/asm/irq.h
>> @@ -54,6 +54,7 @@ extern struct acpi_vector_group pch_group[MAX_IO_PICS];
>> extern struct acpi_vector_group msi_group[MAX_IO_PICS];
>>
>> #define CORES_PER_EIO_NODE 4
>> +#define CORES_PER_VEIO_NODE 256
>>
>> #define LOONGSON_CPU_UART0_VEC 10 /* CPU UART0 */
>> #define LOONGSON_CPU_THSENS_VEC 14 /* CPU Thsens */
>> diff --git a/drivers/irqchip/irq-loongson-eiointc.c b/drivers/irqchip/irq-loongson-eiointc.c
>> index b1f2080be2be..bc54f81ec129 100644
>> --- a/drivers/irqchip/irq-loongson-eiointc.c
>> +++ b/drivers/irqchip/irq-loongson-eiointc.c
>> @@ -14,6 +14,7 @@
>> #include <linux/irqdomain.h>
>> #include <linux/irqchip/chained_irq.h>
>> #include <linux/kernel.h>
>> +#include <linux/kvm_para.h>
>> #include <linux/syscore_ops.h>
>> #include <asm/numa.h>
>>
>> @@ -24,15 +25,37 @@
>> #define EIOINTC_REG_ISR 0x1800
>> #define EIOINTC_REG_ROUTE 0x1c00
>>
>> +#define EXTIOI_VIRT_FEATURES 0x40000000
>> +#define EXTIOI_HAS_VIRT_EXTENSION BIT(0)
>> +#define EXTIOI_HAS_ENABLE_OPTION BIT(1)
>> +#define EXTIOI_HAS_INT_ENCODE BIT(2)
>> +#define EXTIOI_HAS_CPU_ENCODE BIT(3)
>> +#define EXTIOI_VIRT_CONFIG 0x40000004
>> +#define EXTIOI_ENABLE BIT(1)
>> +#define EXTIOI_ENABLE_INT_ENCODE BIT(2)
>> +#define EXTIOI_ENABLE_CPU_ENCODE BIT(3)
> I think qemu hasn't release with v-eiointc? So we still have a chance
> to modify qemu and this driver to simplify registers:
It is already merged in qemu mainline, code is frozen and qemu 9.1 will
release soon. Once it is merged in mainline and even if code is not
frozen, I will modify it only if it is bug.
>
> +#define EXTIOI_VIRT_FEATS 0x40000000
> +#define EXTIOI_HAS_INT_ENCODE BIT(0)
> +#define EXTIOI_HAS_CPU_ENCODE BIT(1)
> +#define EXTIOI_VIRT_CONFIG 0x40000004
> +#define EXTIOI_ENABLE_INT_ENCODE BIT(0)
> +#define EXTIOI_ENABLE_CPU_ENCODE BIT(1)
>
>> +
>> #define VEC_REG_COUNT 4
>> #define VEC_COUNT_PER_REG 64
>> #define VEC_COUNT (VEC_REG_COUNT * VEC_COUNT_PER_REG)
>> #define VEC_REG_IDX(irq_id) ((irq_id) / VEC_COUNT_PER_REG)
>> #define VEC_REG_BIT(irq_id) ((irq_id) % VEC_COUNT_PER_REG)
>> #define EIOINTC_ALL_ENABLE 0xffffffff
>> +#define EIOINTC_ALL_ENABLE_VEC_MASK(vector) (EIOINTC_ALL_ENABLE & ~BIT(vector & 0x1F))
>> +#define EIOINTC_REG_ENABLE_VEC(vector) (EIOINTC_REG_ENABLE + ((vector >> 5) << 2))
>> +#define EIOINTC_USE_CPU_ENCODE BIT(0)
>>
>> #define MAX_EIO_NODES (NR_CPUS / CORES_PER_EIO_NODE)
>>
>> +/*
>> + * Routing registers are 32bit, and there is 8-bit route setting for every
>> + * interrupt vector. So one Route register contains four vectors routing
>> + * information.
>> + */
>> +#define EIOINTC_REG_ROUTE_VEC(vector) (EIOINTC_REG_ROUTE + (vector & ~0x03))
>> +#define EIOINTC_REG_ROUTE_VEC_SHIFT(vector) ((vector & 0x03) << 3)
>> +#define EIOINTC_REG_ROUTE_VEC_MASK(vector) (0xff << EIOINTC_REG_ROUTE_VEC_SHIFT(vector))
>> +
>> static int nr_pics;
>>
>> struct eiointc_priv {
>> @@ -42,6 +65,7 @@ struct eiointc_priv {
>> cpumask_t cpuspan_map;
>> struct fwnode_handle *domain_handle;
>> struct irq_domain *eiointc_domain;
>> + int flags;
>> };
>>
>> static struct eiointc_priv *eiointc_priv[MAX_IO_PICS];
>> @@ -57,7 +81,13 @@ static void eiointc_enable(void)
>>
>> static int cpu_to_eio_node(int cpu)
>> {
>> - return cpu_logical_map(cpu) / CORES_PER_EIO_NODE;
>> + int cores;
>> +
>> + if (kvm_para_has_feature(KVM_FEATURE_VIRT_EXTIOI))
>> + cores = CORES_PER_VEIO_NODE;
>> + else
>> + cores = CORES_PER_EIO_NODE;
>> + return cpu_logical_map(cpu) / cores;
>> }
>>
>> #ifdef CONFIG_SMP
>> @@ -89,6 +119,16 @@ static void eiointc_set_irq_route(int pos, unsigned int cpu, unsigned int mnode,
>>
>> static DEFINE_RAW_SPINLOCK(affinity_lock);
>>
>> +static void virt_extioi_set_irq_route(unsigned int vector, unsigned int cpu)
> Use veiointc_set_irq_route() can keep consistency with real machines.
yes, veiointc_set_irq_route will be added here.
>
>> +{
>> + unsigned long reg = EIOINTC_REG_ROUTE_VEC(vector);
>> + u32 data = iocsr_read32(reg);
>> +
>> + data &= ~EIOINTC_REG_ROUTE_VEC_MASK(vector);
>> + data |= cpu_logical_map(cpu) << EIOINTC_REG_ROUTE_VEC_SHIFT(vector);
>> + iocsr_write32(data, reg);
>> +}
>> +
>> static int eiointc_set_irq_affinity(struct irq_data *d, const struct cpumask *affinity, bool force)
>> {
>> unsigned int cpu;
>> @@ -105,18 +145,24 @@ static int eiointc_set_irq_affinity(struct irq_data *d, const struct cpumask *af
>> }
>>
>> vector = d->hwirq;
>> - regaddr = EIOINTC_REG_ENABLE + ((vector >> 5) << 2);
>> -
>> - /* Mask target vector */
>> - csr_any_send(regaddr, EIOINTC_ALL_ENABLE & (~BIT(vector & 0x1F)),
>> - 0x0, priv->node * CORES_PER_EIO_NODE);
>> -
>> - /* Set route for target vector */
>> - eiointc_set_irq_route(vector, cpu, priv->node, &priv->node_map);
>> -
>> - /* Unmask target vector */
>> - csr_any_send(regaddr, EIOINTC_ALL_ENABLE,
>> - 0x0, priv->node * CORES_PER_EIO_NODE);
>> + regaddr = EIOINTC_REG_ENABLE_VEC(vector);
>> +
>> + if (priv->flags & EIOINTC_USE_CPU_ENCODE) {
>> + iocsr_write32(EIOINTC_ALL_ENABLE_VEC_MASK(vector), regaddr);
>> + virt_extioi_set_irq_route(vector, cpu);
>> + iocsr_write32(EIOINTC_ALL_ENABLE, regaddr);
> We can add the same comments as real machines here.
Similar comments will be added, otherwise there will be duplicated comments.
Hope to this the last review comments about grammar and name -:), else I
will think it is on deliberate. we know every architecture has multiple
maintainers to prevent arbitrary except LoongArch.
Regards
Bibo Mao
>
> Huacai
>
>> + } else {
>> + /* Mask target vector */
>> + csr_any_send(regaddr, EIOINTC_ALL_ENABLE_VEC_MASK(vector),
>> + 0x0, priv->node * CORES_PER_EIO_NODE);
>> +
>> + /* Set route for target vector */
>> + eiointc_set_irq_route(vector, cpu, priv->node, &priv->node_map);
>> +
>> + /* Unmask target vector */
>> + csr_any_send(regaddr, EIOINTC_ALL_ENABLE,
>> + 0x0, priv->node * CORES_PER_EIO_NODE);
>> + }
>>
>> irq_data_update_effective_affinity(d, cpumask_of(cpu));
>>
>> @@ -140,17 +186,23 @@ static int eiointc_index(int node)
>>
>> static int eiointc_router_init(unsigned int cpu)
>> {
>> - int i, bit;
>> - uint32_t data;
>> - uint32_t node = cpu_to_eio_node(cpu);
>> - int index = eiointc_index(node);
>> + int i, bit, cores, index, node;
>> + unsigned int data;
>> +
>> + node = cpu_to_eio_node(cpu);
>> + index = eiointc_index(node);
>>
>> if (index < 0) {
>> pr_err("Error: invalid nodemap!\n");
>> - return -1;
>> + return -EINVAL;
>> }
>>
>> - if ((cpu_logical_map(cpu) % CORES_PER_EIO_NODE) == 0) {
>> + if (eiointc_priv[index]->flags & EIOINTC_USE_CPU_ENCODE)
>> + cores = CORES_PER_VEIO_NODE;
>> + else
>> + cores = CORES_PER_EIO_NODE;
>> +
>> + if ((cpu_logical_map(cpu) % cores) == 0) {
>> eiointc_enable();
>>
>> for (i = 0; i < eiointc_priv[0]->vec_count / 32; i++) {
>> @@ -166,7 +218,9 @@ static int eiointc_router_init(unsigned int cpu)
>>
>> for (i = 0; i < eiointc_priv[0]->vec_count / 4; i++) {
>> /* Route to Node-0 Core-0 */
>> - if (index == 0)
>> + if (eiointc_priv[index]->flags & EIOINTC_USE_CPU_ENCODE)
>> + bit = cpu_logical_map(0);
>> + else if (index == 0)
>> bit = BIT(cpu_logical_map(0));
>> else
>> bit = (eiointc_priv[index]->node << 4) | 1;
>> @@ -370,7 +424,7 @@ static int __init acpi_cascade_irqdomain_init(void)
>> static int __init eiointc_init(struct eiointc_priv *priv, int parent_irq,
>> u64 node_map)
>> {
>> - int i;
>> + int i, val;
>>
>> node_map = node_map ? node_map : -1ULL;
>> for_each_possible_cpu(i) {
>> @@ -390,6 +444,16 @@ static int __init eiointc_init(struct eiointc_priv *priv, int parent_irq,
>> return -ENOMEM;
>> }
>>
>> + if (kvm_para_has_feature(KVM_FEATURE_VIRT_EXTIOI)) {
>> + val = iocsr_read32(EXTIOI_VIRT_FEATURES);
>> + if (val & EXTIOI_HAS_CPU_ENCODE) {
>> + val = iocsr_read32(EXTIOI_VIRT_CONFIG);
>> + val |= EXTIOI_ENABLE_CPU_ENCODE;
>> + iocsr_write32(val, EXTIOI_VIRT_CONFIG);
>> + priv->flags = EIOINTC_USE_CPU_ENCODE;
>> + }
>> + }
>> +
>> eiointc_priv[nr_pics++] = priv;
>> eiointc_router_init(0);
>> irq_set_chained_handler_and_data(parent_irq, eiointc_irq_dispatch, priv);
>> --
>> 2.39.3
>>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v7 3/3] irqchip/loongson-eiointc: Add extioi virt extension support
2024-08-29 1:46 ` maobibo
@ 2024-08-29 12:29 ` Huacai Chen
2024-08-29 14:04 ` Thomas Gleixner
0 siblings, 1 reply; 8+ messages in thread
From: Huacai Chen @ 2024-08-29 12:29 UTC (permalink / raw)
To: maobibo
Cc: Tianrui Zhao, Thomas Gleixner, WANG Xuerui, kvm, loongarch,
linux-kernel, virtualization, x86, Song Gao
On Thu, Aug 29, 2024 at 9:46 AM maobibo <maobibo@loongson.cn> wrote:
>
> Huacai,
>
> On 2024/8/28 下午10:27, Huacai Chen wrote:
> > Hi, Bibo,
> >
> > On Fri, Aug 23, 2024 at 2:39 PM Bibo Mao <maobibo@loongson.cn> wrote:
> >>
> >> Interrupts can be routed to maximal four virtual CPUs with one HW
> >> EIOINTC interrupt controller model, since interrupt routing is encoded with
> >> CPU bitmap and EIOINTC node combined method. Here add the EIOINTC virt
> >> extension support so that interrupts can be routed to 256 vCPUs on
> >> hypervisor mode. CPU bitmap is replaced with normal encoding and EIOINTC
> >> node type is removed, so there are 8 bits for cpu selection, at most 256
> >> vCPUs are supported for interrupt routing.
> >>
> >> Co-developed-by: Song Gao <gaosong@loongson.cn>
> >> Signed-off-by: Song Gao <gaosong@loongson.cn>
> >> Signed-off-by: Bibo Mao <maobibo@loongson.cn>
> >> ---
> >> .../arch/loongarch/irq-chip-model.rst | 64 +++++++++++
> >> .../zh_CN/arch/loongarch/irq-chip-model.rst | 55 +++++++++
> >> arch/loongarch/include/asm/irq.h | 1 +
> >> drivers/irqchip/irq-loongson-eiointc.c | 106 ++++++++++++++----
> >> 4 files changed, 205 insertions(+), 21 deletions(-)
> >>
> >> diff --git a/Documentation/arch/loongarch/irq-chip-model.rst b/Documentation/arch/loongarch/irq-chip-model.rst
> >> index 7988f4192363..d2350780ad1d 100644
> >> --- a/Documentation/arch/loongarch/irq-chip-model.rst
> >> +++ b/Documentation/arch/loongarch/irq-chip-model.rst
> >> @@ -85,6 +85,70 @@ to CPUINTC directly::
> >> | Devices |
> >> +---------+
> >>
> >> +Virtual extended IRQ model
> >> +==========================
> >> +
> >> +In this model, IPI (Inter-Processor Interrupt) and CPU Local Timer interrupt
> >> +go to CPUINTC directly, CPU UARTS interrupts go to PCH-PIC, while all other
> >> +devices interrupts go to PCH-PIC/PCH-MSI and gathered by V-EIOINTC (Virtual
> >> +Extended I/O Interrupt Controller), and then go to CPUINTC directly::
> >> +
> >> + +-----+ +-------------------+ +-------+
> >> + | IPI |--> | CPUINTC(0-255vcpu)| <-- | Timer |
> >> + +-----+ +-------------------+ +-------+
> >> + ^
> >> + |
> >> + +-----------+
> >> + | V-EIOINTC |
> >> + +-----------+
> >> + ^ ^
> >> + | |
> >> + +---------+ +---------+
> >> + | PCH-PIC | | PCH-MSI |
> >> + +---------+ +---------+
> >> + ^ ^ ^
> >> + | | |
> >> + +--------+ +---------+ +---------+
> >> + | UARTs | | Devices | | Devices |
> >> + +--------+ +---------+ +---------+
> >> +
> >> +
> >> +Description
> >> +-----------
> >> +V-EIOINTC (Virtual Extended I/O Interrupt Controller) is an extension of
> >> +EIOINTC, it only works in VM mode which runs in KVM hypervisor. Interrupts can
> >> +be routed to up to four vCPUs via standard EIOINTC, however with V-EIOINTC
> >> +interrupts can be routed to up to 256 virtual cpus.
> >> +
> >> +With standard EIOINTC, interrupt routing setting includes two parts: eight
> >> +bits for CPU selection and four bits for CPU IP (Interrupt Pin) selection.
> >> +For CPU selection there is four bits for EIOINTC node selection, four bits
> >> +for EIOINTC CPU selection. Bitmap method is used for CPU selection and
> >> +CPU IP selection, so interrupt can only route to CPU0 - CPU3 and IP0-IP3 in
> >> +one EIOINTC node.
> >> +
> >> +With V-EIOINTC it supports to route more CPUs and CPU IP (Interrupt Pin),
> >> +there are two newly added registers with V-EIOINTC.
> >> +
> >> +EXTIOI_VIRT_FEATURES
> >> +--------------------
> >> +This register is read-only register, which indicates supported features with
> >> +V-EIOINTC. Feature EXTIOI_HAS_INT_ENCODE and EXTIOI_HAS_CPU_ENCODE is added.
> >> +
> >> +Feature EXTIOI_HAS_INT_ENCODE is part of standard EIOINTC. If it is 1, it
> >> +indicates that CPU Interrupt Pin selection can be normal method rather than
> >> +bitmap method, so interrupt can be routed to IP0 - IP15.
> >> +
> >> +Feature EXTIOI_HAS_CPU_ENCODE is entension of V-EIOINTC. If it is 1, it
> >> +indicates that CPU selection can be normal method rather than bitmap method,
> >> +so interrupt can be routed to CPU0 - CPU255.
> >> +
> >> +EXTIOI_VIRT_CONFIG
> >> +------------------
> >> +This register is read-write register, for compatibility intterupt routed uses
> >> +the default method which is the same with standard EIOINTC. If the bit is set
> >> +with 1, it indicated HW to use normal method rather than bitmap method.
> >> +
> >> ACPI-related definitions
> >> ========================
> >>
> >> diff --git a/Documentation/translations/zh_CN/arch/loongarch/irq-chip-model.rst b/Documentation/translations/zh_CN/arch/loongarch/irq-chip-model.rst
> >> index f1e9ab18206c..d696bd394c02 100644
> >> --- a/Documentation/translations/zh_CN/arch/loongarch/irq-chip-model.rst
> >> +++ b/Documentation/translations/zh_CN/arch/loongarch/irq-chip-model.rst
> >> @@ -87,6 +87,61 @@ PCH-LPC/PCH-MSI，然后被EIOINTC统一收集，再直接到达CPUINTC::
> >> | Devices |
> >> +---------+
> >>
> >> +虚拟扩展IRQ模型
> >> +===============
> >> +
> >> +在这种模型里面, IPI(Inter-Processor Interrupt) 和CPU本地时钟中断直接发送到CPUINTC,
> >> +CPU串口 (UARTs) 中断发送到PCH-PIC, 而其他所有设备的中断则分别发送到所连接的PCH_PIC/
> >> +PCH-MSI, 然后V-EIOINTC统一收集，再直接到达CPUINTC::
> >> +
> >> + +-----+ +-------------------+ +-------+
> >> + | IPI |--> | CPUINTC(0-255vcpu)| <-- | Timer |
> >> + +-----+ +-------------------+ +-------+
> >> + ^
> >> + |
> >> + +-----------+
> >> + | V-EIOINTC |
> >> + +-----------+
> >> + ^ ^
> >> + | |
> >> + +---------+ +---------+
> >> + | PCH-PIC | | PCH-MSI |
> >> + +---------+ +---------+
> >> + ^ ^ ^
> >> + | | |
> >> + +--------+ +---------+ +---------+
> >> + | UARTs | | Devices | | Devices |
> >> + +--------+ +---------+ +---------+
> >> +
> >> +V-EIOINTC 是EIOINTC的扩展, 仅工作在hyperisor模式下, 中断经EIOINTC最多可个路由到４个
> >> +虚拟cpu. 但中断经V-EIOINTC最多可个路由到256个虚拟cpu.
> >> +
> >> +传统的EIOINTC中断控制器，中断路由分为两个部分：8比特用于控制路由到哪个CPU，
> >> +4比特用于控制路由到特定CPU的哪个中断管脚.控制CPU路由的8比特前4比特用于控制
> >> +路由到哪个EIOINTC节点，后4比特用于控制此节点哪个CPU。中断路由在选择CPU路由
> >> +和CPU中断管脚路由时，使用bitmap编码方式而不是正常编码方式，所以对于一个
> >> +EIOINTC中断控制器节点，中断只能路由到CPU0 - CPU3，中断管教IP0-IP3。
> >> +
> >> +V-EIOINTC新增了两个寄存器，支持中断路由到更多CPU个和中断管脚。
> >> +
> >> +V-EIOINTC功能寄存器
> >> +-------------------
> >> +功能寄存器是只读寄存器，用于显示V-EIOINTC支持的特性，目前两个支持两个特性
> >> +EXTIOI_HAS_INT_ENCODE 和 EXTIOI_HAS_CPU_ENCODE。
> >> +
> >> +特性EXTIOI_HAS_INT_ENCODE是传统EIOINTC中断控制器的一个特性，如果此比特为1，
> >> +显示CPU中断管脚路由方式支持正常编码，而不是bitmap编码，所以中断可以路由到
> >> +管脚IP0 - IP15。
> >> +
> >> +特性EXTIOI_HAS_CPU_ENCODE是V-EIOINTC新增特性，如果此比特为1，表示CPU路由
> >> +方式支持正常编码，而不是bitmap编码，所以中断可以路由到CPU0 - CPU255。
> >> +
> >> +V-EIOINTC配置寄存器
> >> +-------------------
> >> +配置寄存器是可读写寄存器，为了兼容性考虑，如果不写此寄存器，中断路由采用
> >> +和传统EIOINTC相同的路由设置。如果对应比特设置为1，表示采用正常路由方式而
> >> +不是bitmap编码的路由方式。
> >> +
> >> ACPI相关的定义
> >> ==============
> >>
> >> diff --git a/arch/loongarch/include/asm/irq.h b/arch/loongarch/include/asm/irq.h
> >> index 480418bc5071..ce85d4c7d225 100644
> >> --- a/arch/loongarch/include/asm/irq.h
> >> +++ b/arch/loongarch/include/asm/irq.h
> >> @@ -54,6 +54,7 @@ extern struct acpi_vector_group pch_group[MAX_IO_PICS];
> >> extern struct acpi_vector_group msi_group[MAX_IO_PICS];
> >>
> >> #define CORES_PER_EIO_NODE 4
> >> +#define CORES_PER_VEIO_NODE 256
> >>
> >> #define LOONGSON_CPU_UART0_VEC 10 /* CPU UART0 */
> >> #define LOONGSON_CPU_THSENS_VEC 14 /* CPU Thsens */
> >> diff --git a/drivers/irqchip/irq-loongson-eiointc.c b/drivers/irqchip/irq-loongson-eiointc.c
> >> index b1f2080be2be..bc54f81ec129 100644
> >> --- a/drivers/irqchip/irq-loongson-eiointc.c
> >> +++ b/drivers/irqchip/irq-loongson-eiointc.c
> >> @@ -14,6 +14,7 @@
> >> #include <linux/irqdomain.h>
> >> #include <linux/irqchip/chained_irq.h>
> >> #include <linux/kernel.h>
> >> +#include <linux/kvm_para.h>
> >> #include <linux/syscore_ops.h>
> >> #include <asm/numa.h>
> >>
> >> @@ -24,15 +25,37 @@
> >> #define EIOINTC_REG_ISR 0x1800
> >> #define EIOINTC_REG_ROUTE 0x1c00
> >>
> >> +#define EXTIOI_VIRT_FEATURES 0x40000000
> >> +#define EXTIOI_HAS_VIRT_EXTENSION BIT(0)
> >> +#define EXTIOI_HAS_ENABLE_OPTION BIT(1)
> >> +#define EXTIOI_HAS_INT_ENCODE BIT(2)
> >> +#define EXTIOI_HAS_CPU_ENCODE BIT(3)
> >> +#define EXTIOI_VIRT_CONFIG 0x40000004
> >> +#define EXTIOI_ENABLE BIT(1)
> >> +#define EXTIOI_ENABLE_INT_ENCODE BIT(2)
> >> +#define EXTIOI_ENABLE_CPU_ENCODE BIT(3)
> > I think qemu hasn't release with v-eiointc? So we still have a chance
> > to modify qemu and this driver to simplify registers:
> It is already merged in qemu mainline, code is frozen and qemu 9.1 will
> release soon. Once it is merged in mainline and even if code is not
> frozen, I will modify it only if it is bug.
If Thomas think the register definition is OK, then I have no objections.
Huacai
> >
> > +#define EXTIOI_VIRT_FEATS 0x40000000
> > +#define EXTIOI_HAS_INT_ENCODE BIT(0)
> > +#define EXTIOI_HAS_CPU_ENCODE BIT(1)
> > +#define EXTIOI_VIRT_CONFIG 0x40000004
> > +#define EXTIOI_ENABLE_INT_ENCODE BIT(0)
> > +#define EXTIOI_ENABLE_CPU_ENCODE BIT(1)
> >
> >> +
> >> #define VEC_REG_COUNT 4
> >> #define VEC_COUNT_PER_REG 64
> >> #define VEC_COUNT (VEC_REG_COUNT * VEC_COUNT_PER_REG)
> >> #define VEC_REG_IDX(irq_id) ((irq_id) / VEC_COUNT_PER_REG)
> >> #define VEC_REG_BIT(irq_id) ((irq_id) % VEC_COUNT_PER_REG)
> >> #define EIOINTC_ALL_ENABLE 0xffffffff
> >> +#define EIOINTC_ALL_ENABLE_VEC_MASK(vector) (EIOINTC_ALL_ENABLE & ~BIT(vector & 0x1F))
> >> +#define EIOINTC_REG_ENABLE_VEC(vector) (EIOINTC_REG_ENABLE + ((vector >> 5) << 2))
> >> +#define EIOINTC_USE_CPU_ENCODE BIT(0)
> >>
> >> #define MAX_EIO_NODES (NR_CPUS / CORES_PER_EIO_NODE)
> >>
> >> +/*
> >> + * Routing registers are 32bit, and there is 8-bit route setting for every
> >> + * interrupt vector. So one Route register contains four vectors routing
> >> + * information.
> >> + */
> >> +#define EIOINTC_REG_ROUTE_VEC(vector) (EIOINTC_REG_ROUTE + (vector & ~0x03))
> >> +#define EIOINTC_REG_ROUTE_VEC_SHIFT(vector) ((vector & 0x03) << 3)
> >> +#define EIOINTC_REG_ROUTE_VEC_MASK(vector) (0xff << EIOINTC_REG_ROUTE_VEC_SHIFT(vector))
> >> +
> >> static int nr_pics;
> >>
> >> struct eiointc_priv {
> >> @@ -42,6 +65,7 @@ struct eiointc_priv {
> >> cpumask_t cpuspan_map;
> >> struct fwnode_handle *domain_handle;
> >> struct irq_domain *eiointc_domain;
> >> + int flags;
> >> };
> >>
> >> static struct eiointc_priv *eiointc_priv[MAX_IO_PICS];
> >> @@ -57,7 +81,13 @@ static void eiointc_enable(void)
> >>
> >> static int cpu_to_eio_node(int cpu)
> >> {
> >> - return cpu_logical_map(cpu) / CORES_PER_EIO_NODE;
> >> + int cores;
> >> +
> >> + if (kvm_para_has_feature(KVM_FEATURE_VIRT_EXTIOI))
> >> + cores = CORES_PER_VEIO_NODE;
> >> + else
> >> + cores = CORES_PER_EIO_NODE;
> >> + return cpu_logical_map(cpu) / cores;
> >> }
> >>
> >> #ifdef CONFIG_SMP
> >> @@ -89,6 +119,16 @@ static void eiointc_set_irq_route(int pos, unsigned int cpu, unsigned int mnode,
> >>
> >> static DEFINE_RAW_SPINLOCK(affinity_lock);
> >>
> >> +static void virt_extioi_set_irq_route(unsigned int vector, unsigned int cpu)
> > Use veiointc_set_irq_route() can keep consistency with real machines.
> yes, veiointc_set_irq_route will be added here.
>
> >
> >> +{
> >> + unsigned long reg = EIOINTC_REG_ROUTE_VEC(vector);
> >> + u32 data = iocsr_read32(reg);
> >> +
> >> + data &= ~EIOINTC_REG_ROUTE_VEC_MASK(vector);
> >> + data |= cpu_logical_map(cpu) << EIOINTC_REG_ROUTE_VEC_SHIFT(vector);
> >> + iocsr_write32(data, reg);
> >> +}
> >> +
> >> static int eiointc_set_irq_affinity(struct irq_data *d, const struct cpumask *affinity, bool force)
> >> {
> >> unsigned int cpu;
> >> @@ -105,18 +145,24 @@ static int eiointc_set_irq_affinity(struct irq_data *d, const struct cpumask *af
> >> }
> >>
> >> vector = d->hwirq;
> >> - regaddr = EIOINTC_REG_ENABLE + ((vector >> 5) << 2);
> >> -
> >> - /* Mask target vector */
> >> - csr_any_send(regaddr, EIOINTC_ALL_ENABLE & (~BIT(vector & 0x1F)),
> >> - 0x0, priv->node * CORES_PER_EIO_NODE);
> >> -
> >> - /* Set route for target vector */
> >> - eiointc_set_irq_route(vector, cpu, priv->node, &priv->node_map);
> >> -
> >> - /* Unmask target vector */
> >> - csr_any_send(regaddr, EIOINTC_ALL_ENABLE,
> >> - 0x0, priv->node * CORES_PER_EIO_NODE);
> >> + regaddr = EIOINTC_REG_ENABLE_VEC(vector);
> >> +
> >> + if (priv->flags & EIOINTC_USE_CPU_ENCODE) {
> >> + iocsr_write32(EIOINTC_ALL_ENABLE_VEC_MASK(vector), regaddr);
> >> + virt_extioi_set_irq_route(vector, cpu);
> >> + iocsr_write32(EIOINTC_ALL_ENABLE, regaddr);
> > We can add the same comments as real machines here.
> Similar comments will be added, otherwise there will be duplicated comments.
>
> Hope to this the last review comments about grammar and name -:), else I
> will think it is on deliberate. we know every architecture has multiple
> maintainers to prevent arbitrary except LoongArch.
>
> Regards
> Bibo Mao
>
> >
> > Huacai
> >
> >> + } else {
> >> + /* Mask target vector */
> >> + csr_any_send(regaddr, EIOINTC_ALL_ENABLE_VEC_MASK(vector),
> >> + 0x0, priv->node * CORES_PER_EIO_NODE);
> >> +
> >> + /* Set route for target vector */
> >> + eiointc_set_irq_route(vector, cpu, priv->node, &priv->node_map);
> >> +
> >> + /* Unmask target vector */
> >> + csr_any_send(regaddr, EIOINTC_ALL_ENABLE,
> >> + 0x0, priv->node * CORES_PER_EIO_NODE);
> >> + }
> >>
> >> irq_data_update_effective_affinity(d, cpumask_of(cpu));
> >>
> >> @@ -140,17 +186,23 @@ static int eiointc_index(int node)
> >>
> >> static int eiointc_router_init(unsigned int cpu)
> >> {
> >> - int i, bit;
> >> - uint32_t data;
> >> - uint32_t node = cpu_to_eio_node(cpu);
> >> - int index = eiointc_index(node);
> >> + int i, bit, cores, index, node;
> >> + unsigned int data;
> >> +
> >> + node = cpu_to_eio_node(cpu);
> >> + index = eiointc_index(node);
> >>
> >> if (index < 0) {
> >> pr_err("Error: invalid nodemap!\n");
> >> - return -1;
> >> + return -EINVAL;
> >> }
> >>
> >> - if ((cpu_logical_map(cpu) % CORES_PER_EIO_NODE) == 0) {
> >> + if (eiointc_priv[index]->flags & EIOINTC_USE_CPU_ENCODE)
> >> + cores = CORES_PER_VEIO_NODE;
> >> + else
> >> + cores = CORES_PER_EIO_NODE;
> >> +
> >> + if ((cpu_logical_map(cpu) % cores) == 0) {
> >> eiointc_enable();
> >>
> >> for (i = 0; i < eiointc_priv[0]->vec_count / 32; i++) {
> >> @@ -166,7 +218,9 @@ static int eiointc_router_init(unsigned int cpu)
> >>
> >> for (i = 0; i < eiointc_priv[0]->vec_count / 4; i++) {
> >> /* Route to Node-0 Core-0 */
> >> - if (index == 0)
> >> + if (eiointc_priv[index]->flags & EIOINTC_USE_CPU_ENCODE)
> >> + bit = cpu_logical_map(0);
> >> + else if (index == 0)
> >> bit = BIT(cpu_logical_map(0));
> >> else
> >> bit = (eiointc_priv[index]->node << 4) | 1;
> >> @@ -370,7 +424,7 @@ static int __init acpi_cascade_irqdomain_init(void)
> >> static int __init eiointc_init(struct eiointc_priv *priv, int parent_irq,
> >> u64 node_map)
> >> {
> >> - int i;
> >> + int i, val;
> >>
> >> node_map = node_map ? node_map : -1ULL;
> >> for_each_possible_cpu(i) {
> >> @@ -390,6 +444,16 @@ static int __init eiointc_init(struct eiointc_priv *priv, int parent_irq,
> >> return -ENOMEM;
> >> }
> >>
> >> + if (kvm_para_has_feature(KVM_FEATURE_VIRT_EXTIOI)) {
> >> + val = iocsr_read32(EXTIOI_VIRT_FEATURES);
> >> + if (val & EXTIOI_HAS_CPU_ENCODE) {
> >> + val = iocsr_read32(EXTIOI_VIRT_CONFIG);
> >> + val |= EXTIOI_ENABLE_CPU_ENCODE;
> >> + iocsr_write32(val, EXTIOI_VIRT_CONFIG);
> >> + priv->flags = EIOINTC_USE_CPU_ENCODE;
> >> + }
> >> + }
> >> +
> >> eiointc_priv[nr_pics++] = priv;
> >> eiointc_router_init(0);
> >> irq_set_chained_handler_and_data(parent_irq, eiointc_irq_dispatch, priv);
> >> --
> >> 2.39.3
> >>
>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v7 3/3] irqchip/loongson-eiointc: Add extioi virt extension support
2024-08-29 12:29 ` Huacai Chen
@ 2024-08-29 14:04 ` Thomas Gleixner
0 siblings, 0 replies; 8+ messages in thread
From: Thomas Gleixner @ 2024-08-29 14:04 UTC (permalink / raw)
To: Huacai Chen, maobibo
Cc: Tianrui Zhao, WANG Xuerui, kvm, loongarch, linux-kernel,
virtualization, x86, Song Gao
On Thu, Aug 29 2024 at 20:29, Huacai Chen wrote:
> On Thu, Aug 29, 2024 at 9:46 AM maobibo <maobibo@loongson.cn> wrote:
>> > I think qemu hasn't release with v-eiointc? So we still have a chance
>> > to modify qemu and this driver to simplify registers:
>> It is already merged in qemu mainline, code is frozen and qemu 9.1 will
>> release soon. Once it is merged in mainline and even if code is not
>> frozen, I will modify it only if it is bug.
> If Thomas think the register definition is OK, then I have no objections.
No objections
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2024-08-29 14:04 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-08-23 6:39 [PATCH v7 0/3] Add extioi virt extension support Bibo Mao
2024-08-23 6:39 ` [PATCH v7 1/3] LoongArch: KVM: Enable paravirt feature control from VMM Bibo Mao
2024-08-23 6:39 ` [PATCH v7 2/3] LoongArch: KVM: Implement function kvm_para_has_feature Bibo Mao
2024-08-23 6:39 ` [PATCH v7 3/3] irqchip/loongson-eiointc: Add extioi virt extension support Bibo Mao
2024-08-28 14:27 ` Huacai Chen
2024-08-29 1:46 ` maobibo
2024-08-29 12:29 ` Huacai Chen
2024-08-29 14:04 ` Thomas Gleixner
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).