[PATCH 00/15] hw/loongarch/virt: Add kvm_irqchip_in_kernel support

[PATCH 00/15] hw/loongarch/virt: Add kvm_irqchip_in_kernel support

[Bibo Mao]

To enable kvm_irqchip_in_kernel option on LoongArch virt machine
platform, property irqchip-in-kernel is added on irqchips including
ExtIOI, IPI, PCH_PCI and PCH_MSI irqchip.

If property irqchip-in-kernel is set, there is special operations with
irqchips in such fields:
  1. During irqchip object realization, kvm_create_device() is used here
     to create irqchip in KVM kernel.
  2. Add pre_save and post_load function, where register states can be
     get and set from KVM kernel.
  3. Interrupt injection to kernel, IRQ line intterupt is injected with
     API kvm_set_irq() and MSI intterrupt is injected with API
     kvm_irqchip_send_msi().
     
Bibo Mao (15):
  hw/intc/loongarch_extioi: Add irqchip-in-kernel property
  hw/intc/loongarch_extioi: Add irqchip-in-kernel realize function
  hw/intc/loongarch_extioi: Add irqchip-in-kernel save/restore function
  hw/intc/loongarch_ipi: Add irqchip-in-kernel property
  hw/intc/loongarch_ipi: Add irqchip-in-kernel realize function
  hw/intc/loongson_ipi: Add load and save interface with ipi_common
    class
  hw/intc/loongarch_ipi: Add irqchip-in-kernel save/restore function
  hw/intc/loongarch_pch_msi: Add irqchip-in-kernel property
  hw/intc/loongarch_pch_msi: Inject MSI interrupt to kernel
  hw/intc/loongarch_pch: Add irqchip-in-kernel property
  hw/intc/loongarch_pch: Add irqchip-in-kernel realize function
  hw/intc/loongarch_pch: Add irqchip-in-kernel save/restore function
  hw/intc/loongarch_pch: Inject irq line interrupt to kernel
  target/loongarch: Report error with split kernel_irqchip option
  hw/loongarch/virt: Add kvm_irqchip_in_kernel support

 hw/intc/loongarch_extioi.c             |  35 ++++++
 hw/intc/loongarch_extioi_kvm.c         | 146 +++++++++++++++++++++++++
 hw/intc/loongarch_ipi.c                |  41 +++++++
 hw/intc/loongarch_ipi_kvm.c            |  83 ++++++++++++++
 hw/intc/loongarch_pch_msi.c            |  17 +++
 hw/intc/loongarch_pch_pic.c            |  50 +++++++++
 hw/intc/loongarch_pic_kvm.c            |  96 ++++++++++++++++
 hw/intc/loongson_ipi_common.c          |  28 +++++
 hw/intc/meson.build                    |   6 +
 hw/loongarch/virt.c                    |  15 +++
 include/hw/intc/loongarch_extioi.h     |   6 +
 include/hw/intc/loongarch_ipi.h        |   6 +
 include/hw/intc/loongarch_pch_msi.h    |   1 +
 include/hw/intc/loongarch_pch_pic.h    |   6 +
 include/hw/intc/loongarch_pic_common.h |   1 +
 include/hw/intc/loongson_ipi_common.h  |   2 +
 target/loongarch/cpu.h                 |   1 +
 target/loongarch/kvm/kvm.c             |  23 +++-
 18 files changed, 562 insertions(+), 1 deletion(-)
 create mode 100644 hw/intc/loongarch_extioi_kvm.c
 create mode 100644 hw/intc/loongarch_ipi_kvm.c
 create mode 100644 hw/intc/loongarch_pic_kvm.c

-- 
2.39.3

[PATCH 01/15] hw/intc/loongarch_extioi: Add irqchip-in-kernel property

[Bibo Mao]

With ExtIOI irqchip, property irqchip-in-kernel is added to indicate
whether feature irqchip_in_kernel is supported or not. This property
can be enabled only if it works in KVM mode.

Signed-off-by: Bibo Mao <maobibo@loongson.cn>
---
 hw/intc/loongarch_extioi.c         | 14 ++++++++++++++
 include/hw/intc/loongarch_extioi.h |  1 +
 2 files changed, 15 insertions(+)

diff --git a/hw/intc/loongarch_extioi.c b/hw/intc/loongarch_extioi.c
index 7c38c4c9b7..a737d49b12 100644
--- a/hw/intc/loongarch_extioi.c
+++ b/hw/intc/loongarch_extioi.c
@@ -11,7 +11,9 @@
 #include "qapi/error.h"
 #include "hw/irq.h"
 #include "hw/loongarch/virt.h"
+#include "hw/qdev-properties.h"
 #include "system/address-spaces.h"
+#include "system/kvm.h"
 #include "hw/intc/loongarch_extioi.h"
 #include "trace.h"
 
@@ -341,6 +343,7 @@ static void loongarch_extioi_realize(DeviceState *dev, Error **errp)
 {
     LoongArchExtIOICommonState *s = LOONGARCH_EXTIOI_COMMON(dev);
     LoongArchExtIOIClass *lec = LOONGARCH_EXTIOI_GET_CLASS(dev);
+    LoongArchExtIOIState *les = LOONGARCH_EXTIOI(dev);
     SysBusDevice *sbd = SYS_BUS_DEVICE(dev);
     Error *local_err = NULL;
     int i;
@@ -351,6 +354,11 @@ static void loongarch_extioi_realize(DeviceState *dev, Error **errp)
         return;
     }
 
+    if (les->irqchip_in_kernel && !kvm_enabled()) {
+        error_setg(errp, "ExtIOI irqchip_in_kernel works only in kvm mode");
+        return;
+    }
+
     for (i = 0; i < EXTIOI_IRQS; i++) {
         sysbus_init_irq(sbd, &s->irq[i]);
     }
@@ -403,6 +411,11 @@ static int vmstate_extioi_post_load(void *opaque, int version_id)
     return 0;
 }
 
+static const Property loongarch_extioi_properties[] = {
+    DEFINE_PROP_BOOL("irqchip-in-kernel", LoongArchExtIOIState,
+                     irqchip_in_kernel, false),
+};
+
 static void loongarch_extioi_class_init(ObjectClass *klass, const void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
@@ -416,6 +429,7 @@ static void loongarch_extioi_class_init(ObjectClass *klass, const void *data)
                                       &lec->parent_unrealize);
     resettable_class_set_parent_phases(rc, NULL, loongarch_extioi_reset_hold,
                                        NULL, &lec->parent_phases);
+    device_class_set_props(dc, loongarch_extioi_properties);
     lecc->post_load = vmstate_extioi_post_load;
 }
 
diff --git a/include/hw/intc/loongarch_extioi.h b/include/hw/intc/loongarch_extioi.h
index 4a6ae903e9..c1d79d0a40 100644
--- a/include/hw/intc/loongarch_extioi.h
+++ b/include/hw/intc/loongarch_extioi.h
@@ -15,6 +15,7 @@ OBJECT_DECLARE_TYPE(LoongArchExtIOIState, LoongArchExtIOIClass, LOONGARCH_EXTIOI
 
 struct LoongArchExtIOIState {
     LoongArchExtIOICommonState parent_obj;
+    bool irqchip_in_kernel;
 };
 
 struct LoongArchExtIOIClass {
-- 
2.39.3

[PATCH 02/15] hw/intc/loongarch_extioi: Add irqchip-in-kernel realize function

[Bibo Mao]

Function kvm_loongarch_extioi_realize() is added if irqchip-in-kernel
property is enabled. It is to notify KVM kernel to create ExtIOI device
in kernel mode.

Signed-off-by: Bibo Mao <maobibo@loongson.cn>
---
 hw/intc/loongarch_extioi.c         |  4 +++
 hw/intc/loongarch_extioi_kvm.c     | 46 ++++++++++++++++++++++++++++++
 hw/intc/meson.build                |  2 ++
 include/hw/intc/loongarch_extioi.h |  3 ++
 4 files changed, 55 insertions(+)
 create mode 100644 hw/intc/loongarch_extioi_kvm.c

diff --git a/hw/intc/loongarch_extioi.c b/hw/intc/loongarch_extioi.c
index a737d49b12..854f54684b 100644
--- a/hw/intc/loongarch_extioi.c
+++ b/hw/intc/loongarch_extioi.c
@@ -376,6 +376,10 @@ static void loongarch_extioi_realize(DeviceState *dev, Error **errp)
     } else {
         s->status |= BIT(EXTIOI_ENABLE);
     }
+
+    if (kvm_enabled() && les->irqchip_in_kernel) {
+        kvm_loongarch_extioi_realize(dev, errp);
+    }
 }
 
 static void loongarch_extioi_unrealize(DeviceState *dev)
diff --git a/hw/intc/loongarch_extioi_kvm.c b/hw/intc/loongarch_extioi_kvm.c
new file mode 100644
index 0000000000..833ec856ee
--- /dev/null
+++ b/hw/intc/loongarch_extioi_kvm.c
@@ -0,0 +1,46 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * LoongArch EXTIOI interrupt kvm support
+ *
+ * Copyright (C) 2025 Loongson Technology Corporation Limited
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/typedefs.h"
+#include "hw/intc/loongarch_extioi.h"
+#include "linux/kvm.h"
+#include "qapi/error.h"
+#include "system/kvm.h"
+
+void kvm_loongarch_extioi_realize(DeviceState *dev, Error **errp)
+{
+    LoongArchExtIOICommonState *lecs = LOONGARCH_EXTIOI_COMMON(dev);
+    LoongArchExtIOIState *les = LOONGARCH_EXTIOI(dev);
+    int ret;
+
+    ret = kvm_create_device(kvm_state, KVM_DEV_TYPE_LOONGARCH_EIOINTC, false);
+    if (ret < 0) {
+        fprintf(stderr, "create KVM_LOONGARCH_EIOINTC failed: %s\n",
+                strerror(-ret));
+        abort();
+    }
+
+    les->dev_fd = ret;
+    ret = kvm_device_access(les->dev_fd, KVM_DEV_LOONGARCH_EXTIOI_GRP_CTRL,
+                            KVM_DEV_LOONGARCH_EXTIOI_CTRL_INIT_NUM_CPU,
+                            &lecs->num_cpu, true, NULL);
+    if (ret < 0) {
+        fprintf(stderr, "KVM_LOONGARCH_EXTIOI_INIT_NUM_CPU failed: %s\n",
+                strerror(-ret));
+        abort();
+    }
+
+    ret = kvm_device_access(les->dev_fd, KVM_DEV_LOONGARCH_EXTIOI_GRP_CTRL,
+                            KVM_DEV_LOONGARCH_EXTIOI_CTRL_INIT_FEATURE,
+                            &lecs->features, true, NULL);
+    if (ret < 0) {
+        fprintf(stderr, "KVM_LOONGARCH_EXTIOI_INIT_FEATURE failed: %s\n",
+                strerror(-ret));
+        abort();
+    }
+}
diff --git a/hw/intc/meson.build b/hw/intc/meson.build
index 602da304b0..70e7548c52 100644
--- a/hw/intc/meson.build
+++ b/hw/intc/meson.build
@@ -74,3 +74,5 @@ specific_ss.add(when: 'CONFIG_LOONGARCH_IPI', if_true: files('loongarch_ipi.c'))
 specific_ss.add(when: 'CONFIG_LOONGARCH_PCH_PIC', if_true: files('loongarch_pch_pic.c', 'loongarch_pic_common.c'))
 specific_ss.add(when: 'CONFIG_LOONGARCH_PCH_MSI', if_true: files('loongarch_pch_msi.c'))
 specific_ss.add(when: 'CONFIG_LOONGARCH_EXTIOI', if_true: files('loongarch_extioi.c', 'loongarch_extioi_common.c'))
+specific_ss.add(when: ['CONFIG_KVM', 'CONFIG_LOONGARCH_EXTIOI'],
+               if_true: files('loongarch_extioi_kvm.c'))
diff --git a/include/hw/intc/loongarch_extioi.h b/include/hw/intc/loongarch_extioi.h
index c1d79d0a40..848a01f52c 100644
--- a/include/hw/intc/loongarch_extioi.h
+++ b/include/hw/intc/loongarch_extioi.h
@@ -16,6 +16,7 @@ OBJECT_DECLARE_TYPE(LoongArchExtIOIState, LoongArchExtIOIClass, LOONGARCH_EXTIOI
 struct LoongArchExtIOIState {
     LoongArchExtIOICommonState parent_obj;
     bool irqchip_in_kernel;
+    int dev_fd;
 };
 
 struct LoongArchExtIOIClass {
@@ -26,4 +27,6 @@ struct LoongArchExtIOIClass {
     ResettablePhases parent_phases;
 };
 
+void kvm_loongarch_extioi_realize(DeviceState *dev, Error **errp);
+
 #endif /* LOONGARCH_EXTIOI_H */
-- 
2.39.3

[PATCH 03/15] hw/intc/loongarch_extioi: Add irqchip-in-kernel save/restore function

[Bibo Mao]

Add save and store funtction if irqchip-in-kernel property is enabled,
it is to get/set ExtIOI irqchip state from KVM kernel.

Signed-off-by: Bibo Mao <maobibo@loongson.cn>
---
 hw/intc/loongarch_extioi.c         |  17 +++++
 hw/intc/loongarch_extioi_kvm.c     | 100 +++++++++++++++++++++++++++++
 include/hw/intc/loongarch_extioi.h |   2 +
 3 files changed, 119 insertions(+)

diff --git a/hw/intc/loongarch_extioi.c b/hw/intc/loongarch_extioi.c
index 854f54684b..e81caf430c 100644
--- a/hw/intc/loongarch_extioi.c
+++ b/hw/intc/loongarch_extioi.c
@@ -398,9 +398,21 @@ static void loongarch_extioi_reset_hold(Object *obj, ResetType type)
     }
 }
 
+static int vmstate_extioi_pre_save(void *opaque)
+{
+    LoongArchExtIOIState *les = LOONGARCH_EXTIOI(opaque);
+
+    if (kvm_enabled() && les->irqchip_in_kernel) {
+        return kvm_loongarch_extioi_pre_save(opaque);
+    }
+
+    return 0;
+}
+
 static int vmstate_extioi_post_load(void *opaque, int version_id)
 {
     LoongArchExtIOICommonState *s = LOONGARCH_EXTIOI_COMMON(opaque);
+    LoongArchExtIOIState *les = LOONGARCH_EXTIOI(opaque);
     int i, start_irq;
 
     for (i = 0; i < (EXTIOI_IRQS / 4); i++) {
@@ -412,6 +424,10 @@ static int vmstate_extioi_post_load(void *opaque, int version_id)
         extioi_update_sw_ipmap(s, i, s->ipmap[i]);
     }
 
+    if (kvm_enabled() && les->irqchip_in_kernel) {
+        return kvm_loongarch_extioi_post_load(opaque, version_id);
+    }
+
     return 0;
 }
 
@@ -434,6 +450,7 @@ static void loongarch_extioi_class_init(ObjectClass *klass, const void *data)
     resettable_class_set_parent_phases(rc, NULL, loongarch_extioi_reset_hold,
                                        NULL, &lec->parent_phases);
     device_class_set_props(dc, loongarch_extioi_properties);
+    lecc->pre_save  = vmstate_extioi_pre_save;
     lecc->post_load = vmstate_extioi_post_load;
 }
 
diff --git a/hw/intc/loongarch_extioi_kvm.c b/hw/intc/loongarch_extioi_kvm.c
index 833ec856ee..9c5649dd1a 100644
--- a/hw/intc/loongarch_extioi_kvm.c
+++ b/hw/intc/loongarch_extioi_kvm.c
@@ -12,6 +12,106 @@
 #include "qapi/error.h"
 #include "system/kvm.h"
 
+static void kvm_extioi_access_regs(int fd, uint64_t addr,
+                                       void *val, bool write)
+{
+    kvm_device_access(fd, KVM_DEV_LOONGARCH_EXTIOI_GRP_REGS,
+                      addr, val, write, &error_abort);
+}
+
+static void kvm_extioi_access_sw_status(int fd, uint64_t addr,
+                                       void *val, bool write)
+{
+    kvm_device_access(fd, KVM_DEV_LOONGARCH_EXTIOI_GRP_SW_STATUS,
+                      addr, val, write, &error_abort);
+}
+
+static void kvm_extioi_save_load_sw_status(void *opaque, bool write)
+{
+    LoongArchExtIOICommonState *lecs = LOONGARCH_EXTIOI_COMMON(opaque);
+    LoongArchExtIOIState *les = LOONGARCH_EXTIOI(opaque);
+    int fd = les->dev_fd;
+    int addr;
+
+    addr = KVM_DEV_LOONGARCH_EXTIOI_SW_STATUS_NUM_CPU;
+    kvm_extioi_access_sw_status(fd, addr, &lecs->num_cpu, write);
+
+    addr = KVM_DEV_LOONGARCH_EXTIOI_SW_STATUS_FEATURE;
+    kvm_extioi_access_sw_status(fd, addr, &lecs->features, write);
+
+    addr = KVM_DEV_LOONGARCH_EXTIOI_SW_STATUS_STATE;
+    kvm_extioi_access_sw_status(fd, addr, &lecs->status, write);
+}
+
+static void kvm_extioi_save_load_regs(void *opaque, bool write)
+{
+    LoongArchExtIOICommonState *lecs = LOONGARCH_EXTIOI_COMMON(opaque);
+    LoongArchExtIOIState *les = LOONGARCH_EXTIOI(opaque);
+    int fd = les->dev_fd;
+    int addr, offset, cpuid;
+
+    for (addr = EXTIOI_NODETYPE_START; addr < EXTIOI_NODETYPE_END; addr += 4) {
+        offset = (addr - EXTIOI_NODETYPE_START) / 4;
+        kvm_extioi_access_regs(fd, addr, &lecs->nodetype[offset], write);
+    }
+
+    for (addr = EXTIOI_IPMAP_START; addr < EXTIOI_IPMAP_END; addr += 4) {
+        offset = (addr - EXTIOI_IPMAP_START) / 4;
+        kvm_extioi_access_regs(fd, addr, &lecs->ipmap[offset], write);
+    }
+
+    for (addr = EXTIOI_ENABLE_START; addr < EXTIOI_ENABLE_END; addr += 4) {
+        offset = (addr - EXTIOI_ENABLE_START) / 4;
+        kvm_extioi_access_regs(fd, addr, &lecs->enable[offset], write);
+    }
+
+    for (addr = EXTIOI_BOUNCE_START; addr < EXTIOI_BOUNCE_END; addr += 4) {
+        offset = (addr - EXTIOI_BOUNCE_START) / 4;
+        kvm_extioi_access_regs(fd, addr, &lecs->bounce[offset], write);
+    }
+
+    for (addr = EXTIOI_ISR_START; addr < EXTIOI_ISR_END; addr += 4) {
+        offset = (addr - EXTIOI_ISR_START) / 4;
+        kvm_extioi_access_regs(fd, addr, &lecs->isr[offset], write);
+    }
+
+    for (addr = EXTIOI_COREMAP_START; addr < EXTIOI_COREMAP_END; addr += 4) {
+        offset = (addr - EXTIOI_COREMAP_START) / 4;
+        kvm_extioi_access_regs(fd, addr, &lecs->coremap[offset], write);
+    }
+
+    for (cpuid = 0; cpuid < lecs->num_cpu; cpuid++) {
+        for (addr = EXTIOI_COREISR_START;
+             addr < EXTIOI_COREISR_END; addr += 4) {
+            offset = (addr - EXTIOI_COREISR_START) / 4;
+            addr = (cpuid << 16) | addr;
+            kvm_extioi_access_regs(fd, addr,
+                                   &lecs->cpu[cpuid].coreisr[offset], write);
+        }
+    }
+}
+
+int kvm_loongarch_extioi_pre_save(void *opaque)
+{
+    kvm_extioi_save_load_regs(opaque, false);
+    kvm_extioi_save_load_sw_status(opaque, false);
+    return 0;
+}
+
+int kvm_loongarch_extioi_post_load(void *opaque, int version_id)
+{
+    LoongArchExtIOIState *les = LOONGARCH_EXTIOI(opaque);
+    int fd = les->dev_fd;
+
+    kvm_extioi_save_load_regs(opaque, true);
+    kvm_extioi_save_load_sw_status(opaque, true);
+
+    kvm_device_access(fd, KVM_DEV_LOONGARCH_EXTIOI_GRP_CTRL,
+                      KVM_DEV_LOONGARCH_EXTIOI_CTRL_LOAD_FINISHED,
+                      NULL, true, &error_abort);
+    return 0;
+}
+
 void kvm_loongarch_extioi_realize(DeviceState *dev, Error **errp)
 {
     LoongArchExtIOICommonState *lecs = LOONGARCH_EXTIOI_COMMON(dev);
diff --git a/include/hw/intc/loongarch_extioi.h b/include/hw/intc/loongarch_extioi.h
index 848a01f52c..69f1c7aedc 100644
--- a/include/hw/intc/loongarch_extioi.h
+++ b/include/hw/intc/loongarch_extioi.h
@@ -28,5 +28,7 @@ struct LoongArchExtIOIClass {
 };
 
 void kvm_loongarch_extioi_realize(DeviceState *dev, Error **errp);
+int kvm_loongarch_extioi_pre_save(void *opaque);
+int kvm_loongarch_extioi_post_load(void *opaque, int version_id);
 
 #endif /* LOONGARCH_EXTIOI_H */
-- 
2.39.3

[PATCH 04/15] hw/intc/loongarch_ipi: Add irqchip-in-kernel property

[Bibo Mao]

With IPI irqchip, property irqchip-in-kernel is added to indicate whether
feature irqchip_in_kernel is supported or not. This property can be
enabled only if it works in KVM mode.

Signed-off-by: Bibo Mao <maobibo@loongson.cn>
---
 hw/intc/loongarch_ipi.c         | 13 +++++++++++++
 include/hw/intc/loongarch_ipi.h |  1 +
 2 files changed, 14 insertions(+)

diff --git a/hw/intc/loongarch_ipi.c b/hw/intc/loongarch_ipi.c
index 74372a2039..64e0250958 100644
--- a/hw/intc/loongarch_ipi.c
+++ b/hw/intc/loongarch_ipi.c
@@ -11,6 +11,7 @@
 #include "qapi/error.h"
 #include "hw/intc/loongarch_ipi.h"
 #include "hw/qdev-properties.h"
+#include "system/kvm.h"
 #include "target/loongarch/cpu.h"
 
 static AddressSpace *get_iocsr_as(CPUState *cpu)
@@ -69,6 +70,7 @@ static void loongarch_ipi_realize(DeviceState *dev, Error **errp)
 {
     LoongsonIPICommonState *lics = LOONGSON_IPI_COMMON(dev);
     LoongarchIPIClass *lic = LOONGARCH_IPI_GET_CLASS(dev);
+    LoongarchIPIState *lis = LOONGARCH_IPI(dev);
     MachineState *machine = MACHINE(qdev_get_machine());
     MachineClass *mc = MACHINE_GET_CLASS(machine);
     const CPUArchIdList *id_list;
@@ -81,6 +83,11 @@ static void loongarch_ipi_realize(DeviceState *dev, Error **errp)
         return;
     }
 
+    if (lis->irqchip_in_kernel && !kvm_enabled()) {
+        error_setg(errp, "IPI irqchip_in_kernel works only in kvm mode");
+        return;
+    }
+
     assert(mc->possible_cpu_arch_ids);
     id_list = mc->possible_cpu_arch_ids(machine);
     lics->num_cpu = id_list->len;
@@ -166,6 +173,11 @@ static void loongarch_ipi_cpu_unplug(HotplugHandler *hotplug_dev,
     core->cpu = NULL;
 }
 
+static const Property loongarch_ipi_properties[] = {
+    DEFINE_PROP_BOOL("irqchip-in-kernel", LoongarchIPIState,
+                     irqchip_in_kernel, false),
+};
+
 static void loongarch_ipi_class_init(ObjectClass *klass, const void *data)
 {
     LoongsonIPICommonClass *licc = LOONGSON_IPI_COMMON_CLASS(klass);
@@ -178,6 +190,7 @@ static void loongarch_ipi_class_init(ObjectClass *klass, const void *data)
                                     &lic->parent_realize);
     resettable_class_set_parent_phases(rc, NULL, loongarch_ipi_reset_hold,
                                        NULL, &lic->parent_phases);
+    device_class_set_props(dc, loongarch_ipi_properties);
     licc->get_iocsr_as = get_iocsr_as;
     licc->cpu_by_arch_id = loongarch_cpu_by_arch_id;
     hc->plug = loongarch_ipi_cpu_plug;
diff --git a/include/hw/intc/loongarch_ipi.h b/include/hw/intc/loongarch_ipi.h
index a7c6bf85d3..d4eebd9a4d 100644
--- a/include/hw/intc/loongarch_ipi.h
+++ b/include/hw/intc/loongarch_ipi.h
@@ -16,6 +16,7 @@ OBJECT_DECLARE_TYPE(LoongarchIPIState, LoongarchIPIClass, LOONGARCH_IPI)
 
 struct LoongarchIPIState {
     LoongsonIPICommonState parent_obj;
+    bool irqchip_in_kernel;
 };
 
 struct LoongarchIPIClass {
-- 
2.39.3

[PATCH 05/15] hw/intc/loongarch_ipi: Add irqchip-in-kernel realize function

[Bibo Mao]

Function kvm_loongarch_ipi_realize() is added if irqchip-in-kernel
property is enabled. It is to notify KVM kernel to create IPI device
in kernel mode.

Signed-off-by: Bibo Mao <maobibo@loongson.cn>
---
 hw/intc/loongarch_ipi.c         |  4 ++++
 hw/intc/loongarch_ipi_kvm.c     | 27 +++++++++++++++++++++++++++
 hw/intc/meson.build             |  2 ++
 include/hw/intc/loongarch_ipi.h |  3 +++
 4 files changed, 36 insertions(+)
 create mode 100644 hw/intc/loongarch_ipi_kvm.c

diff --git a/hw/intc/loongarch_ipi.c b/hw/intc/loongarch_ipi.c
index 64e0250958..5e240382ad 100644
--- a/hw/intc/loongarch_ipi.c
+++ b/hw/intc/loongarch_ipi.c
@@ -98,6 +98,10 @@ static void loongarch_ipi_realize(DeviceState *dev, Error **errp)
         lics->cpu[i].ipi = lics;
         qdev_init_gpio_out(dev, &lics->cpu[i].irq, 1);
     }
+
+    if (kvm_enabled() && lis->irqchip_in_kernel) {
+        kvm_loongarch_ipi_realize(dev, errp);
+    }
 }
 
 static void loongarch_ipi_reset_hold(Object *obj, ResetType type)
diff --git a/hw/intc/loongarch_ipi_kvm.c b/hw/intc/loongarch_ipi_kvm.c
new file mode 100644
index 0000000000..e8fcd3bd2f
--- /dev/null
+++ b/hw/intc/loongarch_ipi_kvm.c
@@ -0,0 +1,27 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * LoongArch IPI interrupt KVM support
+ *
+ * Copyright (C) 2025 Loongson Technology Corporation Limited
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "hw/intc/loongarch_ipi.h"
+#include "system/kvm.h"
+#include "target/loongarch/cpu.h"
+
+void kvm_loongarch_ipi_realize(DeviceState *dev, Error **errp)
+{
+    LoongarchIPIState *lis = LOONGARCH_IPI(dev);
+    int ret;
+
+    ret = kvm_create_device(kvm_state, KVM_DEV_TYPE_LOONGARCH_IPI, false);
+    if (ret < 0) {
+        fprintf(stderr, "IPI KVM_CREATE_DEVICE failed: %s\n",
+                strerror(-ret));
+        abort();
+    }
+
+    lis->dev_fd = ret;
+}
diff --git a/hw/intc/meson.build b/hw/intc/meson.build
index 70e7548c52..1cc999771d 100644
--- a/hw/intc/meson.build
+++ b/hw/intc/meson.build
@@ -71,6 +71,8 @@ specific_ss.add(when: 'CONFIG_M68K_IRQC', if_true: files('m68k_irqc.c'))
 specific_ss.add(when: 'CONFIG_LOONGSON_IPI_COMMON', if_true: files('loongson_ipi_common.c'))
 specific_ss.add(when: 'CONFIG_LOONGSON_IPI', if_true: files('loongson_ipi.c'))
 specific_ss.add(when: 'CONFIG_LOONGARCH_IPI', if_true: files('loongarch_ipi.c'))
+specific_ss.add(when: ['CONFIG_KVM', 'CONFIG_LOONGARCH_IPI'],
+                if_true: files('loongarch_ipi_kvm.c'))
 specific_ss.add(when: 'CONFIG_LOONGARCH_PCH_PIC', if_true: files('loongarch_pch_pic.c', 'loongarch_pic_common.c'))
 specific_ss.add(when: 'CONFIG_LOONGARCH_PCH_MSI', if_true: files('loongarch_pch_msi.c'))
 specific_ss.add(when: 'CONFIG_LOONGARCH_EXTIOI', if_true: files('loongarch_extioi.c', 'loongarch_extioi_common.c'))
diff --git a/include/hw/intc/loongarch_ipi.h b/include/hw/intc/loongarch_ipi.h
index d4eebd9a4d..26076d8062 100644
--- a/include/hw/intc/loongarch_ipi.h
+++ b/include/hw/intc/loongarch_ipi.h
@@ -17,6 +17,7 @@ OBJECT_DECLARE_TYPE(LoongarchIPIState, LoongarchIPIClass, LOONGARCH_IPI)
 struct LoongarchIPIState {
     LoongsonIPICommonState parent_obj;
     bool irqchip_in_kernel;
+    int  dev_fd;
 };
 
 struct LoongarchIPIClass {
@@ -25,4 +26,6 @@ struct LoongarchIPIClass {
     ResettablePhases parent_phases;
 };
 
+void kvm_loongarch_ipi_realize(DeviceState *dev, Error **errp);
+
 #endif
-- 
2.39.3

[PATCH 06/15] hw/intc/loongson_ipi: Add load and save interface with ipi_common class

[Bibo Mao]

Add pre_save and post_load interfaces with ipi_common class, here only
framework ipi_common adds these interfaces. The defailed implementation
is LoongArchIPI child device in later.

Signed-off-by: Bibo Mao <maobibo@loongson.cn>
---
 hw/intc/loongson_ipi_common.c         | 28 +++++++++++++++++++++++++++
 include/hw/intc/loongson_ipi_common.h |  2 ++
 2 files changed, 30 insertions(+)

diff --git a/hw/intc/loongson_ipi_common.c b/hw/intc/loongson_ipi_common.c
index f32661c40f..71e2e26ae1 100644
--- a/hw/intc/loongson_ipi_common.c
+++ b/hw/intc/loongson_ipi_common.c
@@ -277,10 +277,38 @@ static void loongson_ipi_common_unrealize(DeviceState *dev)
     g_free(s->cpu);
 }
 
+static int loongson_ipi_common_pre_save(void *opaque)
+{
+    IPICore *ipicore = (IPICore *)opaque;
+    LoongsonIPICommonState *s = ipicore->ipi;
+    LoongsonIPICommonClass *licc = LOONGSON_IPI_COMMON_GET_CLASS(s);
+
+    if (licc->pre_save) {
+        return licc->pre_save(s);
+    }
+
+    return 0;
+}
+
+static int loongson_ipi_common_post_load(void *opaque, int version_id)
+{
+    IPICore *ipicore = (IPICore *)opaque;
+    LoongsonIPICommonState *s = ipicore->ipi;
+    LoongsonIPICommonClass *licc = LOONGSON_IPI_COMMON_GET_CLASS(s);
+
+    if (licc->post_load) {
+        return licc->post_load(s, version_id);
+    }
+
+    return 0;
+}
+
 static const VMStateDescription vmstate_ipi_core = {
     .name = "ipi-single",
     .version_id = 2,
     .minimum_version_id = 2,
+    .pre_save  = loongson_ipi_common_pre_save,
+    .post_load = loongson_ipi_common_post_load,
     .fields = (const VMStateField[]) {
         VMSTATE_UINT32(status, IPICore),
         VMSTATE_UINT32(en, IPICore),
diff --git a/include/hw/intc/loongson_ipi_common.h b/include/hw/intc/loongson_ipi_common.h
index b587f9c571..e58ce2aa1c 100644
--- a/include/hw/intc/loongson_ipi_common.h
+++ b/include/hw/intc/loongson_ipi_common.h
@@ -48,6 +48,8 @@ struct LoongsonIPICommonClass {
     AddressSpace *(*get_iocsr_as)(CPUState *cpu);
     int (*cpu_by_arch_id)(LoongsonIPICommonState *lics, int64_t id,
                           int *index, CPUState **pcs);
+    int (*pre_save)(void *opaque);
+    int (*post_load)(void *opaque, int version_id);
 };
 
 MemTxResult loongson_ipi_core_readl(void *opaque, hwaddr addr, uint64_t *data,
-- 
2.39.3

[PATCH 07/15] hw/intc/loongarch_ipi: Add irqchip-in-kernel save/restore function

[Bibo Mao]

Add save and store funtction if irqchip-in-kernel property is enabled,
it is to get/set IPI irqchip state from KVM kernel.

Signed-off-by: Bibo Mao <maobibo@loongson.cn>
---
 hw/intc/loongarch_ipi.c         | 24 ++++++++++++++
 hw/intc/loongarch_ipi_kvm.c     | 56 +++++++++++++++++++++++++++++++++
 include/hw/intc/loongarch_ipi.h |  2 ++
 3 files changed, 82 insertions(+)

diff --git a/hw/intc/loongarch_ipi.c b/hw/intc/loongarch_ipi.c
index 5e240382ad..bb28e602fc 100644
--- a/hw/intc/loongarch_ipi.c
+++ b/hw/intc/loongarch_ipi.c
@@ -177,6 +177,28 @@ static void loongarch_ipi_cpu_unplug(HotplugHandler *hotplug_dev,
     core->cpu = NULL;
 }
 
+static int loongarch_ipi_pre_save(void *opaque)
+{
+    LoongarchIPIState *lis = LOONGARCH_IPI(opaque);
+
+    if (kvm_enabled() && lis->irqchip_in_kernel) {
+        return kvm_loongarch_ipi_pre_save(opaque);
+    }
+
+    return 0;
+}
+
+static int loongarch_ipi_post_load(void *opaque, int version_id)
+{
+    LoongarchIPIState *lis = LOONGARCH_IPI(opaque);
+
+    if (kvm_enabled() && lis->irqchip_in_kernel) {
+        return kvm_loongarch_ipi_post_load(opaque, version_id);
+    }
+
+    return 0;
+}
+
 static const Property loongarch_ipi_properties[] = {
     DEFINE_PROP_BOOL("irqchip-in-kernel", LoongarchIPIState,
                      irqchip_in_kernel, false),
@@ -199,6 +221,8 @@ static void loongarch_ipi_class_init(ObjectClass *klass, const void *data)
     licc->cpu_by_arch_id = loongarch_cpu_by_arch_id;
     hc->plug = loongarch_ipi_cpu_plug;
     hc->unplug = loongarch_ipi_cpu_unplug;
+    licc->pre_save = loongarch_ipi_pre_save;
+    licc->post_load = loongarch_ipi_post_load;
 }
 
 static const TypeInfo loongarch_ipi_types[] = {
diff --git a/hw/intc/loongarch_ipi_kvm.c b/hw/intc/loongarch_ipi_kvm.c
index e8fcd3bd2f..b5b2b22045 100644
--- a/hw/intc/loongarch_ipi_kvm.c
+++ b/hw/intc/loongarch_ipi_kvm.c
@@ -11,6 +11,62 @@
 #include "system/kvm.h"
 #include "target/loongarch/cpu.h"
 
+static void kvm_ipi_access_regs(int fd, uint64_t addr,
+                                uint32_t *val, bool write)
+{
+    kvm_device_access(fd, KVM_DEV_LOONGARCH_IPI_GRP_REGS,
+                          addr, val, write, &error_abort);
+}
+
+static void kvm_loongarch_ipi_save_load_regs(void *opaque, bool write)
+{
+    LoongsonIPICommonState *ipi = (LoongsonIPICommonState *)opaque;
+    LoongarchIPIState *lis = LOONGARCH_IPI(opaque);
+    IPICore *cpu;
+    uint64_t attr;
+    int cpu_id = 0;
+    int fd = lis->dev_fd;
+
+    for (cpu_id = 0; cpu_id < ipi->num_cpu; cpu_id++) {
+        cpu = &ipi->cpu[cpu_id];
+        attr = (cpu_id << 16) | CORE_STATUS_OFF;
+        kvm_ipi_access_regs(fd, attr, &cpu->status, write);
+
+        attr = (cpu_id << 16) | CORE_EN_OFF;
+        kvm_ipi_access_regs(fd, attr, &cpu->en, write);
+
+        attr = (cpu_id << 16) | CORE_SET_OFF;
+        kvm_ipi_access_regs(fd, attr, &cpu->set, write);
+
+        attr = (cpu_id << 16) | CORE_CLEAR_OFF;
+        kvm_ipi_access_regs(fd, attr, &cpu->clear, write);
+
+        attr = (cpu_id << 16) | CORE_BUF_20;
+        kvm_ipi_access_regs(fd, attr, &cpu->buf[0], write);
+
+        attr = (cpu_id << 16) | CORE_BUF_28;
+        kvm_ipi_access_regs(fd, attr, &cpu->buf[2], write);
+
+        attr = (cpu_id << 16) | CORE_BUF_30;
+        kvm_ipi_access_regs(fd, attr, &cpu->buf[4], write);
+
+        attr = (cpu_id << 16) | CORE_BUF_38;
+        kvm_ipi_access_regs(fd, attr, &cpu->buf[6], write);
+    }
+}
+
+int kvm_loongarch_ipi_pre_save(void *opaque)
+{
+    kvm_loongarch_ipi_save_load_regs(opaque, false);
+    return 0;
+}
+
+int kvm_loongarch_ipi_post_load(void *opaque, int version_id)
+{
+    kvm_loongarch_ipi_save_load_regs(opaque, true);
+    return 0;
+}
+
 void kvm_loongarch_ipi_realize(DeviceState *dev, Error **errp)
 {
     LoongarchIPIState *lis = LOONGARCH_IPI(dev);
diff --git a/include/hw/intc/loongarch_ipi.h b/include/hw/intc/loongarch_ipi.h
index 26076d8062..921cbb0fb5 100644
--- a/include/hw/intc/loongarch_ipi.h
+++ b/include/hw/intc/loongarch_ipi.h
@@ -27,5 +27,7 @@ struct LoongarchIPIClass {
 };
 
 void kvm_loongarch_ipi_realize(DeviceState *dev, Error **errp);
+int kvm_loongarch_ipi_pre_save(void *opaque);
+int kvm_loongarch_ipi_post_load(void *opaque, int version_id);
 
 #endif
-- 
2.39.3

[PATCH 08/15] hw/intc/loongarch_pch_msi: Add irqchip-in-kernel property

[Bibo Mao]

With PCH MSI irqchip, property irqchip-in-kernel is added to indicate
whether feature irqchip_in_kernel is supported or not. This property can
be enabled only if it works in KVM mode.

Signed-off-by: Bibo Mao <maobibo@loongson.cn>
---
 hw/intc/loongarch_pch_msi.c         | 8 ++++++++
 include/hw/intc/loongarch_pch_msi.h | 1 +
 2 files changed, 9 insertions(+)

diff --git a/hw/intc/loongarch_pch_msi.c b/hw/intc/loongarch_pch_msi.c
index 06eb944da0..e2dacc39bf 100644
--- a/hw/intc/loongarch_pch_msi.c
+++ b/hw/intc/loongarch_pch_msi.c
@@ -13,6 +13,7 @@
 #include "hw/pci/msi.h"
 #include "hw/misc/unimp.h"
 #include "migration/vmstate.h"
+#include "system/kvm.h"
 #include "trace.h"
 
 static uint64_t loongarch_msi_mem_read(void *opaque, hwaddr addr, unsigned size)
@@ -51,6 +52,11 @@ static void loongarch_pch_msi_realize(DeviceState *dev, Error **errp)
         return;
     }
 
+    if (s->irqchip_in_kernel && !kvm_enabled()) {
+        error_setg(errp, "irqchip_in_kernel works only in kvm mode");
+        return;
+    }
+
     s->pch_msi_irq = g_new(qemu_irq, s->irq_num);
     qdev_init_gpio_out(dev, s->pch_msi_irq, s->irq_num);
 }
@@ -77,6 +83,8 @@ static void loongarch_pch_msi_init(Object *obj)
 static const Property loongarch_msi_properties[] = {
     DEFINE_PROP_UINT32("msi_irq_base", LoongArchPCHMSI, irq_base, 0),
     DEFINE_PROP_UINT32("msi_irq_num",  LoongArchPCHMSI, irq_num, 0),
+    DEFINE_PROP_BOOL("irqchip-in-kernel", LoongArchPCHMSI,
+                     irqchip_in_kernel, false),
 };
 
 static void loongarch_pch_msi_class_init(ObjectClass *klass, const void *data)
diff --git a/include/hw/intc/loongarch_pch_msi.h b/include/hw/intc/loongarch_pch_msi.h
index b8586fb3b6..25f290d81a 100644
--- a/include/hw/intc/loongarch_pch_msi.h
+++ b/include/hw/intc/loongarch_pch_msi.h
@@ -22,4 +22,5 @@ struct LoongArchPCHMSI {
     /* irq base passed to upper extioi intc */
     unsigned int irq_base;
     unsigned int irq_num;
+    bool irqchip_in_kernel;
 };
-- 
2.39.3

[PATCH 09/15] hw/intc/loongarch_pch_msi: Inject MSI interrupt to kernel

[Bibo Mao]

If property irqchip_in_kernel is enabled, MSI interrupt can be injected
with API kvm_irqchip_send_msi() to KVM.

Signed-off-by: Bibo Mao <maobibo@loongson.cn>
---
 hw/intc/loongarch_pch_msi.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/hw/intc/loongarch_pch_msi.c b/hw/intc/loongarch_pch_msi.c
index e2dacc39bf..d196ee39a8 100644
--- a/hw/intc/loongarch_pch_msi.c
+++ b/hw/intc/loongarch_pch_msi.c
@@ -27,6 +27,15 @@ static void loongarch_msi_mem_write(void *opaque, hwaddr addr,
     LoongArchPCHMSI *s = (LoongArchPCHMSI *)opaque;
     int irq_num;
 
+    if (kvm_enabled() && s->irqchip_in_kernel) {
+        MSIMessage msg;
+
+        msg.address = addr;
+        msg.data = val;
+        kvm_irqchip_send_msi(kvm_state, msg);
+        return;
+    }
+
     /*
      * vector number is irq number from upper extioi intc
      * need subtract irq base to get msi vector offset
-- 
2.39.3

[PATCH 10/15] hw/intc/loongarch_pch: Add irqchip-in-kernel property

[Bibo Mao]

With PCH PCI irqchip, property irqchip-in-kernel is added to indicate
whether feature irqchip_in_kernel is supported or not. This property can
be enabled only if it works in KVM mode.

Signed-off-by: Bibo Mao <maobibo@loongson.cn>
---
 hw/intc/loongarch_pch_pic.c         | 14 ++++++++++++++
 include/hw/intc/loongarch_pch_pic.h |  1 +
 2 files changed, 15 insertions(+)

diff --git a/hw/intc/loongarch_pch_pic.c b/hw/intc/loongarch_pch_pic.c
index cbba2fc284..3729ab9700 100644
--- a/hw/intc/loongarch_pch_pic.c
+++ b/hw/intc/loongarch_pch_pic.c
@@ -10,6 +10,8 @@
 #include "qemu/log.h"
 #include "hw/irq.h"
 #include "hw/intc/loongarch_pch_pic.h"
+#include "hw/qdev-properties.h"
+#include "system/kvm.h"
 #include "trace.h"
 #include "qapi/error.h"
 
@@ -264,6 +266,7 @@ static void loongarch_pic_realize(DeviceState *dev, Error **errp)
 {
     LoongArchPICCommonState *s = LOONGARCH_PIC_COMMON(dev);
     LoongarchPICClass *lpc = LOONGARCH_PIC_GET_CLASS(dev);
+    LoongarchPICState *lps = LOONGARCH_PIC(dev);
     SysBusDevice *sbd = SYS_BUS_DEVICE(dev);
     Error *local_err = NULL;
 
@@ -273,6 +276,11 @@ static void loongarch_pic_realize(DeviceState *dev, Error **errp)
         return;
     }
 
+    if (lps->irqchip_in_kernel && !kvm_enabled()) {
+        error_setg(errp, "PCH_PCI irqchip_in_kernel works only in kvm mode");
+        return;
+    }
+
     qdev_init_gpio_out(dev, s->parent_irq, s->irq_num);
     qdev_init_gpio_in(dev, pch_pic_irq_handler, s->irq_num);
     memory_region_init_io(&s->iomem, OBJECT(dev),
@@ -281,6 +289,11 @@ static void loongarch_pic_realize(DeviceState *dev, Error **errp)
     sysbus_init_mmio(sbd, &s->iomem);
 }
 
+static const Property loongarch_pic_properties[] = {
+    DEFINE_PROP_BOOL("irqchip-in-kernel", LoongarchPICState,
+                     irqchip_in_kernel, false),
+};
+
 static void loongarch_pic_class_init(ObjectClass *klass, const void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
@@ -291,6 +304,7 @@ static void loongarch_pic_class_init(ObjectClass *klass, const void *data)
                                        NULL, &lpc->parent_phases);
     device_class_set_parent_realize(dc, loongarch_pic_realize,
                                     &lpc->parent_realize);
+    device_class_set_props(dc, loongarch_pic_properties);
 }
 
 static const TypeInfo loongarch_pic_types[] = {
diff --git a/include/hw/intc/loongarch_pch_pic.h b/include/hw/intc/loongarch_pch_pic.h
index 839a59a43b..40fe550c0b 100644
--- a/include/hw/intc/loongarch_pch_pic.h
+++ b/include/hw/intc/loongarch_pch_pic.h
@@ -16,6 +16,7 @@ OBJECT_DECLARE_TYPE(LoongarchPICState, LoongarchPICClass, LOONGARCH_PIC)
 
 struct LoongarchPICState {
     LoongArchPICCommonState parent_obj;
+    bool irqchip_in_kernel;
 };
 
 struct LoongarchPICClass {
-- 
2.39.3

[PATCH 11/15] hw/intc/loongarch_pch: Add irqchip-in-kernel realize function

[Bibo Mao]

Function kvm_loongarch_pic_realize() is added if irqchip-in-kernel
property is enabled. It is to notify KVM kernel to create PCH PCI device
in kernel mode.

Signed-off-by: Bibo Mao <maobibo@loongson.cn>
---
 hw/intc/loongarch_pch_pic.c         |  4 +++
 hw/intc/loongarch_pic_kvm.c         | 38 +++++++++++++++++++++++++++++
 hw/intc/meson.build                 |  2 ++
 include/hw/intc/loongarch_pch_pic.h |  3 +++
 4 files changed, 47 insertions(+)
 create mode 100644 hw/intc/loongarch_pic_kvm.c

diff --git a/hw/intc/loongarch_pch_pic.c b/hw/intc/loongarch_pch_pic.c
index 3729ab9700..481a303cc1 100644
--- a/hw/intc/loongarch_pch_pic.c
+++ b/hw/intc/loongarch_pch_pic.c
@@ -287,6 +287,10 @@ static void loongarch_pic_realize(DeviceState *dev, Error **errp)
                           &loongarch_pch_pic_ops,
                           s, TYPE_LOONGARCH_PIC, VIRT_PCH_REG_SIZE);
     sysbus_init_mmio(sbd, &s->iomem);
+
+    if (kvm_enabled() && lps->irqchip_in_kernel) {
+        kvm_loongarch_pic_realize(dev, errp);
+    }
 }
 
 static const Property loongarch_pic_properties[] = {
diff --git a/hw/intc/loongarch_pic_kvm.c b/hw/intc/loongarch_pic_kvm.c
new file mode 100644
index 0000000000..696dabb0b2
--- /dev/null
+++ b/hw/intc/loongarch_pic_kvm.c
@@ -0,0 +1,38 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * LoongArch kvm pch pic interrupt support
+ *
+ * Copyright (C) 2024 Loongson Technology Corporation Limited
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "hw/boards.h"
+#include "hw/intc/loongarch_pch_pic.h"
+#include "hw/loongarch/virt.h"
+#include "hw/pci-host/ls7a.h"
+#include "system/kvm.h"
+
+void kvm_loongarch_pic_realize(DeviceState *dev, Error **errp)
+{
+    LoongarchPICState *lps = LOONGARCH_PIC(dev);
+    uint64_t pch_pic_base = VIRT_PCH_REG_BASE;
+    int ret;
+
+    ret = kvm_create_device(kvm_state, KVM_DEV_TYPE_LOONGARCH_PCHPIC, false);
+    if (ret < 0) {
+        fprintf(stderr, "Create KVM_LOONGARCH_PCHPIC failed: %s\n",
+                strerror(-ret));
+        abort();
+    }
+
+    lps->dev_fd = ret;
+    ret = kvm_device_access(lps->dev_fd, KVM_DEV_LOONGARCH_PCH_PIC_GRP_CTRL,
+                            KVM_DEV_LOONGARCH_PCH_PIC_CTRL_INIT,
+                            &pch_pic_base, true, NULL);
+    if (ret < 0) {
+        fprintf(stderr, "KVM_LOONGARCH_PCH_PIC_INIT failed: %s\n",
+                strerror(-ret));
+        abort();
+    }
+}
diff --git a/hw/intc/meson.build b/hw/intc/meson.build
index 1cc999771d..3137521a4a 100644
--- a/hw/intc/meson.build
+++ b/hw/intc/meson.build
@@ -74,6 +74,8 @@ specific_ss.add(when: 'CONFIG_LOONGARCH_IPI', if_true: files('loongarch_ipi.c'))
 specific_ss.add(when: ['CONFIG_KVM', 'CONFIG_LOONGARCH_IPI'],
                 if_true: files('loongarch_ipi_kvm.c'))
 specific_ss.add(when: 'CONFIG_LOONGARCH_PCH_PIC', if_true: files('loongarch_pch_pic.c', 'loongarch_pic_common.c'))
+specific_ss.add(when: ['CONFIG_KVM', 'CONFIG_LOONGARCH_PCH_PIC'],
+                if_true: files('loongarch_pic_kvm.c'))
 specific_ss.add(when: 'CONFIG_LOONGARCH_PCH_MSI', if_true: files('loongarch_pch_msi.c'))
 specific_ss.add(when: 'CONFIG_LOONGARCH_EXTIOI', if_true: files('loongarch_extioi.c', 'loongarch_extioi_common.c'))
 specific_ss.add(when: ['CONFIG_KVM', 'CONFIG_LOONGARCH_EXTIOI'],
diff --git a/include/hw/intc/loongarch_pch_pic.h b/include/hw/intc/loongarch_pch_pic.h
index 40fe550c0b..91a20dd624 100644
--- a/include/hw/intc/loongarch_pch_pic.h
+++ b/include/hw/intc/loongarch_pch_pic.h
@@ -17,6 +17,7 @@ OBJECT_DECLARE_TYPE(LoongarchPICState, LoongarchPICClass, LOONGARCH_PIC)
 struct LoongarchPICState {
     LoongArchPICCommonState parent_obj;
     bool irqchip_in_kernel;
+    int dev_fd;
 };
 
 struct LoongarchPICClass {
@@ -26,4 +27,6 @@ struct LoongarchPICClass {
     ResettablePhases parent_phases;
 };
 
+void kvm_loongarch_pic_realize(DeviceState *dev, Error **errp);
+
 #endif /* HW_LOONGARCH_PCH_PIC_H */
-- 
2.39.3

[PATCH 12/15] hw/intc/loongarch_pch: Add irqchip-in-kernel save/restore function

[Bibo Mao]

Add save and store funtction if irqchip-in-kernel property is enabled,
it is to get/set PCH PCI irqchip state from KVM kernel.

Signed-off-by: Bibo Mao <maobibo@loongson.cn>
`
---
 hw/intc/loongarch_pch_pic.c            | 26 ++++++++++++
 hw/intc/loongarch_pic_kvm.c            | 58 ++++++++++++++++++++++++++
 include/hw/intc/loongarch_pch_pic.h    |  2 +
 include/hw/intc/loongarch_pic_common.h |  1 +
 4 files changed, 87 insertions(+)

diff --git a/hw/intc/loongarch_pch_pic.c b/hw/intc/loongarch_pch_pic.c
index 481a303cc1..e8dfd0ed75 100644
--- a/hw/intc/loongarch_pch_pic.c
+++ b/hw/intc/loongarch_pch_pic.c
@@ -293,6 +293,29 @@ static void loongarch_pic_realize(DeviceState *dev, Error **errp)
     }
 }
 
+static int loongarch_pic_pre_save(LoongArchPICCommonState *opaque)
+{
+    LoongarchPICState *lps = LOONGARCH_PIC(opaque);
+
+    if (kvm_enabled() && lps->irqchip_in_kernel) {
+        return kvm_loongarch_pic_pre_save(opaque);
+    }
+
+    return 0;
+}
+
+static int loongarch_pic_post_load(LoongArchPICCommonState *opaque,
+                                   int version_id)
+{
+    LoongarchPICState *lps = LOONGARCH_PIC(opaque);
+
+    if (kvm_enabled() && lps->irqchip_in_kernel) {
+        return kvm_loongarch_pic_post_load(opaque, version_id);
+    }
+
+    return 0;
+}
+
 static const Property loongarch_pic_properties[] = {
     DEFINE_PROP_BOOL("irqchip-in-kernel", LoongarchPICState,
                      irqchip_in_kernel, false),
@@ -302,6 +325,7 @@ static void loongarch_pic_class_init(ObjectClass *klass, const void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
     LoongarchPICClass *lpc = LOONGARCH_PIC_CLASS(klass);
+    LoongArchPICCommonClass *lpcc = LOONGARCH_PIC_COMMON_CLASS(klass);
     ResettableClass *rc = RESETTABLE_CLASS(klass);
 
     resettable_class_set_parent_phases(rc, NULL, loongarch_pic_reset_hold,
@@ -309,6 +333,8 @@ static void loongarch_pic_class_init(ObjectClass *klass, const void *data)
     device_class_set_parent_realize(dc, loongarch_pic_realize,
                                     &lpc->parent_realize);
     device_class_set_props(dc, loongarch_pic_properties);
+    lpcc->pre_save = loongarch_pic_pre_save;
+    lpcc->post_load = loongarch_pic_post_load;
 }
 
 static const TypeInfo loongarch_pic_types[] = {
diff --git a/hw/intc/loongarch_pic_kvm.c b/hw/intc/loongarch_pic_kvm.c
index 696dabb0b2..84942a86cd 100644
--- a/hw/intc/loongarch_pic_kvm.c
+++ b/hw/intc/loongarch_pic_kvm.c
@@ -13,6 +13,64 @@
 #include "hw/pci-host/ls7a.h"
 #include "system/kvm.h"
 
+static void kvm_pch_pic_access_regs(int fd, uint64_t addr,
+                                       void *val, bool write)
+{
+    kvm_device_access(fd, KVM_DEV_LOONGARCH_PCH_PIC_GRP_REGS,
+                      addr, val, write, &error_abort);
+}
+
+static void kvm_loongarch_pch_pic_save_load(void *opaque, bool write)
+{
+    LoongArchPICCommonState *s = LOONGARCH_PIC_COMMON(opaque);
+    LoongarchPICState *lps = LOONGARCH_PIC(opaque);
+    int fd = lps->dev_fd;
+    int addr, offset;
+
+    kvm_pch_pic_access_regs(fd, VIRT_PCH_REG_BASE + PCH_PIC_INT_MASK,
+                            &s->int_mask, write);
+    kvm_pch_pic_access_regs(fd, VIRT_PCH_REG_BASE + PCH_PIC_HTMSI_EN,
+                            &s->htmsi_en, write);
+    kvm_pch_pic_access_regs(fd, VIRT_PCH_REG_BASE + PCH_PIC_INT_EDGE,
+                            &s->intedge, write);
+    kvm_pch_pic_access_regs(fd, VIRT_PCH_REG_BASE + PCH_PIC_AUTO_CTRL0,
+                            &s->auto_crtl0, write);
+    kvm_pch_pic_access_regs(fd, VIRT_PCH_REG_BASE + PCH_PIC_AUTO_CTRL1,
+                            &s->auto_crtl1, write);
+
+    for (addr = PCH_PIC_ROUTE_ENTRY;
+        addr < PCH_PIC_ROUTE_ENTRY_END; addr++) {
+        offset = addr - PCH_PIC_ROUTE_ENTRY;
+        kvm_pch_pic_access_regs(fd, addr + VIRT_PCH_REG_BASE,
+                                &s->route_entry[offset], write);
+    }
+
+    for (addr = PCH_PIC_HTMSI_VEC; addr < PCH_PIC_HTMSI_VEC_END; addr++) {
+        offset = addr - PCH_PIC_HTMSI_VEC;
+        kvm_pch_pic_access_regs(fd, addr + VIRT_PCH_REG_BASE,
+                                &s->htmsi_vector[offset], write);
+    }
+
+    kvm_pch_pic_access_regs(fd, VIRT_PCH_REG_BASE + PCH_PIC_INT_REQUEST,
+                            &s->intirr, write);
+    kvm_pch_pic_access_regs(fd, VIRT_PCH_REG_BASE + PCH_PIC_INT_STATUS,
+                            &s->intisr, write);
+    kvm_pch_pic_access_regs(fd, VIRT_PCH_REG_BASE + PCH_PIC_INT_POL,
+                            &s->int_polarity, write);
+}
+
+int kvm_loongarch_pic_pre_save(void *opaque)
+{
+    kvm_loongarch_pch_pic_save_load(opaque, false);
+    return 0;
+}
+
+int kvm_loongarch_pic_post_load(void *opaque, int version_id)
+{
+    kvm_loongarch_pch_pic_save_load(opaque, true);
+    return 0;
+}
+
 void kvm_loongarch_pic_realize(DeviceState *dev, Error **errp)
 {
     LoongarchPICState *lps = LOONGARCH_PIC(dev);
diff --git a/include/hw/intc/loongarch_pch_pic.h b/include/hw/intc/loongarch_pch_pic.h
index 91a20dd624..822d9d3544 100644
--- a/include/hw/intc/loongarch_pch_pic.h
+++ b/include/hw/intc/loongarch_pch_pic.h
@@ -28,5 +28,7 @@ struct LoongarchPICClass {
 };
 
 void kvm_loongarch_pic_realize(DeviceState *dev, Error **errp);
+int kvm_loongarch_pic_pre_save(void *opaque);
+int kvm_loongarch_pic_post_load(void *opaque, int version_id);
 
 #endif /* HW_LOONGARCH_PCH_PIC_H */
diff --git a/include/hw/intc/loongarch_pic_common.h b/include/hw/intc/loongarch_pic_common.h
index 9349a055d0..f774c975d4 100644
--- a/include/hw/intc/loongarch_pic_common.h
+++ b/include/hw/intc/loongarch_pic_common.h
@@ -23,6 +23,7 @@
 #define PCH_PIC_ROUTE_ENTRY_END         0x13f
 #define PCH_PIC_HTMSI_VEC               0x200
 #define PCH_PIC_HTMSI_VEC_END           0x23f
+#define PCH_PIC_INT_REQUEST             0x380
 #define PCH_PIC_INT_STATUS              0x3a0
 #define PCH_PIC_INT_POL                 0x3e0
 
-- 
2.39.3

[PATCH 13/15] hw/intc/loongarch_pch: Inject irq line interrupt to kernel

[Bibo Mao]

If property irqchip_in_kernel is enabled, irq line interrupt can be
injected with API kvm_set_irq() to KVM.

Signed-off-by: Bibo Mao <maobibo@loongson.cn>
---
 hw/intc/loongarch_pch_pic.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/hw/intc/loongarch_pch_pic.c b/hw/intc/loongarch_pch_pic.c
index e8dfd0ed75..fb063173b7 100644
--- a/hw/intc/loongarch_pch_pic.c
+++ b/hw/intc/loongarch_pch_pic.c
@@ -45,11 +45,17 @@ static void pch_pic_update_irq(LoongArchPICCommonState *s, uint64_t mask,
 static void pch_pic_irq_handler(void *opaque, int irq, int level)
 {
     LoongArchPICCommonState *s = LOONGARCH_PIC_COMMON(opaque);
+    LoongarchPICState *lps = LOONGARCH_PIC(opaque);
     uint64_t mask = 1ULL << irq;
 
     assert(irq < s->irq_num);
     trace_loongarch_pch_pic_irq_handler(irq, level);
 
+    if (kvm_enabled() && lps->irqchip_in_kernel) {
+        kvm_set_irq(kvm_state, irq, !!level);
+        return;
+    }
+
     if (s->intedge & mask) {
         /* Edge triggered */
         if (level) {
-- 
2.39.3

[PATCH 14/15] target/loongarch: Report error with split kernel_irqchip option

[Bibo Mao]

Option kernel_irqchip=split is not supported on LoongArch virt machine,
report error and exit if detect split kernel_irqchip option.

Signed-off-by: Bibo Mao <maobibo@loongson.cn>
---
 target/loongarch/kvm/kvm.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/target/loongarch/kvm/kvm.c b/target/loongarch/kvm/kvm.c
index 1bda570482..a8e5724b21 100644
--- a/target/loongarch/kvm/kvm.c
+++ b/target/loongarch/kvm/kvm.c
@@ -1249,7 +1249,12 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
 
 int kvm_arch_irqchip_create(KVMState *s)
 {
-    return 0;
+    if (kvm_kernel_irqchip_split()) {
+        error_report("-machine kernel_irqchip=split is not supported on ARM.");
+        exit(1);
+    }
+
+    return kvm_check_extension(s, KVM_CAP_DEVICE_CTRL);
 }
 
 void kvm_arch_pre_run(CPUState *cs, struct kvm_run *run)
-- 
2.39.3

[PATCH 15/15] hw/loongarch/virt: Add kvm_irqchip_in_kernel support

[Bibo Mao]

If kvm_irqchip_in_kernel option is set, set property irqchip-in-kernel
with ExtIOI, IPI, PCH_PCI and PCH_MSI interrupt controller, so that
irqchip in kernel is supported.

Signed-off-by: Bibo Mao <maobibo@loongson.cn>
---
 hw/loongarch/virt.c        | 15 +++++++++++++++
 target/loongarch/cpu.h     |  1 +
 target/loongarch/kvm/kvm.c | 16 ++++++++++++++++
 3 files changed, 32 insertions(+)

diff --git a/hw/loongarch/virt.c b/hw/loongarch/virt.c
index 1b504047db..a79e77e663 100644
--- a/hw/loongarch/virt.c
+++ b/hw/loongarch/virt.c
@@ -398,6 +398,9 @@ static void virt_irq_init(LoongArchVirtMachineState *lvms)
 
     /* Create IPI device */
     ipi = qdev_new(TYPE_LOONGARCH_IPI);
+    if (kvm_irqchip_in_kernel()) {
+        qdev_prop_set_bit(ipi, "irqchip-in-kernel", true);
+    }
     lvms->ipi = ipi;
     sysbus_realize_and_unref(SYS_BUS_DEVICE(ipi), &error_fatal);
 
@@ -413,6 +416,9 @@ static void virt_irq_init(LoongArchVirtMachineState *lvms)
     if (virt_is_veiointc_enabled(lvms)) {
         qdev_prop_set_bit(extioi, "has-virtualization-extension", true);
     }
+    if (kvm_irqchip_in_kernel()) {
+        qdev_prop_set_bit(extioi, "irqchip-in-kernel", true);
+    }
     sysbus_realize_and_unref(SYS_BUS_DEVICE(extioi), &error_fatal);
     memory_region_add_subregion(&lvms->system_iocsr, APIC_BASE,
                     sysbus_mmio_get_region(SYS_BUS_DEVICE(extioi), 0));
@@ -423,6 +429,9 @@ static void virt_irq_init(LoongArchVirtMachineState *lvms)
 
     virt_cpu_irq_init(lvms);
     pch_pic = qdev_new(TYPE_LOONGARCH_PIC);
+    if (kvm_irqchip_in_kernel()) {
+        qdev_prop_set_bit(pch_pic, "irqchip-in-kernel", true);
+    }
     num = VIRT_PCH_PIC_IRQ_NUM;
     qdev_prop_set_uint32(pch_pic, "pch_pic_irq_num", num);
     d = SYS_BUS_DEVICE(pch_pic);
@@ -436,6 +445,9 @@ static void virt_irq_init(LoongArchVirtMachineState *lvms)
     }
 
     pch_msi = qdev_new(TYPE_LOONGARCH_PCH_MSI);
+    if (kvm_irqchip_in_kernel()) {
+        qdev_prop_set_bit(pch_msi, "irqchip-in-kernel", true);
+    }
     start   =  num;
     num = EXTIOI_IRQS - start;
     qdev_prop_set_uint32(pch_msi, "msi_irq_base", start);
@@ -449,6 +461,9 @@ static void virt_irq_init(LoongArchVirtMachineState *lvms)
                               qdev_get_gpio_in(extioi, i + start));
     }
 
+    if (kvm_irqchip_in_kernel()) {
+        kvm_loongarch_init_irq_routing();
+    }
     virt_devices_init(pch_pic, lvms);
 }
 
diff --git a/target/loongarch/cpu.h b/target/loongarch/cpu.h
index 262bf87f7b..9538e8d61d 100644
--- a/target/loongarch/cpu.h
+++ b/target/loongarch/cpu.h
@@ -503,5 +503,6 @@ static inline void kvm_loongarch_cpu_post_init(LoongArchCPU *cpu)
 {
 }
 #endif
+void kvm_loongarch_init_irq_routing(void);
 
 #endif /* LOONGARCH_CPU_H */
diff --git a/target/loongarch/kvm/kvm.c b/target/loongarch/kvm/kvm.c
index a8e5724b21..52a54f286b 100644
--- a/target/loongarch/kvm/kvm.c
+++ b/target/loongarch/kvm/kvm.c
@@ -1236,6 +1236,22 @@ void kvm_arch_init_irq_routing(KVMState *s)
 {
 }
 
+void kvm_loongarch_init_irq_routing(void)
+{
+    int i;
+
+    kvm_async_interrupts_allowed = true;
+    kvm_msi_via_irqfd_allowed = kvm_irqfds_enabled();
+    if (kvm_has_gsi_routing()) {
+        for (i = 0; i < 64; ++i) {
+            kvm_irqchip_add_irq_route(kvm_state, i, 0, i);
+        }
+
+        kvm_gsi_routing_allowed = true;
+        kvm_irqchip_commit_routes(kvm_state);
+    }
+}
+
 int kvm_arch_get_default_type(MachineState *ms)
 {
     return 0;
-- 
2.39.3

Re: [PATCH 14/15] target/loongarch: Report error with split kernel_irqchip option

[Jiaxun Yang]

在2025年5月9日周五 上午11:12，Bibo Mao写道：
> Option kernel_irqchip=split is not supported on LoongArch virt machine,
> report error and exit if detect split kernel_irqchip option.
>
> Signed-off-by: Bibo Mao <maobibo@loongson.cn>
> ---
>  target/loongarch/kvm/kvm.c | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/target/loongarch/kvm/kvm.c b/target/loongarch/kvm/kvm.c
> index 1bda570482..a8e5724b21 100644
> --- a/target/loongarch/kvm/kvm.c
> +++ b/target/loongarch/kvm/kvm.c
> @@ -1249,7 +1249,12 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
> 
>  int kvm_arch_irqchip_create(KVMState *s)
>  {
> -    return 0;
> +    if (kvm_kernel_irqchip_split()) {
> +        error_report("-machine kernel_irqchip=split is not supported on ARM.");
                                                                           ^ ARM :)

Thanks

-- 
- Jiaxun

Re: [PATCH 03/15] hw/intc/loongarch_extioi: Add irqchip-in-kernel save/restore function

[Jiaxun Yang]

在2025年5月9日周五 上午11:07，Bibo Mao写道：
> Add save and store funtction if irqchip-in-kernel property is enabled,
> it is to get/set ExtIOI irqchip state from KVM kernel.
>
> Signed-off-by: Bibo Mao <maobibo@loongson.cn>
> ---
>  hw/intc/loongarch_extioi.c         |  17 +++++
>  hw/intc/loongarch_extioi_kvm.c     | 100 +++++++++++++++++++++++++++++
>  include/hw/intc/loongarch_extioi.h |   2 +
>  3 files changed, 119 insertions(+)
>
> diff --git a/hw/intc/loongarch_extioi.c b/hw/intc/loongarch_extioi.c
> index 854f54684b..e81caf430c 100644
> --- a/hw/intc/loongarch_extioi.c
> +++ b/hw/intc/loongarch_extioi.c
> @@ -398,9 +398,21 @@ static void loongarch_extioi_reset_hold(Object 
> *obj, ResetType type)
>      }
>  }
> 
> +static int vmstate_extioi_pre_save(void *opaque)
> +{
> +    LoongArchExtIOIState *les = LOONGARCH_EXTIOI(opaque);
> +
> +    if (kvm_enabled() && les->irqchip_in_kernel) {
> +        return kvm_loongarch_extioi_pre_save(opaque);
> +    }
> +
> +    return 0;
> +}
> +

Hi Bibo,

I believe hijacking loongarch_extioi.c is not the proper way to do it.
The sensible solution is to create a TYPE_LOONGARCH_EXTIOI_KVM, which
inherits TYPE_LOONGARCH_EXTIOI_COMMON, and let machine create
TYPE_LOONGARCH_EXTIOI_KVM vs TYPE_LOONGARCH_EXTIOI as necessary.

In this way you can avoid ugly "irqchip-in-kernel" property.
Also you don't really want all those emulation functions in
loongarch_extioi.c to kick in when irqchip is in kernel. If
IOCSR VMEXIT happens on extioi range, it's a hypervisor error
rather than something needs to be emulated.

Also I think EXTIOI_VIRT_CONFIG range MMIO needs to be handled
differently on KVM vs userspace irqchip. EXTIOI_VIRT_CONFIG needs
to be relayed to kernel, as virt_iocsr_misc_write will perform
IOCSR read/write in userspace, this needs to be translated to
KVM_DEV_LOONGARCH_EXTIOI_SW_STATUS_STATE.

Hope it makes sense.

Thanks
-- 
- Jiaxun

Re: [PATCH 14/15] target/loongarch: Report error with split kernel_irqchip option

[Bibo Mao]

On 2025/5/9 下午8:14, Jiaxun Yang wrote:
> 
> 
> 在2025年5月9日周五 上午11:12，Bibo Mao写道：
>> Option kernel_irqchip=split is not supported on LoongArch virt machine,
>> report error and exit if detect split kernel_irqchip option.
>>
>> Signed-off-by: Bibo Mao <maobibo@loongson.cn>
>> ---
>>   target/loongarch/kvm/kvm.c | 7 ++++++-
>>   1 file changed, 6 insertions(+), 1 deletion(-)
>>
>> diff --git a/target/loongarch/kvm/kvm.c b/target/loongarch/kvm/kvm.c
>> index 1bda570482..a8e5724b21 100644
>> --- a/target/loongarch/kvm/kvm.c
>> +++ b/target/loongarch/kvm/kvm.c
>> @@ -1249,7 +1249,12 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
>>
>>   int kvm_arch_irqchip_create(KVMState *s)
>>   {
>> -    return 0;
>> +    if (kvm_kernel_irqchip_split()) {
>> +        error_report("-machine kernel_irqchip=split is not supported on ARM.");
>                                                                             ^ ARM :)
oops, silly copy and past here :)
will do.

Regards
Bibo Mao
> 
> Thanks
> 

Re: [PATCH 03/15] hw/intc/loongarch_extioi: Add irqchip-in-kernel save/restore function

[Bibo Mao]

On 2025/5/9 下午8:36, Jiaxun Yang wrote:
> 
> 
> 在2025年5月9日周五 上午11:07，Bibo Mao写道：
>> Add save and store funtction if irqchip-in-kernel property is enabled,
>> it is to get/set ExtIOI irqchip state from KVM kernel.
>>
>> Signed-off-by: Bibo Mao <maobibo@loongson.cn>
>> ---
>>   hw/intc/loongarch_extioi.c         |  17 +++++
>>   hw/intc/loongarch_extioi_kvm.c     | 100 +++++++++++++++++++++++++++++
>>   include/hw/intc/loongarch_extioi.h |   2 +
>>   3 files changed, 119 insertions(+)
>>
>> diff --git a/hw/intc/loongarch_extioi.c b/hw/intc/loongarch_extioi.c
>> index 854f54684b..e81caf430c 100644
>> --- a/hw/intc/loongarch_extioi.c
>> +++ b/hw/intc/loongarch_extioi.c
>> @@ -398,9 +398,21 @@ static void loongarch_extioi_reset_hold(Object
>> *obj, ResetType type)
>>       }
>>   }
>>
>> +static int vmstate_extioi_pre_save(void *opaque)
>> +{
>> +    LoongArchExtIOIState *les = LOONGARCH_EXTIOI(opaque);
>> +
>> +    if (kvm_enabled() && les->irqchip_in_kernel) {
>> +        return kvm_loongarch_extioi_pre_save(opaque);
>> +    }
>> +
>> +    return 0;
>> +}
>> +
> 
> Hi Bibo,
> 
> I believe hijacking loongarch_extioi.c is not the proper way to do it.
> The sensible solution is to create a TYPE_LOONGARCH_EXTIOI_KVM, which
> inherits TYPE_LOONGARCH_EXTIOI_COMMON, and let machine create
> TYPE_LOONGARCH_EXTIOI_KVM vs TYPE_LOONGARCH_EXTIOI as necessary.
what is advantage about creating TYPE_LOONGARCH_EXTIOI_KVM device in KVM 
node and TYPE_LOONGARCH_EXTIOI device in TCG mode?

That means there will be two virt machine types since device name is 
different in different accel mode.
> 
> In this way you can avoid ugly "irqchip-in-kernel" property.
> Also you don't really want all those emulation functions in
> loongarch_extioi.c to kick in when irqchip is in kernel. If
> IOCSR VMEXIT happens on extioi range, it's a hypervisor error
> rather than something needs to be emulated.
> 
> Also I think EXTIOI_VIRT_CONFIG range MMIO needs to be handled
> differently on KVM vs userspace irqchip. EXTIOI_VIRT_CONFIG needs
> to be relayed to kernel, as virt_iocsr_misc_write will perform
> IOCSR read/write in userspace, this needs to be translated to
> KVM_DEV_LOONGARCH_EXTIOI_SW_STATUS_STATE.
Will handle it, misc iocsr should be emulated in kernel also.

Regards
Bibo Mao
> 
> Hope it makes sense.
> 
> Thanks
> 

Re: [PATCH 03/15] hw/intc/loongarch_extioi: Add irqchip-in-kernel save/restore function

[Jiaxun Yang]

在2025年5月19日周一 上午3:56，Bibo Mao写道：
[...]
>> 
>> Hi Bibo,
>> 
>> I believe hijacking loongarch_extioi.c is not the proper way to do it.
>> The sensible solution is to create a TYPE_LOONGARCH_EXTIOI_KVM, which
>> inherits TYPE_LOONGARCH_EXTIOI_COMMON, and let machine create
>> TYPE_LOONGARCH_EXTIOI_KVM vs TYPE_LOONGARCH_EXTIOI as necessary.
> what is advantage about creating TYPE_LOONGARCH_EXTIOI_KVM device in KVM 
> node and TYPE_LOONGARCH_EXTIOI device in TCG mode?

Cleaner, less error-prone, isolate unnecessary emulation functions to
reduce attack surface...

>
> That means there will be two virt machine types since device name is 
> different in different accel mode.

Yes I do think you shouldn't aim migration capability between different
accel mode. It's not doable given we can't emulate full set of h/w behaviour.

>> 
>> In this way you can avoid ugly "irqchip-in-kernel" property.
>> Also you don't really want all those emulation functions in
>> loongarch_extioi.c to kick in when irqchip is in kernel. If
>> IOCSR VMEXIT happens on extioi range, it's a hypervisor error
>> rather than something needs to be emulated.
>> 
>> Also I think EXTIOI_VIRT_CONFIG range MMIO needs to be handled
>> differently on KVM vs userspace irqchip. EXTIOI_VIRT_CONFIG needs
>> to be relayed to kernel, as virt_iocsr_misc_write will perform
>> IOCSR read/write in userspace, this needs to be translated to
>> KVM_DEV_LOONGARCH_EXTIOI_SW_STATUS_STATE.
> Will handle it, misc iocsr should be emulated in kernel also.

IMHO misc IOCSR should be in user space because it described
the capability of machine instead of CPU, it's not a part of
irqchip.

>
> Regards
> Bibo Mao

Thanks
-- 
- Jiaxun

Re: [PATCH 03/15] hw/intc/loongarch_extioi: Add irqchip-in-kernel save/restore function

[Bibo Mao]

On 2025/5/19 下午2:50, Jiaxun Yang wrote:
> 
> 
> 在2025年5月19日周一 上午3:56，Bibo Mao写道：
> [...]
>>>
>>> Hi Bibo,
>>>
>>> I believe hijacking loongarch_extioi.c is not the proper way to do it.
>>> The sensible solution is to create a TYPE_LOONGARCH_EXTIOI_KVM, which
>>> inherits TYPE_LOONGARCH_EXTIOI_COMMON, and let machine create
>>> TYPE_LOONGARCH_EXTIOI_KVM vs TYPE_LOONGARCH_EXTIOI as necessary.
>> what is advantage about creating TYPE_LOONGARCH_EXTIOI_KVM device in KVM
>> node and TYPE_LOONGARCH_EXTIOI device in TCG mode?
> 
> Cleaner, less error-prone, isolate unnecessary emulation functions to
> reduce attack surface...
yes, there is a beautiful code logic internal, however from user the 
device tree will be different because of irqchip-in-kernel feature, such 
as different output of *info qom-tree*. It will bring out illusions of 
different virt machine type for users.

Regards
Bibo Mao
> 
>>
>> That means there will be two virt machine types since device name is
>> different in different accel mode.
> 
> Yes I do think you shouldn't aim migration capability between different
> accel mode. It's not doable given we can't emulate full set of h/w behaviour.
> 
>>>
>>> In this way you can avoid ugly "irqchip-in-kernel" property.
>>> Also you don't really want all those emulation functions in
>>> loongarch_extioi.c to kick in when irqchip is in kernel. If
>>> IOCSR VMEXIT happens on extioi range, it's a hypervisor error
>>> rather than something needs to be emulated.
>>>
>>> Also I think EXTIOI_VIRT_CONFIG range MMIO needs to be handled
>>> differently on KVM vs userspace irqchip. EXTIOI_VIRT_CONFIG needs
>>> to be relayed to kernel, as virt_iocsr_misc_write will perform
>>> IOCSR read/write in userspace, this needs to be translated to
>>> KVM_DEV_LOONGARCH_EXTIOI_SW_STATUS_STATE.
>> Will handle it, misc iocsr should be emulated in kernel also.
> 
> IMHO misc IOCSR should be in user space because it described
> the capability of machine instead of CPU, it's not a part of
> irqchip.
> 
>>
>> Regards
>> Bibo Mao
> 
> Thanks
> 

Re: [PATCH 03/15] hw/intc/loongarch_extioi: Add irqchip-in-kernel save/restore function

[Jiaxun Yang]

在2025年5月19日周一 上午8:09，Bibo Mao写道：
> On 2025/5/19 下午2:50, Jiaxun Yang wrote:
>> 
>> 
>> 在2025年5月19日周一 上午3:56，Bibo Mao写道：
>> [...]
>>>>
>>>> Hi Bibo,
>>>>
>>>> I believe hijacking loongarch_extioi.c is not the proper way to do it.
>>>> The sensible solution is to create a TYPE_LOONGARCH_EXTIOI_KVM, which
>>>> inherits TYPE_LOONGARCH_EXTIOI_COMMON, and let machine create
>>>> TYPE_LOONGARCH_EXTIOI_KVM vs TYPE_LOONGARCH_EXTIOI as necessary.
>>> what is advantage about creating TYPE_LOONGARCH_EXTIOI_KVM device in KVM
>>> node and TYPE_LOONGARCH_EXTIOI device in TCG mode?
>> 
>> Cleaner, less error-prone, isolate unnecessary emulation functions to
>> reduce attack surface...
> yes, there is a beautiful code logic internal, however from user the 
> device tree will be different because of irqchip-in-kernel feature, such 
> as different output of *info qom-tree*. It will bring out illusions of 
> different virt machine type for users.

It's actually different machine as kernel irqchip is never on par with usermode
emulation. This approach is taken by i386 (TYPE_KVM_IOAPIC vs TYPE_IOAPIC),
Arm (TYPE_KVM_ARM_ITS vs TYPE_ARM_GICV3_ITS), PowerPC (TYPE_KVM_OPENPIC vs
TYPE_OPENPIC) and I see no reason that LoongArch should not follow.

I'm actually planning to bring user space EXTIOI emulation closer to actual
hardware behaviour and I wound not expect such change to be accepted by
in-kernel irqchip.

Thanks

>
> Regards
> Bibo Mao
>> 

-- 
- Jiaxun

Re: [PATCH 03/15] hw/intc/loongarch_extioi: Add irqchip-in-kernel save/restore function

[Bibo Mao]

On 2025/5/19 下午4:49, Jiaxun Yang wrote:
> 
> 
> 在2025年5月19日周一 上午8:09，Bibo Mao写道：
>> On 2025/5/19 下午2:50, Jiaxun Yang wrote:
>>>
>>>
>>> 在2025年5月19日周一 上午3:56，Bibo Mao写道：
>>> [...]
>>>>>
>>>>> Hi Bibo,
>>>>>
>>>>> I believe hijacking loongarch_extioi.c is not the proper way to do it.
>>>>> The sensible solution is to create a TYPE_LOONGARCH_EXTIOI_KVM, which
>>>>> inherits TYPE_LOONGARCH_EXTIOI_COMMON, and let machine create
>>>>> TYPE_LOONGARCH_EXTIOI_KVM vs TYPE_LOONGARCH_EXTIOI as necessary.
>>>> what is advantage about creating TYPE_LOONGARCH_EXTIOI_KVM device in KVM
>>>> node and TYPE_LOONGARCH_EXTIOI device in TCG mode?
>>>
>>> Cleaner, less error-prone, isolate unnecessary emulation functions to
>>> reduce attack surface...
>> yes, there is a beautiful code logic internal, however from user the
>> device tree will be different because of irqchip-in-kernel feature, such
>> as different output of *info qom-tree*. It will bring out illusions of
>> different virt machine type for users.
> 
> It's actually different machine as kernel irqchip is never on par with usermode
> emulation. This approach is taken by i386 (TYPE_KVM_IOAPIC vs TYPE_IOAPIC),
> Arm (TYPE_KVM_ARM_ITS vs TYPE_ARM_GICV3_ITS), PowerPC (TYPE_KVM_OPENPIC vs
> TYPE_OPENPIC) and I see no reason that LoongArch should not follow.
So what is the advantage and disadvantage from yourself understanding here?

Regards
Bibo Mao
> 
> I'm actually planning to bring user space EXTIOI emulation closer to actual
> hardware behaviour and I wound not expect such change to be accepted by
> in-kernel irqchip.
> 
> Thanks
> 
>>
>> Regards
>> Bibo Mao
>>>
> 

Re: [PATCH 03/15] hw/intc/loongarch_extioi: Add irqchip-in-kernel save/restore function

[Jiaxun Yang]

在2025年5月19日周一 上午9:55，Bibo Mao写道：
[...]
>> It's actually different machine as kernel irqchip is never on par with usermode
>> emulation. This approach is taken by i386 (TYPE_KVM_IOAPIC vs TYPE_IOAPIC),
>> Arm (TYPE_KVM_ARM_ITS vs TYPE_ARM_GICV3_ITS), PowerPC (TYPE_KVM_OPENPIC vs
>> TYPE_OPENPIC) and I see no reason that LoongArch should not follow.
> So what is the advantage and disadvantage from yourself understanding here?

I think I made myself pretty clear in previous replies, in case you missed that. 

The advantage is clean design, clean interface, clean vmstates (user space emulation
tends to have more states vs in-kernel irqchip), proper signalling to user that
migration between user-space/in-kernel irqchip is not feasible, perhaps some performance
advantage on reducing number of user space IOCSR ranges, reducing attack surface
exposed by userspace emulation, reducing the chance of hypervisor error being covered
up by userspace fallback....

I don't think there is any disadvantage. I don't really buy the "different machine"
justification you made. Paravirt solution tends to have its own behaviour and I don't
think it's a bad thing to expose it to users.

Thanks
-- 
- Jiaxun

Re: [PATCH 03/15] hw/intc/loongarch_extioi: Add irqchip-in-kernel save/restore function

[bibo mao]

Jiaxun Yang <jiaxun.yang@flygoat.com> 于2025年5月19日周一 17:17写道：
>
>
>
> 在2025年5月19日周一 上午9:55，Bibo Mao写道：
> [...]
> >> It's actually different machine as kernel irqchip is never on par with usermode
> >> emulation. This approach is taken by i386 (TYPE_KVM_IOAPIC vs TYPE_IOAPIC),
> >> Arm (TYPE_KVM_ARM_ITS vs TYPE_ARM_GICV3_ITS), PowerPC (TYPE_KVM_OPENPIC vs
> >> TYPE_OPENPIC) and I see no reason that LoongArch should not follow.
> > So what is the advantage and disadvantage from yourself understanding here?
>
> I think I made myself pretty clear in previous replies, in case you missed that.
>
> The advantage is clean design, clean interface, clean vmstates (user space emulation
> tends to have more states vs in-kernel irqchip), proper signalling to user that
> migration between user-space/in-kernel irqchip is not feasible, perhaps some performance
> advantage on reducing number of user space IOCSR ranges, reducing attack surface
> exposed by userspace emulation, reducing the chance of hypervisor error being covered
> up by userspace fallback....
>
> I don't think there is any disadvantage. I don't really buy the "different machine"
> justification you made. Paravirt solution tends to have its own behaviour and I don't
> think it's a bad thing to expose it to users.
irqchip-in-kernel is some kind of optimization, register layout and
function is the same whatever it is
emulated in kernel or user mode.

The same for cpu type, do you think that cpu type la464 should be
named as la464-kvm in
KVM mode? Or you do not care about name at all？

Regards
Bibo Mao

>
> Thanks
> --
> - Jiaxun
>

Re: [PATCH 03/15] hw/intc/loongarch_extioi: Add irqchip-in-kernel save/restore function

[Jiaxun Yang]

在2025年5月19日周一 下午2:24，bibo mao写道：
[...]
>> I don't think there is any disadvantage. I don't really buy the "different machine"
>> justification you made. Paravirt solution tends to have its own behaviour and I don't
>> think it's a bad thing to expose it to users.
> irqchip-in-kernel is some kind of optimization, register layout and
> function is the same whatever it is
> emulated in kernel or user mode.

I can already observe some differences, for example with in-kernel irqchip
PCH MSI can be delivered to any EXTIOI vector while in user space it's only
possible to do so for 64+ vector.

I'm also planning to bring user space EXTIOI emulation closer to hardware,
as I found many issues when I was trying to bring up SylixOS BSP in QEMU,
and it's unlikely in-kernel one will follow due to performance considerations.

>
> The same for cpu type, do you think that cpu type la464 should be
> named as la464-kvm in
> KVM mode? Or you do not care about name at all？
            ^ 

I do think it for user interface it should be "host" or "max" whenever
possible in KVM mode. From QOM perspective, TCG vs KVM is handled by
TYPE_ACCEL_OPS, this makes clear distinctions at higher level.

Also, EXTIOI device is not user creatable, QOM tree information is a deep
internal detail. I searched mailing list and gitlab issues and I was unable to
find any user reports about "qom tree information confused users", given that
other architectures had taken this approach for a while.

QOM is severing QEMU's internal design, not vice versa.

Thanks
-- 
- Jiaxun