[RFC PATCH 0/6] iommu/riscv: Add MRIF support

[RFC PATCH 0/6] iommu/riscv: Add MRIF support

[fangyu.yu]

From: Fangyu Yu <fangyu.yu@linux.alibaba.com>

According to the RISC-V IOMMU Spec, an IOMMU may optionally support
memory-resident interrupt files  (MRIFs). When the guest  interrupt 
files are used up, an MRIF can record an incoming MSI.

At present, the hypervisor has allocated an MRIF for each IMSIC, we
only need to configure the PPN of the MRIF into the MSI PTE of  the
IOMMU in MRIF mode.At the same time, we also need to configure NPPN
and NID for notice MSIs, in these patches,we use the host interrupt
(allocated  via  VFIO) as the notice MSIs, so that we don't need to 
allocate a new MSI interrupt,  and we can easily redirect the guest
interrupt back to the host interrupt when MRIF is not  supported on
the IOMMU hardware.

This RFC series are based on [1] by Andrew Jones.

Self Test:
-----------
1. Key parameters for starting host QEMU:
./qemu-system-riscv64  \
-M virt,aia=aplic-imsic,aia-guests=1 -m 8G -smp 2  \
-nographic -device riscv-iommu-pci,vendor-id=0x1efd,device-id=0x0008 \
-netdev user,id=net1,hostfwd=tcp::2323-:22 \
-device e1000e,netdev=net1 \
-drive file=./nvme_disk.qcow2,if=none,id=nvm \
-device nvme,serial=deadbeef,drive=nvm \
...

2. Steps to start a virtual machine：
# lspci
00:00.0 Host bridge: Red Hat, Inc. QEMU PCIe Host bridge
00:01.0 IOMMU: Device 1efd:0008 (rev 01)
00:02.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
00:03.0 Non-Volatile memory controller: Red Hat, Inc. QEMU NVM Express Controller (rev 02)
00:04.0 Unclassified device [0002]: Red Hat, Inc. Virtio filesystem
# echo 0000:00:02.0 > /sys/bus/pci/drivers/e1000e/unbind
# echo 0000:00:03.0 > /sys/bus/pci/drivers/nvme/unbind
# echo 8086 10d3 > /sys/bus/pci/drivers/vfio-pci/new_id
# echo 1b36 0010 > /sys/bus/pci/drivers/vfio-pci/new_id

qemu-system-riscv64 -M virt,aia=aplic-imsic --enable-kvm -m 2G -smp 4 \
-device vfio-pci,host=0000:00:02.0 \
-device vfio-pci,host=0000:00:03.0 \
...

3. Test within guest os:
root@qemu:/mnt/nvme# lspci
00:00.0 Host bridge: Red Hat, Inc. QEMU PCIe Host bridge
00:01.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
00:02.0 Non-Volatile memory controller: Red Hat, Inc. QEMU NVM Express Controller (rev 02)
root@qemu:~# mount /dev/nvme0n1p1 /mnt/nvme/
root@qemu:~# cd /mnt/nvme/
root@qemu:/mnt/nvme# ping 11.122.129.243 -i 0.1 | tee c.txt
64 bytes from 11.122.129.243: icmp_seq=18533 ttl=255 time=1.18 ms
64 bytes from 11.122.129.243: icmp_seq=18534 ttl=255 time=1.60 ms
^C
--- 11.122.129.243 ping statistics ---
18534 packets transmitted, 18534 received, 0% packet loss, time 1934380ms
rtt min/avg/max/mdev = 0.437/11.986/3451.393/118.494 ms, pipe 34

root@qemu:/mnt/nvme# cat /proc/interrupts
	CPU0       CPU1       CPU2       CPU3
10:      49856     218638     192244      58721 RISC-V INTC   5 Edge      riscv-timer
12:          0     105519          0          0 PCI-MSIX-0000:00:01.0   0 Edge      eth0-rx-0
13:          0          0      97134          0 PCI-MSIX-0000:00:01.0   1 Edge      eth0-tx-0
14:          0          0          0          2 PCI-MSIX-0000:00:01.0   2 Edge      eth0
16:          0      42752          0          0 PCI-MSIX-0000:00:02.0   0 Edge      nvme0q0
17:        376          0          0          0 PCI-MSIX-0000:00:02.0   1 Edge      nvme0q1
18:          0       1308          0          0 PCI-MSIX-0000:00:02.0   2 Edge      nvme0q2
19:          0          0      49757          0 PCI-MSIX-0000:00:02.0   3 Edge      nvme0q3
20:          0          0          0       1282 PCI-MSIX-0000:00:02.0   4 Edge      nvme0q4

[1] https://github.com/jones-drew/linux/tree/riscv/iommu-irqbypass-rfc-v2-rc1

Fangyu Yu (6):
  RISC-V: Add more elements to irqbypass vcpu_info
  RISC-V: KVM: Transfer the physical address of MRIF to iommu-ir
  RISC-V: KVM: Add a xarray to record host irq msg
  iommu/riscv: Add irq_mask and irq_ack configure for iommu-ir
  iommu/riscv: Add MRIF mode support
  RISC-V: KVM: Check the MRIF in notice MSI irq handler

 arch/riscv/include/asm/irq.h     |   3 +
 arch/riscv/kvm/aia_imsic.c       | 119 ++++++++++++++++++++++++++++---
 drivers/iommu/riscv/iommu-bits.h |   6 ++
 drivers/iommu/riscv/iommu-ir.c   |  40 +++++++++--
 4 files changed, 156 insertions(+), 12 deletions(-)

-- 
2.49.0


-- 
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv

[RFC PATCH 1/6] RISC-V: Add more elements to irqbypass vcpu_info

[fangyu.yu]

From: Fangyu Yu <fangyu.yu@linux.alibaba.com>

To support MRIF mode, we need to add more elements to
let the iommu driver get the ppn of MRIF.

Signed-off-by: Fangyu Yu <fangyu.yu@linux.alibaba.com>
---
 arch/riscv/include/asm/irq.h | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/arch/riscv/include/asm/irq.h b/arch/riscv/include/asm/irq.h
index 8588667cbb5f..6293ac00e051 100644
--- a/arch/riscv/include/asm/irq.h
+++ b/arch/riscv/include/asm/irq.h
@@ -30,6 +30,9 @@ struct riscv_iommu_vcpu_info {
 	u32 group_index_shift;
 	u64 gpa;
 	u64 hpa;
+	u32 host_irq;
+	bool mrif;
+	struct msi_msg *host_msg;
 };
 
 #ifdef CONFIG_ACPI
-- 
2.49.0


-- 
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv

[RFC PATCH 2/6] RISC-V: KVM: Transfer the physical address of MRIF to iommu-ir

[fangyu.yu]

From: Fangyu Yu <fangyu.yu@linux.alibaba.com>

According to the RISC-V IOMMU Spec, an IOMMU may optionally
support memory-resident interrupt files (MRIFs).

When the guest interrupt files are used up, we transfer the
physical address of MRIF to iommu-ir, and enable MRIF  mode
if the iommu-ir supports.

Signed-off-by: Fangyu Yu <fangyu.yu@linux.alibaba.com>
---
 arch/riscv/kvm/aia_imsic.c | 23 +++++++++++++++++------
 1 file changed, 17 insertions(+), 6 deletions(-)

diff --git a/arch/riscv/kvm/aia_imsic.c b/arch/riscv/kvm/aia_imsic.c
index 8f5703d9112a..e91164742fd0 100644
--- a/arch/riscv/kvm/aia_imsic.c
+++ b/arch/riscv/kvm/aia_imsic.c
@@ -21,6 +21,8 @@
 
 #define IMSIC_MAX_EIX	(IMSIC_MAX_ID / BITS_PER_TYPE(u64))
 
+static int kvm_riscv_vcpu_irq_update(struct kvm_vcpu *vcpu);
+
 struct imsic_mrif_eix {
 	unsigned long eip[BITS_PER_TYPE(u64) / BITS_PER_LONG];
 	unsigned long eie[BITS_PER_TYPE(u64) / BITS_PER_LONG];
@@ -717,7 +719,8 @@ void kvm_riscv_vcpu_aia_imsic_release(struct kvm_vcpu *vcpu)
 				 vcpu->arch.aia_context.imsic_addr,
 				 IMSIC_MMIO_PAGE_SZ);
 
-	/* TODO: Purge the IOMMU mapping ??? */
+	/* Update the IOMMU mapping */
+	kvm_riscv_vcpu_irq_update(vcpu);
 
 	/*
 	 * At this point, all interrupt producers have been re-directed
@@ -795,13 +798,14 @@ int kvm_arch_update_irqfd_routing(struct kvm *kvm, unsigned int host_irq,
 
 			read_lock_irqsave(&imsic->vsfile_lock, flags);
 
-			if (WARN_ON_ONCE(imsic->vsfile_cpu < 0)) {
-				read_unlock_irqrestore(&imsic->vsfile_lock, flags);
-				goto out;
+			if (imsic->vsfile_cpu < 0) {
+				vcpu_info.hpa = imsic->swfile_pa;
+				vcpu_info.mrif = true;
+			} else {
+				vcpu_info.hpa = imsic->vsfile_pa;
+				vcpu_info.mrif = false;
 			}
 
-			vcpu_info.hpa = imsic->vsfile_pa;
-
 			ret = irq_set_vcpu_affinity(host_irq, &vcpu_info);
 			if (ret) {
 				read_unlock_irqrestore(&imsic->vsfile_lock, flags);
@@ -844,6 +848,13 @@ static int kvm_riscv_vcpu_irq_update(struct kvm_vcpu *vcpu)
 		if (!irqfd->producer)
 			continue;
 		host_irq = irqfd->producer->irq;
+
+		if (imsic->vsfile_cpu < 0) {
+			vcpu_info.hpa = imsic->swfile_pa;
+			vcpu_info.mrif = true;
+		} else {
+			vcpu_info.mrif = false;
+		}
 		ret = irq_set_vcpu_affinity(host_irq, &vcpu_info);
 		if (ret) {
 			spin_unlock_irq(&kvm->irqfds.lock);
-- 
2.49.0


-- 
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv

[RFC PATCH 3/6] RISC-V: KVM: Add a xarray to record host irq msg

[fangyu.yu]

From: Fangyu Yu <fangyu.yu@linux.alibaba.com>

In the irq bypass scenario,the host interrupt comes from VFIO, and
it is an enabled MSI/MSI-X interrupt.  Due to the  reconfiguration
of the PCI-e BAR space during the irq bypass process,this host irq
will not be triggered in the host system.

We can use this host irq as a notice MSI in IOMMU MRIF mode.

Signed-off-by: Fangyu Yu <fangyu.yu@linux.alibaba.com>
---
 arch/riscv/kvm/aia_imsic.c | 69 +++++++++++++++++++++++++++++++++++++-
 1 file changed, 68 insertions(+), 1 deletion(-)

diff --git a/arch/riscv/kvm/aia_imsic.c b/arch/riscv/kvm/aia_imsic.c
index e91164742fd0..58807e68a3dd 100644
--- a/arch/riscv/kvm/aia_imsic.c
+++ b/arch/riscv/kvm/aia_imsic.c
@@ -60,6 +60,9 @@ struct imsic {
 	struct imsic_mrif *swfile;
 	phys_addr_t swfile_pa;
 	raw_spinlock_t swfile_extirq_lock;
+
+	bool mrif_support;
+	struct xarray hostirq_array;	/* Attached host irq array */
 };
 
 #define imsic_vs_csr_read(__c)			\
@@ -740,6 +743,57 @@ void kvm_riscv_vcpu_aia_imsic_release(struct kvm_vcpu *vcpu)
 	kvm_riscv_aia_free_hgei(old_vsfile_cpu, old_vsfile_hgei);
 }
 
+static int kvm_arch_update_irqfd_unset(struct kvm *kvm, unsigned int host_irq)
+{
+	struct kvm_vcpu *vcpu;
+	unsigned long tmp;
+
+	kvm_for_each_vcpu(tmp, vcpu, kvm) {
+		struct imsic *imsic = vcpu->arch.aia_context.imsic_state;
+		struct msi_msg *curr = xa_load(&imsic->hostirq_array, host_irq);
+
+		if (!curr)
+			continue;
+
+		xa_erase(&imsic->hostirq_array, host_irq);
+		kfree(curr);
+		break;
+	}
+
+	return irq_set_vcpu_affinity(host_irq, NULL);
+}
+
+static struct msi_msg *kvm_arch_update_irqfd_hostirq(struct imsic *imsic,
+						     unsigned int host_irq, int *ret,
+						     struct kvm_kernel_irq_routing_entry *e)
+{
+	struct msi_msg *priv_msg = xa_load(&imsic->hostirq_array, host_irq);
+
+	if (!priv_msg) {
+		priv_msg = kzalloc(sizeof(*priv_msg), GFP_KERNEL);
+		if (!priv_msg) {
+			*ret = -ENOMEM;
+			goto out;
+		}
+
+		struct msi_msg host_msg, *curr;
+
+		get_cached_msi_msg(host_irq, &host_msg);
+		priv_msg[0] = host_msg;
+		curr = xa_cmpxchg(&imsic->hostirq_array, host_irq,
+				NULL, priv_msg, GFP_ATOMIC);
+		if (WARN_ON_ONCE(curr)) {
+			*ret = xa_err(curr) ? : -EBUSY;
+			kfree(priv_msg);
+			goto out;
+		}
+	}
+	*ret = 0;
+
+out:
+	return priv_msg;
+}
+
 int kvm_arch_update_irqfd_routing(struct kvm *kvm, unsigned int host_irq,
 				  uint32_t guest_irq, bool set)
 {
@@ -750,7 +804,7 @@ int kvm_arch_update_irqfd_routing(struct kvm *kvm, unsigned int host_irq,
 	int idx, ret = -ENXIO;
 
 	if (!set)
-		return irq_set_vcpu_affinity(host_irq, NULL);
+		return kvm_arch_update_irqfd_unset(kvm, host_irq);
 
 	idx = srcu_read_lock(&kvm->irq_srcu);
 	irq_rt = srcu_dereference(kvm->irq_routing, &kvm->irq_srcu);
@@ -795,6 +849,11 @@ int kvm_arch_update_irqfd_routing(struct kvm *kvm, unsigned int host_irq,
 
 			vcpu_info.msi_addr_pattern = tppn & ~vcpu_info.msi_addr_mask;
 			vcpu_info.gpa = target;
+			vcpu_info.host_irq = host_irq;
+			vcpu_info.host_msg =
+				    kvm_arch_update_irqfd_hostirq(imsic, host_irq, &ret, e);
+			if (ret)
+				goto out;
 
 			read_lock_irqsave(&imsic->vsfile_lock, flags);
 
@@ -848,6 +907,10 @@ static int kvm_riscv_vcpu_irq_update(struct kvm_vcpu *vcpu)
 		if (!irqfd->producer)
 			continue;
 		host_irq = irqfd->producer->irq;
+		vcpu_info.host_irq = host_irq;
+		vcpu_info.host_msg = xa_load(&imsic->hostirq_array, host_irq);
+		if (!vcpu_info.host_msg)
+			continue;
 
 		if (imsic->vsfile_cpu < 0) {
 			vcpu_info.hpa = imsic->swfile_pa;
@@ -855,6 +918,7 @@ static int kvm_riscv_vcpu_irq_update(struct kvm_vcpu *vcpu)
 		} else {
 			vcpu_info.mrif = false;
 		}
+
 		ret = irq_set_vcpu_affinity(host_irq, &vcpu_info);
 		if (ret) {
 			spin_unlock_irq(&kvm->irqfds.lock);
@@ -1195,6 +1259,9 @@ int kvm_riscv_vcpu_aia_imsic_init(struct kvm_vcpu *vcpu)
 	imsic->swfile_pa = page_to_phys(swfile_page);
 	raw_spin_lock_init(&imsic->swfile_extirq_lock);
 
+	xa_init(&imsic->hostirq_array);
+	imsic->mrif_support = false;
+
 	/* Setup IO device */
 	kvm_iodevice_init(&imsic->iodev, &imsic_iodoev_ops);
 	mutex_lock(&kvm->slots_lock);
-- 
2.49.0


-- 
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv

[RFC PATCH 4/6] iommu/riscv: Add irq_mask and irq_ack configure for iommu-ir

[fangyu.yu]

From: Fangyu Yu <fangyu.yu@linux.alibaba.com>

The irq_mask and irq_ack are required for host irq to be triggered
under the host system.

Signed-off-by: Fangyu Yu <fangyu.yu@linux.alibaba.com>
---
 drivers/iommu/riscv/iommu-ir.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/drivers/iommu/riscv/iommu-ir.c b/drivers/iommu/riscv/iommu-ir.c
index 5461dbe18159..73f552ed5b65 100644
--- a/drivers/iommu/riscv/iommu-ir.c
+++ b/drivers/iommu/riscv/iommu-ir.c
@@ -208,6 +208,7 @@ static struct irq_chip riscv_iommu_irq_chip = {
 	.irq_unmask		= irq_chip_unmask_parent,
 	.irq_set_affinity	= irq_chip_set_affinity_parent,
 	.irq_set_vcpu_affinity	= riscv_iommu_irq_set_vcpu_affinity,
+	.irq_ack		= irq_chip_ack_parent,
 };
 
 static int riscv_iommu_irq_domain_alloc_irqs(struct irq_domain *irqdomain,
@@ -239,7 +240,9 @@ static const struct msi_parent_ops riscv_iommu_msi_parent_ops = {
 	.supported_flags	= MSI_GENERIC_FLAGS_MASK |
 				  MSI_FLAG_PCI_MSIX,
 	.required_flags		= MSI_FLAG_USE_DEF_DOM_OPS |
-				  MSI_FLAG_USE_DEF_CHIP_OPS,
+				  MSI_FLAG_USE_DEF_CHIP_OPS |
+				  MSI_FLAG_PCI_MSI_MASK_PARENT,
+	.chip_flags		= MSI_CHIP_FLAG_SET_ACK,
 	.init_dev_msi_info	= msi_parent_init_dev_msi_info,
 };
 
-- 
2.49.0


-- 
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv

[RFC PATCH 5/6] iommu/riscv: Add MRIF mode support

[fangyu.yu]

From: Fangyu Yu <fangyu.yu@linux.alibaba.com>

If the guest interrupt files are exhausted and the MRIF is supported
we configure the MSI PTE with MRIF mode,  and set  the  NPPN and NID
using notice MSI from host irq.

Otherwise, we redirect the guest interrupt back to the original host
irq and inject the interrupt into the guest machine through irqfd.

Signed-off-by: Fangyu Yu <fangyu.yu@linux.alibaba.com>
---
 drivers/iommu/riscv/iommu-bits.h |  6 ++++++
 drivers/iommu/riscv/iommu-ir.c   | 35 +++++++++++++++++++++++++++++---
 2 files changed, 38 insertions(+), 3 deletions(-)

diff --git a/drivers/iommu/riscv/iommu-bits.h b/drivers/iommu/riscv/iommu-bits.h
index d3d98dbed709..3af6436d5c5c 100644
--- a/drivers/iommu/riscv/iommu-bits.h
+++ b/drivers/iommu/riscv/iommu-bits.h
@@ -39,6 +39,12 @@
 /* RISC-V IOMMU PPN <> PHYS address conversions, PHYS <=> PPN[53:10] */
 #define riscv_iommu_phys_to_ppn(pa)	(((pa) >> 2) & (((1ULL << 44) - 1) << 10))
 #define riscv_iommu_ppn_to_phys(pn)	(((pn) << 2) & (((1ULL << 44) - 1) << 12))
+/* RISC-V IOMMU MRIF Address <> PHYS address conversions, PHYS <=> MRIF[53:7] */
+#define riscv_iommu_phys_to_mrif(pa)	(((pa) >> 2) & (((1ULL << 47) - 1) << 7))
+/* RISC-V IOMMU nppn <> PHYS address conversions, PHYS <=> nppn[53:10] */
+#define riscv_iommu_phys_to_nppn(pa)	(((pa) >> 2) & (((1ULL << 44) - 1) << 10))
+#define riscv_iommu_data_to_nid(data)   \
+		((((data) & 0x3FFULL)) | (((((data) >> 10) & 1ULL)) << 60))
 
 /* 5.3 IOMMU Capabilities (64bits) */
 #define RISCV_IOMMU_REG_CAPABILITIES		0x0000
diff --git a/drivers/iommu/riscv/iommu-ir.c b/drivers/iommu/riscv/iommu-ir.c
index 73f552ed5b65..f3ebf62de53e 100644
--- a/drivers/iommu/riscv/iommu-ir.c
+++ b/drivers/iommu/riscv/iommu-ir.c
@@ -150,9 +150,12 @@ static int riscv_iommu_irq_set_vcpu_affinity(struct irq_data *data, void *info)
 {
 	struct riscv_iommu_vcpu_info *vcpu_info = info;
 	struct riscv_iommu_domain *domain = data->domain->host_data;
+	struct device *dev = msi_desc_to_dev(irq_data_get_msi_desc(data));
+	struct riscv_iommu_device *iommu = dev_to_iommu(dev);
 	struct riscv_iommu_msipte *pte;
 	int ret = -EINVAL;
-	u64 pteval;
+	u64 pteval, mrifval = 0;
+	bool mrif_support = (iommu->caps & RISCV_IOMMU_CAPABILITIES_MSI_MRIF);
 
 	if (WARN_ON(domain->domain.type != IOMMU_DOMAIN_UNMANAGED))
 		return ret;
@@ -186,12 +189,38 @@ static int riscv_iommu_irq_set_vcpu_affinity(struct irq_data *data, void *info)
 	if (!pte)
 		goto out_unlock;
 
-	pteval = FIELD_PREP(RISCV_IOMMU_MSIPTE_M, 3) |
-		 riscv_iommu_phys_to_ppn(vcpu_info->hpa) |
+	if (!vcpu_info->mrif) {
+		pteval = FIELD_PREP(RISCV_IOMMU_MSIPTE_M, 3) |
+			 riscv_iommu_phys_to_ppn(vcpu_info->hpa) |
+			 FIELD_PREP(RISCV_IOMMU_MSIPTE_V, 1);
+		goto update_pte;
+	}
+
+	pteval = FIELD_PREP(RISCV_IOMMU_MSIPTE_M, 1) |
+		 riscv_iommu_phys_to_mrif(vcpu_info->hpa) |
 		 FIELD_PREP(RISCV_IOMMU_MSIPTE_V, 1);
+	if (mrif_support) {
+		mrifval = riscv_iommu_data_to_nid(vcpu_info->host_msg->data) |
+				riscv_iommu_phys_to_nppn(
+				(u64)vcpu_info->host_msg->address_hi << 32 |
+				vcpu_info->host_msg->address_lo);
+	} else {
+		/* If the guest interrupt file is exhausted and MRIF is not supported, we
+		 * redirect the guest interrupt back to the original host interrupt and
+		 * inject the interrupt into the guest machine through irqfd.
+		 */
+		struct irq_data *irqdata = irq_get_irq_data(vcpu_info->host_irq);
+
+		irq_data_get_irq_chip(irqdata)->irq_write_msi_msg(irqdata,
+						vcpu_info->host_msg);
+		ret = -ENODEV;
+		goto out_unlock;
+	}
 
+update_pte:
 	if (pte->pte != pteval) {
 		pte->pte = pteval;
+		pte->mrif_info = mrifval;
 		riscv_iommu_ir_msitbl_inval(domain, pte);
 	}
 
-- 
2.49.0


-- 
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv

[RFC PATCH 6/6] RISC-V: KVM: Check the MRIF in notice MSI irq handler

[fangyu.yu]

From: Fangyu Yu <fangyu.yu@linux.alibaba.com>

In MRIF mode,  the Advanced Interrupt Architecture Specification
defines the operation to store the  incoming MSIs into the  MRIF
and to generate the notice MSI,the software shold check the MRIF
in the notice MSI irq handler.

And without MRIF support,we redirect the guest interrupt back to
the original host interrupt,  the software update and check MRIF
in host irq handler.

Signed-off-by: Fangyu Yu <fangyu.yu@linux.alibaba.com>
---
 arch/riscv/kvm/aia_imsic.c | 29 +++++++++++++++++++++++++++--
 1 file changed, 27 insertions(+), 2 deletions(-)

diff --git a/arch/riscv/kvm/aia_imsic.c b/arch/riscv/kvm/aia_imsic.c
index 58807e68a3dd..f0d1acde0dd4 100644
--- a/arch/riscv/kvm/aia_imsic.c
+++ b/arch/riscv/kvm/aia_imsic.c
@@ -867,11 +867,16 @@ int kvm_arch_update_irqfd_routing(struct kvm *kvm, unsigned int host_irq,
 
 			ret = irq_set_vcpu_affinity(host_irq, &vcpu_info);
 			if (ret) {
+				if (ret == -ENODEV) {
+					imsic->mrif_support = false;
+					ret = 0;
+				}
 				read_unlock_irqrestore(&imsic->vsfile_lock, flags);
 				goto out;
 			}
 
-			irq_data_get_irq_chip(irqdata)->irq_write_msi_msg(irqdata, msg);
+			if (imsic->mrif_support)
+				irq_data_get_irq_chip(irqdata)->irq_write_msi_msg(irqdata, msg);
 
 			read_unlock_irqrestore(&imsic->vsfile_lock, flags);
 		}
@@ -921,6 +926,10 @@ static int kvm_riscv_vcpu_irq_update(struct kvm_vcpu *vcpu)
 
 		ret = irq_set_vcpu_affinity(host_irq, &vcpu_info);
 		if (ret) {
+			if (ret == -ENODEV) {
+				imsic->mrif_support = false;
+				ret = 0;
+			}
 			spin_unlock_irq(&kvm->irqfds.lock);
 			return ret;
 		}
@@ -1182,8 +1191,24 @@ int kvm_riscv_vcpu_aia_imsic_inject(struct kvm_vcpu *vcpu,
 	if (imsic->vsfile_cpu >= 0) {
 		writel(iid, imsic->vsfile_va + IMSIC_MMIO_SETIPNUM_LE);
 	} else {
+		if (imsic->mrif_support) {
+			struct msi_msg *msg;
+			unsigned long idx;
+
+			/* In MRIF mode, the noticed MSI irq handler will call here to
+			 * determine whether the MRIF has been updated.Since the IOMMU
+			 * hardware has updated the MRIF,the software does not need to
+			 * update the MRIF file again.
+			 */
+			xa_for_each(&imsic->hostirq_array, idx, msg) {
+				if (msg->data == iid)
+					goto skip_update_swfile;
+			}
+		}
 		eix = &imsic->swfile->eix[iid / BITS_PER_TYPE(u64)];
 		set_bit(iid & (BITS_PER_TYPE(u64) - 1), eix->eip);
+
+skip_update_swfile:
 		imsic_swfile_extirq_update(vcpu);
 	}
 
@@ -1260,7 +1285,7 @@ int kvm_riscv_vcpu_aia_imsic_init(struct kvm_vcpu *vcpu)
 	raw_spin_lock_init(&imsic->swfile_extirq_lock);
 
 	xa_init(&imsic->hostirq_array);
-	imsic->mrif_support = false;
+	imsic->mrif_support = true;
 
 	/* Setup IO device */
 	kvm_iodevice_init(&imsic->iodev, &imsic_iodoev_ops);
-- 
2.49.0


-- 
kvm-riscv mailing list
kvm-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kvm-riscv