[PATCH 0/2] iommu/riscv: Enable IOMMU_DMA

[PATCH 0/2] iommu/riscv: Enable IOMMU_DMA

[Andrew Jones]

Arguably long overdue, let's start using paging domains. One blocker
to enabling IOMMU_DMA was that platforms with IMSICs would fault on
MSIs - Patch1 handles that. And, since QEMU is still one of the most-
used riscv platforms, another issue is that commit 69541898b71a
("iommu/riscv: Enable SVNAPOT support for contiguous ptes") exposes
a bug in the QEMU RISC-V IOMMU model. A patch for that is now on the
QEMU list[1].

Rest assured that the irqbypass work will get a v3 posted soon. This
series can be independently merged though since we don't need irqbypass
to enable paging domains and deliver MSIs for host devices.

[1] https://lore.kernel.org/all/20260508205129.377032-1-andrew.jones@oss.qualcomm.com/
 

Andrew Jones (1):
  iommu/riscv: Map IMSIC addresses for paging domains

Tomasz Jeznach (1):
  iommu/dma: enable IOMMU_DMA for RISC-V

 drivers/iommu/Kconfig               |  2 +-
 drivers/iommu/riscv/iommu.c         | 34 +++++++++++++++++++++++++++++
 include/linux/irqchip/riscv-imsic.h |  7 ++++++
 3 files changed, 42 insertions(+), 1 deletion(-)

-- 
2.43.0


_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

[PATCH 1/2] iommu/riscv: Map IMSIC addresses for paging domains

[Andrew Jones]

When IOMMU_DMA is enabled, devices get paging domains and MSI writes
to IMSIC interrupt files must be handled correctly in the s-stage.
As the device always writes to the host physical IMSIC addresses,
which the IMSIC irqchip programs directly, install s-stage identity
mappings for the host IMSICs. But, use IOMMU_RESV_DIRECT_RELAXABLE
since the 1:1 mappings aren't required for device assignment.

Loop over the cpus rather than imsic groups to handle asymmetric
configurations.

Signed-off-by: Andrew Jones <andrew.jones@oss.qualcomm.com>
---
 drivers/iommu/riscv/iommu.c         | 34 +++++++++++++++++++++++++++++
 include/linux/irqchip/riscv-imsic.h |  7 ++++++
 2 files changed, 41 insertions(+)

diff --git a/drivers/iommu/riscv/iommu.c b/drivers/iommu/riscv/iommu.c
index a31f50bbad35..3c6aa9d69f95 100644
--- a/drivers/iommu/riscv/iommu.c
+++ b/drivers/iommu/riscv/iommu.c
@@ -19,6 +19,7 @@
 #include <linux/init.h>
 #include <linux/iommu.h>
 #include <linux/iopoll.h>
+#include <linux/irqchip/riscv-imsic.h>
 #include <linux/kernel.h>
 #include <linux/pci.h>
 #include <linux/generic_pt/iommu.h>
@@ -1286,6 +1287,38 @@ static struct iommu_domain *riscv_iommu_alloc_paging_domain(struct device *dev)
 	return &domain->domain;
 }
 
+static void riscv_iommu_get_resv_regions(struct device *dev, struct list_head *head)
+{
+	const struct imsic_global_config *imsic_global;
+	unsigned int cpu;
+
+	if (!imsic_enabled())
+		return;
+
+	imsic_global = imsic_get_global_config();
+
+	for_each_possible_cpu(cpu) {
+		const struct imsic_local_config *local;
+		struct iommu_resv_region *reg;
+
+		local = per_cpu_ptr(imsic_global->local, cpu);
+		if (!local->msi_va)
+			continue;
+
+		/*
+		 * The device always writes to the host physical IMSIC address, so install
+		 * identity mappings directly. Use IOMMU_RESV_DIRECT_RELAXABLE instead of
+		 * IOMMU_RESV_DIRECT since these 1:1 mappings are not required for assigned
+		 * devices.
+		 */
+		reg = iommu_alloc_resv_region(local->msi_pa, IMSIC_MMIO_PAGE_SZ,
+					      IOMMU_WRITE | IOMMU_NOEXEC | IOMMU_MMIO,
+					      IOMMU_RESV_DIRECT_RELAXABLE, GFP_KERNEL);
+		if (reg)
+			list_add_tail(&reg->list, head);
+	}
+}
+
 static int riscv_iommu_attach_blocking_domain(struct iommu_domain *iommu_domain,
 					      struct device *dev,
 					      struct iommu_domain *old)
@@ -1401,6 +1434,7 @@ static const struct iommu_ops riscv_iommu_ops = {
 	.blocked_domain = &riscv_iommu_blocking_domain,
 	.release_domain = &riscv_iommu_blocking_domain,
 	.domain_alloc_paging = riscv_iommu_alloc_paging_domain,
+	.get_resv_regions = riscv_iommu_get_resv_regions,
 	.device_group = riscv_iommu_device_group,
 	.probe_device = riscv_iommu_probe_device,
 	.release_device	= riscv_iommu_release_device,
diff --git a/include/linux/irqchip/riscv-imsic.h b/include/linux/irqchip/riscv-imsic.h
index 4b348836de7a..ba3000f047b0 100644
--- a/include/linux/irqchip/riscv-imsic.h
+++ b/include/linux/irqchip/riscv-imsic.h
@@ -88,6 +88,13 @@ static inline const struct imsic_global_config *imsic_get_global_config(void)
 
 #endif
 
+static inline bool imsic_enabled(void)
+{
+	const struct imsic_global_config *imsic_global = imsic_get_global_config();
+
+	return imsic_global && imsic_global->nr_ids;
+}
+
 #if IS_ENABLED(CONFIG_ACPI) && IS_ENABLED(CONFIG_RISCV_IMSIC)
 int imsic_platform_acpi_probe(struct fwnode_handle *fwnode);
 struct fwnode_handle *imsic_acpi_get_fwnode(struct device *dev);
-- 
2.43.0


_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH 1/2] iommu/riscv: Map IMSIC addresses for paging domains

[fangyu.yu]

>When IOMMU_DMA is enabled, devices get paging domains and MSI writes
>to IMSIC interrupt files must be handled correctly in the s-stage.
>As the device always writes to the host physical IMSIC addresses,
>which the IMSIC irqchip programs directly, install s-stage identity
>mappings for the host IMSICs. But, use IOMMU_RESV_DIRECT_RELAXABLE
>since the 1:1 mappings aren't required for device assignment.
>
>Loop over the cpus rather than imsic groups to handle asymmetric
>configurations.
>
>Signed-off-by: Andrew Jones <andrew.jones@oss.qualcomm.com>
>---
> drivers/iommu/riscv/iommu.c         | 34 +++++++++++++++++++++++++++++
> include/linux/irqchip/riscv-imsic.h |  7 ++++++
> 2 files changed, 41 insertions(+)
>
>diff --git a/drivers/iommu/riscv/iommu.c b/drivers/iommu/riscv/iommu.c
>index a31f50bbad35..3c6aa9d69f95 100644
>--- a/drivers/iommu/riscv/iommu.c
>+++ b/drivers/iommu/riscv/iommu.c
>@@ -19,6 +19,7 @@
> #include <linux/init.h>
> #include <linux/iommu.h>
> #include <linux/iopoll.h>
>+#include <linux/irqchip/riscv-imsic.h>
> #include <linux/kernel.h>
> #include <linux/pci.h>
> #include <linux/generic_pt/iommu.h>
>@@ -1286,6 +1287,38 @@ static struct iommu_domain *riscv_iommu_alloc_paging_domain(struct device *dev)
> 	return &domain->domain;
> }
>
>+static void riscv_iommu_get_resv_regions(struct device *dev, struct list_head *head)
>+{
>+	const struct imsic_global_config *imsic_global;
>+	unsigned int cpu;
>+
>+	if (!imsic_enabled())
>+		return;
>+
>+	imsic_global = imsic_get_global_config();
>+
>+	for_each_possible_cpu(cpu) {
>+		const struct imsic_local_config *local;
>+		struct iommu_resv_region *reg;
>+
>+		local = per_cpu_ptr(imsic_global->local, cpu);
>+		if (!local->msi_va)
>+			continue;
>+
>+		/*
>+		 * The device always writes to the host physical IMSIC address, so install
>+		 * identity mappings directly. Use IOMMU_RESV_DIRECT_RELAXABLE instead of
>+		 * IOMMU_RESV_DIRECT since these 1:1 mappings are not required for assigned
>+		 * devices.
>+		 */
>+		reg = iommu_alloc_resv_region(local->msi_pa, IMSIC_MMIO_PAGE_SZ,
>+					      IOMMU_WRITE | IOMMU_NOEXEC | IOMMU_MMIO,
>+					      IOMMU_RESV_DIRECT_RELAXABLE, GFP_KERNEL);
>+		if (reg)
>+			list_add_tail(&reg->list, head);
>+	}
>+}
>+

Hi Andrew,

Thanks for picking this up -- enabling IOMMU_DMA on RISC-V has been
along-standing gap, and handling the IMSIC MSI mapping is the missing
piece that finally unblocks it.

One concern is that the current implementation emits one 4 KiB
RESV_DIRECT_RELAXABLE region for each possible CPU. On platforms
with hundreds of harts, this noticeably increases the cost of both
.get_resv_regions() and the iommu_create_device_direct_mappings()
walk.

Since interrupt files within one IMSIC group occupy a physically
contiguous range, would it make sense to emit one region per IMSIC
group covering the full group stride, aligned down/up to 2 MiB so
the core can map it as a superpage? This would over-map some padding
within the IMSIC PA window, but RESV_DIRECT_RELAXABLE keeps the
padding out of assigned-device IOVA space, so it looks harmless.

Thanks,
Fangyu

> static int riscv_iommu_attach_blocking_domain(struct iommu_domain *iommu_domain,
> 					      struct device *dev,
> 					      struct iommu_domain *old)
>@@ -1401,6 +1434,7 @@ static const struct iommu_ops riscv_iommu_ops = {
> 	.blocked_domain = &riscv_iommu_blocking_domain,
> 	.release_domain = &riscv_iommu_blocking_domain,
> 	.domain_alloc_paging = riscv_iommu_alloc_paging_domain,
>+	.get_resv_regions = riscv_iommu_get_resv_regions,
> 	.device_group = riscv_iommu_device_group,
> 	.probe_device = riscv_iommu_probe_device,
> 	.release_device	= riscv_iommu_release_device,
>diff --git a/include/linux/irqchip/riscv-imsic.h b/include/linux/irqchip/riscv-imsic.h
>index 4b348836de7a..ba3000f047b0 100644
>--- a/include/linux/irqchip/riscv-imsic.h
>+++ b/include/linux/irqchip/riscv-imsic.h
>@@ -88,6 +88,13 @@ static inline const struct imsic_global_config *imsic_get_global_config(void)
>
> #endif
>
>+static inline bool imsic_enabled(void)
>+{
>+	const struct imsic_global_config *imsic_global = imsic_get_global_config();
>+
>+	return imsic_global && imsic_global->nr_ids;
>+}
>+
> #if IS_ENABLED(CONFIG_ACPI) && IS_ENABLED(CONFIG_RISCV_IMSIC)
> int imsic_platform_acpi_probe(struct fwnode_handle *fwnode);
> struct fwnode_handle *imsic_acpi_get_fwnode(struct device *dev);
>--
>2.43.0

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH 1/2] iommu/riscv: Map IMSIC addresses for paging domains

[Andrew Jones]

On Sat, May 09, 2026 at 10:21:13AM +0800, fangyu.yu@linux.alibaba.com wrote:
> >When IOMMU_DMA is enabled, devices get paging domains and MSI writes
> >to IMSIC interrupt files must be handled correctly in the s-stage.
> >As the device always writes to the host physical IMSIC addresses,
> >which the IMSIC irqchip programs directly, install s-stage identity
> >mappings for the host IMSICs. But, use IOMMU_RESV_DIRECT_RELAXABLE
> >since the 1:1 mappings aren't required for device assignment.
> >
> >Loop over the cpus rather than imsic groups to handle asymmetric
> >configurations.
> >
> >Signed-off-by: Andrew Jones <andrew.jones@oss.qualcomm.com>
> >---
> > drivers/iommu/riscv/iommu.c         | 34 +++++++++++++++++++++++++++++
> > include/linux/irqchip/riscv-imsic.h |  7 ++++++
> > 2 files changed, 41 insertions(+)
> >
> >diff --git a/drivers/iommu/riscv/iommu.c b/drivers/iommu/riscv/iommu.c
> >index a31f50bbad35..3c6aa9d69f95 100644
> >--- a/drivers/iommu/riscv/iommu.c
> >+++ b/drivers/iommu/riscv/iommu.c
> >@@ -19,6 +19,7 @@
> > #include <linux/init.h>
> > #include <linux/iommu.h>
> > #include <linux/iopoll.h>
> >+#include <linux/irqchip/riscv-imsic.h>
> > #include <linux/kernel.h>
> > #include <linux/pci.h>
> > #include <linux/generic_pt/iommu.h>
> >@@ -1286,6 +1287,38 @@ static struct iommu_domain *riscv_iommu_alloc_paging_domain(struct device *dev)
> > 	return &domain->domain;
> > }
> >
> >+static void riscv_iommu_get_resv_regions(struct device *dev, struct list_head *head)
> >+{
> >+	const struct imsic_global_config *imsic_global;
> >+	unsigned int cpu;
> >+
> >+	if (!imsic_enabled())
> >+		return;
> >+
> >+	imsic_global = imsic_get_global_config();
> >+
> >+	for_each_possible_cpu(cpu) {
> >+		const struct imsic_local_config *local;
> >+		struct iommu_resv_region *reg;
> >+
> >+		local = per_cpu_ptr(imsic_global->local, cpu);
> >+		if (!local->msi_va)
> >+			continue;
> >+
> >+		/*
> >+		 * The device always writes to the host physical IMSIC address, so install
> >+		 * identity mappings directly. Use IOMMU_RESV_DIRECT_RELAXABLE instead of
> >+		 * IOMMU_RESV_DIRECT since these 1:1 mappings are not required for assigned
> >+		 * devices.
> >+		 */
> >+		reg = iommu_alloc_resv_region(local->msi_pa, IMSIC_MMIO_PAGE_SZ,
> >+					      IOMMU_WRITE | IOMMU_NOEXEC | IOMMU_MMIO,
> >+					      IOMMU_RESV_DIRECT_RELAXABLE, GFP_KERNEL);
> >+		if (reg)
> >+			list_add_tail(&reg->list, head);
> >+	}
> >+}
> >+
> 
> Hi Andrew,
> 
> Thanks for picking this up -- enabling IOMMU_DMA on RISC-V has been
> along-standing gap, and handling the IMSIC MSI mapping is the missing
> piece that finally unblocks it.
> 
> One concern is that the current implementation emits one 4 KiB
> RESV_DIRECT_RELAXABLE region for each possible CPU. On platforms
> with hundreds of harts, this noticeably increases the cost of both
> .get_resv_regions() and the iommu_create_device_direct_mappings()
> walk.
> 
> Since interrupt files within one IMSIC group occupy a physically
> contiguous range, would it make sense to emit one region per IMSIC
> group covering the full group stride, aligned down/up to 2 MiB so
> the core can map it as a superpage? This would over-map some padding
> within the IMSIC PA window, but RESV_DIRECT_RELAXABLE keeps the
> padding out of assigned-device IOVA space, so it looks harmless.
> 

Hi Fangyu,

Thanks for pointing out this issue. We'll need to decide how much we
want to isolate the devices from VMs in order to address it, though,
because, if we do group mappings, then we'll also be exposing the guest
interrupt files to the devices.

I'll certainly send a v2 to do larger mappings when s-mode imsics are
contiguous (nr-guest-files = 0) and then we can consider creating a
way to opt-in to group mappings even when nr-guest-files != 0. How's
that sound?

Thanks,
drew

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: Re: [PATCH 1/2] iommu/riscv: Map IMSIC addresses for paging domains

[fangyu.yu]

>> >When IOMMU_DMA is enabled, devices get paging domains and MSI writes
>> >to IMSIC interrupt files must be handled correctly in the s-stage.
>> >As the device always writes to the host physical IMSIC addresses,
>> >which the IMSIC irqchip programs directly, install s-stage identity
>> >mappings for the host IMSICs. But, use IOMMU_RESV_DIRECT_RELAXABLE
>> >since the 1:1 mappings aren't required for device assignment.
>> >
>> >Loop over the cpus rather than imsic groups to handle asymmetric
>> >configurations.
>> >
>> >Signed-off-by: Andrew Jones <andrew.jones@oss.qualcomm.com>
>> >---
>> > drivers/iommu/riscv/iommu.c         | 34 +++++++++++++++++++++++++++++
>> > include/linux/irqchip/riscv-imsic.h |  7 ++++++
>> > 2 files changed, 41 insertions(+)
>> >
>> >diff --git a/drivers/iommu/riscv/iommu.c b/drivers/iommu/riscv/iommu.c
>> >index a31f50bbad35..3c6aa9d69f95 100644
>> >--- a/drivers/iommu/riscv/iommu.c
>> >+++ b/drivers/iommu/riscv/iommu.c
>> >@@ -19,6 +19,7 @@
>> > #include <linux/init.h>
>> > #include <linux/iommu.h>
>> > #include <linux/iopoll.h>
>> >+#include <linux/irqchip/riscv-imsic.h>
>> > #include <linux/kernel.h>
>> > #include <linux/pci.h>
>> > #include <linux/generic_pt/iommu.h>
>> >@@ -1286,6 +1287,38 @@ static struct iommu_domain *riscv_iommu_alloc_paging_domain(struct device *dev)
>> > 	return &domain->domain;
>> > }
>> >
>> >+static void riscv_iommu_get_resv_regions(struct device *dev, struct list_head *head)
>> >+{
>> >+	const struct imsic_global_config *imsic_global;
>> >+	unsigned int cpu;
>> >+
>> >+	if (!imsic_enabled())
>> >+		return;
>> >+
>> >+	imsic_global = imsic_get_global_config();
>> >+
>> >+	for_each_possible_cpu(cpu) {
>> >+		const struct imsic_local_config *local;
>> >+		struct iommu_resv_region *reg;
>> >+
>> >+		local = per_cpu_ptr(imsic_global->local, cpu);
>> >+		if (!local->msi_va)
>> >+			continue;
>> >+
>> >+		/*
>> >+		 * The device always writes to the host physical IMSIC address, so install
>> >+		 * identity mappings directly. Use IOMMU_RESV_DIRECT_RELAXABLE instead of
>> >+		 * IOMMU_RESV_DIRECT since these 1:1 mappings are not required for assigned
>> >+		 * devices.
>> >+		 */
>> >+		reg = iommu_alloc_resv_region(local->msi_pa, IMSIC_MMIO_PAGE_SZ,
>> >+					      IOMMU_WRITE | IOMMU_NOEXEC | IOMMU_MMIO,
>> >+					      IOMMU_RESV_DIRECT_RELAXABLE, GFP_KERNEL);
>> >+		if (reg)
>> >+			list_add_tail(&reg->list, head);
>> >+	}
>> >+}
>> >+
>> 
>> Hi Andrew,
>> 
>> Thanks for picking this up -- enabling IOMMU_DMA on RISC-V has been
>> along-standing gap, and handling the IMSIC MSI mapping is the missing
>> piece that finally unblocks it.
>> 
>> One concern is that the current implementation emits one 4 KiB
>> RESV_DIRECT_RELAXABLE region for each possible CPU. On platforms
>> with hundreds of harts, this noticeably increases the cost of both
>> .get_resv_regions() and the iommu_create_device_direct_mappings()
>> walk.
>> 
>> Since interrupt files within one IMSIC group occupy a physically
>> contiguous range, would it make sense to emit one region per IMSIC
>> group covering the full group stride, aligned down/up to 2 MiB so
>> the core can map it as a superpage? This would over-map some padding
>> within the IMSIC PA window, but RESV_DIRECT_RELAXABLE keeps the
>> padding out of assigned-device IOVA space, so it looks harmless.
>> 
>
>Hi Fangyu,
>
>Thanks for pointing out this issue. We'll need to decide how much we
>want to isolate the devices from VMs in order to address it, though,
>because, if we do group mappings, then we'll also be exposing the guest
>interrupt files to the devices.
>
>I'll certainly send a v2 to do larger mappings when s-mode imsics are
>contiguous (nr-guest-files = 0) and then we can consider creating a
>way to opt-in to group mappings even when nr-guest-files != 0. How's
>that sound?
>

Hi Andrew,

As you mentioned, my suggestion could also expose the guest interrupt
files to the device-visible/mappable range, but the current approach
already appears to provide only limited isolation. So I think this
change would not materially increase the risk.

In any case, I respect your decision, and we can certainly proceed
with your current suggestion for now.

Thanks,
Fangyu

>Thanks,
>drew

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

[PATCH 2/2] iommu/dma: enable IOMMU_DMA for RISC-V

[Andrew Jones]

From: Tomasz Jeznach <tjeznach@rivosinc.com>

With iommu/riscv driver available we can enable IOMMU_DMA support
for RISC-V architecture.

Signed-off-by: Tomasz Jeznach <tjeznach@rivosinc.com>
Signed-off-by: Andrew Jones <andrew.jones@oss.qualcomm.com>
Reviewed-by: Nutty Liu <nutty.liu@hotmail.com>
---
 drivers/iommu/Kconfig | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig
index f86262b11416..34d8a792339f 100644
--- a/drivers/iommu/Kconfig
+++ b/drivers/iommu/Kconfig
@@ -151,7 +151,7 @@ config OF_IOMMU
 
 # IOMMU-agnostic DMA-mapping layer
 config IOMMU_DMA
-	def_bool ARM64 || X86 || S390
+	def_bool ARM64 || X86 || S390 || RISCV
 	select DMA_OPS_HELPERS
 	select IOMMU_API
 	select IOMMU_IOVA
-- 
2.43.0


_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv