[PATCH 0/9] Initial DMABUF support for iommufd

[PATCH 0/9] Initial DMABUF support for iommufd

[Jason Gunthorpe]

This series is the start of adding full DMABUF support to
iommufd. Currently it is limited to only work with VFIO's DMABUF exporter.
It sits on top of Leon's series to add a DMABUF exporter to VFIO:

  https://lore.kernel.org/r/20251106-dmabuf-vfio-v7-0-2503bf390699@nvidia.com

The existing IOMMU_IOAS_MAP_FILE is enhanced to detect DMABUF fd's, but
otherwise works the same as it does today for a memfd. The user can select
a slice of the FD to map into the ioas and if the underliyng alignment
requirements are met it will be placed in the iommu_domain.

Though limited, it is enough to allow a VMM like QEMU to connect MMIO BAR
memory from VFIO to an iommu_domain controlled by iommufd. This is used
for PCI Peer to Peer support in VMs, and is the last feature that the VFIO
type 1 container has that iommufd couldn't do.

The VFIO type1 version extracts raw PFNs from VMAs, which has no lifetime
control and is a use-after-free security problem.

Instead iommufd relies on revokable DMABUFs. Whenever VFIO thinks there
should be no access to the MMIO it can shoot down the mapping in iommufd
which will unmap it from the iommu_domain. There is no automatic remap,
this is a safety protocol so the kernel doesn't get stuck. Userspace is
expected to know it is doing something that will revoke the dmabuf and
map/unmap it around the activity. Eg when QEMU goes to issue FLR it should
do the map/unmap to iommufd.

Since DMABUF is missing some key general features for this use case it
relies on a "private interconnect" between VFIO and iommufd via the
vfio_pci_dma_buf_iommufd_map() call.

The call confirms the DMABUF has revoke semantics and delivers a phys_addr
for the memory suitable for use with iommu_map().

Medium term there is a desire to expand the supported DMABUFs to include
GPU drivers to support DPDK/SPDK type use cases so future series will work
to add a general concept of revoke and a general negotiation of
interconnect to remove vfio_pci_dma_buf_iommufd_map().

I also plan another series to modify iommufd's vfio_compat to
transparently pull a dmabuf out of a VFIO VMA to emulate more of the uAPI
of type1.

The latest series for interconnect negotation to exchange a phys_addr is:
 https://lore.kernel.org/r/20251027044712.1676175-1-vivek.kasireddy@intel.com

And the discussion for design of revoke is here:
 https://lore.kernel.org/dri-devel/20250114173103.GE5556@nvidia.com/

This is on github: https://github.com/jgunthorpe/linux/commits/iommufd_dmabuf

v2:
 - Rebase on Leon's v7
 - Fix mislocking in an iopt_fill_domain() error path
v1: https://patch.msgid.link/r/0-v1-64bed2430cdb+31b-iommufd_dmabuf_jgg@nvidia.com

Jason Gunthorpe (9):
  vfio/pci: Add vfio_pci_dma_buf_iommufd_map()
  iommufd: Add DMABUF to iopt_pages
  iommufd: Do not map/unmap revoked DMABUFs
  iommufd: Allow a DMABUF to be revoked
  iommufd: Allow MMIO pages in a batch
  iommufd: Have pfn_reader process DMABUF iopt_pages
  iommufd: Have iopt_map_file_pages convert the fd to a file
  iommufd: Accept a DMABUF through IOMMU_IOAS_MAP_FILE
  iommufd/selftest: Add some tests for the dmabuf flow

 drivers/iommu/iommufd/io_pagetable.c          |  78 +++-
 drivers/iommu/iommufd/io_pagetable.h          |  53 ++-
 drivers/iommu/iommufd/ioas.c                  |   8 +-
 drivers/iommu/iommufd/iommufd_private.h       |  14 +-
 drivers/iommu/iommufd/iommufd_test.h          |  10 +
 drivers/iommu/iommufd/main.c                  |  10 +
 drivers/iommu/iommufd/pages.c                 | 407 ++++++++++++++++--
 drivers/iommu/iommufd/selftest.c              | 142 ++++++
 drivers/vfio/pci/vfio_pci_dmabuf.c            |  34 ++
 include/linux/vfio_pci_core.h                 |   4 +
 tools/testing/selftests/iommu/iommufd.c       |  43 ++
 tools/testing/selftests/iommu/iommufd_utils.h |  44 ++
 12 files changed, 781 insertions(+), 66 deletions(-)


base-commit: bb04e92c86b44b3e36532099b68de1e889acfee7
-- 
2.43.0

[PATCH 1/9] vfio/pci: Add vfio_pci_dma_buf_iommufd_map()

[Jason Gunthorpe]

This function is used to establish the "private interconnect" between the
VFIO DMABUF exporter and the iommufd DMABUF importer. This is intended to
be a temporary API until the core DMABUF interface is improved to natively
support a private interconnect and revocable negotiation.

This function should only be called by iommufd when trying to map a
DMABUF. For now iommufd will only support VFIO DMABUFs.

The following improvements are needed in the DMABUF API to generically
support more exporters with iommufd/kvm type importers that cannot use the
DMA API:

 1) Revoke semantics. VFIO needs to be able to prevent access to the MMIO
    during FLR, and so it will use dma_buf_move_notify() to prevent
    access. iommmufd does not support fault handling so it cannot
    implement the full move_notify. Instead if revoke is negotiated the
    exporter promises not to use move_notify() unless the importer can
    experiance failures. iommufd will unmap the dmabuf from the iommu page
    tables while it is revoked.

 2) Private interconnect negotiation. iommufd will only be able to map
    a "private interconnect" that provides a phys_addr_t and a
    struct p2pdma_provider * to describe the memory. It cannot use a DMA
    mapped scatterlist since it is directly calling iommu_map().

 3) NULL device during dma_buf_dynamic_attach(). Since iommufd doesn't use
    the DMA API it doesn't have a DMAable struct device to pass here.

Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
 drivers/vfio/pci/vfio_pci_dmabuf.c | 34 ++++++++++++++++++++++++++++++
 include/linux/vfio_pci_core.h      |  4 ++++
 2 files changed, 38 insertions(+)

diff --git a/drivers/vfio/pci/vfio_pci_dmabuf.c b/drivers/vfio/pci/vfio_pci_dmabuf.c
index cbf502b14e3c02..7cafd944ce5d38 100644
--- a/drivers/vfio/pci/vfio_pci_dmabuf.c
+++ b/drivers/vfio/pci/vfio_pci_dmabuf.c
@@ -81,6 +81,40 @@ static const struct dma_buf_ops vfio_pci_dmabuf_ops = {
 	.release = vfio_pci_dma_buf_release,
 };
 
+/*
+ * This is a temporary "private interconnect" between VFIO DMABUF and iommufd.
+ * It allows the two co-operating drivers to exchange the physical address of
+ * the BAR. This is to be replaced with a formal DMABUF system for negotiated
+ * interconnect types.
+ *
+ * If this function succeeds the following are true:
+ *  - There is one physical range and it is pointing to MMIO
+ *  - When move_notify is called it means revoke, not move, vfio_dma_buf_map
+ *    will fail if it is currently revoked
+ */
+int vfio_pci_dma_buf_iommufd_map(struct dma_buf_attachment *attachment,
+				 struct dma_buf_phys_vec *phys)
+{
+	struct vfio_pci_dma_buf *priv;
+
+	dma_resv_assert_held(attachment->dmabuf->resv);
+
+	if (attachment->dmabuf->ops != &vfio_pci_dmabuf_ops)
+		return -EOPNOTSUPP;
+
+	priv = attachment->dmabuf->priv;
+	if (priv->revoked)
+		return -ENODEV;
+
+	/* More than one range to iommufd will require proper DMABUF support */
+	if (priv->nr_ranges != 1)
+		return -EOPNOTSUPP;
+
+	*phys = priv->phys_vec[0];
+	return 0;
+}
+EXPORT_SYMBOL_FOR_MODULES(vfio_pci_dma_buf_iommufd_map, "iommufd");
+
 int vfio_pci_core_fill_phys_vec(struct dma_buf_phys_vec *phys_vec,
 				struct vfio_region_dma_range *dma_ranges,
 				size_t nr_ranges, phys_addr_t start,
diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h
index c9466ba323fa9c..6a3074f2cf1cea 100644
--- a/include/linux/vfio_pci_core.h
+++ b/include/linux/vfio_pci_core.h
@@ -28,6 +28,7 @@ struct vfio_pci_core_device;
 struct vfio_pci_region;
 struct p2pdma_provider;
 struct dma_buf_phys_vec;
+struct dma_buf_attachment;
 
 struct vfio_pci_regops {
 	ssize_t (*rw)(struct vfio_pci_core_device *vdev, char __user *buf,
@@ -203,4 +204,7 @@ VFIO_IOREAD_DECLARATION(32)
 VFIO_IOREAD_DECLARATION(64)
 #endif
 
+int vfio_pci_dma_buf_iommufd_map(struct dma_buf_attachment *attachment,
+				 struct dma_buf_phys_vec *phys);
+
 #endif /* VFIO_PCI_CORE_H */
-- 
2.43.0

RE: [PATCH 1/9] vfio/pci: Add vfio_pci_dma_buf_iommufd_map()

[Tian, Kevin]

> From: Jason Gunthorpe <jgg@nvidia.com>
> Sent: Saturday, November 8, 2025 12:50 AM
> 
> This function is used to establish the "private interconnect" between the
> VFIO DMABUF exporter and the iommufd DMABUF importer. This is
> intended to
> be a temporary API until the core DMABUF interface is improved to natively
> support a private interconnect and revocable negotiation.
> 
> This function should only be called by iommufd when trying to map a
> DMABUF. For now iommufd will only support VFIO DMABUFs.
> 
> The following improvements are needed in the DMABUF API to generically
> support more exporters with iommufd/kvm type importers that cannot use
> the
> DMA API:
> 
>  1) Revoke semantics. VFIO needs to be able to prevent access to the MMIO
>     during FLR, and so it will use dma_buf_move_notify() to prevent
>     access. iommmufd does not support fault handling so it cannot
>     implement the full move_notify. Instead if revoke is negotiated the
>     exporter promises not to use move_notify() unless the importer can
>     experiance failures. iommufd will unmap the dmabuf from the iommu
> page
>     tables while it is revoked.
> 
>  2) Private interconnect negotiation. iommufd will only be able to map
>     a "private interconnect" that provides a phys_addr_t and a
>     struct p2pdma_provider * to describe the memory. It cannot use a DMA
>     mapped scatterlist since it is directly calling iommu_map().
> 
>  3) NULL device during dma_buf_dynamic_attach(). Since iommufd doesn't
> use
>     the DMA API it doesn't have a DMAable struct device to pass here.
> 
> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>

Reviewed-by: Kevin Tian <kevin.tian@intel.com>

Re: [PATCH 1/9] vfio/pci: Add vfio_pci_dma_buf_iommufd_map()

[Nicolin Chen]

On Fri, Nov 07, 2025 at 12:49:33PM -0400, Jason Gunthorpe wrote:
> This function is used to establish the "private interconnect" between the
> VFIO DMABUF exporter and the iommufd DMABUF importer. This is intended to
> be a temporary API until the core DMABUF interface is improved to natively
> support a private interconnect and revocable negotiation.
> 
> This function should only be called by iommufd when trying to map a
> DMABUF. For now iommufd will only support VFIO DMABUFs.
> 
> The following improvements are needed in the DMABUF API to generically
> support more exporters with iommufd/kvm type importers that cannot use the
> DMA API:
> 
>  1) Revoke semantics. VFIO needs to be able to prevent access to the MMIO
>     during FLR, and so it will use dma_buf_move_notify() to prevent
>     access. iommmufd does not support fault handling so it cannot
>     implement the full move_notify. Instead if revoke is negotiated the
>     exporter promises not to use move_notify() unless the importer can
>     experiance failures. iommufd will unmap the dmabuf from the iommu page
>     tables while it is revoked.
> 
>  2) Private interconnect negotiation. iommufd will only be able to map
>     a "private interconnect" that provides a phys_addr_t and a
>     struct p2pdma_provider * to describe the memory. It cannot use a DMA
>     mapped scatterlist since it is directly calling iommu_map().
> 
>  3) NULL device during dma_buf_dynamic_attach(). Since iommufd doesn't use
>     the DMA API it doesn't have a DMAable struct device to pass here.
> 
> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
 
Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>

[PATCH 2/9] iommufd: Add DMABUF to iopt_pages

[Jason Gunthorpe]

Add IOPT_ADDRESS_DMABUF to the iopt_pages and the basic infrastructure to
create an iopt_pages from a struct dma_buf *.

DMABUF pages are not supported for accesses, and for now can only be used
with the VFIO DMABUF exporter.

The overall flow will be similar to memfd where the user can pass in a
DMABUF file descriptor to IOMMU_IOAS_MAP_FILE and create an area and
pages. Like other areas it can be copied and otherwise manipulated, though
there is little point in doing so.

There is no pinned page accounting done for DMABUF maps.

The DMABUF attachment exists so long as the dmabuf is mapped into an IOAS,
even if the IOAS is not mapped to any domains.

Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
 drivers/iommu/iommufd/io_pagetable.c    |   3 +
 drivers/iommu/iommufd/io_pagetable.h    |  24 ++++-
 drivers/iommu/iommufd/iommufd_private.h |   2 +
 drivers/iommu/iommufd/main.c            |  10 ++
 drivers/iommu/iommufd/pages.c           | 133 +++++++++++++++++++++++-
 5 files changed, 168 insertions(+), 4 deletions(-)

diff --git a/drivers/iommu/iommufd/io_pagetable.c b/drivers/iommu/iommufd/io_pagetable.c
index c0360c450880b8..b3cf3825a88c7f 100644
--- a/drivers/iommu/iommufd/io_pagetable.c
+++ b/drivers/iommu/iommufd/io_pagetable.c
@@ -284,6 +284,9 @@ static int iopt_alloc_area_pages(struct io_pagetable *iopt,
 		case IOPT_ADDRESS_FILE:
 			start = elm->start_byte + elm->pages->start;
 			break;
+		case IOPT_ADDRESS_DMABUF:
+			start = elm->start_byte + elm->pages->dmabuf.start;
+			break;
 		}
 		rc = iopt_alloc_iova(iopt, dst_iova, start, length);
 		if (rc)
diff --git a/drivers/iommu/iommufd/io_pagetable.h b/drivers/iommu/iommufd/io_pagetable.h
index b6064f4ce4af91..389e3c3c80f335 100644
--- a/drivers/iommu/iommufd/io_pagetable.h
+++ b/drivers/iommu/iommufd/io_pagetable.h
@@ -5,6 +5,7 @@
 #ifndef __IO_PAGETABLE_H
 #define __IO_PAGETABLE_H
 
+#include <linux/dma-buf.h>
 #include <linux/interval_tree.h>
 #include <linux/kref.h>
 #include <linux/mutex.h>
@@ -179,7 +180,14 @@ enum {
 
 enum iopt_address_type {
 	IOPT_ADDRESS_USER = 0,
-	IOPT_ADDRESS_FILE = 1,
+	IOPT_ADDRESS_FILE,
+	IOPT_ADDRESS_DMABUF,
+};
+
+struct iopt_pages_dmabuf {
+	struct dma_buf_attachment *attach;
+	struct dma_buf_phys_vec phys;
+	unsigned long start;
 };
 
 /*
@@ -209,6 +217,8 @@ struct iopt_pages {
 			struct file *file;
 			unsigned long start;
 		};
+		/* IOPT_ADDRESS_DMABUF */
+		struct iopt_pages_dmabuf dmabuf;
 	};
 	bool writable:1;
 	u8 account_mode;
@@ -220,10 +230,22 @@ struct iopt_pages {
 	struct rb_root_cached domains_itree;
 };
 
+static inline bool iopt_is_dmabuf(struct iopt_pages *pages)
+{
+	if (!IS_ENABLED(CONFIG_DMA_SHARED_BUFFER))
+		return false;
+	return pages->type == IOPT_ADDRESS_DMABUF;
+}
+
 struct iopt_pages *iopt_alloc_user_pages(void __user *uptr,
 					 unsigned long length, bool writable);
 struct iopt_pages *iopt_alloc_file_pages(struct file *file, unsigned long start,
 					 unsigned long length, bool writable);
+struct iopt_pages *iopt_alloc_dmabuf_pages(struct iommufd_ctx *ictx,
+					   struct dma_buf *dmabuf,
+					   unsigned long start_byte,
+					   unsigned long start,
+					   unsigned long length, bool writable);
 void iopt_release_pages(struct kref *kref);
 static inline void iopt_put_pages(struct iopt_pages *pages)
 {
diff --git a/drivers/iommu/iommufd/iommufd_private.h b/drivers/iommu/iommufd/iommufd_private.h
index 627f9b78483a0e..10ee3510312132 100644
--- a/drivers/iommu/iommufd/iommufd_private.h
+++ b/drivers/iommu/iommufd/iommufd_private.h
@@ -504,6 +504,8 @@ void iommufd_device_pre_destroy(struct iommufd_object *obj);
 void iommufd_device_destroy(struct iommufd_object *obj);
 int iommufd_get_hw_info(struct iommufd_ucmd *ucmd);
 
+struct device *iommufd_global_device(void);
+
 struct iommufd_access {
 	struct iommufd_object obj;
 	struct iommufd_ctx *ictx;
diff --git a/drivers/iommu/iommufd/main.c b/drivers/iommu/iommufd/main.c
index ce775fbbae94e7..5cc4b08c25f585 100644
--- a/drivers/iommu/iommufd/main.c
+++ b/drivers/iommu/iommufd/main.c
@@ -751,6 +751,15 @@ static struct miscdevice vfio_misc_dev = {
 	.mode = 0666,
 };
 
+/*
+ * Used only by DMABUF, returns a valid struct device to use as a dummy struct
+ * device for attachment.
+ */
+struct device *iommufd_global_device(void)
+{
+	return iommu_misc_dev.this_device;
+}
+
 static int __init iommufd_init(void)
 {
 	int ret;
@@ -794,5 +803,6 @@ MODULE_ALIAS("devname:vfio/vfio");
 #endif
 MODULE_IMPORT_NS("IOMMUFD_INTERNAL");
 MODULE_IMPORT_NS("IOMMUFD");
+MODULE_IMPORT_NS("DMA_BUF");
 MODULE_DESCRIPTION("I/O Address Space Management for passthrough devices");
 MODULE_LICENSE("GPL");
diff --git a/drivers/iommu/iommufd/pages.c b/drivers/iommu/iommufd/pages.c
index c3433b84556172..7bc4731501cc21 100644
--- a/drivers/iommu/iommufd/pages.c
+++ b/drivers/iommu/iommufd/pages.c
@@ -45,6 +45,8 @@
  * last_iova + 1 can overflow. An iopt_pages index will always be much less than
  * ULONG_MAX so last_index + 1 cannot overflow.
  */
+#include <linux/dma-buf.h>
+#include <linux/dma-resv.h>
 #include <linux/file.h>
 #include <linux/highmem.h>
 #include <linux/iommu.h>
@@ -53,6 +55,7 @@
 #include <linux/overflow.h>
 #include <linux/slab.h>
 #include <linux/sched/mm.h>
+#include <linux/vfio_pci_core.h>
 
 #include "double_span.h"
 #include "io_pagetable.h"
@@ -272,6 +275,7 @@ struct pfn_batch {
 	unsigned int end;
 	unsigned int total_pfns;
 };
+enum { MAX_NPFNS = type_max(typeof(((struct pfn_batch *)0)->npfns[0])) };
 
 static void batch_clear(struct pfn_batch *batch)
 {
@@ -350,7 +354,6 @@ static void batch_destroy(struct pfn_batch *batch, void *backup)
 static bool batch_add_pfn_num(struct pfn_batch *batch, unsigned long pfn,
 			      u32 nr)
 {
-	const unsigned int MAX_NPFNS = type_max(typeof(*batch->npfns));
 	unsigned int end = batch->end;
 
 	if (end && pfn == batch->pfns[end - 1] + batch->npfns[end - 1] &&
@@ -1360,6 +1363,121 @@ struct iopt_pages *iopt_alloc_file_pages(struct file *file, unsigned long start,
 	return pages;
 }
 
+static void iopt_revoke_notify(struct dma_buf_attachment *attach)
+{
+	struct iopt_pages *pages = attach->importer_priv;
+
+	guard(mutex)(&pages->mutex);
+	pages->dmabuf.phys.len = 0;
+}
+
+static struct dma_buf_attach_ops iopt_dmabuf_attach_revoke_ops = {
+	.allow_peer2peer = true,
+	.move_notify = iopt_revoke_notify,
+};
+
+/*
+ * iommufd and vfio have a circular dependency. Future work for a phys
+ * based private interconnect will remove this.
+ */
+static int
+sym_vfio_pci_dma_buf_iommufd_map(struct dma_buf_attachment *attachment,
+				 struct dma_buf_phys_vec *phys)
+{
+	typeof(&vfio_pci_dma_buf_iommufd_map) fn;
+	int rc;
+
+	if (!IS_ENABLED(CONFIG_VFIO_PCI_DMABUF))
+		return -EOPNOTSUPP;
+
+	fn = symbol_get(vfio_pci_dma_buf_iommufd_map);
+	if (!fn)
+		return -EOPNOTSUPP;
+	rc = fn(attachment, phys);
+	symbol_put(vfio_pci_dma_buf_iommufd_map);
+	return rc;
+}
+
+static int iopt_map_dmabuf(struct iommufd_ctx *ictx, struct iopt_pages *pages,
+			   struct dma_buf *dmabuf)
+{
+	struct dma_buf_attachment *attach;
+	int rc;
+
+	attach = dma_buf_dynamic_attach(dmabuf, iommufd_global_device(),
+					&iopt_dmabuf_attach_revoke_ops, pages);
+	if (IS_ERR(attach))
+		return PTR_ERR(attach);
+
+	dma_resv_lock(dmabuf->resv, NULL);
+	/*
+	 * Lock ordering requires the mutex to be taken inside the reservation,
+	 * make sure lockdep sees this.
+	 */
+	if (IS_ENABLED(CONFIG_LOCKDEP)) {
+		mutex_lock(&pages->mutex);
+		mutex_unlock(&pages->mutex);
+	}
+
+	rc = sym_vfio_pci_dma_buf_iommufd_map(attach, &pages->dmabuf.phys);
+	if (rc)
+		goto err_detach;
+
+	dma_resv_unlock(dmabuf->resv);
+
+	/* On success iopt_release_pages() will detach and put the dmabuf. */
+	pages->dmabuf.attach = attach;
+	return 0;
+
+err_detach:
+	dma_resv_unlock(dmabuf->resv);
+	dma_buf_detach(dmabuf, attach);
+	return rc;
+}
+
+struct iopt_pages *iopt_alloc_dmabuf_pages(struct iommufd_ctx *ictx,
+					   struct dma_buf *dmabuf,
+					   unsigned long start_byte,
+					   unsigned long start,
+					   unsigned long length, bool writable)
+{
+	static struct lock_class_key pages_dmabuf_mutex_key;
+	struct iopt_pages *pages;
+	int rc;
+
+	if (!IS_ENABLED(CONFIG_DMA_SHARED_BUFFER))
+		return ERR_PTR(-EOPNOTSUPP);
+
+	if (dmabuf->size <= (start + length - 1) ||
+	    length / PAGE_SIZE >= MAX_NPFNS)
+		return ERR_PTR(-EINVAL);
+
+	pages = iopt_alloc_pages(start_byte, length, writable);
+	if (IS_ERR(pages))
+		return pages;
+
+	/*
+	 * The mmap_lock can be held when obtaining the dmabuf reservation lock
+	 * which creates a locking cycle with the pages mutex which is held
+	 * while obtaining the mmap_lock. This locking path is not present for
+	 * IOPT_ADDRESS_DMABUF so split the lock class.
+	 */
+	lockdep_set_class(&pages->mutex, &pages_dmabuf_mutex_key);
+
+	/* dmabuf does not use pinned page accounting. */
+	pages->account_mode = IOPT_PAGES_ACCOUNT_NONE;
+	pages->type = IOPT_ADDRESS_DMABUF;
+	pages->dmabuf.start = start - start_byte;
+
+	rc = iopt_map_dmabuf(ictx, pages, dmabuf);
+	if (rc) {
+		iopt_put_pages(pages);
+		return ERR_PTR(rc);
+	}
+
+	return pages;
+}
+
 void iopt_release_pages(struct kref *kref)
 {
 	struct iopt_pages *pages = container_of(kref, struct iopt_pages, kref);
@@ -1372,8 +1490,14 @@ void iopt_release_pages(struct kref *kref)
 	mutex_destroy(&pages->mutex);
 	put_task_struct(pages->source_task);
 	free_uid(pages->source_user);
-	if (pages->type == IOPT_ADDRESS_FILE)
+	if (iopt_is_dmabuf(pages) && pages->dmabuf.attach) {
+		struct dma_buf *dmabuf = pages->dmabuf.attach->dmabuf;
+
+		dma_buf_detach(dmabuf, pages->dmabuf.attach);
+		dma_buf_put(dmabuf);
+	} else if (pages->type == IOPT_ADDRESS_FILE) {
 		fput(pages->file);
+	}
 	kfree(pages);
 }
 
@@ -2031,7 +2155,10 @@ int iopt_pages_rw_access(struct iopt_pages *pages, unsigned long start_byte,
 	if ((flags & IOMMUFD_ACCESS_RW_WRITE) && !pages->writable)
 		return -EPERM;
 
-	if (pages->type == IOPT_ADDRESS_FILE)
+	if (iopt_is_dmabuf(pages))
+		return -EINVAL;
+
+	if (pages->type != IOPT_ADDRESS_USER)
 		return iopt_pages_rw_slow(pages, start_index, last_index,
 					  start_byte % PAGE_SIZE, data, length,
 					  flags);
-- 
2.43.0

Re: [PATCH 2/9] iommufd: Add DMABUF to iopt_pages

[Nicolin Chen]

On Fri, Nov 07, 2025 at 12:49:34PM -0400, Jason Gunthorpe wrote:
> Add IOPT_ADDRESS_DMABUF to the iopt_pages and the basic infrastructure to
> create an iopt_pages from a struct dma_buf *.
> 
> DMABUF pages are not supported for accesses, and for now can only be used
> with the VFIO DMABUF exporter.
> 
> The overall flow will be similar to memfd where the user can pass in a
> DMABUF file descriptor to IOMMU_IOAS_MAP_FILE and create an area and
> pages. Like other areas it can be copied and otherwise manipulated, though
> there is little point in doing so.
> 
> There is no pinned page accounting done for DMABUF maps.
> 
> The DMABUF attachment exists so long as the dmabuf is mapped into an IOAS,
> even if the IOAS is not mapped to any domains.
> 
> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
 
Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>

RE: [PATCH 2/9] iommufd: Add DMABUF to iopt_pages

[Tian, Kevin]

> From: Jason Gunthorpe <jgg@nvidia.com>
> Sent: Saturday, November 8, 2025 12:50 AM
> 
> 
> @@ -2031,7 +2155,10 @@ int iopt_pages_rw_access(struct iopt_pages
> *pages, unsigned long start_byte,
>  	if ((flags & IOMMUFD_ACCESS_RW_WRITE) && !pages->writable)
>  		return -EPERM;
> 
> -	if (pages->type == IOPT_ADDRESS_FILE)
> +	if (iopt_is_dmabuf(pages))
> +		return -EINVAL;
> +

probably also add helpers for other types, e.g.:

  iopt_is_user()
  iopt_is_memfd()

> +	if (pages->type != IOPT_ADDRESS_USER)
>  		return iopt_pages_rw_slow(pages, start_index, last_index,
>  					  start_byte % PAGE_SIZE, data,
> length,
>  					  flags);
> --

then the following WARN_ON() becomes useless:

	if (IS_ENABLED(CONFIG_IOMMUFD_TEST) &&
	    WARN_ON(pages->type != IOPT_ADDRESS_USER))
		return -EINVAL;

Reviewed-by: Kevin Tian <kevin.tian@intel.com>

Re: [PATCH 2/9] iommufd: Add DMABUF to iopt_pages

[Jason Gunthorpe]

On Thu, Nov 20, 2025 at 07:55:04AM +0000, Tian, Kevin wrote:
> > From: Jason Gunthorpe <jgg@nvidia.com>
> > Sent: Saturday, November 8, 2025 12:50 AM
> > 
> > 
> > @@ -2031,7 +2155,10 @@ int iopt_pages_rw_access(struct iopt_pages
> > *pages, unsigned long start_byte,
> >  	if ((flags & IOMMUFD_ACCESS_RW_WRITE) && !pages->writable)
> >  		return -EPERM;
> > 
> > -	if (pages->type == IOPT_ADDRESS_FILE)
> > +	if (iopt_is_dmabuf(pages))
> > +		return -EINVAL;
> > +
> 
> probably also add helpers for other types, e.g.:
> 
>   iopt_is_user()
>   iopt_is_memfd()

The helper was to integrate the IS_ENABLED() check for DMABUF, there
are not so many others uses, I think leave it to not bloat the patch.

> > +	if (pages->type != IOPT_ADDRESS_USER)
> >  		return iopt_pages_rw_slow(pages, start_index, last_index,
> >  					  start_byte % PAGE_SIZE, data,
> > length,
> >  					  flags);
> > --
> 
> then the following WARN_ON() becomes useless:
> 
> 	if (IS_ENABLED(CONFIG_IOMMUFD_TEST) &&
> 	    WARN_ON(pages->type != IOPT_ADDRESS_USER))
> 		return -EINVAL;

Yep

Thanks,
Jason

[PATCH 3/9] iommufd: Do not map/unmap revoked DMABUFs

[Jason Gunthorpe]

Once a DMABUF is revoked the domain will be unmapped under the pages
mutex. Double unmapping will trigger a WARN, and mapping while revoked
will fail.

Check for revoked DMABUFs along all the map and unmap paths to resolve
this. Ensure that map/unmap is always done under the pages mutex so it is
synchronized with the revoke notifier.

If a revoke happens between allocating the iopt_pages and the population
to a domain then the population will succeed, and leave things unmapped as
though revoke had happened immediately after.

Currently there is no way to repopulate the domains. Userspace is expected
to know if it is going to do something that would trigger revoke (eg if it
is about to do a FLR) then it should go and remove the DMABUF mappings
before and put the back after. The revoke is only to protect the kernel
from mis-behaving userspace.

Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
 drivers/iommu/iommufd/io_pagetable.c | 11 +++++-
 drivers/iommu/iommufd/io_pagetable.h |  8 +++++
 drivers/iommu/iommufd/pages.c        | 54 +++++++++++++++++-----------
 3 files changed, 52 insertions(+), 21 deletions(-)

diff --git a/drivers/iommu/iommufd/io_pagetable.c b/drivers/iommu/iommufd/io_pagetable.c
index b3cf3825a88c7f..38c5fdc6c82128 100644
--- a/drivers/iommu/iommufd/io_pagetable.c
+++ b/drivers/iommu/iommufd/io_pagetable.c
@@ -970,9 +970,14 @@ static void iopt_unfill_domain(struct io_pagetable *iopt,
 				WARN_ON(!area->storage_domain);
 			if (area->storage_domain == domain)
 				area->storage_domain = storage_domain;
+			if (iopt_is_dmabuf(pages)) {
+				if (!iopt_dmabuf_revoked(pages))
+					iopt_area_unmap_domain(area, domain);
+			}
 			mutex_unlock(&pages->mutex);
 
-			iopt_area_unmap_domain(area, domain);
+			if (!iopt_is_dmabuf(pages))
+				iopt_area_unmap_domain(area, domain);
 		}
 		return;
 	}
@@ -1261,6 +1266,10 @@ static int iopt_area_split(struct iopt_area *area, unsigned long iova)
 	if (!pages || area->prevent_access)
 		return -EBUSY;
 
+	/* Maintaining the domains_itree below is a bit complicated */
+	if (iopt_is_dmabuf(pages))
+		return -EOPNOTSUPP;
+
 	if (new_start & (alignment - 1) ||
 	    iopt_area_start_byte(area, new_start) & (alignment - 1))
 		return -EINVAL;
diff --git a/drivers/iommu/iommufd/io_pagetable.h b/drivers/iommu/iommufd/io_pagetable.h
index 389e3c3c80f335..00fe06bf533d62 100644
--- a/drivers/iommu/iommufd/io_pagetable.h
+++ b/drivers/iommu/iommufd/io_pagetable.h
@@ -237,6 +237,14 @@ static inline bool iopt_is_dmabuf(struct iopt_pages *pages)
 	return pages->type == IOPT_ADDRESS_DMABUF;
 }
 
+static inline bool iopt_dmabuf_revoked(struct iopt_pages *pages)
+{
+	lockdep_assert_held(&pages->mutex);
+	if (iopt_is_dmabuf(pages))
+		return pages->dmabuf.phys.len == 0;
+	return false;
+}
+
 struct iopt_pages *iopt_alloc_user_pages(void __user *uptr,
 					 unsigned long length, bool writable);
 struct iopt_pages *iopt_alloc_file_pages(struct file *file, unsigned long start,
diff --git a/drivers/iommu/iommufd/pages.c b/drivers/iommu/iommufd/pages.c
index 7bc4731501cc21..2c1a59ab1cc123 100644
--- a/drivers/iommu/iommufd/pages.c
+++ b/drivers/iommu/iommufd/pages.c
@@ -1650,6 +1650,9 @@ void iopt_area_unmap_domain(struct iopt_area *area, struct iommu_domain *domain)
 void iopt_area_unfill_domain(struct iopt_area *area, struct iopt_pages *pages,
 			     struct iommu_domain *domain)
 {
+	if (iopt_dmabuf_revoked(pages))
+		return;
+
 	__iopt_area_unfill_domain(area, pages, domain,
 				  iopt_area_last_index(area));
 }
@@ -1670,6 +1673,9 @@ int iopt_area_fill_domain(struct iopt_area *area, struct iommu_domain *domain)
 
 	lockdep_assert_held(&area->pages->mutex);
 
+	if (iopt_dmabuf_revoked(area->pages))
+		return 0;
+
 	rc = pfn_reader_first(&pfns, area->pages, iopt_area_index(area),
 			      iopt_area_last_index(area));
 	if (rc)
@@ -1729,33 +1735,38 @@ int iopt_area_fill_domains(struct iopt_area *area, struct iopt_pages *pages)
 		return 0;
 
 	mutex_lock(&pages->mutex);
-	rc = pfn_reader_first(&pfns, pages, iopt_area_index(area),
-			      iopt_area_last_index(area));
-	if (rc)
-		goto out_unlock;
+	if (!iopt_dmabuf_revoked(pages)) {
+		rc = pfn_reader_first(&pfns, pages, iopt_area_index(area),
+				      iopt_area_last_index(area));
+		if (rc)
+			goto out_unlock;
 
-	while (!pfn_reader_done(&pfns)) {
-		done_first_end_index = pfns.batch_end_index;
-		done_all_end_index = pfns.batch_start_index;
-		xa_for_each(&area->iopt->domains, index, domain) {
-			rc = batch_to_domain(&pfns.batch, domain, area,
-					     pfns.batch_start_index);
+		while (!pfn_reader_done(&pfns)) {
+			done_first_end_index = pfns.batch_end_index;
+			done_all_end_index = pfns.batch_start_index;
+			xa_for_each(&area->iopt->domains, index, domain) {
+				rc = batch_to_domain(&pfns.batch, domain, area,
+						     pfns.batch_start_index);
+				if (rc)
+					goto out_unmap;
+			}
+			done_all_end_index = done_first_end_index;
+
+			rc = pfn_reader_next(&pfns);
 			if (rc)
 				goto out_unmap;
 		}
-		done_all_end_index = done_first_end_index;
-
-		rc = pfn_reader_next(&pfns);
+		rc = pfn_reader_update_pinned(&pfns);
 		if (rc)
 			goto out_unmap;
+
+		pfn_reader_destroy(&pfns);
 	}
-	rc = pfn_reader_update_pinned(&pfns);
-	if (rc)
-		goto out_unmap;
 
 	area->storage_domain = xa_load(&area->iopt->domains, 0);
 	interval_tree_insert(&area->pages_node, &pages->domains_itree);
-	goto out_destroy;
+	mutex_unlock(&pages->mutex);
+	return 0;
 
 out_unmap:
 	pfn_reader_release_pins(&pfns);
@@ -1782,7 +1793,6 @@ int iopt_area_fill_domains(struct iopt_area *area, struct iopt_pages *pages)
 							end_index);
 		}
 	}
-out_destroy:
 	pfn_reader_destroy(&pfns);
 out_unlock:
 	mutex_unlock(&pages->mutex);
@@ -1809,11 +1819,15 @@ void iopt_area_unfill_domains(struct iopt_area *area, struct iopt_pages *pages)
 	if (!area->storage_domain)
 		goto out_unlock;
 
-	xa_for_each(&iopt->domains, index, domain)
-		if (domain != area->storage_domain)
+	xa_for_each(&iopt->domains, index, domain) {
+		if (domain == area->storage_domain)
+			continue;
+
+		if (!iopt_dmabuf_revoked(pages))
 			iopt_area_unmap_domain_range(
 				area, domain, iopt_area_index(area),
 				iopt_area_last_index(area));
+	}
 
 	if (IS_ENABLED(CONFIG_IOMMUFD_TEST))
 		WARN_ON(RB_EMPTY_NODE(&area->pages_node.rb));
-- 
2.43.0

Re: [PATCH 3/9] iommufd: Do not map/unmap revoked DMABUFs

[Nicolin Chen]

On Fri, Nov 07, 2025 at 12:49:35PM -0400, Jason Gunthorpe wrote:
> Once a DMABUF is revoked the domain will be unmapped under the pages
> mutex. Double unmapping will trigger a WARN, and mapping while revoked
> will fail.
> 
> Check for revoked DMABUFs along all the map and unmap paths to resolve
> this. Ensure that map/unmap is always done under the pages mutex so it is
> synchronized with the revoke notifier.
> 
> If a revoke happens between allocating the iopt_pages and the population
> to a domain then the population will succeed, and leave things unmapped as
> though revoke had happened immediately after.
> 
> Currently there is no way to repopulate the domains. Userspace is expected
> to know if it is going to do something that would trigger revoke (eg if it
> is about to do a FLR) then it should go and remove the DMABUF mappings
> before and put the back after. The revoke is only to protect the kernel
> from mis-behaving userspace.
> 
> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>

Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>

RE: [PATCH 3/9] iommufd: Do not map/unmap revoked DMABUFs

[Tian, Kevin]

> From: Jason Gunthorpe <jgg@nvidia.com>
> Sent: Saturday, November 8, 2025 12:50 AM
> 
> Once a DMABUF is revoked the domain will be unmapped under the pages
> mutex. Double unmapping will trigger a WARN, and mapping while revoked
> will fail.
> 
> Check for revoked DMABUFs along all the map and unmap paths to resolve
> this. Ensure that map/unmap is always done under the pages mutex so it is
> synchronized with the revoke notifier.
> 
> If a revoke happens between allocating the iopt_pages and the population
> to a domain then the population will succeed, and leave things unmapped as
> though revoke had happened immediately after.
> 
> Currently there is no way to repopulate the domains. Userspace is expected
> to know if it is going to do something that would trigger revoke (eg if it
> is about to do a FLR) then it should go and remove the DMABUF mappings
> before and put the back after. The revoke is only to protect the kernel
> from mis-behaving userspace.
> 
> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>

Reviewed-by: Kevin Tian <kevin.tian@intel.com>

[PATCH 4/9] iommufd: Allow a DMABUF to be revoked

[Jason Gunthorpe]

When connected to VFIO, the only DMABUF exporter that is accepted, the
move_notify callback will be made when VFIO wants to remove access to the
MMIO. This is being called revoke.

Wire up revoke to go through all the iommu_domain's that have mapped the
DMABUF and unmap them.

The locking here is unpleasant, since the existing locking scheme was
designed to come from the iopt through the area to the pages we cannot use
pages as starting point for the locking. There is no way to obtain the
domains_rwsem before obtaining the pages mutex to reliably use the
existing domains_itree.

Solve this problem by adding a new tracking structure just for DMABUF
revoke. Record a linked list of areas and domains inside the pages
mutex. Clean the entries on the list during revoke. The map/unmaps are now
all done under a pages mutex while updating the tracking linked list so
nothing can get out of sync. Only one lock is required for revoke
processing.

Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
 drivers/iommu/iommufd/io_pagetable.c |  16 +++-
 drivers/iommu/iommufd/io_pagetable.h |  17 +++++
 drivers/iommu/iommufd/pages.c        | 106 ++++++++++++++++++++++++++-
 3 files changed, 135 insertions(+), 4 deletions(-)

diff --git a/drivers/iommu/iommufd/io_pagetable.c b/drivers/iommu/iommufd/io_pagetable.c
index 38c5fdc6c82128..a120c2ae942a8b 100644
--- a/drivers/iommu/iommufd/io_pagetable.c
+++ b/drivers/iommu/iommufd/io_pagetable.c
@@ -973,6 +973,7 @@ static void iopt_unfill_domain(struct io_pagetable *iopt,
 			if (iopt_is_dmabuf(pages)) {
 				if (!iopt_dmabuf_revoked(pages))
 					iopt_area_unmap_domain(area, domain);
+				iopt_dmabuf_untrack_domain(pages, area, domain);
 			}
 			mutex_unlock(&pages->mutex);
 
@@ -994,6 +995,8 @@ static void iopt_unfill_domain(struct io_pagetable *iopt,
 		WARN_ON(area->storage_domain != domain);
 		area->storage_domain = NULL;
 		iopt_area_unfill_domain(area, pages, domain);
+		if (iopt_is_dmabuf(pages))
+			iopt_dmabuf_untrack_domain(pages, area, domain);
 		mutex_unlock(&pages->mutex);
 	}
 }
@@ -1023,10 +1026,16 @@ static int iopt_fill_domain(struct io_pagetable *iopt,
 		if (!pages)
 			continue;
 
-		mutex_lock(&pages->mutex);
+		guard(mutex)(&pages->mutex);
+		if (iopt_is_dmabuf(pages)) {
+			rc = iopt_dmabuf_track_domain(pages, area, domain);
+			if (rc)
+				goto out_unfill;
+		}
 		rc = iopt_area_fill_domain(area, domain);
 		if (rc) {
-			mutex_unlock(&pages->mutex);
+			if (iopt_is_dmabuf(pages))
+				iopt_dmabuf_untrack_domain(pages, area, domain);
 			goto out_unfill;
 		}
 		if (!area->storage_domain) {
@@ -1035,7 +1044,6 @@ static int iopt_fill_domain(struct io_pagetable *iopt,
 			interval_tree_insert(&area->pages_node,
 					     &pages->domains_itree);
 		}
-		mutex_unlock(&pages->mutex);
 	}
 	return 0;
 
@@ -1056,6 +1064,8 @@ static int iopt_fill_domain(struct io_pagetable *iopt,
 			area->storage_domain = NULL;
 		}
 		iopt_area_unfill_domain(area, pages, domain);
+		if (iopt_is_dmabuf(pages))
+			iopt_dmabuf_untrack_domain(pages, area, domain);
 		mutex_unlock(&pages->mutex);
 	}
 	return rc;
diff --git a/drivers/iommu/iommufd/io_pagetable.h b/drivers/iommu/iommufd/io_pagetable.h
index 00fe06bf533d62..81c6093beee2a5 100644
--- a/drivers/iommu/iommufd/io_pagetable.h
+++ b/drivers/iommu/iommufd/io_pagetable.h
@@ -70,6 +70,16 @@ void iopt_area_unfill_domain(struct iopt_area *area, struct iopt_pages *pages,
 void iopt_area_unmap_domain(struct iopt_area *area,
 			    struct iommu_domain *domain);
 
+int iopt_dmabuf_track_domain(struct iopt_pages *pages, struct iopt_area *area,
+			     struct iommu_domain *domain);
+void iopt_dmabuf_untrack_domain(struct iopt_pages *pages,
+				struct iopt_area *area,
+				struct iommu_domain *domain);
+int iopt_dmabuf_track_all_domains(struct iopt_area *area,
+				  struct iopt_pages *pages);
+void iopt_dmabuf_untrack_all_domains(struct iopt_area *area,
+				     struct iopt_pages *pages);
+
 static inline unsigned long iopt_area_index(struct iopt_area *area)
 {
 	return area->pages_node.start;
@@ -184,10 +194,17 @@ enum iopt_address_type {
 	IOPT_ADDRESS_DMABUF,
 };
 
+struct iopt_pages_dmabuf_track {
+	struct iommu_domain *domain;
+	struct iopt_area *area;
+	struct list_head elm;
+};
+
 struct iopt_pages_dmabuf {
 	struct dma_buf_attachment *attach;
 	struct dma_buf_phys_vec phys;
 	unsigned long start;
+	struct list_head tracker;
 };
 
 /*
diff --git a/drivers/iommu/iommufd/pages.c b/drivers/iommu/iommufd/pages.c
index 2c1a59ab1cc123..97f5a173122b4a 100644
--- a/drivers/iommu/iommufd/pages.c
+++ b/drivers/iommu/iommufd/pages.c
@@ -1366,8 +1366,19 @@ struct iopt_pages *iopt_alloc_file_pages(struct file *file, unsigned long start,
 static void iopt_revoke_notify(struct dma_buf_attachment *attach)
 {
 	struct iopt_pages *pages = attach->importer_priv;
+	struct iopt_pages_dmabuf_track *track;
 
 	guard(mutex)(&pages->mutex);
+	if (iopt_dmabuf_revoked(pages))
+		return;
+
+	list_for_each_entry(track, &pages->dmabuf.tracker, elm) {
+		struct iopt_area *area = track->area;
+
+		iopt_area_unmap_domain_range(area, track->domain,
+					     iopt_area_index(area),
+					     iopt_area_last_index(area));
+	}
 	pages->dmabuf.phys.len = 0;
 }
 
@@ -1468,6 +1479,7 @@ struct iopt_pages *iopt_alloc_dmabuf_pages(struct iommufd_ctx *ictx,
 	pages->account_mode = IOPT_PAGES_ACCOUNT_NONE;
 	pages->type = IOPT_ADDRESS_DMABUF;
 	pages->dmabuf.start = start - start_byte;
+	INIT_LIST_HEAD(&pages->dmabuf.tracker);
 
 	rc = iopt_map_dmabuf(ictx, pages, dmabuf);
 	if (rc) {
@@ -1478,6 +1490,86 @@ struct iopt_pages *iopt_alloc_dmabuf_pages(struct iommufd_ctx *ictx,
 	return pages;
 }
 
+int iopt_dmabuf_track_domain(struct iopt_pages *pages, struct iopt_area *area,
+			     struct iommu_domain *domain)
+{
+	struct iopt_pages_dmabuf_track *track;
+
+	lockdep_assert_held(&pages->mutex);
+	if (WARN_ON(!iopt_is_dmabuf(pages)))
+		return -EINVAL;
+
+	list_for_each_entry(track, &pages->dmabuf.tracker, elm)
+		if (WARN_ON(track->domain == domain && track->area == area))
+			return -EINVAL;
+
+	track = kzalloc(sizeof(*track), GFP_KERNEL);
+	if (!track)
+		return -ENOMEM;
+	track->domain = domain;
+	track->area = area;
+	list_add_tail(&track->elm, &pages->dmabuf.tracker);
+
+	return 0;
+}
+
+void iopt_dmabuf_untrack_domain(struct iopt_pages *pages,
+				struct iopt_area *area,
+				struct iommu_domain *domain)
+{
+	struct iopt_pages_dmabuf_track *track;
+
+	lockdep_assert_held(&pages->mutex);
+	WARN_ON(!iopt_is_dmabuf(pages));
+
+	list_for_each_entry(track, &pages->dmabuf.tracker, elm) {
+		if (track->domain == domain && track->area == area) {
+			list_del(&track->elm);
+			kfree(track);
+			return;
+		}
+	}
+	WARN_ON(true);
+}
+
+int iopt_dmabuf_track_all_domains(struct iopt_area *area,
+				  struct iopt_pages *pages)
+{
+	struct iopt_pages_dmabuf_track *track;
+	struct iommu_domain *domain;
+	unsigned long index;
+	int rc;
+
+	list_for_each_entry(track, &pages->dmabuf.tracker, elm)
+		if (WARN_ON(track->area == area))
+			return -EINVAL;
+
+	xa_for_each(&area->iopt->domains, index, domain) {
+		rc = iopt_dmabuf_track_domain(pages, area, domain);
+		if (rc)
+			goto err_untrack;
+	}
+	return 0;
+err_untrack:
+	iopt_dmabuf_untrack_all_domains(area, pages);
+	return rc;
+}
+
+void iopt_dmabuf_untrack_all_domains(struct iopt_area *area,
+				     struct iopt_pages *pages)
+{
+	struct iopt_pages_dmabuf_track *track;
+	struct iopt_pages_dmabuf_track *tmp;
+
+	list_for_each_entry_safe(track, tmp, &pages->dmabuf.tracker,
+				 elm) {
+		if (track->area == area) {
+			list_del(&track->elm);
+			kfree(track);
+		}
+	}
+}
+
 void iopt_release_pages(struct kref *kref)
 {
 	struct iopt_pages *pages = container_of(kref, struct iopt_pages, kref);
@@ -1495,6 +1587,7 @@ void iopt_release_pages(struct kref *kref)
 
 		dma_buf_detach(dmabuf, pages->dmabuf.attach);
 		dma_buf_put(dmabuf);
+		WARN_ON(!list_empty(&pages->dmabuf.tracker));
 	} else if (pages->type == IOPT_ADDRESS_FILE) {
 		fput(pages->file);
 	}
@@ -1735,11 +1828,17 @@ int iopt_area_fill_domains(struct iopt_area *area, struct iopt_pages *pages)
 		return 0;
 
 	mutex_lock(&pages->mutex);
+	if (iopt_is_dmabuf(pages)) {
+		rc = iopt_dmabuf_track_all_domains(area, pages);
+		if (rc)
+			goto out_unlock;
+	}
+
 	if (!iopt_dmabuf_revoked(pages)) {
 		rc = pfn_reader_first(&pfns, pages, iopt_area_index(area),
 				      iopt_area_last_index(area));
 		if (rc)
-			goto out_unlock;
+			goto out_untrack;
 
 		while (!pfn_reader_done(&pfns)) {
 			done_first_end_index = pfns.batch_end_index;
@@ -1794,6 +1893,9 @@ int iopt_area_fill_domains(struct iopt_area *area, struct iopt_pages *pages)
 		}
 	}
 	pfn_reader_destroy(&pfns);
+out_untrack:
+	if (iopt_is_dmabuf(pages))
+		iopt_dmabuf_untrack_all_domains(area, pages);
 out_unlock:
 	mutex_unlock(&pages->mutex);
 	return rc;
@@ -1833,6 +1935,8 @@ void iopt_area_unfill_domains(struct iopt_area *area, struct iopt_pages *pages)
 		WARN_ON(RB_EMPTY_NODE(&area->pages_node.rb));
 	interval_tree_remove(&area->pages_node, &pages->domains_itree);
 	iopt_area_unfill_domain(area, pages, area->storage_domain);
+	if (iopt_is_dmabuf(pages))
+		iopt_dmabuf_untrack_all_domains(area, pages);
 	area->storage_domain = NULL;
 out_unlock:
 	mutex_unlock(&pages->mutex);
-- 
2.43.0

Re: [PATCH 4/9] iommufd: Allow a DMABUF to be revoked

[Nicolin Chen]

On Fri, Nov 07, 2025 at 12:49:36PM -0400, Jason Gunthorpe wrote:
> When connected to VFIO, the only DMABUF exporter that is accepted, the
> move_notify callback will be made when VFIO wants to remove access to the
> MMIO. This is being called revoke.
> 
> Wire up revoke to go through all the iommu_domain's that have mapped the
> DMABUF and unmap them.
> 
> The locking here is unpleasant, since the existing locking scheme was
> designed to come from the iopt through the area to the pages we cannot use
> pages as starting point for the locking. There is no way to obtain the
> domains_rwsem before obtaining the pages mutex to reliably use the
> existing domains_itree.
> 
> Solve this problem by adding a new tracking structure just for DMABUF
> revoke. Record a linked list of areas and domains inside the pages
> mutex. Clean the entries on the list during revoke. The map/unmaps are now
> all done under a pages mutex while updating the tracking linked list so
> nothing can get out of sync. Only one lock is required for revoke
> processing.
> 
> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
 
Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>

RE: [PATCH 4/9] iommufd: Allow a DMABUF to be revoked

[Tian, Kevin]

> From: Jason Gunthorpe <jgg@nvidia.com>
> Sent: Saturday, November 8, 2025 12:50 AM
> 
> When connected to VFIO, the only DMABUF exporter that is accepted, the
> move_notify callback will be made when VFIO wants to remove access to the
> MMIO. This is being called revoke.
> 
> Wire up revoke to go through all the iommu_domain's that have mapped the
> DMABUF and unmap them.
> 
> The locking here is unpleasant, since the existing locking scheme was
> designed to come from the iopt through the area to the pages we cannot use
> pages as starting point for the locking. There is no way to obtain the
> domains_rwsem before obtaining the pages mutex to reliably use the
> existing domains_itree.
> 
> Solve this problem by adding a new tracking structure just for DMABUF
> revoke. Record a linked list of areas and domains inside the pages
> mutex. Clean the entries on the list during revoke. The map/unmaps are now
> all done under a pages mutex while updating the tracking linked list so
> nothing can get out of sync. Only one lock is required for revoke
> processing.
> 
> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>

Reviewed-by: Kevin Tian <kevin.tian@intel.com>

[PATCH 5/9] iommufd: Allow MMIO pages in a batch

[Jason Gunthorpe]

Addresses intended for MMIO should be propagated through to the iommu with
the IOMMU_MMIO flag set.

Keep track in the batch if all the pfns are cachable or mmio and flush the
batch out of it ever needs to be changed. Switch to IOMMU_MMIO if the
batch is MMIO when mapping the iommu.

Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
 drivers/iommu/iommufd/pages.c | 29 ++++++++++++++++++++++++-----
 1 file changed, 24 insertions(+), 5 deletions(-)

diff --git a/drivers/iommu/iommufd/pages.c b/drivers/iommu/iommufd/pages.c
index 97f5a173122b4a..e71796b0c4bf44 100644
--- a/drivers/iommu/iommufd/pages.c
+++ b/drivers/iommu/iommufd/pages.c
@@ -261,6 +261,11 @@ static struct iopt_area *iopt_pages_find_domain_area(struct iopt_pages *pages,
 	return container_of(node, struct iopt_area, pages_node);
 }
 
+enum batch_kind {
+	BATCH_CPU_MEMORY = 0,
+	BATCH_MMIO,
+};
+
 /*
  * A simple datastructure to hold a vector of PFNs, optimized for contiguous
  * PFNs. This is used as a temporary holding memory for shuttling pfns from one
@@ -274,6 +279,7 @@ struct pfn_batch {
 	unsigned int array_size;
 	unsigned int end;
 	unsigned int total_pfns;
+	enum batch_kind kind;
 };
 enum { MAX_NPFNS = type_max(typeof(((struct pfn_batch *)0)->npfns[0])) };
 
@@ -352,10 +358,17 @@ static void batch_destroy(struct pfn_batch *batch, void *backup)
 }
 
 static bool batch_add_pfn_num(struct pfn_batch *batch, unsigned long pfn,
-			      u32 nr)
+			      u32 nr, enum batch_kind kind)
 {
 	unsigned int end = batch->end;
 
+	if (batch->kind != kind) {
+		/* One kind per batch */
+		if (batch->end != 0)
+			return false;
+		batch->kind = kind;
+	}
+
 	if (end && pfn == batch->pfns[end - 1] + batch->npfns[end - 1] &&
 	    nr <= MAX_NPFNS - batch->npfns[end - 1]) {
 		batch->npfns[end - 1] += nr;
@@ -382,7 +395,7 @@ static void batch_remove_pfn_num(struct pfn_batch *batch, unsigned long nr)
 /* true if the pfn was added, false otherwise */
 static bool batch_add_pfn(struct pfn_batch *batch, unsigned long pfn)
 {
-	return batch_add_pfn_num(batch, pfn, 1);
+	return batch_add_pfn_num(batch, pfn, 1, BATCH_CPU_MEMORY);
 }
 
 /*
@@ -495,6 +508,7 @@ static int batch_to_domain(struct pfn_batch *batch, struct iommu_domain *domain,
 {
 	bool disable_large_pages = area->iopt->disable_large_pages;
 	unsigned long last_iova = iopt_area_last_iova(area);
+	int iommu_prot = area->iommu_prot;
 	unsigned int page_offset = 0;
 	unsigned long start_iova;
 	unsigned long next_iova;
@@ -502,6 +516,11 @@ static int batch_to_domain(struct pfn_batch *batch, struct iommu_domain *domain,
 	unsigned long iova;
 	int rc;
 
+	if (batch->kind == BATCH_MMIO) {
+		iommu_prot &= ~IOMMU_CACHE;
+		iommu_prot |= IOMMU_MMIO;
+	}
+
 	/* The first index might be a partial page */
 	if (start_index == iopt_area_index(area))
 		page_offset = area->page_offset;
@@ -515,11 +534,11 @@ static int batch_to_domain(struct pfn_batch *batch, struct iommu_domain *domain,
 			rc = batch_iommu_map_small(
 				domain, iova,
 				PFN_PHYS(batch->pfns[cur]) + page_offset,
-				next_iova - iova, area->iommu_prot);
+				next_iova - iova, iommu_prot);
 		else
 			rc = iommu_map(domain, iova,
 				       PFN_PHYS(batch->pfns[cur]) + page_offset,
-				       next_iova - iova, area->iommu_prot,
+				       next_iova - iova, iommu_prot,
 				       GFP_KERNEL_ACCOUNT);
 		if (rc)
 			goto err_unmap;
@@ -655,7 +674,7 @@ static int batch_from_folios(struct pfn_batch *batch, struct folio ***folios_p,
 		nr = min(nr, npages);
 		npages -= nr;
 
-		if (!batch_add_pfn_num(batch, pfn, nr))
+		if (!batch_add_pfn_num(batch, pfn, nr, BATCH_CPU_MEMORY))
 			break;
 		if (nr > 1) {
 			rc = folio_add_pins(folio, nr - 1);
-- 
2.43.0

Re: [PATCH 5/9] iommufd: Allow MMIO pages in a batch

[Nicolin Chen]

On Fri, Nov 07, 2025 at 12:49:37PM -0400, Jason Gunthorpe wrote:
> Addresses intended for MMIO should be propagated through to the iommu with
> the IOMMU_MMIO flag set.
> 
> Keep track in the batch if all the pfns are cachable or mmio and flush the
> batch out of it ever needs to be changed. Switch to IOMMU_MMIO if the
> batch is MMIO when mapping the iommu.
> 
> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>

Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>

RE: [PATCH 5/9] iommufd: Allow MMIO pages in a batch

[Tian, Kevin]

> From: Jason Gunthorpe <jgg@nvidia.com>
> Sent: Saturday, November 8, 2025 12:50 AM
> 
> +enum batch_kind {
> +	BATCH_CPU_MEMORY = 0,
> +	BATCH_MMIO,
> +};

with 'CPU_MEMORY' (instead of plain 'MEMORY') implies future
support of 'DEV_MEMORY'?

Reviewed-by: Kevin Tian <kevin.tian@intel.com>

Re: [PATCH 5/9] iommufd: Allow MMIO pages in a batch

[Jason Gunthorpe]

On Thu, Nov 20, 2025 at 07:59:19AM +0000, Tian, Kevin wrote:
> > From: Jason Gunthorpe <jgg@nvidia.com>
> > Sent: Saturday, November 8, 2025 12:50 AM
> > 
> > +enum batch_kind {
> > +	BATCH_CPU_MEMORY = 0,
> > +	BATCH_MMIO,
> > +};
> 
> with 'CPU_MEMORY' (instead of plain 'MEMORY') implies future
> support of 'DEV_MEMORY'?

Maybe, but I don't have an immediate thought on this. CXL "MMIO" that
is cachable is a thing but we can also label it as CPU_MEMORY.

We might have something for CC shared/protected memory down the road.

Thanks,
Jason

[PATCH 6/9] iommufd: Have pfn_reader process DMABUF iopt_pages

[Jason Gunthorpe]

Make another sub implementation of pfn_reader for DMABUF. This version
will fill the batch using the struct phys_vec recorded during the
attachment.

Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
 drivers/iommu/iommufd/pages.c | 70 +++++++++++++++++++++++++++++++----
 1 file changed, 63 insertions(+), 7 deletions(-)

diff --git a/drivers/iommu/iommufd/pages.c b/drivers/iommu/iommufd/pages.c
index e71796b0c4bf44..935184dc68be32 100644
--- a/drivers/iommu/iommufd/pages.c
+++ b/drivers/iommu/iommufd/pages.c
@@ -1076,6 +1076,40 @@ static int pfn_reader_user_update_pinned(struct pfn_reader_user *user,
 	return iopt_pages_update_pinned(pages, npages, inc, user);
 }
 
+struct pfn_reader_dmabuf {
+	struct dma_buf_phys_vec phys;
+	unsigned long start_offset;
+};
+
+static int pfn_reader_dmabuf_init(struct pfn_reader_dmabuf *dmabuf,
+				  struct iopt_pages *pages)
+{
+	/* Callers must not get here if the dmabuf was already revoked */
+	if (WARN_ON(iopt_dmabuf_revoked(pages)))
+		return -EINVAL;
+
+	dmabuf->phys = pages->dmabuf.phys;
+	dmabuf->start_offset = pages->dmabuf.start;
+	return 0;
+}
+
+static int pfn_reader_fill_dmabuf(struct pfn_reader_dmabuf *dmabuf,
+				  struct pfn_batch *batch,
+				  unsigned long start_index,
+				  unsigned long last_index)
+{
+	unsigned long start = dmabuf->start_offset + start_index * PAGE_SIZE;
+
+	/*
+	 * This works in PAGE_SIZE indexes, if the dmabuf is sliced and
+	 * starts/ends at a sub page offset then the batch to domain code will
+	 * adjust it.
+	 */
+	batch_add_pfn_num(batch, PHYS_PFN(dmabuf->phys.paddr + start),
+			  last_index - start_index + 1, BATCH_MMIO);
+	return 0;
+}
+
 /*
  * PFNs are stored in three places, in order of preference:
  * - The iopt_pages xarray. This is only populated if there is a
@@ -1094,7 +1128,10 @@ struct pfn_reader {
 	unsigned long batch_end_index;
 	unsigned long last_index;
 
-	struct pfn_reader_user user;
+	union {
+		struct pfn_reader_user user;
+		struct pfn_reader_dmabuf dmabuf;
+	};
 };
 
 static int pfn_reader_update_pinned(struct pfn_reader *pfns)
@@ -1130,7 +1167,7 @@ static int pfn_reader_fill_span(struct pfn_reader *pfns)
 {
 	struct interval_tree_double_span_iter *span = &pfns->span;
 	unsigned long start_index = pfns->batch_end_index;
-	struct pfn_reader_user *user = &pfns->user;
+	struct pfn_reader_user *user;
 	unsigned long npages;
 	struct iopt_area *area;
 	int rc;
@@ -1162,8 +1199,13 @@ static int pfn_reader_fill_span(struct pfn_reader *pfns)
 		return 0;
 	}
 
-	if (start_index >= pfns->user.upages_end) {
-		rc = pfn_reader_user_pin(&pfns->user, pfns->pages, start_index,
+	if (iopt_is_dmabuf(pfns->pages))
+		return pfn_reader_fill_dmabuf(&pfns->dmabuf, &pfns->batch,
+					      start_index, span->last_hole);
+
+	user = &pfns->user;
+	if (start_index >= user->upages_end) {
+		rc = pfn_reader_user_pin(user, pfns->pages, start_index,
 					 span->last_hole);
 		if (rc)
 			return rc;
@@ -1231,7 +1273,10 @@ static int pfn_reader_init(struct pfn_reader *pfns, struct iopt_pages *pages,
 	pfns->batch_start_index = start_index;
 	pfns->batch_end_index = start_index;
 	pfns->last_index = last_index;
-	pfn_reader_user_init(&pfns->user, pages);
+	if (iopt_is_dmabuf(pages))
+		pfn_reader_dmabuf_init(&pfns->dmabuf, pages);
+	else
+		pfn_reader_user_init(&pfns->user, pages);
 	rc = batch_init(&pfns->batch, last_index - start_index + 1);
 	if (rc)
 		return rc;
@@ -1252,8 +1297,12 @@ static int pfn_reader_init(struct pfn_reader *pfns, struct iopt_pages *pages,
 static void pfn_reader_release_pins(struct pfn_reader *pfns)
 {
 	struct iopt_pages *pages = pfns->pages;
-	struct pfn_reader_user *user = &pfns->user;
+	struct pfn_reader_user *user;
 
+	if (iopt_is_dmabuf(pages))
+		return;
+
+	user = &pfns->user;
 	if (user->upages_end > pfns->batch_end_index) {
 		/* Any pages not transferred to the batch are just unpinned */
 
@@ -1283,7 +1332,8 @@ static void pfn_reader_destroy(struct pfn_reader *pfns)
 	struct iopt_pages *pages = pfns->pages;
 
 	pfn_reader_release_pins(pfns);
-	pfn_reader_user_destroy(&pfns->user, pfns->pages);
+	if (!iopt_is_dmabuf(pfns->pages))
+		pfn_reader_user_destroy(&pfns->user, pfns->pages);
 	batch_destroy(&pfns->batch, NULL);
 	WARN_ON(pages->last_npinned != pages->npinned);
 }
@@ -1687,6 +1737,12 @@ static void __iopt_area_unfill_domain(struct iopt_area *area,
 
 	lockdep_assert_held(&pages->mutex);
 
+	if (iopt_is_dmabuf(pages)) {
+		iopt_area_unmap_domain_range(area, domain, start_index,
+					     last_index);
+		return;
+	}
+
 	/*
 	 * For security we must not unpin something that is still DMA mapped,
 	 * so this must unmap any IOVA before we go ahead and unpin the pages.
-- 
2.43.0

Re: [PATCH 6/9] iommufd: Have pfn_reader process DMABUF iopt_pages

[Nicolin Chen]

On Fri, Nov 07, 2025 at 12:49:38PM -0400, Jason Gunthorpe wrote:
> Make another sub implementation of pfn_reader for DMABUF. This version
> will fill the batch using the struct phys_vec recorded during the
> attachment.
> 
> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>

Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>

> @@ -1687,6 +1737,12 @@ static void __iopt_area_unfill_domain(struct iopt_area *area,
>  
>  	lockdep_assert_held(&pages->mutex);
>  
> +	if (iopt_is_dmabuf(pages)) {
> +		iopt_area_unmap_domain_range(area, domain, start_index,
> +					     last_index);
> +		return;
> +	}

Should it be:
	if (iopt_is_dmabuf(pages) && !iopt_dmabuf_revoked(pages)) {
?

Nicolin

Re: [PATCH 6/9] iommufd: Have pfn_reader process DMABUF iopt_pages

[Jason Gunthorpe]

On Thu, Nov 13, 2025 at 03:39:46PM -0800, Nicolin Chen wrote:
> > @@ -1687,6 +1737,12 @@ static void __iopt_area_unfill_domain(struct iopt_area *area,
> >  
> >  	lockdep_assert_held(&pages->mutex);
> >  
> > +	if (iopt_is_dmabuf(pages)) {
> > +		iopt_area_unmap_domain_range(area, domain, start_index,
> > +					     last_index);
> > +		return;
> > +	}
> 
> Should it be:
> 	if (iopt_is_dmabuf(pages) && !iopt_dmabuf_revoked(pages)) {
> ?

All callers have already done it, let's add an assertion though..

@@ -1873,6 +1873,8 @@ static void __iopt_area_unfill_domain(struct iopt_area *area,
        lockdep_assert_held(&pages->mutex);
 
        if (iopt_is_dmabuf(pages)) {
+               if (WARN_ON(iopt_dmabuf_revoked(pages)))
+                       return;
                iopt_area_unmap_domain_range(area, domain, start_index,
                                             last_index);
                return;

Jason

RE: [PATCH 6/9] iommufd: Have pfn_reader process DMABUF iopt_pages

[Tian, Kevin]

> From: Jason Gunthorpe <jgg@nvidia.com>
> Sent: Saturday, November 8, 2025 12:50 AM
> +
> +static int pfn_reader_fill_dmabuf(struct pfn_reader_dmabuf *dmabuf,
> +				  struct pfn_batch *batch,
> +				  unsigned long start_index,
> +				  unsigned long last_index)
> +{
> +	unsigned long start = dmabuf->start_offset + start_index * PAGE_SIZE;
> +
> +	/*
> +	 * This works in PAGE_SIZE indexes, if the dmabuf is sliced and
> +	 * starts/ends at a sub page offset then the batch to domain code will
> +	 * adjust it.
> +	 */

dmabuf->start_offset comes from pages->dmabuf.start, which is initialized as:

	pages->dmabuf.start = start - start_byte;

so it's always page-aligned. Where is the sub-page offset coming from?

> @@ -1687,6 +1737,12 @@ static void __iopt_area_unfill_domain(struct
> iopt_area *area,
> 
>  	lockdep_assert_held(&pages->mutex);
> 
> +	if (iopt_is_dmabuf(pages)) {
> +		iopt_area_unmap_domain_range(area, domain, start_index,
> +					     last_index);
> +		return;
> +	}
> +

this belongs to patch3?

Reviewed-by: Kevin Tian <kevin.tian@intel.com>

Re: [PATCH 6/9] iommufd: Have pfn_reader process DMABUF iopt_pages

[Jason Gunthorpe]

On Thu, Nov 20, 2025 at 08:04:37AM +0000, Tian, Kevin wrote:
> > From: Jason Gunthorpe <jgg@nvidia.com>
> > Sent: Saturday, November 8, 2025 12:50 AM
> > +
> > +static int pfn_reader_fill_dmabuf(struct pfn_reader_dmabuf *dmabuf,
> > +				  struct pfn_batch *batch,
> > +				  unsigned long start_index,
> > +				  unsigned long last_index)
> > +{
> > +	unsigned long start = dmabuf->start_offset + start_index * PAGE_SIZE;
> > +
> > +	/*
> > +	 * This works in PAGE_SIZE indexes, if the dmabuf is sliced and
> > +	 * starts/ends at a sub page offset then the batch to domain code will
> > +	 * adjust it.
> > +	 */
> 
> dmabuf->start_offset comes from pages->dmabuf.start, which is initialized as:
> 
> 	pages->dmabuf.start = start - start_byte;
> 
> so it's always page-aligned. Where is the sub-page offset coming from?

I need to go over this again to check it, this sub-page stuff is
a bit convoluted. start_offset should include the sub page offset
here..

> > @@ -1687,6 +1737,12 @@ static void __iopt_area_unfill_domain(struct
> > iopt_area *area,
> > 
> >  	lockdep_assert_held(&pages->mutex);
> > 
> > +	if (iopt_is_dmabuf(pages)) {
> > +		iopt_area_unmap_domain_range(area, domain, start_index,
> > +					     last_index);
> > +		return;
> > +	}
> > +
> 
> this belongs to patch3?

This is part of programming the domain with the dmabuf, the patch3 was
about the revoke which is a slightly different topic though they are
both similar.

Thanks,
Jason

Re: [PATCH 6/9] iommufd: Have pfn_reader process DMABUF iopt_pages

[Jason Gunthorpe]

On Thu, Nov 20, 2025 at 08:47:20PM -0400, Jason Gunthorpe wrote:
> On Thu, Nov 20, 2025 at 08:04:37AM +0000, Tian, Kevin wrote:
> > > From: Jason Gunthorpe <jgg@nvidia.com>
> > > Sent: Saturday, November 8, 2025 12:50 AM
> > > +
> > > +static int pfn_reader_fill_dmabuf(struct pfn_reader_dmabuf *dmabuf,
> > > +				  struct pfn_batch *batch,
> > > +				  unsigned long start_index,
> > > +				  unsigned long last_index)
> > > +{
> > > +	unsigned long start = dmabuf->start_offset + start_index * PAGE_SIZE;
> > > +
> > > +	/*
> > > +	 * This works in PAGE_SIZE indexes, if the dmabuf is sliced and
> > > +	 * starts/ends at a sub page offset then the batch to domain code will
> > > +	 * adjust it.
> > > +	 */
> > 
> > dmabuf->start_offset comes from pages->dmabuf.start, which is initialized as:
> > 
> > 	pages->dmabuf.start = start - start_byte;
> > 
> > so it's always page-aligned. Where is the sub-page offset coming from?
> 
> I need to go over this again to check it, this sub-page stuff is
> a bit convoluted. start_offset should include the sub page offset
> here..

So in the pfn reader section everything is page aligned, that means
the start location is also page aligned. The comment is just a bit
confusing, maybe this is better:

	/*
	 * start/last_index and start_offset are all PAGE_SIZE aligned, the
	 * batch is always filled using page size aligned PFNs just like the
	 * other types. If the dmabuf has been sliced with a sub page offset
	 * then the common batch to domain code will adjust it before mapping to
	 * the domain.
	 */

The adjustment is this code in batch_to_domain():

	/* The first index might be a partial page */
	if (start_index == iopt_area_index(area))
		page_offset = area->page_offset;

Thanks,
Jason

[PATCH 7/9] iommufd: Have iopt_map_file_pages convert the fd to a file

[Jason Gunthorpe]

Since dmabuf only has APIs that work on a int fd and not a struct file *,
pass the fd deeper into the call chain so we can use the dmabuf APIs as
is.

Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
 drivers/iommu/iommufd/io_pagetable.c    | 15 +++++++++++----
 drivers/iommu/iommufd/ioas.c            |  8 +-------
 drivers/iommu/iommufd/iommufd_private.h |  2 +-
 3 files changed, 13 insertions(+), 12 deletions(-)

diff --git a/drivers/iommu/iommufd/io_pagetable.c b/drivers/iommu/iommufd/io_pagetable.c
index a120c2ae942a8b..0bf6ca77888c0f 100644
--- a/drivers/iommu/iommufd/io_pagetable.c
+++ b/drivers/iommu/iommufd/io_pagetable.c
@@ -10,6 +10,7 @@
  */
 #include <linux/err.h>
 #include <linux/errno.h>
+#include <linux/file.h>
 #include <linux/iommu.h>
 #include <linux/iommufd.h>
 #include <linux/lockdep.h>
@@ -471,21 +472,27 @@ int iopt_map_user_pages(struct iommufd_ctx *ictx, struct io_pagetable *iopt,
  * @iopt: io_pagetable to act on
  * @iova: If IOPT_ALLOC_IOVA is set this is unused on input and contains
  *        the chosen iova on output. Otherwise is the iova to map to on input
- * @file: file to map
+ * @fd: fdno of a file to map
  * @start: map file starting at this byte offset
  * @length: Number of bytes to map
  * @iommu_prot: Combination of IOMMU_READ/WRITE/etc bits for the mapping
  * @flags: IOPT_ALLOC_IOVA or zero
  */
 int iopt_map_file_pages(struct iommufd_ctx *ictx, struct io_pagetable *iopt,
-			unsigned long *iova, struct file *file,
-			unsigned long start, unsigned long length,
-			int iommu_prot, unsigned int flags)
+			unsigned long *iova, int fd, unsigned long start,
+			unsigned long length, int iommu_prot,
+			unsigned int flags)
 {
 	struct iopt_pages *pages;
+	struct file *file;
+
+	file = fget(fd);
+	if (!file)
+		return -EBADF;
 
 	pages = iopt_alloc_file_pages(file, start, length,
 				      iommu_prot & IOMMU_WRITE);
+	fput(file);
 	if (IS_ERR(pages))
 		return PTR_ERR(pages);
 	return iopt_map_common(ictx, iopt, pages, iova, length,
diff --git a/drivers/iommu/iommufd/ioas.c b/drivers/iommu/iommufd/ioas.c
index 1542c5fd10a85c..0dee38d7252df4 100644
--- a/drivers/iommu/iommufd/ioas.c
+++ b/drivers/iommu/iommufd/ioas.c
@@ -207,7 +207,6 @@ int iommufd_ioas_map_file(struct iommufd_ucmd *ucmd)
 	unsigned long iova = cmd->iova;
 	struct iommufd_ioas *ioas;
 	unsigned int flags = 0;
-	struct file *file;
 	int rc;
 
 	if (cmd->flags &
@@ -229,11 +228,7 @@ int iommufd_ioas_map_file(struct iommufd_ucmd *ucmd)
 	if (!(cmd->flags & IOMMU_IOAS_MAP_FIXED_IOVA))
 		flags = IOPT_ALLOC_IOVA;
 
-	file = fget(cmd->fd);
-	if (!file)
-		return -EBADF;
-
-	rc = iopt_map_file_pages(ucmd->ictx, &ioas->iopt, &iova, file,
+	rc = iopt_map_file_pages(ucmd->ictx, &ioas->iopt, &iova, cmd->fd,
 				 cmd->start, cmd->length,
 				 conv_iommu_prot(cmd->flags), flags);
 	if (rc)
@@ -243,7 +238,6 @@ int iommufd_ioas_map_file(struct iommufd_ucmd *ucmd)
 	rc = iommufd_ucmd_respond(ucmd, sizeof(*cmd));
 out_put:
 	iommufd_put_object(ucmd->ictx, &ioas->obj);
-	fput(file);
 	return rc;
 }
 
diff --git a/drivers/iommu/iommufd/iommufd_private.h b/drivers/iommu/iommufd/iommufd_private.h
index 10ee3510312132..a2786fee7603f5 100644
--- a/drivers/iommu/iommufd/iommufd_private.h
+++ b/drivers/iommu/iommufd/iommufd_private.h
@@ -108,7 +108,7 @@ int iopt_map_user_pages(struct iommufd_ctx *ictx, struct io_pagetable *iopt,
 			unsigned long length, int iommu_prot,
 			unsigned int flags);
 int iopt_map_file_pages(struct iommufd_ctx *ictx, struct io_pagetable *iopt,
-			unsigned long *iova, struct file *file,
+			unsigned long *iova, int fd,
 			unsigned long start, unsigned long length,
 			int iommu_prot, unsigned int flags);
 int iopt_map_pages(struct io_pagetable *iopt, struct list_head *pages_list,
-- 
2.43.0

Re: [PATCH 7/9] iommufd: Have iopt_map_file_pages convert the fd to a file

[Nicolin Chen]

On Fri, Nov 07, 2025 at 12:49:39PM -0400, Jason Gunthorpe wrote:
> Since dmabuf only has APIs that work on a int fd and not a struct file *,

"an int fd"

> pass the fd deeper into the call chain so we can use the dmabuf APIs as
> is.
> 
> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>

Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>

RE: [PATCH 7/9] iommufd: Have iopt_map_file_pages convert the fd to a file

[Tian, Kevin]

> From: Jason Gunthorpe <jgg@nvidia.com>
> Sent: Saturday, November 8, 2025 12:50 AM
> 
> Since dmabuf only has APIs that work on a int fd and not a struct file *,
> pass the fd deeper into the call chain so we can use the dmabuf APIs as
> is.
> 
> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>

Reviewed-by: Kevin Tian <kevin.tian@intel.com>

[PATCH 8/9] iommufd: Accept a DMABUF through IOMMU_IOAS_MAP_FILE

[Jason Gunthorpe]

Finally call iopt_alloc_dmabuf_pages() if the user passed in a DMABUF
through IOMMU_IOAS_MAP_FILE. This makes the feature visible to userspace.

Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
 drivers/iommu/iommufd/io_pagetable.c | 43 +++++++++++++++++++++-------
 drivers/iommu/iommufd/io_pagetable.h |  4 ++-
 drivers/iommu/iommufd/pages.c        | 13 ++++-----
 3 files changed, 41 insertions(+), 19 deletions(-)

diff --git a/drivers/iommu/iommufd/io_pagetable.c b/drivers/iommu/iommufd/io_pagetable.c
index 0bf6ca77888c0f..44a0a7c79388d8 100644
--- a/drivers/iommu/iommufd/io_pagetable.c
+++ b/drivers/iommu/iommufd/io_pagetable.c
@@ -8,6 +8,7 @@
  * The datastructure uses the iopt_pages to optimize the storage of the PFNs
  * between the domains and xarray.
  */
+#include <linux/dma-buf.h>
 #include <linux/err.h>
 #include <linux/errno.h>
 #include <linux/file.h>
@@ -484,19 +485,41 @@ int iopt_map_file_pages(struct iommufd_ctx *ictx, struct io_pagetable *iopt,
 			unsigned int flags)
 {
 	struct iopt_pages *pages;
-	struct file *file;
+	struct dma_buf *dmabuf;
+	unsigned long start_byte;
+	unsigned long last;
 
-	file = fget(fd);
-	if (!file)
-		return -EBADF;
+	if (!length)
+		return -EINVAL;
+	if (check_add_overflow(start, length - 1, &last))
+		return -EOVERFLOW;
+
+	start_byte = start - ALIGN_DOWN(start, PAGE_SIZE);
+	dmabuf = dma_buf_get(fd);
+	if (!IS_ERR(dmabuf)) {
+		pages = iopt_alloc_dmabuf_pages(ictx, dmabuf, start_byte, start,
+						length,
+						iommu_prot & IOMMU_WRITE);
+		if (IS_ERR(pages)) {
+			dma_buf_put(dmabuf);
+			return PTR_ERR(pages);
+		}
+	} else {
+		struct file *file;
+
+		file = fget(fd);
+		if (!file)
+			return -EBADF;
+
+		pages = iopt_alloc_file_pages(file, start_byte, start, length,
+					      iommu_prot & IOMMU_WRITE);
+		fput(file);
+		if (IS_ERR(pages))
+			return PTR_ERR(pages);
+	}
 
-	pages = iopt_alloc_file_pages(file, start, length,
-				      iommu_prot & IOMMU_WRITE);
-	fput(file);
-	if (IS_ERR(pages))
-		return PTR_ERR(pages);
 	return iopt_map_common(ictx, iopt, pages, iova, length,
-			       start - pages->start, iommu_prot, flags);
+			       start_byte, iommu_prot, flags);
 }
 
 struct iova_bitmap_fn_arg {
diff --git a/drivers/iommu/iommufd/io_pagetable.h b/drivers/iommu/iommufd/io_pagetable.h
index 81c6093beee2a5..e033d44b0feb60 100644
--- a/drivers/iommu/iommufd/io_pagetable.h
+++ b/drivers/iommu/iommufd/io_pagetable.h
@@ -264,7 +264,9 @@ static inline bool iopt_dmabuf_revoked(struct iopt_pages *pages)
 
 struct iopt_pages *iopt_alloc_user_pages(void __user *uptr,
 					 unsigned long length, bool writable);
-struct iopt_pages *iopt_alloc_file_pages(struct file *file, unsigned long start,
+struct iopt_pages *iopt_alloc_file_pages(struct file *file,
+					 unsigned long start_byte,
+					 unsigned long start,
 					 unsigned long length, bool writable);
 struct iopt_pages *iopt_alloc_dmabuf_pages(struct iommufd_ctx *ictx,
 					   struct dma_buf *dmabuf,
diff --git a/drivers/iommu/iommufd/pages.c b/drivers/iommu/iommufd/pages.c
index 935184dc68be32..410ddce4e99d8f 100644
--- a/drivers/iommu/iommufd/pages.c
+++ b/drivers/iommu/iommufd/pages.c
@@ -1412,22 +1412,19 @@ struct iopt_pages *iopt_alloc_user_pages(void __user *uptr,
 	return pages;
 }
 
-struct iopt_pages *iopt_alloc_file_pages(struct file *file, unsigned long start,
+struct iopt_pages *iopt_alloc_file_pages(struct file *file,
+					 unsigned long start_byte,
+					 unsigned long start,
 					 unsigned long length, bool writable)
 
 {
 	struct iopt_pages *pages;
-	unsigned long start_down = ALIGN_DOWN(start, PAGE_SIZE);
-	unsigned long end;
 
-	if (length && check_add_overflow(start, length - 1, &end))
-		return ERR_PTR(-EOVERFLOW);
-
-	pages = iopt_alloc_pages(start - start_down, length, writable);
+	pages = iopt_alloc_pages(start_byte, length, writable);
 	if (IS_ERR(pages))
 		return pages;
 	pages->file = get_file(file);
-	pages->start = start_down;
+	pages->start = start - start_byte;
 	pages->type = IOPT_ADDRESS_FILE;
 	return pages;
 }
-- 
2.43.0

Re: [PATCH 8/9] iommufd: Accept a DMABUF through IOMMU_IOAS_MAP_FILE

[Nicolin Chen]

On Fri, Nov 07, 2025 at 12:49:40PM -0400, Jason Gunthorpe wrote:
> Finally call iopt_alloc_dmabuf_pages() if the user passed in a DMABUF
> through IOMMU_IOAS_MAP_FILE. This makes the feature visible to userspace.
> 
> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>

Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>

With a nit:

> -struct iopt_pages *iopt_alloc_file_pages(struct file *file, unsigned long start,
> +struct iopt_pages *iopt_alloc_file_pages(struct file *file,
> +					 unsigned long start_byte,
> +					 unsigned long start,
>  					 unsigned long length, bool writable);

Passing in start_byte looks like a cleanup to me, aligning with
what iopt_map_common() has.

Since we are doing this cleanup, maybe we could follow the same
sequence: xxx, start, length, start_byte, writable?

Thanks
Nicolin

Re: [PATCH 8/9] iommufd: Accept a DMABUF through IOMMU_IOAS_MAP_FILE

[Jason Gunthorpe]

On Thu, Nov 13, 2025 at 04:05:09PM -0800, Nicolin Chen wrote:
> > -struct iopt_pages *iopt_alloc_file_pages(struct file *file, unsigned long start,
> > +struct iopt_pages *iopt_alloc_file_pages(struct file *file,
> > +					 unsigned long start_byte,
> > +					 unsigned long start,
> >  					 unsigned long length, bool writable);
> 
> Passing in start_byte looks like a cleanup to me, aligning with
> what iopt_map_common() has.

> Since we are doing this cleanup, maybe we could follow the same
> sequence: xxx, start, length, start_byte, writable?

??

static int iopt_map_common(struct iommufd_ctx *ictx, struct io_pagetable *iopt,
			   struct iopt_pages *pages, unsigned long *iova,
			   unsigned long length, unsigned long start_byte,
			   int iommu_prot, unsigned int flags)

Not the same arguments, we don't pass start and start_byte there?

Jason

Re: [PATCH 8/9] iommufd: Accept a DMABUF through IOMMU_IOAS_MAP_FILE

[Nicolin Chen]

On Tue, Nov 18, 2025 at 03:44:01PM -0400, Jason Gunthorpe wrote:
> On Thu, Nov 13, 2025 at 04:05:09PM -0800, Nicolin Chen wrote:
> > > -struct iopt_pages *iopt_alloc_file_pages(struct file *file, unsigned long start,
> > > +struct iopt_pages *iopt_alloc_file_pages(struct file *file,
> > > +					 unsigned long start_byte,
> > > +					 unsigned long start,
> > >  					 unsigned long length, bool writable);
> > 
> > Passing in start_byte looks like a cleanup to me, aligning with
> > what iopt_map_common() has.
> 
> > Since we are doing this cleanup, maybe we could follow the same
> > sequence: xxx, start, length, start_byte, writable?
> 
> ??
> 
> static int iopt_map_common(struct iommufd_ctx *ictx, struct io_pagetable *iopt,
> 			   struct iopt_pages *pages, unsigned long *iova,
> 			   unsigned long length, unsigned long start_byte,
> 			   int iommu_prot, unsigned int flags)
> 
> Not the same arguments, we don't pass start and start_byte there?

I found that iopt_map_common() has a "start_byte" following the
"length". So, wondering if it'd be cleaner to do the same.

But, yes, they are slightly different. Let's keep it as-is then.

Thanks
Nicolin

RE: [PATCH 8/9] iommufd: Accept a DMABUF through IOMMU_IOAS_MAP_FILE

[Tian, Kevin]

> From: Jason Gunthorpe <jgg@nvidia.com>
> Sent: Saturday, November 8, 2025 12:50 AM
> 
> Finally call iopt_alloc_dmabuf_pages() if the user passed in a DMABUF
> through IOMMU_IOAS_MAP_FILE. This makes the feature visible to
> userspace.
> 
> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>

Reviewed-by: Kevin Tian <kevin.tian@intel.com>

[PATCH 9/9] iommufd/selftest: Add some tests for the dmabuf flow

[Jason Gunthorpe]

Basic tests of establishing a dmabuf and revoking it. The selftest kernel
side provides a basic small dmabuf for this testing.

Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
 drivers/iommu/iommufd/iommufd_private.h       |  10 ++
 drivers/iommu/iommufd/iommufd_test.h          |  10 ++
 drivers/iommu/iommufd/pages.c                 |   4 +
 drivers/iommu/iommufd/selftest.c              | 142 ++++++++++++++++++
 tools/testing/selftests/iommu/iommufd.c       |  43 ++++++
 tools/testing/selftests/iommu/iommufd_utils.h |  44 ++++++
 6 files changed, 253 insertions(+)

diff --git a/drivers/iommu/iommufd/iommufd_private.h b/drivers/iommu/iommufd/iommufd_private.h
index a2786fee7603f5..ef2db82e3d7bf5 100644
--- a/drivers/iommu/iommufd/iommufd_private.h
+++ b/drivers/iommu/iommufd/iommufd_private.h
@@ -19,6 +19,8 @@ struct iommu_domain;
 struct iommu_group;
 struct iommu_option;
 struct iommufd_device;
+struct dma_buf_attachment;
+struct dma_buf_phys_vec;
 
 struct iommufd_sw_msi_map {
 	struct list_head sw_msi_item;
@@ -713,6 +715,8 @@ bool iommufd_should_fail(void);
 int __init iommufd_test_init(void);
 void iommufd_test_exit(void);
 bool iommufd_selftest_is_mock_dev(struct device *dev);
+int iommufd_test_dma_buf_iommufd_map(struct dma_buf_attachment *attachment,
+				     struct dma_buf_phys_vec *phys);
 #else
 static inline void iommufd_test_syz_conv_iova_id(struct iommufd_ucmd *ucmd,
 						 unsigned int ioas_id,
@@ -734,5 +738,11 @@ static inline bool iommufd_selftest_is_mock_dev(struct device *dev)
 {
 	return false;
 }
+static inline int
+iommufd_test_dma_buf_iommufd_map(struct dma_buf_attachment *attachment,
+				 struct dma_buf_phys_vec *phys)
+{
+	return -EOPNOTSUPP;
+}
 #endif
 #endif
diff --git a/drivers/iommu/iommufd/iommufd_test.h b/drivers/iommu/iommufd/iommufd_test.h
index 8fc618b2bcf967..9166c39eb0c8b2 100644
--- a/drivers/iommu/iommufd/iommufd_test.h
+++ b/drivers/iommu/iommufd/iommufd_test.h
@@ -29,6 +29,8 @@ enum {
 	IOMMU_TEST_OP_PASID_REPLACE,
 	IOMMU_TEST_OP_PASID_DETACH,
 	IOMMU_TEST_OP_PASID_CHECK_HWPT,
+	IOMMU_TEST_OP_DMABUF_GET,
+	IOMMU_TEST_OP_DMABUF_REVOKE,
 };
 
 enum {
@@ -176,6 +178,14 @@ struct iommu_test_cmd {
 			__u32 hwpt_id;
 			/* @id is stdev_id */
 		} pasid_check;
+		struct {
+			__u32 length;
+			__u32 open_flags;
+		} dmabuf_get;
+		struct {
+			__s32 dmabuf_fd;
+			__u32 revoked;
+		} dmabuf_revoke;
 	};
 	__u32 last;
 };
diff --git a/drivers/iommu/iommufd/pages.c b/drivers/iommu/iommufd/pages.c
index 410ddce4e99d8f..01c55e6519ee84 100644
--- a/drivers/iommu/iommufd/pages.c
+++ b/drivers/iommu/iommufd/pages.c
@@ -1464,6 +1464,10 @@ sym_vfio_pci_dma_buf_iommufd_map(struct dma_buf_attachment *attachment,
 	typeof(&vfio_pci_dma_buf_iommufd_map) fn;
 	int rc;
 
+	rc = iommufd_test_dma_buf_iommufd_map(attachment, phys);
+	if (rc != -EOPNOTSUPP)
+		return rc;
+
 	if (!IS_ENABLED(CONFIG_VFIO_PCI_DMABUF))
 		return -EOPNOTSUPP;
 
diff --git a/drivers/iommu/iommufd/selftest.c b/drivers/iommu/iommufd/selftest.c
index de178827a078a9..ccd90cf73a73d6 100644
--- a/drivers/iommu/iommufd/selftest.c
+++ b/drivers/iommu/iommufd/selftest.c
@@ -5,6 +5,8 @@
  */
 #include <linux/anon_inodes.h>
 #include <linux/debugfs.h>
+#include <linux/dma-buf.h>
+#include <linux/dma-resv.h>
 #include <linux/fault-inject.h>
 #include <linux/file.h>
 #include <linux/iommu.h>
@@ -2031,6 +2033,139 @@ void iommufd_selftest_destroy(struct iommufd_object *obj)
 	}
 }
 
+struct iommufd_test_dma_buf {
+	void *memory;
+	size_t length;
+	bool revoked;
+};
+
+static int iommufd_test_dma_buf_attach(struct dma_buf *dmabuf,
+				       struct dma_buf_attachment *attachment)
+{
+	return 0;
+}
+
+static void iommufd_test_dma_buf_detach(struct dma_buf *dmabuf,
+					struct dma_buf_attachment *attachment)
+{
+}
+
+static struct sg_table *
+iommufd_test_dma_buf_map(struct dma_buf_attachment *attachment,
+			 enum dma_data_direction dir)
+{
+	return ERR_PTR(-EOPNOTSUPP);
+}
+
+static void iommufd_test_dma_buf_unmap(struct dma_buf_attachment *attachment,
+				       struct sg_table *sgt,
+				       enum dma_data_direction dir)
+{
+}
+
+static void iommufd_test_dma_buf_release(struct dma_buf *dmabuf)
+{
+	struct iommufd_test_dma_buf *priv = dmabuf->priv;
+
+	kfree(priv);
+}
+
+static const struct dma_buf_ops iommufd_test_dmabuf_ops = {
+	.attach = iommufd_test_dma_buf_attach,
+	.detach = iommufd_test_dma_buf_detach,
+	.map_dma_buf = iommufd_test_dma_buf_map,
+	.release = iommufd_test_dma_buf_release,
+	.unmap_dma_buf = iommufd_test_dma_buf_unmap,
+};
+
+int iommufd_test_dma_buf_iommufd_map(struct dma_buf_attachment *attachment,
+				     struct dma_buf_phys_vec *phys)
+{
+	struct iommufd_test_dma_buf *priv = attachment->dmabuf->priv;
+
+	dma_resv_assert_held(attachment->dmabuf->resv);
+
+	if (attachment->dmabuf->ops != &iommufd_test_dmabuf_ops)
+		return -EOPNOTSUPP;
+
+	if (priv->revoked)
+		return -ENODEV;
+
+	phys->paddr = virt_to_phys(priv->memory);
+	phys->len = priv->length;
+	return 0;
+}
+
+static int iommufd_test_dmabuf_get(struct iommufd_ucmd *ucmd,
+				   unsigned int open_flags,
+				   size_t len)
+{
+	DEFINE_DMA_BUF_EXPORT_INFO(exp_info);
+	struct iommufd_test_dma_buf *priv;
+	struct dma_buf *dmabuf;
+	int rc;
+
+	len = ALIGN(len, PAGE_SIZE);
+	if (len == 0 || len > PAGE_SIZE * 512)
+		return -EINVAL;
+
+	priv = kzalloc(sizeof(*priv), GFP_KERNEL);
+	if (!priv)
+		return -ENOMEM;
+
+	priv->length = len;
+	priv->memory = kzalloc(len, GFP_KERNEL);
+	if (!priv->memory) {
+		rc = -ENOMEM;
+		goto err_free;
+	}
+
+	exp_info.ops = &iommufd_test_dmabuf_ops;
+	exp_info.size = len;
+	exp_info.flags = open_flags;
+	exp_info.priv = priv;
+
+	dmabuf = dma_buf_export(&exp_info);
+	if (IS_ERR(dmabuf)) {
+		rc = PTR_ERR(dmabuf);
+		goto err_free;
+	}
+
+	return dma_buf_fd(dmabuf, open_flags);
+
+err_free:
+	kfree(priv->memory);
+	kfree(priv);
+	return rc;
+}
+
+static int iommufd_test_dmabuf_revoke(struct iommufd_ucmd *ucmd, int fd,
+				      bool revoked)
+{
+	struct iommufd_test_dma_buf *priv;
+	struct dma_buf *dmabuf;
+	int rc = 0;
+
+	dmabuf = dma_buf_get(fd);
+	if (IS_ERR(dmabuf))
+		return PTR_ERR(dmabuf);
+
+	if (dmabuf->ops != &iommufd_test_dmabuf_ops) {
+		rc = -EOPNOTSUPP;
+		goto err_put;
+	}
+
+	priv = dmabuf->priv;
+	dma_resv_lock(dmabuf->resv, NULL);
+	priv->revoked = revoked;
+	dma_buf_move_notify(dmabuf);
+	dma_resv_unlock(dmabuf->resv);
+
+err_put:
+	dma_buf_put(dmabuf);
+	return rc;
+}
+
 int iommufd_test(struct iommufd_ucmd *ucmd)
 {
 	struct iommu_test_cmd *cmd = ucmd->cmd;
@@ -2109,6 +2244,13 @@ int iommufd_test(struct iommufd_ucmd *ucmd)
 		return iommufd_test_pasid_detach(ucmd, cmd);
 	case IOMMU_TEST_OP_PASID_CHECK_HWPT:
 		return iommufd_test_pasid_check_hwpt(ucmd, cmd);
+	case IOMMU_TEST_OP_DMABUF_GET:
+		return iommufd_test_dmabuf_get(ucmd, cmd->dmabuf_get.open_flags,
+					       cmd->dmabuf_get.length);
+	case IOMMU_TEST_OP_DMABUF_REVOKE:
+		return iommufd_test_dmabuf_revoke(ucmd,
+						  cmd->dmabuf_revoke.dmabuf_fd,
+						  cmd->dmabuf_revoke.revoked);
 	default:
 		return -EOPNOTSUPP;
 	}
diff --git a/tools/testing/selftests/iommu/iommufd.c b/tools/testing/selftests/iommu/iommufd.c
index 3eebf5e3b974f4..de348d6412791b 100644
--- a/tools/testing/selftests/iommu/iommufd.c
+++ b/tools/testing/selftests/iommu/iommufd.c
@@ -1574,6 +1574,49 @@ TEST_F(iommufd_ioas, copy_sweep)
 	test_ioctl_destroy(dst_ioas_id);
 }
 
+TEST_F(iommufd_ioas, dmabuf_simple)
+{
+	size_t buf_size = PAGE_SIZE*4;
+	__u64 iova;
+	int dfd;
+
+	test_cmd_get_dmabuf(buf_size, &dfd);
+	test_err_ioctl_ioas_map_file(EINVAL, dfd, 0, 0, &iova);
+	test_err_ioctl_ioas_map_file(EINVAL, dfd, buf_size, buf_size, &iova);
+	test_err_ioctl_ioas_map_file(EINVAL, dfd, 0, buf_size + 1, &iova);
+	test_ioctl_ioas_map_file(dfd, 0, buf_size, &iova);
+
+	close(dfd);
+}
+
+TEST_F(iommufd_ioas, dmabuf_revoke)
+{
+	size_t buf_size = PAGE_SIZE*4;
+	__u32 hwpt_id;
+	__u64 iova;
+	__u64 iova2;
+	int dfd;
+
+	test_cmd_get_dmabuf(buf_size, &dfd);
+	test_ioctl_ioas_map_file(dfd, 0, buf_size, &iova);
+	test_cmd_revoke_dmabuf(dfd, true);
+
+	if (variant->mock_domains)
+		test_cmd_hwpt_alloc(self->device_id, self->ioas_id, 0,
+				    &hwpt_id);
+
+	test_err_ioctl_ioas_map_file(ENODEV, dfd, 0, buf_size, &iova2);
+
+	test_cmd_revoke_dmabuf(dfd, false);
+	test_ioctl_ioas_map_file(dfd, 0, buf_size, &iova2);
+
+	/* Restore the iova back */
+	test_ioctl_ioas_unmap(iova, buf_size);
+	test_ioctl_ioas_map_fixed_file(dfd, 0, buf_size, iova);
+
+	close(dfd);
+}
+
 FIXTURE(iommufd_mock_domain)
 {
 	int fd;
diff --git a/tools/testing/selftests/iommu/iommufd_utils.h b/tools/testing/selftests/iommu/iommufd_utils.h
index 772ca1db6e5971..f6d1678b310e1f 100644
--- a/tools/testing/selftests/iommu/iommufd_utils.h
+++ b/tools/testing/selftests/iommu/iommufd_utils.h
@@ -548,6 +548,39 @@ static int _test_cmd_destroy_access_pages(int fd, unsigned int access_id,
 	EXPECT_ERRNO(_errno, _test_cmd_destroy_access_pages(              \
 				     self->fd, access_id, access_pages_id))
 
+static int _test_cmd_get_dmabuf(int fd, size_t len, int *out_fd)
+{
+	struct iommu_test_cmd cmd = {
+		.size = sizeof(cmd),
+		.op = IOMMU_TEST_OP_DMABUF_GET,
+		.dmabuf_get = { .length = len, .open_flags = O_CLOEXEC },
+	};
+
+	*out_fd = ioctl(fd, IOMMU_TEST_CMD, &cmd);
+	if (*out_fd < 0)
+		return -1;
+	return 0;
+}
+#define test_cmd_get_dmabuf(len, out_fd) \
+	ASSERT_EQ(0, _test_cmd_get_dmabuf(self->fd, len, out_fd))
+
+static int _test_cmd_revoke_dmabuf(int fd, int dmabuf_fd, bool revoked)
+{
+	struct iommu_test_cmd cmd = {
+		.size = sizeof(cmd),
+		.op = IOMMU_TEST_OP_DMABUF_REVOKE,
+		.dmabuf_revoke = { .dmabuf_fd = dmabuf_fd, .revoked = revoked },
+	};
+	int ret;
+
+	ret = ioctl(fd, IOMMU_TEST_CMD, &cmd);
+	if (ret < 0)
+		return -1;
+	return 0;
+}
+#define test_cmd_revoke_dmabuf(dmabuf_fd, revoke) \
+	ASSERT_EQ(0, _test_cmd_revoke_dmabuf(self->fd, dmabuf_fd, revoke))
+
 static int _test_ioctl_destroy(int fd, unsigned int id)
 {
 	struct iommu_destroy cmd = {
@@ -718,6 +751,17 @@ static int _test_ioctl_ioas_map_file(int fd, unsigned int ioas_id, int mfd,
 			  self->fd, ioas_id, mfd, start, length, iova_p,     \
 			  IOMMU_IOAS_MAP_WRITEABLE | IOMMU_IOAS_MAP_READABLE))
 
+#define test_ioctl_ioas_map_fixed_file(mfd, start, length, iova)          \
+	({                                                                \
+		__u64 __iova = iova;                                      \
+		ASSERT_EQ(0, _test_ioctl_ioas_map_file(                   \
+				     self->fd, self->ioas_id, mfd, start, \
+				     length, &__iova,                     \
+				     IOMMU_IOAS_MAP_FIXED_IOVA |          \
+					     IOMMU_IOAS_MAP_WRITEABLE |   \
+					     IOMMU_IOAS_MAP_READABLE));   \
+	})
+
 static int _test_ioctl_set_temp_memory_limit(int fd, unsigned int limit)
 {
 	struct iommu_test_cmd memlimit_cmd = {
-- 
2.43.0

Re: [PATCH 9/9] iommufd/selftest: Add some tests for the dmabuf flow

[Nicolin Chen]

On Fri, Nov 07, 2025 at 12:49:41PM -0400, Jason Gunthorpe wrote:
> Basic tests of establishing a dmabuf and revoking it. The selftest kernel
> side provides a basic small dmabuf for this testing.
> 
> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>

Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>

> +static void iommufd_test_dma_buf_release(struct dma_buf *dmabuf)
> +{
> +	struct iommufd_test_dma_buf *priv = dmabuf->priv;
> +
> +	kfree(priv);
> +}

Missing
	kfree(priv->memory);
?

Re: [PATCH 9/9] iommufd/selftest: Add some tests for the dmabuf flow

[Jason Gunthorpe]

On Fri, Nov 07, 2025 at 11:43:56AM -0800, Nicolin Chen wrote:
> > +static void iommufd_test_dma_buf_release(struct dma_buf *dmabuf)
> > +{
> > +	struct iommufd_test_dma_buf *priv = dmabuf->priv;
> > +
> > +	kfree(priv);
> > +}
> 
> Missing
> 	kfree(priv->memory);
> ?

Yes, thanks

Jason

RE: [PATCH 9/9] iommufd/selftest: Add some tests for the dmabuf flow

[Tian, Kevin]

> From: Jason Gunthorpe <jgg@nvidia.com>
> Sent: Wednesday, November 19, 2025 3:25 AM
> 
> On Fri, Nov 07, 2025 at 11:43:56AM -0800, Nicolin Chen wrote:
> > > +static void iommufd_test_dma_buf_release(struct dma_buf *dmabuf)
> > > +{
> > > +	struct iommufd_test_dma_buf *priv = dmabuf->priv;
> > > +
> > > +	kfree(priv);
> > > +}
> >
> > Missing
> > 	kfree(priv->memory);
> > ?
> 
> Yes, thanks
> 

Reviewed-by: Kevin Tian <kevin.tian@intel.com>

Re: [PATCH 0/9] Initial DMABUF support for iommufd

[Nicolin Chen]

On Fri, Nov 07, 2025 at 12:49:32PM -0400, Jason Gunthorpe wrote:
> This is on github: https://github.com/jgunthorpe/linux/commits/iommufd_dmabuf
> 
> v2:
>  - Rebase on Leon's v7
>  - Fix mislocking in an iopt_fill_domain() error path

I have verified this v2 using the branch above, by drafting a QEMU
patch for dmabuf on top of Shameer's vSMMU v5 series:
https://github.com/nicolinc/qemu/commits/wip/iommufd_dmabuf/

With that, I see GPU BAR memory be correctly fetched in the QEMU:
vfio_region_dmabuf Device 0009:01:00.0, region "0009:01:00.0 BAR 0", offset: 0x0, size: 0x1000000
vfio_region_dmabuf Device 0009:01:00.0, region "0009:01:00.0 BAR 2", offset: 0x0, size: 0x44f00000
vfio_region_dmabuf Device 0009:01:00.0, region "0009:01:00.0 BAR 4", offset: 0x0, size: 0x17a0000000

Tested-by: Nicolin Chen <nicolinc@nvidia.com>

Re: [PATCH 0/9] Initial DMABUF support for iommufd

[Shuai Xue]

Hi, Nicolin

在 2025/11/8 01:52, Nicolin Chen 写道:
> On Fri, Nov 07, 2025 at 12:49:32PM -0400, Jason Gunthorpe wrote:
>> This is on github: https://github.com/jgunthorpe/linux/commits/iommufd_dmabuf
>>
>> v2:
>>   - Rebase on Leon's v7
>>   - Fix mislocking in an iopt_fill_domain() error path
> 
> I have verified this v2 using the branch above, by drafting a QEMU
> patch for dmabuf on top of Shameer's vSMMU v5 series:
> https://github.com/nicolinc/qemu/commits/wip/iommufd_dmabuf/
> 
> With that, I see GPU BAR memory be correctly fetched in the QEMU:
> vfio_region_dmabuf Device 0009:01:00.0, region "0009:01:00.0 BAR 0", offset: 0x0, size: 0x1000000
> vfio_region_dmabuf Device 0009:01:00.0, region "0009:01:00.0 BAR 2", offset: 0x0, size: 0x44f00000
> vfio_region_dmabuf Device 0009:01:00.0, region "0009:01:00.0 BAR 4", offset: 0x0, size: 0x17a0000000
> 
> Tested-by: Nicolin Chen <nicolinc@nvidia.com>

I tested with your QEMU branch and this v2 branch, and observed
the following:

**QEMU successfully maps the BAR regions via dmabuf:**

vfio_region_mmap Region 0008:01:00.0 BAR 0 mmaps[0] [0x0 - 0x3ffffff]
vfio_region_dmabuf Device 0008:01:00.0, region "0008:01:00.0 BAR 0", offset: 0x0, size: 0x4000000
vfio_region_mmap Region 0008:01:00.0 BAR 2 mmaps[0] [0x0 - 0x3fffffffff]
vfio_region_dmabuf Device 0008:01:00.0, region "0008:01:00.0 BAR 2", offset: 0x0, size: 0x4000000000
vfio_region_mmap Region 0008:01:00.0 BAR 4 mmaps[0] [0x0 - 0x2e41efffff]
vfio_region_dmabuf Device 0008:01:00.0, region "0008:01:00.0 BAR 4", offset: 0x0, size: 0x2e41f00000

**IOMMUFD backend successfully maps the DMA regions:**

iommufd_backend_map_file_dma  iommufd=303 ioas=2 iova=0x30000000000 size=0xb90000 fd=306 start=0 readonly=0 (0)
vfio_listener_region_add_ram region_add [ram] 0x30000b91000 - 0x30003ffffff [0xffcae8b91000]
iommufd_backend_map_file_dma  iommufd=303 ioas=2 iova=0x30000b91000 size=0x346f000 fd=306 start=12128256 readonly=0 (0)
vfio_listener_region_add_ram region_add [ram] 0x2c000000000 - 0x2ffffffffff [0xff8ac0000000]
iommufd_backend_map_file_dma  iommufd=303 ioas=2 iova=0x2c000000000 size=0x4000000000 fd=307 start=0 readonly=0 (0)
vfio_listener_region_add_ram region_add [ram] 0x28000000000 - 0x2ae41efffff [0xff5c7e000000]
iommufd_backend_map_file_dma  iommufd=303 ioas=2 iova=0x28000000000 size=0x2e41f00000 fd=308 start=0 readonly=0 (0)

**However, the guest kernel crashes during PCI initialization:**

[    1.422280] acpi PNP0A08:01: ECAM area [mem 0x4010100000-0x40102fffff] reserved by PNP0C02:00^M
[    1.422907] acpi PNP0A08:01: ECAM at [mem 0x4010100000-0x40102fffff] for [bus 01-02]^M
[    1.423541] ------------[ cut here ]------------^M
[    1.423905] kernel BUG at mm/vmalloc.c:109!^M
[    1.424240] Internal error: Oops - BUG: 00000000f2000800 [#1] SMP^M
[    1.424724] Modules linked in:^M
[    1.424972] CPU: 19 PID: 1 Comm: swapper/0 Not tainted 6.6.102-5.100_rc1.1.al8.aarch64+64k #1^M
[    1.425647] Hardware name: QEMU KVM Virtual Machine, BIOS 0.0.0 02/06/2015^M
[    1.426201] pstate: 01401005 (nzcv daif +PAN -UAO -TCO +DIT +SSBS BTYPE=--)^M
[    1.426751] pc : vmap_pte_range+0x158/0x2b8^M
[    1.427101] lr : vmap_pte_range+0x178/0x2b8^M
[    1.427455] sp : ffff800083fcf6d0^M
[    1.427758] x29: ffff800083fcf6d0 x28: 0068000040820713 x27: 0000000000004083^M
[    1.428328] x26: ffff800083fcf7d4 x25: 0068000040820713 x24: 000000000000003f^M
[    1.428895] x23: 0068000000000713 x22: 0010000000000001 x21: ffffffbffe801000^M
[    1.429460] x20: ffffffc000040000 x19: ffff0018f9010020 x18: ffffffffffffffff^M
[    1.430034] x17: 75625b20726f6620 x16: 5d66666666663230 x15: 0000000000000002^M
[    1.430599] x14: 0000000000000000 x13: ffff0000c96ccf36 x12: 75625b20726f6620^M
[    1.431189] x11: 000000000000002e x10: 000003bffe800000 x9 : ffff800080341544^M
[    1.431802] x8 : ffff0000c96b8b80 x7 : ffff0018f9000000 x6 : ffff800083fcf7d4^M
[    1.432376] x5 : 000000000000003f x4 : 0068000000000713 x3 : 0000000000000000^M
[    1.432936] x2 : fffffffffe7d1000 x1 : ffff0031fff81380 x0 : 0000000000000008^M
[    1.433495] Call trace:^M
[    1.433689]  vmap_pte_range+0x158/0x2b8^M
[    1.433996]  vmap_range_noflush+0x19c/0x260^M
[    1.434323]  ioremap_page_range+0x20/0x30^M
[    1.434636]  pci_remap_iospace+0x78/0x90^M
[    1.434960]  acpi_pci_probe_root_resources+0x124/0x260^M
[    1.435378]  pci_acpi_root_prepare_resources+0x20/0xd0^M
[    1.435814]  acpi_pci_root_create+0x94/0x2e8^M
[    1.436154]  pci_acpi_scan_root+0x98/0x130^M
[    1.436474]  acpi_pci_root_add+0x15c/0x2f8^M
[    1.436795]  acpi_bus_attach+0x1bc/0x338^M
[    1.437104]  acpi_dev_for_one_check+0x38/0x48^M
[    1.437444]  device_for_each_child+0x60/0xb8^M
[    1.437799]  acpi_dev_for_each_child+0x40/0x70^M
[    1.438148]  acpi_bus_attach+0x234/0x338^M
[    1.438454]  acpi_dev_for_one_check+0x38/0x48^M
[    1.438796]  device_for_each_child+0x60/0xb8^M
[    1.439148]  acpi_dev_for_each_child+0x40/0x70^M
[    1.439531]  acpi_bus_attach+0x234/0x338^M
[    1.439854]  acpi_bus_scan+0x68/0x208^M
[    1.440142]  acpi_scan_init+0xd0/0x2b8^M
[    1.440445]  acpi_init+0x80/0x140^M
[    1.440704]  do_one_initcall+0x48/0x348^M
[    1.441010]  do_initcalls+0x104/0x200^M
[    1.441305]  kernel_init_freeable+0x128/0x2a0^M
[    1.441645]  kernel_init+0x28/0x1e8^M
[    1.441943]  ret_from_fork+0x10/0x20^M
[    1.442231] Code: 97f3ddf6 aa0003e1 b4fffa41 d503201f (d4210000) ^M

Could you share more details for testing?, e.g. QEMU cmdline,
Guest kernel version.

I'd like to compare with my setup to identify the difference.

Thanks.
Shuai

Re: [PATCH 0/9] Initial DMABUF support for iommufd

[Nicolin Chen]

On Thu, Nov 13, 2025 at 02:33:01PM +0800, Shuai Xue wrote:
> Could you share more details for testing?, e.g. QEMU cmdline,
> Guest kernel version.

My v6.8 and v6.14 kernels can boot cleanly in the VM.

vm_image=PATH-to-FILE
e1000_rom=PATH-to-FILE
qemu-system-aarch64 \
	-object iommufd,id=iommufd0 \
	-machine hmat=on -machine virt,accel=kvm,gic-version=3,ras=on,highmem-mmio-size=4T \
	-cpu host -smp cpus=62 -m size=16G,slots=2,maxmem=256G -nographic \
	-object memory-backend-ram,size=8G,id=m0 -object memory-backend-ram,size=8G,id=m1 \
	-numa node,memdev=m0,cpus=0-61,nodeid=0 -numa node,memdev=m1,nodeid=1 \
	-numa node,nodeid=2 -numa node,nodeid=3 -numa node,nodeid=4 -numa node,nodeid=5 \
	-numa node,nodeid=6 -numa node,nodeid=7 -numa node,nodeid=8 -numa node,nodeid=9 \
	-device pxb-pcie,id=pcie.1,bus_nr=1,bus=pcie.0 -device arm-smmuv3,primary-bus=pcie.1,id=smmuv3.1,accel=on,ats=on,ril=off,pasid=on,oas=48 \
	-device pcie-root-port,id=pcie.port1,bus=pcie.1,chassis=1,io-reserve=0 \
	-device vfio-pci-nohotplug,host=0009:01:00.0,bus=pcie.port1,rombar=0,id=dev0,iommufd=iommufd0 \
	-object acpi-generic-initiator,id=gi0,pci-dev=dev0,node=2 \
	-object acpi-generic-initiator,id=gi1,pci-dev=dev0,node=3 \
	-object acpi-generic-initiator,id=gi2,pci-dev=dev0,node=4 \
	-object acpi-generic-initiator,id=gi3,pci-dev=dev0,node=5 \
	-object acpi-generic-initiator,id=gi4,pci-dev=dev0,node=6 \
	-object acpi-generic-initiator,id=gi5,pci-dev=dev0,node=7 \
	-object acpi-generic-initiator,id=gi6,pci-dev=dev0,node=8 \
	-object acpi-generic-initiator,id=gi7,pci-dev=dev0,node=9 \
	-bios /usr/share/AAVMF/AAVMF_CODE.fd \
	-device nvme,drive=nvme0,serial=deadbeaf1,bus=pcie.0 \
	-drive file=${vm_image},index=0,media=disk,format=qcow2,if=none,id=nvme0 \
	-device e1000,romfile=${e1000_rom},netdev=net0,bus=pcie.0 \
	-netdev user,id=net0,hostfwd=tcp::5558-:22,hostfwd=tcp::5586-:5586

Nicolin

Re: [PATCH 0/9] Initial DMABUF support for iommufd

[Shuai Xue]

Hi, Nicolin,

在 2025/11/13 15:34, Nicolin Chen 写道:
> On Thu, Nov 13, 2025 at 02:33:01PM +0800, Shuai Xue wrote:
>> Could you share more details for testing?, e.g. QEMU cmdline,
>> Guest kernel version.
> 
> My v6.8 and v6.14 kernels can boot cleanly in the VM.
> 
> vm_image=PATH-to-FILE
> e1000_rom=PATH-to-FILE
> qemu-system-aarch64 \
> 	-object iommufd,id=iommufd0 \
> 	-machine hmat=on -machine virt,accel=kvm,gic-version=3,ras=on,highmem-mmio-size=4T \
> 	-cpu host -smp cpus=62 -m size=16G,slots=2,maxmem=256G -nographic \
> 	-object memory-backend-ram,size=8G,id=m0 -object memory-backend-ram,size=8G,id=m1 \
> 	-numa node,memdev=m0,cpus=0-61,nodeid=0 -numa node,memdev=m1,nodeid=1 \
> 	-numa node,nodeid=2 -numa node,nodeid=3 -numa node,nodeid=4 -numa node,nodeid=5 \
> 	-numa node,nodeid=6 -numa node,nodeid=7 -numa node,nodeid=8 -numa node,nodeid=9 \
> 	-device pxb-pcie,id=pcie.1,bus_nr=1,bus=pcie.0 -device arm-smmuv3,primary-bus=pcie.1,id=smmuv3.1,accel=on,ats=on,ril=off,pasid=on,oas=48 \
> 	-device pcie-root-port,id=pcie.port1,bus=pcie.1,chassis=1,io-reserve=0 \
> 	-device vfio-pci-nohotplug,host=0009:01:00.0,bus=pcie.port1,rombar=0,id=dev0,iommufd=iommufd0 \
> 	-object acpi-generic-initiator,id=gi0,pci-dev=dev0,node=2 \
> 	-object acpi-generic-initiator,id=gi1,pci-dev=dev0,node=3 \
> 	-object acpi-generic-initiator,id=gi2,pci-dev=dev0,node=4 \
> 	-object acpi-generic-initiator,id=gi3,pci-dev=dev0,node=5 \
> 	-object acpi-generic-initiator,id=gi4,pci-dev=dev0,node=6 \
> 	-object acpi-generic-initiator,id=gi5,pci-dev=dev0,node=7 \
> 	-object acpi-generic-initiator,id=gi6,pci-dev=dev0,node=8 \
> 	-object acpi-generic-initiator,id=gi7,pci-dev=dev0,node=9 \
> 	-bios /usr/share/AAVMF/AAVMF_CODE.fd \
> 	-device nvme,drive=nvme0,serial=deadbeaf1,bus=pcie.0 \
> 	-drive file=${vm_image},index=0,media=disk,format=qcow2,if=none,id=nvme0 \
> 	-device e1000,romfile=${e1000_rom},netdev=net0,bus=pcie.0 \
> 	-netdev user,id=net0,hostfwd=tcp::5558-:22,hostfwd=tcp::5586-:5586
> 

Thanks for sharing the command line. The issue was with my QEMU command
line configuration.

Using your command line, I can now boot my v6.6 kernel cleanly in the
VM, and the PCIe passthrough device works correctly.

Tested-by: Shuai Xue <xueshuai@linux.alibaba.com>

Thanks.
Shuai

Re: [PATCH 0/9] Initial DMABUF support for iommufd

[Nicolin Chen]

On Thu, Nov 13, 2025 at 07:32:46PM +0800, Shuai Xue wrote:
> Thanks for sharing the command line. The issue was with my QEMU command
> line configuration.
> 
> Using your command line, I can now boot my v6.6 kernel cleanly in the
> VM, and the PCIe passthrough device works correctly.
> 
> Tested-by: Shuai Xue <xueshuai@linux.alibaba.com>

Glad to hear that. Thanks for testing!

Nicolin

Re: [PATCH 0/9] Initial DMABUF support for iommufd

[Alex Williamson]

On Fri,  7 Nov 2025 12:49:32 -0400
Jason Gunthorpe <jgg@nvidia.com> wrote:

> This series is the start of adding full DMABUF support to
> iommufd. Currently it is limited to only work with VFIO's DMABUF exporter.
> It sits on top of Leon's series to add a DMABUF exporter to VFIO:
> 
>   https://lore.kernel.org/r/20251106-dmabuf-vfio-v7-0-2503bf390699@nvidia.com
> 
> The existing IOMMU_IOAS_MAP_FILE is enhanced to detect DMABUF fd's, but
> otherwise works the same as it does today for a memfd. The user can select
> a slice of the FD to map into the ioas and if the underliyng alignment
> requirements are met it will be placed in the iommu_domain.
> 
> Though limited, it is enough to allow a VMM like QEMU to connect MMIO BAR
> memory from VFIO to an iommu_domain controlled by iommufd. This is used
> for PCI Peer to Peer support in VMs, and is the last feature that the VFIO
> type 1 container has that iommufd couldn't do.
> 
> The VFIO type1 version extracts raw PFNs from VMAs, which has no lifetime
> control and is a use-after-free security problem.
> 
> Instead iommufd relies on revokable DMABUFs. Whenever VFIO thinks there
> should be no access to the MMIO it can shoot down the mapping in iommufd
> which will unmap it from the iommu_domain. There is no automatic remap,
> this is a safety protocol so the kernel doesn't get stuck. Userspace is
> expected to know it is doing something that will revoke the dmabuf and
> map/unmap it around the activity. Eg when QEMU goes to issue FLR it should
> do the map/unmap to iommufd.
> 
> Since DMABUF is missing some key general features for this use case it
> relies on a "private interconnect" between VFIO and iommufd via the
> vfio_pci_dma_buf_iommufd_map() call.
> 
> The call confirms the DMABUF has revoke semantics and delivers a phys_addr
> for the memory suitable for use with iommu_map().
> 
> Medium term there is a desire to expand the supported DMABUFs to include
> GPU drivers to support DPDK/SPDK type use cases so future series will work
> to add a general concept of revoke and a general negotiation of
> interconnect to remove vfio_pci_dma_buf_iommufd_map().
> 
> I also plan another series to modify iommufd's vfio_compat to
> transparently pull a dmabuf out of a VFIO VMA to emulate more of the uAPI
> of type1.
> 
> The latest series for interconnect negotation to exchange a phys_addr is:
>  https://lore.kernel.org/r/20251027044712.1676175-1-vivek.kasireddy@intel.com

If this is in development, why are we pursuing a vfio specific
temporary "private interconnect" here rather than building on that
work?  What are the gaps/barriers/timeline?

I don't see any uAPI changes here, is there any visibility to userspace
whether IOMMUFD supports this feature or is it simply a try and fail
approach?  The latter makes it difficult for management tools to select
whether to choose a VM configuration based on IOMMUFD or legacy vfio if
p2p DMA is a requirement.  Thanks,

Alex

Re: [PATCH 0/9] Initial DMABUF support for iommufd

[Jason Gunthorpe]

On Thu, Nov 13, 2025 at 11:37:12AM -0700, Alex Williamson wrote:
> > The latest series for interconnect negotation to exchange a phys_addr is:
> >  https://lore.kernel.org/r/20251027044712.1676175-1-vivek.kasireddy@intel.com
> 
> If this is in development, why are we pursuing a vfio specific
> temporary "private interconnect" here rather than building on that
> work?  What are the gaps/barriers/timeline?

I broadly don't expect to see an agreement on the above for probably
half a year, and I see no reason to hold this up for it. Many people
are asking for this P2P support to be completed in iommufd.

Further, I think the above will be easier to work on when we have this
merged as an example that can consume it in a different way. Right now
it is too theoretical, IMHO.

> I don't see any uAPI changes here, is there any visibility to userspace
> whether IOMMUFD supports this feature or is it simply a try and fail
> approach?  

So far we haven't done discoverably things beyond try and fail.

I'd be happy if the userspace folks doing libvirt or whatever came
with some requests/patches for discoverability. It is not just this
feature, but things like nesting and IOMMU driver support and so on.

> The latter makes it difficult for management tools to select
> whether to choose a VM configuration based on IOMMUFD or legacy vfio if
> p2p DMA is a requirement.  Thanks,

In alot of cases it isn't really a choice as you need iommufd to do an
accelerated vIOMMU.

But yes, it would be nice to eventually automatically use iommufd
whenever possible.

Thanks,
Jason

RE: [PATCH 0/9] Initial DMABUF support for iommufd

[Kasireddy, Vivek]

Hi Jason,

> Subject: Re: [PATCH 0/9] Initial DMABUF support for iommufd
> 
> On Thu, Nov 13, 2025 at 11:37:12AM -0700, Alex Williamson wrote:
> > > The latest series for interconnect negotation to exchange a phys_addr is:
> > >  https://lore.kernel.org/r/20251027044712.1676175-1-
> vivek.kasireddy@intel.com
> >
> > If this is in development, why are we pursuing a vfio specific
> > temporary "private interconnect" here rather than building on that
> > work?  What are the gaps/barriers/timeline?
> 
> I broadly don't expect to see an agreement on the above for probably
Are you planning to post your SGT mapping type patches soon, so that we
can start discussion on the design?

I went ahead and tested your patches and did not notice any regressions
with my test-cases (after adding some minor fixups). I have also added/tested
support for IOV mapping type based on your design:
https://gitlab.freedesktop.org/Vivek/drm-tip/-/commits/dmabuf_iov_v1

Thanks,
Vivek

> half a year, and I see no reason to hold this up for it. Many people
> are asking for this P2P support to be completed in iommufd.
> 
> Further, I think the above will be easier to work on when we have this
> merged as an example that can consume it in a different way. Right now
> it is too theoretical, IMHO.
> 
> > I don't see any uAPI changes here, is there any visibility to userspace
> > whether IOMMUFD supports this feature or is it simply a try and fail
> > approach?
> 
> So far we haven't done discoverably things beyond try and fail.
> 
> I'd be happy if the userspace folks doing libvirt or whatever came
> with some requests/patches for discoverability. It is not just this
> feature, but things like nesting and IOMMU driver support and so on.
> 
> > The latter makes it difficult for management tools to select
> > whether to choose a VM configuration based on IOMMUFD or legacy vfio if
> > p2p DMA is a requirement.  Thanks,
> 
> In alot of cases it isn't really a choice as you need iommufd to do an
> accelerated vIOMMU.
> 
> But yes, it would be nice to eventually automatically use iommufd
> whenever possible.
> 
> Thanks,
> Jason

Re: [PATCH 0/9] Initial DMABUF support for iommufd

[Jason Gunthorpe]

On Tue, Nov 18, 2025 at 05:37:59AM +0000, Kasireddy, Vivek wrote:
> Hi Jason,
> 
> > Subject: Re: [PATCH 0/9] Initial DMABUF support for iommufd
> > 
> > On Thu, Nov 13, 2025 at 11:37:12AM -0700, Alex Williamson wrote:
> > > > The latest series for interconnect negotation to exchange a phys_addr is:
> > > >  https://lore.kernel.org/r/20251027044712.1676175-1-
> > vivek.kasireddy@intel.com
> > >
> > > If this is in development, why are we pursuing a vfio specific
> > > temporary "private interconnect" here rather than building on that
> > > work?  What are the gaps/barriers/timeline?
> >
> > I broadly don't expect to see an agreement on the above for probably
> Are you planning to post your SGT mapping type patches soon, so that we
> can start discussion on the design?

It is on my list, but probably not soon enough :\

I wanted to address the remarks given and I still have to conclude
some urgent things for this merge window.

> I went ahead and tested your patches and did not notice any regressions
> with my test-cases (after adding some minor fixups). I have also added/tested
> support for IOV mapping type based on your design:
> https://gitlab.freedesktop.org/Vivek/drm-tip/-/commits/dmabuf_iov_v1

Wow, that's great!

Jason