[PATCH v2 0/3] drm/amdgpu: Handle MMIO_REMAP as fixed I/O via dma-buf

[PATCH v2 0/3] drm/amdgpu: Handle MMIO_REMAP as fixed I/O via dma-buf

[Srinivasan Shanmugam]

This series makes the amdgpu dma-buf exporter handle
AMDGPU_PL_MMIO_REMAP (the HDP flush page) as a BAR-mapped register
window (MMIO)

The HDP flush “MMIO_REMAP” page is a BAR-backed I/O window (not RAM).
When another PCIe device (GPU)  needs to poke that window (e.g.,
device-to-device HDP flush), we must share it as a stable I/O span that
never migrates, and we must map it for importers using the correct API
for I/O resources (not page-backed memory).

This series makes MMIO_REMAP sharable through dma-buf by:
representing it as a singleton, pinned 4 KB BO for the device lifetime,
and mapping/unmapping it for importers via
dma_map_resource()/dma_unmap_resource() using a 1-entry sg_table

v2:
Design-level adjustments (Christian’s feedback):
 - Pin at creation, not at export. We now pin the MMIO_REMAP BO in TTM
   during init using ttm_bo_pin(). This enforces “never migrate/evict”
   from birth and removes the need for export-time pinning and release-time
   unpinning.
 - No begin_cpu_access hacks. Since the object is pinned and treated as
   fixed I/O, we removed the special-casing in begin_cpu_access();
   migration heuristics are irrelevant.
 - No pin/unpin no-ops in dma-buf paths. Those were a workaround for
   export-time pinning. With creation-time pinning, they’re unnecessary
   and are dropped.

* Only compilation tested so far (x86_64, defconfig + amdgpu enabled).

Suggested-by: Christian König <christian.koenig@amd.com>
Suggested-by: Alex Deucher <alexander.deucher@amd.com>
Cc: Christian König <christian.koenig@amd.com>
Cc: Alex Deucher <alexander.deucher@amd.com>

Srinivasan Shanmugam (3):
  drm/amdgpu/ttm: Pin 4K MMIO_REMAP Singleton BO at Init
  drm/amdgpu/dma-buf: Add helpers to map/unmap BAR I/O with
    dma_map_resource()
  drm/amdgpu/dma-buf: Map/Unmap MMIO_REMAP as BAR register window
    (dma_map_resource)

 drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c | 86 +++++++++++++++++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c     | 26 +++++++
 2 files changed, 112 insertions(+)


base-commit: bc52ed7845660e03827131505f95fcb4dd2fcf4f
-- 
2.34.1

[PATCH v2 1/3] drm/amdgpu/ttm: Pin 4K MMIO_REMAP Singleton BO at Init

[Srinivasan Shanmugam]

MMIO_REMAP (HDP flush page) is a hardware I/O window exposed via a PCI
BAR.  It must not migrate or be evicted.

Allocate a single 4 KB GEM BO in AMDGPU_GEM_DOMAIN_MMIO_REMAP during TTM
initialization when the hardware exposes a remap bus address and the
host page size is <= 4 KiB. Reserve the BO and pin it at the TTM level
so it remains fixed for its lifetime. No CPU mapping is established
here.

On teardown, reserve, unpin, and free the BO if present.

This prepares the object to be shared (e.g., via dma-buf) without
triggering placement changes or no CPU-access migration

Suggested-by: Christian König <christian.koenig@amd.com>
Suggested-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 26 +++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
index f38bc9542cd6..5ce7c8b9ff39 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
@@ -1862,6 +1862,10 @@ static void amdgpu_ttm_pools_fini(struct amdgpu_device *adev)
  * hardware exposes a remap base (adev->rmmio_remap.bus_addr) and the host
  * PAGE_SIZE is <= AMDGPU_GPU_PAGE_SIZE (4K). The BO is created as a regular
  * GEM object (amdgpu_bo_create).
+ * 
+ * The BO is created as a normal GEM object via amdgpu_bo_create(), then
+ * reserved and pinned at the TTM level (ttm_bo_pin()) so it can never be
+ * migrated or evicted. No CPU mapping is established here.
  *
  * Return:
  *  * 0 on success or intentional skip (feature not present/unsupported)
@@ -1891,7 +1895,25 @@ static int amdgpu_ttm_mmio_remap_bo_init(struct amdgpu_device *adev)
 	if (r)
 		return r;
 
+	r = amdgpu_bo_reserve(adev->rmmio_remap.bo, true);
+	if (r)
+		goto err_unref;
+
+	/*
+	 * MMIO_REMAP is a fixed I/O placement (AMDGPU_PL_MMIO_REMAP).
+	 * Use TTM-level pin so the BO cannot be evicted/migrated,
+	 * independent of GEM domains. This
+	 * enforces the “fixed I/O window”
+	 */
+	ttm_bo_pin(&adev->rmmio_remap.bo->tbo);
+
+	amdgpu_bo_unreserve(adev->rmmio_remap.bo);
 	return 0;
+
+err_unref:
+	amdgpu_bo_unref(&adev->rmmio_remap.bo);
+	adev->rmmio_remap.bo = NULL;
+	return r;
 }
 
 /**
@@ -1903,6 +1925,10 @@ static int amdgpu_ttm_mmio_remap_bo_init(struct amdgpu_device *adev)
  */
 static void amdgpu_ttm_mmio_remap_bo_fini(struct amdgpu_device *adev)
 {
+	if (!amdgpu_bo_reserve(adev->rmmio_remap.bo, true)) {
+		ttm_bo_unpin(&adev->rmmio_remap.bo->tbo);
+		amdgpu_bo_unreserve(adev->rmmio_remap.bo);
+	}
 	amdgpu_bo_unref(&adev->rmmio_remap.bo);
 	adev->rmmio_remap.bo = NULL;
 }
-- 
2.34.1

Re: [PATCH v2 1/3] drm/amdgpu/ttm: Pin 4K MMIO_REMAP Singleton BO at Init

[Christian König]

On 12.09.25 13:11, Srinivasan Shanmugam wrote:
> MMIO_REMAP (HDP flush page) is a hardware I/O window exposed via a PCI
> BAR.  It must not migrate or be evicted.
> 
> Allocate a single 4 KB GEM BO in AMDGPU_GEM_DOMAIN_MMIO_REMAP during TTM
> initialization when the hardware exposes a remap bus address and the
> host page size is <= 4 KiB. Reserve the BO and pin it at the TTM level
> so it remains fixed for its lifetime. No CPU mapping is established
> here.
> 
> On teardown, reserve, unpin, and free the BO if present.
> 
> This prepares the object to be shared (e.g., via dma-buf) without
> triggering placement changes or no CPU-access migration
> 
> Suggested-by: Christian König <christian.koenig@amd.com>
> Suggested-by: Alex Deucher <alexander.deucher@amd.com>
> Signed-off-by: Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>

Reviewed-by: Christian König <christian.koenig@amd.com>

> ---
>  drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 26 +++++++++++++++++++++++++
>  1 file changed, 26 insertions(+)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> index f38bc9542cd6..5ce7c8b9ff39 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> @@ -1862,6 +1862,10 @@ static void amdgpu_ttm_pools_fini(struct amdgpu_device *adev)
>   * hardware exposes a remap base (adev->rmmio_remap.bus_addr) and the host
>   * PAGE_SIZE is <= AMDGPU_GPU_PAGE_SIZE (4K). The BO is created as a regular
>   * GEM object (amdgpu_bo_create).
> + * 
> + * The BO is created as a normal GEM object via amdgpu_bo_create(), then
> + * reserved and pinned at the TTM level (ttm_bo_pin()) so it can never be
> + * migrated or evicted. No CPU mapping is established here.
>   *
>   * Return:
>   *  * 0 on success or intentional skip (feature not present/unsupported)
> @@ -1891,7 +1895,25 @@ static int amdgpu_ttm_mmio_remap_bo_init(struct amdgpu_device *adev)
>  	if (r)
>  		return r;
>  
> +	r = amdgpu_bo_reserve(adev->rmmio_remap.bo, true);
> +	if (r)
> +		goto err_unref;
> +
> +	/*
> +	 * MMIO_REMAP is a fixed I/O placement (AMDGPU_PL_MMIO_REMAP).
> +	 * Use TTM-level pin so the BO cannot be evicted/migrated,
> +	 * independent of GEM domains. This
> +	 * enforces the “fixed I/O window”
> +	 */
> +	ttm_bo_pin(&adev->rmmio_remap.bo->tbo);
> +
> +	amdgpu_bo_unreserve(adev->rmmio_remap.bo);
>  	return 0;
> +
> +err_unref:
> +	amdgpu_bo_unref(&adev->rmmio_remap.bo);
> +	adev->rmmio_remap.bo = NULL;
> +	return r;
>  }
>  
>  /**
> @@ -1903,6 +1925,10 @@ static int amdgpu_ttm_mmio_remap_bo_init(struct amdgpu_device *adev)
>   */
>  static void amdgpu_ttm_mmio_remap_bo_fini(struct amdgpu_device *adev)
>  {
> +	if (!amdgpu_bo_reserve(adev->rmmio_remap.bo, true)) {
> +		ttm_bo_unpin(&adev->rmmio_remap.bo->tbo);
> +		amdgpu_bo_unreserve(adev->rmmio_remap.bo);
> +	}
>  	amdgpu_bo_unref(&adev->rmmio_remap.bo);
>  	adev->rmmio_remap.bo = NULL;
>  }

[PATCH v2 2/3] drm/amdgpu/dma-buf: Add helpers to map/unmap BAR I/O with dma_map_resource()

[Srinivasan Shanmugam]

Add helpers to map/unmap a hardware MMIO register window (PCI BAR) as a
1-entry sg_table using dma_map_resource()/dma_unmap_resource(). This
will be used by MMIO_REMAP.

Cc: Christian König <christian.koenig@amd.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c | 51 +++++++++++++++++++++
 1 file changed, 51 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
index ff98c87b2e0b..33fa17a927ce 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
@@ -37,11 +37,15 @@
 #include "amdgpu_dma_buf.h"
 #include "amdgpu_xgmi.h"
 #include "amdgpu_vm.h"
+#include "amdgpu_object.h"
 #include <drm/amdgpu_drm.h>
 #include <drm/ttm/ttm_tt.h>
 #include <linux/dma-buf.h>
 #include <linux/dma-fence-array.h>
 #include <linux/pci-p2pdma.h>
+#include <linux/dma-mapping.h>
+#include <linux/scatterlist.h>
+#include <linux/slab.h>
 
 static const struct dma_buf_attach_ops amdgpu_dma_buf_attach_ops;
 
@@ -146,6 +150,53 @@ static void amdgpu_dma_buf_unpin(struct dma_buf_attachment *attach)
 	amdgpu_bo_unpin(bo);
 }
 
+/* Map a BAR-backed I/O span as a 1-entry sg_table via dma_map_resource(). */
+static __maybe_unused struct sg_table *
+amdgpu_dmabuf_map_iomem(struct device *dev, resource_size_t phys,
+			size_t size, enum dma_data_direction dir)
+{
+	struct sg_table *sgt;
+	unsigned long attrs = DMA_ATTR_SKIP_CPU_SYNC; /* no P2PDMA attr */
+	dma_addr_t dma;
+
+	sgt = kzalloc(sizeof(*sgt), GFP_KERNEL);
+	if (!sgt)
+		return ERR_PTR(-ENOMEM);
+
+	if (sg_alloc_table(sgt, 1, GFP_KERNEL)) {
+		kfree(sgt);
+		return ERR_PTR(-ENOMEM);
+	}
+
+	/* No struct page backing for I/O regions. */
+	sg_set_page(sgt->sgl, NULL, size, 0);
+
+	dma = dma_map_resource(dev, phys, size, dir, attrs);
+	if (dma_mapping_error(dev, dma)) {
+		sg_free_table(sgt);
+		kfree(sgt);
+		return ERR_PTR(-EIO);
+	}
+
+	sg_dma_address(sgt->sgl) = dma;
+	sg_dma_len(sgt->sgl) = size;
+	return sgt;
+}
+
+static __maybe_unused void
+amdgpu_dmabuf_unmap_iomem(struct device *dev, struct sg_table *sgt,
+			  enum dma_data_direction dir)
+{
+	/* attrs must match map side; we only used SKIP_CPU_SYNC above */
+	dma_unmap_resource(dev,
+			   sg_dma_address(sgt->sgl),
+			   sg_dma_len(sgt->sgl),
+			   dir,
+			   0);
+	sg_free_table(sgt);
+	kfree(sgt);
+}
+
 /**
  * amdgpu_dma_buf_map - &dma_buf_ops.map_dma_buf implementation
  * @attach: DMA-buf attachment
-- 
2.34.1

Re: [PATCH v2 2/3] drm/amdgpu/dma-buf: Add helpers to map/unmap BAR I/O with dma_map_resource()

[Christian König]

Please squash that together with patch #3 in this series.

On 12.09.25 13:11, Srinivasan Shanmugam wrote:
> Add helpers to map/unmap a hardware MMIO register window (PCI BAR) as a
> 1-entry sg_table using dma_map_resource()/dma_unmap_resource(). This
> will be used by MMIO_REMAP.
> 
> Cc: Christian König <christian.koenig@amd.com>
> Cc: Alex Deucher <alexander.deucher@amd.com>
> Signed-off-by: Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>
> ---
>  drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c | 51 +++++++++++++++++++++
>  1 file changed, 51 insertions(+)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
> index ff98c87b2e0b..33fa17a927ce 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
> @@ -37,11 +37,15 @@
>  #include "amdgpu_dma_buf.h"
>  #include "amdgpu_xgmi.h"
>  #include "amdgpu_vm.h"
> +#include "amdgpu_object.h"
>  #include <drm/amdgpu_drm.h>
>  #include <drm/ttm/ttm_tt.h>
>  #include <linux/dma-buf.h>
>  #include <linux/dma-fence-array.h>
>  #include <linux/pci-p2pdma.h>
> +#include <linux/dma-mapping.h>
> +#include <linux/scatterlist.h>
> +#include <linux/slab.h>
>  
>  static const struct dma_buf_attach_ops amdgpu_dma_buf_attach_ops;
>  
> @@ -146,6 +150,53 @@ static void amdgpu_dma_buf_unpin(struct dma_buf_attachment *attach)
>  	amdgpu_bo_unpin(bo);
>  }
>  
> +/* Map a BAR-backed I/O span as a 1-entry sg_table via dma_map_resource(). */
> +static __maybe_unused struct sg_table *
> +amdgpu_dmabuf_map_iomem(struct device *dev, resource_size_t phys,
> +			size_t size, enum dma_data_direction dir)
> +{
> +	struct sg_table *sgt;
> +	unsigned long attrs = DMA_ATTR_SKIP_CPU_SYNC; /* no P2PDMA attr */
> +	dma_addr_t dma;
> +
> +	sgt = kzalloc(sizeof(*sgt), GFP_KERNEL);
> +	if (!sgt)
> +		return ERR_PTR(-ENOMEM);
> +
> +	if (sg_alloc_table(sgt, 1, GFP_KERNEL)) {
> +		kfree(sgt);
> +		return ERR_PTR(-ENOMEM);
> +	}
> +
> +	/* No struct page backing for I/O regions. */
> +	sg_set_page(sgt->sgl, NULL, size, 0);
> +
> +	dma = dma_map_resource(dev, phys, size, dir, attrs);
> +	if (dma_mapping_error(dev, dma)) {
> +		sg_free_table(sgt);
> +		kfree(sgt);
> +		return ERR_PTR(-EIO);
> +	}
> +
> +	sg_dma_address(sgt->sgl) = dma;
> +	sg_dma_len(sgt->sgl) = size;
> +	return sgt;
> +}
> +
> +static __maybe_unused void
> +amdgpu_dmabuf_unmap_iomem(struct device *dev, struct sg_table *sgt,
> +			  enum dma_data_direction dir)
> +{
> +	/* attrs must match map side; we only used SKIP_CPU_SYNC above */
> +	dma_unmap_resource(dev,
> +			   sg_dma_address(sgt->sgl),
> +			   sg_dma_len(sgt->sgl),
> +			   dir,
> +			   0);
> +	sg_free_table(sgt);
> +	kfree(sgt);
> +}
> +
>  /**
>   * amdgpu_dma_buf_map - &dma_buf_ops.map_dma_buf implementation
>   * @attach: DMA-buf attachment

[PATCH v2 3/3] drm/amdgpu/dma-buf: Map/Unmap MMIO_REMAP as BAR register window (dma_map_resource)

[Srinivasan Shanmugam]

MMIO_REMAP (HDP flush page) exposes a hardware MMIO register window via a PCI BAR.

Handle it as fixed I/O:
- map(): if MMIO_REMAP, require P2P, compute the BAR address (bus_addr + page
  offset), and build a 1-entry sg_table with dma_map_resource().
- unmap(): if MMIO_REMAP, call dma_unmap_resource() and return.

Cc: Christian König <christian.koenig@amd.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c | 39 +++++++++++++++++++--
 1 file changed, 37 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
index 33fa17a927ce..f85e16be438f 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
@@ -151,7 +151,7 @@ static void amdgpu_dma_buf_unpin(struct dma_buf_attachment *attach)
 }
 
 /* Map a BAR-backed I/O span as a 1-entry sg_table via dma_map_resource(). */
-static __maybe_unused struct sg_table *
+static struct sg_table *
 amdgpu_dmabuf_map_iomem(struct device *dev, resource_size_t phys,
 			size_t size, enum dma_data_direction dir)
 {
@@ -183,7 +183,7 @@ amdgpu_dmabuf_map_iomem(struct device *dev, resource_size_t phys,
 	return sgt;
 }
 
-static __maybe_unused void
+static void
 amdgpu_dmabuf_unmap_iomem(struct device *dev, struct sg_table *sgt,
 			  enum dma_data_direction dir)
 {
@@ -218,6 +218,9 @@ static struct sg_table *amdgpu_dma_buf_map(struct dma_buf_attachment *attach,
 	struct amdgpu_bo *bo = gem_to_amdgpu_bo(obj);
 	struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
 	struct sg_table *sgt;
+	resource_size_t phys;
+	u64 off;
+	size_t len;
 	long r;
 
 	if (!bo->tbo.pin_count) {
@@ -261,6 +264,29 @@ static struct sg_table *amdgpu_dma_buf_map(struct dma_buf_attachment *attach,
 		if (r)
 			return ERR_PTR(r);
 		break;
+
+	case AMDGPU_PL_MMIO_REMAP:
+		/* Only allow when importer can reach exporter via P2P. */
+		if (!attach->peer2peer ||
+		    pci_p2pdma_distance(adev->pdev, attach->dev, false) < 0)
+			return ERR_PTR(-EOPNOTSUPP);
+
+		if (!adev->rmmio_remap.bus_addr)
+			return ERR_PTR(-ENODEV);
+
+		if (bo->tbo.base.size != AMDGPU_GPU_PAGE_SIZE)
+			return ERR_PTR(-EINVAL);
+
+		/* TTM start is in pages → convert to byte offset. */
+		off  = (u64)bo->tbo.resource->start << PAGE_SHIFT;
+		len  = AMDGPU_GPU_PAGE_SIZE;
+		phys = adev->rmmio_remap.bus_addr + off;
+
+		sgt = amdgpu_dmabuf_map_iomem(attach->dev, phys, len, dir);
+		if (IS_ERR(sgt))
+			return sgt;
+		break;
+
 	default:
 		return ERR_PTR(-EINVAL);
 	}
@@ -286,6 +312,15 @@ static void amdgpu_dma_buf_unmap(struct dma_buf_attachment *attach,
 				 struct sg_table *sgt,
 				 enum dma_data_direction dir)
 {
+	struct drm_gem_object *obj = attach->dmabuf->priv;
+	struct amdgpu_bo *bo = gem_to_amdgpu_bo(obj);
+
+	if (bo->tbo.resource &&
+	    bo->tbo.resource->mem_type == AMDGPU_PL_MMIO_REMAP) {
+		amdgpu_dmabuf_unmap_iomem(attach->dev, sgt, dir);
+		return;
+	}
+
 	if (sg_page(sgt->sgl)) {
 		dma_unmap_sgtable(attach->dev, sgt, dir, 0);
 		sg_free_table(sgt);
-- 
2.34.1

Re: [PATCH v2 3/3] drm/amdgpu/dma-buf: Map/Unmap MMIO_REMAP as BAR register window (dma_map_resource)

[Christian König]

On 12.09.25 13:11, Srinivasan Shanmugam wrote:
> MMIO_REMAP (HDP flush page) exposes a hardware MMIO register window via a PCI BAR.
> 
> Handle it as fixed I/O:
> - map(): if MMIO_REMAP, require P2P, compute the BAR address (bus_addr + page
>   offset), and build a 1-entry sg_table with dma_map_resource().
> - unmap(): if MMIO_REMAP, call dma_unmap_resource() and return.
> 
> Cc: Christian König <christian.koenig@amd.com>
> Cc: Alex Deucher <alexander.deucher@amd.com>
> Signed-off-by: Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>
> ---
>  drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c | 39 +++++++++++++++++++--
>  1 file changed, 37 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
> index 33fa17a927ce..f85e16be438f 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
> @@ -151,7 +151,7 @@ static void amdgpu_dma_buf_unpin(struct dma_buf_attachment *attach)
>  }
>  
>  /* Map a BAR-backed I/O span as a 1-entry sg_table via dma_map_resource(). */
> -static __maybe_unused struct sg_table *
> +static struct sg_table *
>  amdgpu_dmabuf_map_iomem(struct device *dev, resource_size_t phys,
>  			size_t size, enum dma_data_direction dir)
>  {
> @@ -183,7 +183,7 @@ amdgpu_dmabuf_map_iomem(struct device *dev, resource_size_t phys,
>  	return sgt;
>  }
>  
> -static __maybe_unused void
> +static void
>  amdgpu_dmabuf_unmap_iomem(struct device *dev, struct sg_table *sgt,
>  			  enum dma_data_direction dir)
>  {
> @@ -218,6 +218,9 @@ static struct sg_table *amdgpu_dma_buf_map(struct dma_buf_attachment *attach,
>  	struct amdgpu_bo *bo = gem_to_amdgpu_bo(obj);
>  	struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
>  	struct sg_table *sgt;
> +	resource_size_t phys;
> +	u64 off;
> +	size_t len;
>  	long r;
>  
>  	if (!bo->tbo.pin_count) {
> @@ -261,6 +264,29 @@ static struct sg_table *amdgpu_dma_buf_map(struct dma_buf_attachment *attach,
>  		if (r)
>  			return ERR_PTR(r);
>  		break;
> +
> +	case AMDGPU_PL_MMIO_REMAP:
> +		/* Only allow when importer can reach exporter via P2P. */
> +		if (!attach->peer2peer ||
> +		    pci_p2pdma_distance(adev->pdev, attach->dev, false) < 0)
> +			return ERR_PTR(-EOPNOTSUPP);
> +

> +		if (!adev->rmmio_remap.bus_addr)
> +			return ERR_PTR(-ENODEV);
> +
> +		if (bo->tbo.base.size != AMDGPU_GPU_PAGE_SIZE)
> +			return ERR_PTR(-EINVAL);

I think you can drop those checks.

> +
> +		/* TTM start is in pages → convert to byte offset. */
> +		off  = (u64)bo->tbo.resource->start << PAGE_SHIFT;

Please don't use resource->start for that. Instead use the functions in amdgpu_res_cursor.h.

> +		len  = AMDGPU_GPU_PAGE_SIZE;
> +		phys = adev->rmmio_remap.bus_addr + off;
> +
> +		sgt = amdgpu_dmabuf_map_iomem(attach->dev, phys, len, dir);

Move the amdgpu_dmabuf_map_iomem() function as well as the off and length calculation into amdgpu_ttm.c.

That is still not an ideal placement, but better than here.

Apart from that looks good to me.

Regards,
Christian.

> +		if (IS_ERR(sgt))
> +			return sgt;
> +		break;
> +
>  	default:
>  		return ERR_PTR(-EINVAL);
>  	}
> @@ -286,6 +312,15 @@ static void amdgpu_dma_buf_unmap(struct dma_buf_attachment *attach,
>  				 struct sg_table *sgt,
>  				 enum dma_data_direction dir)
>  {
> +	struct drm_gem_object *obj = attach->dmabuf->priv;
> +	struct amdgpu_bo *bo = gem_to_amdgpu_bo(obj);
> +
> +	if (bo->tbo.resource &&
> +	    bo->tbo.resource->mem_type == AMDGPU_PL_MMIO_REMAP) {
> +		amdgpu_dmabuf_unmap_iomem(attach->dev, sgt, dir);
> +		return;
> +	}
> +
>  	if (sg_page(sgt->sgl)) {
>  		dma_unmap_sgtable(attach->dev, sgt, dir, 0);
>  		sg_free_table(sgt);

[PATCH v3 0/2] drm/amdgpu: Handle MMIO_REMAP as fixed I/O via dma-buf v3

[Srinivasan Shanmugam]

v3: Addressed Christian’s feedbacks from v2

Move BAR I/O mapping into TTM:

New helpers in amdgpu_ttm.c: amdgpu_ttm_mmio_remap_alloc_sgt() /
amdgpu_ttm_mmio_remap_free_sgt().  Export prototypes in amdgpu_ttm.h and
include it from amdgpu_dma_buf.c.

Stop doing offset calculations in dma-buf: Use amdgpu_res_cursor
(amdgpu_res_first/next) in the TTM helper to get start/size.  Compute
BAR phys as adev->rmmio_remap.bus_addr + start inside the helper.

Simplify dma-buf MMIO_REMAP path: amdgpu_dma_buf_map()/unmap() now just
call the TTM helpers.  Drop ad-hoc checks (e.g., resource->start,
size/page checks) from dma-buf per review; policy checks remain at
callers when needed.

Keep the mapping: Build a minimal 1-entry sg_table (sg_alloc_table(...,
1, ...)) with sg_set_page(NULL, ...) since this is BAR I/O (no struct
pages).  Map with dma_map_resource(..., DMA_ATTR_SKIP_CPU_SYNC); unmap
with dma_unmap_resource().

* Only compilation tested so far (x86_64, defconfig + amdgpu enabled).

Suggested-by: Christian König <christian.koenig@amd.com>
Suggested-by: Alex Deucher <alexander.deucher@amd.com>
Cc: Christian König <christian.koenig@amd.com>
Cc: Alex Deucher <alexander.deucher@amd.com>

Srinivasan Shanmugam (2):
  drm/amdgpu/ttm: Pin 4K MMIO_REMAP Singleton BO at Init
  drm/amdgpu: Map/Unmap MMIO_REMAP as BAR register window; add TTM sg
    helpers; wire dma-buf

 drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c |  18 ++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c     | 106 ++++++++++++++++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h     |   9 ++
 3 files changed, 133 insertions(+)


base-commit: 40f4510b661af74cd4b32b368747867c00ad7490
-- 
2.34.1

[PATCH v3 1/2] drm/amdgpu/ttm: Pin 4K MMIO_REMAP Singleton BO at Init

[Srinivasan Shanmugam]

MMIO_REMAP (HDP flush page) is a hardware I/O window exposed via a PCI
BAR.  It must not migrate or be evicted.

Allocate a single 4 KB GEM BO in AMDGPU_GEM_DOMAIN_MMIO_REMAP during TTM
initialization when the hardware exposes a remap bus address and the
host page size is <= 4 KiB. Reserve the BO and pin it at the TTM level
so it remains fixed for its lifetime. No CPU mapping is established
here.

On teardown, reserve, unpin, and free the BO if present.

This prepares the object to be shared (e.g., via dma-buf) without
triggering placement changes or no CPU-access migration

Suggested-by: Christian König <christian.koenig@amd.com>
Suggested-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 26 +++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
index 046ff2346dab..d4c93c78b80a 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
@@ -1861,6 +1861,10 @@ static void amdgpu_ttm_pools_fini(struct amdgpu_device *adev)
  * PAGE_SIZE is <= AMDGPU_GPU_PAGE_SIZE (4K). The BO is created as a regular
  * GEM object (amdgpu_bo_create).
  *
+ * The BO is created as a normal GEM object via amdgpu_bo_create(), then
+ * reserved and pinned at the TTM level (ttm_bo_pin()) so it can never be
+ * migrated or evicted. No CPU mapping is established here.
+ *
  * Return:
  *  * 0 on success or intentional skip (feature not present/unsupported)
  *  * negative errno on allocation failure
@@ -1889,7 +1893,25 @@ static int amdgpu_ttm_mmio_remap_bo_init(struct amdgpu_device *adev)
 	if (r)
 		return r;
 
+	r = amdgpu_bo_reserve(adev->rmmio_remap.bo, true);
+	if (r)
+		goto err_unref;
+
+	/*
+	 * MMIO_REMAP is a fixed I/O placement (AMDGPU_PL_MMIO_REMAP).
+	 * Use TTM-level pin so the BO cannot be evicted/migrated,
+	 * independent of GEM domains. This
+	 * enforces the “fixed I/O window”
+	 */
+	ttm_bo_pin(&adev->rmmio_remap.bo->tbo);
+
+	amdgpu_bo_unreserve(adev->rmmio_remap.bo);
 	return 0;
+
+err_unref:
+	amdgpu_bo_unref(&adev->rmmio_remap.bo);
+	adev->rmmio_remap.bo = NULL;
+	return r;
 }
 
 /**
@@ -1901,6 +1923,10 @@ static int amdgpu_ttm_mmio_remap_bo_init(struct amdgpu_device *adev)
  */
 static void amdgpu_ttm_mmio_remap_bo_fini(struct amdgpu_device *adev)
 {
+	if (!amdgpu_bo_reserve(adev->rmmio_remap.bo, true)) {
+		ttm_bo_unpin(&adev->rmmio_remap.bo->tbo);
+		amdgpu_bo_unreserve(adev->rmmio_remap.bo);
+	}
 	amdgpu_bo_unref(&adev->rmmio_remap.bo);
 	adev->rmmio_remap.bo = NULL;
 }
-- 
2.34.1

[PATCH v3 2/2] drm/amdgpu: Map/Unmap MMIO_REMAP as BAR register window; add TTM sg helpers; wire dma-buf

[Srinivasan Shanmugam]

MMIO_REMAP (HDP flush page) exposes a hardware MMIO register window via
a PCI BAR; there are no struct pages backing it (not normal RAM).  But
when one device shares memory with another through dma-buf, the receiver
still expects a delivery route—a list of DMA-able chunks—called an
sg_table. For the BAR window, we can’t (no pages!), so we instead create
a one-entry list that points directly to the BAR’s physical bus address
and tell DMA: “use this I/O span.” - A single, contiguous byte range on
the PCI bus (start DMA address + length)). That’s why we map it with
dma_map_resource() and set sg_set_page(..., NULL, ...). Perform DMA
reads/writes directly to that range so we build an sg_table from a BAR
physical span and map it with dma_map_resource().

This patch centralizes the BAR-I/O mapping in TTM and wires dma-buf to
it:

Add amdgpu_ttm_mmio_remap_alloc_sgt() /
amdgpu_ttm_mmio_remap_free_sgt(). They walk the TTM resource via
amdgpu_res_cursor, add the byte offset to adev->rmmio_remap.bus_addr,
build a one-entry sg_table with sg_set_page(NULL, …), and map/unmap it
with dma_map_resource().

In dma-buf map/unmap, if the BO is in AMDGPU_PL_MMIO_REMAP, call the new
helpers.

Single place for BAR-I/O handling: amdgpu_ttm.c in
amdgpu_ttm_mmio_remap_alloc_sgt() and ..._free_sgt().
No struct pages: sg_set_page(sg, NULL, cur.size, 0); inside
amdgpu_ttm_mmio_remap_alloc_sgt().
Minimal sg_table: sg_alloc_table(*sgt, 1, GFP_KERNEL); inside
amdgpu_ttm_mmio_remap_alloc_sgt().
Hooked into dma-buf: amdgpu_dma_buf_map()/unmap() in amdgpu_dma_buf.c
call these helpers for AMDGPU_PL_MMIO_REMAP.

Suggested-by: Christian König <christian.koenig@amd.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c | 18 +++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c     | 80 +++++++++++++++++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h     |  9 +++
 3 files changed, 107 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
index ff98c87b2e0b..2fbd6d458a6f 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
@@ -37,6 +37,7 @@
 #include "amdgpu_dma_buf.h"
 #include "amdgpu_xgmi.h"
 #include "amdgpu_vm.h"
+#include "amdgpu_ttm.h"
 #include <drm/amdgpu_drm.h>
 #include <drm/ttm/ttm_tt.h>
 #include <linux/dma-buf.h>
@@ -210,6 +211,14 @@ static struct sg_table *amdgpu_dma_buf_map(struct dma_buf_attachment *attach,
 		if (r)
 			return ERR_PTR(r);
 		break;
+
+	case AMDGPU_PL_MMIO_REMAP:
+		r = amdgpu_ttm_mmio_remap_alloc_sgt(adev, bo->tbo.resource,
+						    attach->dev, dir, &sgt);
+		if (r)
+			return ERR_PTR(r);
+		break;
+
 	default:
 		return ERR_PTR(-EINVAL);
 	}
@@ -235,6 +244,15 @@ static void amdgpu_dma_buf_unmap(struct dma_buf_attachment *attach,
 				 struct sg_table *sgt,
 				 enum dma_data_direction dir)
 {
+	struct drm_gem_object *obj = attach->dmabuf->priv;
+	struct amdgpu_bo *bo = gem_to_amdgpu_bo(obj);
+
+	if (bo->tbo.resource &&
+	    bo->tbo.resource->mem_type == AMDGPU_PL_MMIO_REMAP) {
+		amdgpu_ttm_mmio_remap_free_sgt(attach->dev, dir, sgt);
+		return;
+	}
+
 	if (sg_page(sgt->sgl)) {
 		dma_unmap_sgtable(attach->dev, sgt, dir, 0);
 		sg_free_table(sgt);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
index d4c93c78b80a..17ea079bd96f 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
@@ -1102,6 +1102,86 @@ static void amdgpu_ttm_backend_destroy(struct ttm_device *bdev,
 	kfree(gtt);
 }
 
+/**
+ * amdgpu_ttm_mmio_remap_alloc_sgt - build an sg_table for MMIO_REMAP I/O aperture
+ * @adev: amdgpu device providing the remap BAR base (adev->rmmio_remap.bus_addr)
+ * @res:  TTM resource of the BO to export; expected to live in AMDGPU_PL_MMIO_REMAP
+ * @dev:  importing device to map for (typically @attach->dev in dma-buf paths)
+ * @dir:  DMA data direction for the importer (passed to dma_map_resource())
+ * @sgt:  output; on success, set to a newly allocated sg_table describing the I/O span
+ *
+ * The HDP flush page (AMDGPU_PL_MMIO_REMAP) is a fixed hardware I/O window in a PCI
+ * BAR—there are no struct pages to back it. Importers still need a DMA address list,
+ * so we synthesize a minimal sg_table and populate it from dma_map_resource(), not
+ * from pages. Using the common amdgpu_res_cursor walker keeps the offset/size math
+ * consistent with other TTM/manager users.
+ *
+ * - @res is assumed to be a small, contiguous I/O region (typically a single 4 KiB
+ *   page) in AMDGPU_PL_MMIO_REMAP. Callers should validate placement before calling.
+ * - The sg entry is created with sg_set_page(sg, NULL, …) to reflect I/O space.
+ * - The mapping uses DMA_ATTR_SKIP_CPU_SYNC because this is MMIO, not cacheable RAM.
+ * - Peer reachability / p2pdma policy checks must be done by the caller.
+ *
+ * Return:
+ * * 0 on success, with *@sgt set to a valid table that must be freed via
+ *   amdgpu_ttm_mmio_remap_free_sgt().
+ * * -ENOMEM if allocation of the sg_table fails.
+ * * -EIO if dma_map_resource() fails.
+ *
+ */
+int amdgpu_ttm_mmio_remap_alloc_sgt(struct amdgpu_device *adev,
+				    struct ttm_resource *res,
+				    struct device *dev,
+				    enum dma_data_direction dir,
+				    struct sg_table **sgt)
+{
+	struct amdgpu_res_cursor cur;
+	dma_addr_t dma;
+	resource_size_t phys;
+	struct scatterlist *sg;
+	int r;
+
+	/* Walk the resource once; MMIO_REMAP is expected to be contiguous+small. */
+	amdgpu_res_first(res, 0, res->size, &cur);
+
+	/* Translate byte offset in the remap window into a host physical BAR address. */
+	phys = adev->rmmio_remap.bus_addr + cur.start;
+
+	/* Build a single-entry sg_table mapped as I/O (no struct page backing). */
+	*sgt = kzalloc(sizeof(**sgt), GFP_KERNEL);
+	if (!*sgt)
+		return -ENOMEM;
+	r = sg_alloc_table(*sgt, 1, GFP_KERNEL);
+	if (r) {
+		kfree(*sgt);
+		return r;
+	}
+	sg = (*sgt)->sgl;
+	sg_set_page(sg, NULL, cur.size, 0);  /* WHY: I/O space → no pages */
+
+	dma = dma_map_resource(dev, phys, cur.size, dir, DMA_ATTR_SKIP_CPU_SYNC);
+	if (dma_mapping_error(dev, dma)) {
+		sg_free_table(*sgt);
+		kfree(*sgt);
+		return -EIO;
+	}
+	sg_dma_address(sg) = dma;
+	sg_dma_len(sg) = cur.size;
+	return 0;
+}
+
+void amdgpu_ttm_mmio_remap_free_sgt(struct device *dev,
+				    enum dma_data_direction dir,
+				    struct sg_table *sgt)
+{
+	struct scatterlist *sg = sgt->sgl;
+
+	dma_unmap_resource(dev, sg_dma_address(sg), sg_dma_len(sg),
+			   dir, DMA_ATTR_SKIP_CPU_SYNC);
+	sg_free_table(sgt);
+	kfree(sgt);
+}
+
 /**
  * amdgpu_ttm_tt_create - Create a ttm_tt object for a given BO
  *
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h
index a8379b925878..116f3bb1d64b 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h
@@ -233,4 +233,13 @@ int amdgpu_ttm_evict_resources(struct amdgpu_device *adev, int mem_type);
 
 void amdgpu_ttm_debugfs_init(struct amdgpu_device *adev);
 
+int amdgpu_ttm_mmio_remap_alloc_sgt(struct amdgpu_device *adev,
+				    struct ttm_resource *res,
+				    struct device *dev,
+				    enum dma_data_direction dir,
+				    struct sg_table **sgt);
+void amdgpu_ttm_mmio_remap_free_sgt(struct device *dev,
+				    enum dma_data_direction dir,
+				    struct sg_table *sgt);
+
 #endif
-- 
2.34.1

Re: [PATCH v3 2/2] drm/amdgpu: Map/Unmap MMIO_REMAP as BAR register window; add TTM sg helpers; wire dma-buf

[Alex Deucher]

On Mon, Oct 6, 2025 at 10:36 AM Srinivasan Shanmugam
<srinivasan.shanmugam@amd.com> wrote:
>
> MMIO_REMAP (HDP flush page) exposes a hardware MMIO register window via
> a PCI BAR; there are no struct pages backing it (not normal RAM).  But
> when one device shares memory with another through dma-buf, the receiver
> still expects a delivery route—a list of DMA-able chunks—called an
> sg_table. For the BAR window, we can’t (no pages!), so we instead create
> a one-entry list that points directly to the BAR’s physical bus address
> and tell DMA: “use this I/O span.” - A single, contiguous byte range on
> the PCI bus (start DMA address + length)). That’s why we map it with
> dma_map_resource() and set sg_set_page(..., NULL, ...). Perform DMA
> reads/writes directly to that range so we build an sg_table from a BAR
> physical span and map it with dma_map_resource().
>
> This patch centralizes the BAR-I/O mapping in TTM and wires dma-buf to
> it:
>
> Add amdgpu_ttm_mmio_remap_alloc_sgt() /
> amdgpu_ttm_mmio_remap_free_sgt(). They walk the TTM resource via
> amdgpu_res_cursor, add the byte offset to adev->rmmio_remap.bus_addr,
> build a one-entry sg_table with sg_set_page(NULL, …), and map/unmap it
> with dma_map_resource().
>
> In dma-buf map/unmap, if the BO is in AMDGPU_PL_MMIO_REMAP, call the new
> helpers.
>
> Single place for BAR-I/O handling: amdgpu_ttm.c in
> amdgpu_ttm_mmio_remap_alloc_sgt() and ..._free_sgt().
> No struct pages: sg_set_page(sg, NULL, cur.size, 0); inside
> amdgpu_ttm_mmio_remap_alloc_sgt().
> Minimal sg_table: sg_alloc_table(*sgt, 1, GFP_KERNEL); inside
> amdgpu_ttm_mmio_remap_alloc_sgt().
> Hooked into dma-buf: amdgpu_dma_buf_map()/unmap() in amdgpu_dma_buf.c
> call these helpers for AMDGPU_PL_MMIO_REMAP.
>
> Suggested-by: Christian König <christian.koenig@amd.com>
> Cc: Alex Deucher <alexander.deucher@amd.com>
> Signed-off-by: Srinivasan Shanmugam <srinivasan.shanmugam@amd.com>

Series is:
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>

> ---
>  drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c | 18 +++++
>  drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c     | 80 +++++++++++++++++++++
>  drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h     |  9 +++
>  3 files changed, 107 insertions(+)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
> index ff98c87b2e0b..2fbd6d458a6f 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
> @@ -37,6 +37,7 @@
>  #include "amdgpu_dma_buf.h"
>  #include "amdgpu_xgmi.h"
>  #include "amdgpu_vm.h"
> +#include "amdgpu_ttm.h"
>  #include <drm/amdgpu_drm.h>
>  #include <drm/ttm/ttm_tt.h>
>  #include <linux/dma-buf.h>
> @@ -210,6 +211,14 @@ static struct sg_table *amdgpu_dma_buf_map(struct dma_buf_attachment *attach,
>                 if (r)
>                         return ERR_PTR(r);
>                 break;
> +
> +       case AMDGPU_PL_MMIO_REMAP:
> +               r = amdgpu_ttm_mmio_remap_alloc_sgt(adev, bo->tbo.resource,
> +                                                   attach->dev, dir, &sgt);
> +               if (r)
> +                       return ERR_PTR(r);
> +               break;
> +
>         default:
>                 return ERR_PTR(-EINVAL);
>         }
> @@ -235,6 +244,15 @@ static void amdgpu_dma_buf_unmap(struct dma_buf_attachment *attach,
>                                  struct sg_table *sgt,
>                                  enum dma_data_direction dir)
>  {
> +       struct drm_gem_object *obj = attach->dmabuf->priv;
> +       struct amdgpu_bo *bo = gem_to_amdgpu_bo(obj);
> +
> +       if (bo->tbo.resource &&
> +           bo->tbo.resource->mem_type == AMDGPU_PL_MMIO_REMAP) {
> +               amdgpu_ttm_mmio_remap_free_sgt(attach->dev, dir, sgt);
> +               return;
> +       }
> +
>         if (sg_page(sgt->sgl)) {
>                 dma_unmap_sgtable(attach->dev, sgt, dir, 0);
>                 sg_free_table(sgt);
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> index d4c93c78b80a..17ea079bd96f 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> @@ -1102,6 +1102,86 @@ static void amdgpu_ttm_backend_destroy(struct ttm_device *bdev,
>         kfree(gtt);
>  }
>
> +/**
> + * amdgpu_ttm_mmio_remap_alloc_sgt - build an sg_table for MMIO_REMAP I/O aperture
> + * @adev: amdgpu device providing the remap BAR base (adev->rmmio_remap.bus_addr)
> + * @res:  TTM resource of the BO to export; expected to live in AMDGPU_PL_MMIO_REMAP
> + * @dev:  importing device to map for (typically @attach->dev in dma-buf paths)
> + * @dir:  DMA data direction for the importer (passed to dma_map_resource())
> + * @sgt:  output; on success, set to a newly allocated sg_table describing the I/O span
> + *
> + * The HDP flush page (AMDGPU_PL_MMIO_REMAP) is a fixed hardware I/O window in a PCI
> + * BAR—there are no struct pages to back it. Importers still need a DMA address list,
> + * so we synthesize a minimal sg_table and populate it from dma_map_resource(), not
> + * from pages. Using the common amdgpu_res_cursor walker keeps the offset/size math
> + * consistent with other TTM/manager users.
> + *
> + * - @res is assumed to be a small, contiguous I/O region (typically a single 4 KiB
> + *   page) in AMDGPU_PL_MMIO_REMAP. Callers should validate placement before calling.
> + * - The sg entry is created with sg_set_page(sg, NULL, …) to reflect I/O space.
> + * - The mapping uses DMA_ATTR_SKIP_CPU_SYNC because this is MMIO, not cacheable RAM.
> + * - Peer reachability / p2pdma policy checks must be done by the caller.
> + *
> + * Return:
> + * * 0 on success, with *@sgt set to a valid table that must be freed via
> + *   amdgpu_ttm_mmio_remap_free_sgt().
> + * * -ENOMEM if allocation of the sg_table fails.
> + * * -EIO if dma_map_resource() fails.
> + *
> + */
> +int amdgpu_ttm_mmio_remap_alloc_sgt(struct amdgpu_device *adev,
> +                                   struct ttm_resource *res,
> +                                   struct device *dev,
> +                                   enum dma_data_direction dir,
> +                                   struct sg_table **sgt)
> +{
> +       struct amdgpu_res_cursor cur;
> +       dma_addr_t dma;
> +       resource_size_t phys;
> +       struct scatterlist *sg;
> +       int r;
> +
> +       /* Walk the resource once; MMIO_REMAP is expected to be contiguous+small. */
> +       amdgpu_res_first(res, 0, res->size, &cur);
> +
> +       /* Translate byte offset in the remap window into a host physical BAR address. */
> +       phys = adev->rmmio_remap.bus_addr + cur.start;
> +
> +       /* Build a single-entry sg_table mapped as I/O (no struct page backing). */
> +       *sgt = kzalloc(sizeof(**sgt), GFP_KERNEL);
> +       if (!*sgt)
> +               return -ENOMEM;
> +       r = sg_alloc_table(*sgt, 1, GFP_KERNEL);
> +       if (r) {
> +               kfree(*sgt);
> +               return r;
> +       }
> +       sg = (*sgt)->sgl;
> +       sg_set_page(sg, NULL, cur.size, 0);  /* WHY: I/O space → no pages */
> +
> +       dma = dma_map_resource(dev, phys, cur.size, dir, DMA_ATTR_SKIP_CPU_SYNC);
> +       if (dma_mapping_error(dev, dma)) {
> +               sg_free_table(*sgt);
> +               kfree(*sgt);
> +               return -EIO;
> +       }
> +       sg_dma_address(sg) = dma;
> +       sg_dma_len(sg) = cur.size;
> +       return 0;
> +}
> +
> +void amdgpu_ttm_mmio_remap_free_sgt(struct device *dev,
> +                                   enum dma_data_direction dir,
> +                                   struct sg_table *sgt)
> +{
> +       struct scatterlist *sg = sgt->sgl;
> +
> +       dma_unmap_resource(dev, sg_dma_address(sg), sg_dma_len(sg),
> +                          dir, DMA_ATTR_SKIP_CPU_SYNC);
> +       sg_free_table(sgt);
> +       kfree(sgt);
> +}
> +
>  /**
>   * amdgpu_ttm_tt_create - Create a ttm_tt object for a given BO
>   *
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h
> index a8379b925878..116f3bb1d64b 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h
> @@ -233,4 +233,13 @@ int amdgpu_ttm_evict_resources(struct amdgpu_device *adev, int mem_type);
>
>  void amdgpu_ttm_debugfs_init(struct amdgpu_device *adev);
>
> +int amdgpu_ttm_mmio_remap_alloc_sgt(struct amdgpu_device *adev,
> +                                   struct ttm_resource *res,
> +                                   struct device *dev,
> +                                   enum dma_data_direction dir,
> +                                   struct sg_table **sgt);
> +void amdgpu_ttm_mmio_remap_free_sgt(struct device *dev,
> +                                   enum dma_data_direction dir,
> +                                   struct sg_table *sgt);
> +
>  #endif
> --
> 2.34.1
>