[PATCH 00/14] drm/amdgpu: Improve retry fault handling (v3)

[PATCH 00/14] drm/amdgpu: Improve retry fault handling (v3)

[Timur Kristóf]

With this series, the kernel is able to mitigate
most page faults on Navi 3 and 4 without causing a
hang and without a need to reset the GPU, when the
amdgpu.noretry=0 module parameter is set.

Fix various issues with retry fault handling.

Enable filter CAM on Navi 3 and 4 which is a
HW block that filters the repeated page fault
interrupts that happen when retry faults are
enabled, making the handling more efficient.

Changes in v2:

* Reordered patches in the series to put bug fixes first
* Enable retry fault interrupt in init_system_aperture_regs()
  instead of in set_fault_enable_default()
* Added a patch to respect the noretry flag on GFX12.1 too

Changes in v3:

* This series now includes fixes for Navi 4 too which
  were previously submitted in a separate series.
* Dropped patch that improved handling of timestamps
  on the page fault interrupts. I'll come up with
  a different solution for that problem later.
* Dropped patches that solve race condition between
  retry fault handling and filter CAM ACK because
  they conflict with Christian's ongoing work.
  I'll revisit that later.
* Reworked the series into smaller patches for
  easier reviews.
* Addressed comments from Tvrtko's review.
  Thank you Tvrtko for taking the time to review!

Timur Kristóf (14):
  drm/amdgpu: Respect noretry flag for retry faults on GFX12.1
  drm/amdgpu/gfxhub: Enable retry fault interrupts when needed
  drm/amdgpu/ih: Don't perturb HW registers when accessing soft IH ring
  drm/amdgpu/ih: Add retry_cam_ack IH function pointer
  drm/amdgpu/ih6.1: Use IH_SW_RING_SIZE for soft IH ring instead of
    PAGE_SIZE
  drm/amdgpu/ih7.0: Use IH_SW_RING_SIZE for soft IH ring instead of
    PAGE_SIZE
  drm/amdgpu/gmc11: Pass cam_index to retry fault handler
  drm/amdgpu/gmc12: Pass cam_index to retry fault handler
  drm/amdgpu/gmc12: Use AMDGPU_PTE_IS_PTE flag for init_pte_flags on
    GFX12.0
  drm/amdgpu/vm: Use init PTE flags and NOALLOC in
    amdgpu_vm_handle_fault()
  drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH 6.0
  drm/amdgpu/ih7.0: Use MMIO ACK instead of doorbell for retry CAM on IH
    7.0
  drm/amdgpu/ih6.0: Enable retry CAM on Navi 3 dGPUs
  drm/amdgpu/ih7.0: Enable retry CAM on Navi 4 dGPUs

 drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c     |  7 ++++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ih.h      |  1 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c      |  8 ++++--
 drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c |  9 +++++--
 drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c   |  9 +++++--
 drivers/gpu/drm/amd/amdgpu/gfxhub_v12_1.c   |  4 +--
 drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c    |  9 +++++--
 drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c    |  2 ++
 drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c    |  9 +++++--
 drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c    |  9 +++++--
 drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c    |  9 +++++--
 drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c  |  9 +++++--
 drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c      |  5 +++-
 drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c      |  8 ++++--
 drivers/gpu/drm/amd/amdgpu/ih_v6_0.c        | 25 ++++++++++++++++-
 drivers/gpu/drm/amd/amdgpu/ih_v6_1.c        |  9 ++++++-
 drivers/gpu/drm/amd/amdgpu/ih_v7_0.c        | 30 +++++++++------------
 drivers/gpu/drm/amd/amdgpu/mmhub_v4_2_0.c   |  2 +-
 drivers/gpu/drm/amd/amdgpu/navi10_ih.c      |  4 +++
 drivers/gpu/drm/amd/amdgpu/vega20_ih.c      |  8 +++++-
 20 files changed, 133 insertions(+), 43 deletions(-)

-- 
2.54.0

[PATCH 01/14] drm/amdgpu: Respect noretry flag for retry faults on GFX12.1

[Timur Kristóf]

When retry faults are disabled (amdgpu.noretry=1),
the ENABLE_RETRY_FAULT_INTERRUPT bit should be programmed to 0.

Note that retry faults are enabled by default on GFX12.1
so this just fixes the case when they are explicitly disabled.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
---
 drivers/gpu/drm/amd/amdgpu/gfxhub_v12_1.c | 4 ++--
 drivers/gpu/drm/amd/amdgpu/mmhub_v4_2_0.c | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v12_1.c b/drivers/gpu/drm/amd/amdgpu/gfxhub_v12_1.c
index 4c2fd1e6616e..0cada13d92a4 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v12_1.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v12_1.c
@@ -243,7 +243,7 @@ static void gfxhub_v12_1_xcc_init_system_aperture_regs(struct amdgpu_device *ade
 		tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
 				    ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
 		tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
-				    ENABLE_RETRY_FAULT_INTERRUPT, 0x1);
+				    ENABLE_RETRY_FAULT_INTERRUPT, !adev->gmc.noretry);
 		WREG32_SOC15(GC, GET_INST(GC, i),
 			     regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp);
 	}
@@ -447,7 +447,7 @@ static void gfxhub_v12_1_xcc_setup_vmid_config(struct amdgpu_device *adev,
 			/* Send no-retry XNACK on fault to suppress VM fault storm */
 			tmp = REG_SET_FIELD(tmp, GCVM_CONTEXT1_CNTL,
 					    RETRY_PERMISSION_OR_INVALID_PAGE_FAULT,
-					    1);
+					    !adev->gmc.noretry);
 			WREG32_SOC15_OFFSET(GC, GET_INST(GC, j), regGCVM_CONTEXT1_CNTL,
 					    i * hub->ctx_distance, tmp);
 			WREG32_SOC15_OFFSET(GC, GET_INST(GC, j),
diff --git a/drivers/gpu/drm/amd/amdgpu/mmhub_v4_2_0.c b/drivers/gpu/drm/amd/amdgpu/mmhub_v4_2_0.c
index 49b7f16a941f..f2c549737e02 100644
--- a/drivers/gpu/drm/amd/amdgpu/mmhub_v4_2_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/mmhub_v4_2_0.c
@@ -320,7 +320,7 @@ static void mmhub_v4_2_0_mid_init_system_aperture_regs(struct amdgpu_device *ade
 		tmp = REG_SET_FIELD(tmp, MMVM_L2_PROTECTION_FAULT_CNTL2,
 				    ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
 		tmp = REG_SET_FIELD(tmp, MMVM_L2_PROTECTION_FAULT_CNTL2,
-				    ENABLE_RETRY_FAULT_INTERRUPT, 0x1);
+				    ENABLE_RETRY_FAULT_INTERRUPT, !adev->gmc.noretry);
 		WREG32_SOC15(MMHUB, GET_INST(MMHUB, i),
 			     regMMVM_L2_PROTECTION_FAULT_CNTL2, tmp);
 	}
-- 
2.54.0

[PATCH 02/14] drm/amdgpu/gfxhub: Enable retry fault interrupts when needed

[Timur Kristóf]

Enable retry fault interrupts when initializing the GFXHUB
system aperture registers according to whether retrying
page faults is enabled in amdgpu (ie. amdgpu.noretry=0).

Needs to be done for each GFXHUB version at once,
because none of them actually enabled this interrupt.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
---
 drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c | 9 +++++++--
 drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c   | 9 +++++++--
 drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c    | 9 +++++++--
 drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c    | 2 ++
 drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c    | 9 +++++++--
 drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c    | 9 +++++++--
 drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c    | 9 +++++++--
 drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c  | 9 +++++++--
 8 files changed, 51 insertions(+), 14 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c b/drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c
index 652eea6eae4a..ef20eafd59ae 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c
@@ -155,6 +155,7 @@ static void gfxhub_v11_5_0_init_gart_aperture_regs(struct amdgpu_device *adev)
 static void gfxhub_v11_5_0_init_system_aperture_regs(struct amdgpu_device *adev)
 {
 	uint64_t value;
+	u32 tmp;
 
 	WREG32_SOC15(GC, 0, regGCMC_VM_AGP_BASE, 0);
 	WREG32_SOC15(GC, 0, regGCMC_VM_AGP_BOT, adev->gmc.agp_start >> 24);
@@ -180,8 +181,12 @@ static void gfxhub_v11_5_0_init_system_aperture_regs(struct amdgpu_device *adev)
 	WREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
 		     (u32)((u64)adev->dummy_page_addr >> 44));
 
-	WREG32_FIELD15_PREREG(GC, 0, GCVM_L2_PROTECTION_FAULT_CNTL2,
-		       ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
+	tmp = RREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_CNTL2);
+	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
+				 ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
+	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
+				 ENABLE_RETRY_FAULT_INTERRUPT, !adev->gmc.noretry);
+	WREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp);
 }
 
 static void gfxhub_v11_5_0_init_tlb_regs(struct amdgpu_device *adev)
diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c b/drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c
index 6cbf837d50dd..ec3ff4dec674 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c
@@ -158,6 +158,7 @@ static void gfxhub_v12_0_init_gart_aperture_regs(struct amdgpu_device *adev)
 static void gfxhub_v12_0_init_system_aperture_regs(struct amdgpu_device *adev)
 {
 	uint64_t value;
+	u32 tmp;
 
 	/* Program the AGP BAR */
 	WREG32_SOC15(GC, 0, regGCMC_VM_AGP_BASE, 0);
@@ -184,8 +185,12 @@ static void gfxhub_v12_0_init_system_aperture_regs(struct amdgpu_device *adev)
 	WREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
 		     (u32)((u64)adev->dummy_page_addr >> 44));
 
-	WREG32_FIELD15_PREREG(GC, 0, GCVM_L2_PROTECTION_FAULT_CNTL2,
-		       ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
+	tmp = RREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_CNTL2);
+	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
+				 ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
+	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
+				 ENABLE_RETRY_FAULT_INTERRUPT, !adev->gmc.noretry);
+	WREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp);
 }
 
 
diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c b/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c
index bfe247b1a333..27d7f7cb903f 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c
@@ -91,6 +91,7 @@ static void gfxhub_v1_0_init_gart_aperture_regs(struct amdgpu_device *adev)
 static void gfxhub_v1_0_init_system_aperture_regs(struct amdgpu_device *adev)
 {
 	uint64_t value;
+	u32 tmp;
 
 	if (!amdgpu_sriov_vf(adev) || adev->asic_type <= CHIP_VEGA10) {
 		/* Program the AGP BAR */
@@ -134,8 +135,12 @@ static void gfxhub_v1_0_init_system_aperture_regs(struct amdgpu_device *adev)
 		WREG32_SOC15(GC, 0, mmVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
 			     (u32)((u64)adev->dummy_page_addr >> 44));
 
-		WREG32_FIELD15(GC, 0, VM_L2_PROTECTION_FAULT_CNTL2,
-			       ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
+		tmp = RREG32_SOC15(GC, 0, mmVM_L2_PROTECTION_FAULT_CNTL2);
+		tmp = REG_SET_FIELD(tmp, VM_L2_PROTECTION_FAULT_CNTL2,
+					 ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
+		tmp = REG_SET_FIELD(tmp, VM_L2_PROTECTION_FAULT_CNTL2,
+					 ENABLE_RETRY_FAULT_INTERRUPT, !adev->gmc.noretry);
+		WREG32_SOC15(GC, 0, mmVM_L2_PROTECTION_FAULT_CNTL2, tmp);
 	}
 
 	/* In the case squeezing vram into GART aperture, we don't use
diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c b/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c
index fbdf46070b38..ed9a64bc5aaa 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c
@@ -176,6 +176,8 @@ gfxhub_v1_2_xcc_init_system_aperture_regs(struct amdgpu_device *adev,
 			tmp = RREG32_SOC15(GC, GET_INST(GC, i), regVM_L2_PROTECTION_FAULT_CNTL2);
 			tmp = REG_SET_FIELD(tmp, VM_L2_PROTECTION_FAULT_CNTL2,
 					    ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
+			tmp = REG_SET_FIELD(tmp, VM_L2_PROTECTION_FAULT_CNTL2,
+					    ENABLE_RETRY_FAULT_INTERRUPT, !adev->gmc.noretry);
 			WREG32_SOC15(GC, GET_INST(GC, i), regVM_L2_PROTECTION_FAULT_CNTL2, tmp);
 		}
 
diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c b/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c
index 9ea593e2c719..152b2735d360 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c
@@ -151,6 +151,7 @@ static void gfxhub_v2_0_init_gart_aperture_regs(struct amdgpu_device *adev)
 static void gfxhub_v2_0_init_system_aperture_regs(struct amdgpu_device *adev)
 {
 	uint64_t value;
+	u32 tmp;
 
 	if (!amdgpu_sriov_vf(adev)) {
 		/* Program the AGP BAR */
@@ -178,8 +179,12 @@ static void gfxhub_v2_0_init_system_aperture_regs(struct amdgpu_device *adev)
 	WREG32_SOC15(GC, 0, mmGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
 		     (u32)((u64)adev->dummy_page_addr >> 44));
 
-	WREG32_FIELD15(GC, 0, GCVM_L2_PROTECTION_FAULT_CNTL2,
-		       ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
+	tmp = RREG32_SOC15(GC, 0, mmGCVM_L2_PROTECTION_FAULT_CNTL2);
+	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
+				 ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
+	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
+				 ENABLE_RETRY_FAULT_INTERRUPT, !adev->gmc.noretry);
+	WREG32_SOC15(GC, 0, mmGCVM_L2_PROTECTION_FAULT_CNTL2, tmp);
 }
 
 
diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c b/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c
index 30b90d35abd0..83c2ddbbd292 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c
@@ -154,6 +154,7 @@ static void gfxhub_v2_1_init_gart_aperture_regs(struct amdgpu_device *adev)
 static void gfxhub_v2_1_init_system_aperture_regs(struct amdgpu_device *adev)
 {
 	uint64_t value;
+	u32 tmp;
 
 	if (amdgpu_sriov_vf(adev))
 		return;
@@ -182,8 +183,12 @@ static void gfxhub_v2_1_init_system_aperture_regs(struct amdgpu_device *adev)
 	WREG32_SOC15(GC, 0, mmGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
 		     (u32)((u64)adev->dummy_page_addr >> 44));
 
-	WREG32_FIELD15(GC, 0, GCVM_L2_PROTECTION_FAULT_CNTL2,
-		       ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
+	tmp = RREG32_SOC15(GC, 0, mmGCVM_L2_PROTECTION_FAULT_CNTL2);
+	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
+				 ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
+	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
+				 ENABLE_RETRY_FAULT_INTERRUPT, !adev->gmc.noretry);
+	WREG32_SOC15(GC, 0, mmGCVM_L2_PROTECTION_FAULT_CNTL2, tmp);
 }
 
 
diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c b/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c
index 9e6a6e13dec0..90bbb2fe4884 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c
@@ -150,6 +150,7 @@ static void gfxhub_v3_0_init_gart_aperture_regs(struct amdgpu_device *adev)
 static void gfxhub_v3_0_init_system_aperture_regs(struct amdgpu_device *adev)
 {
 	uint64_t value;
+	u32 tmp;
 
 	/* Program the AGP BAR */
 	WREG32_SOC15(GC, 0, regGCMC_VM_AGP_BASE, 0);
@@ -176,8 +177,12 @@ static void gfxhub_v3_0_init_system_aperture_regs(struct amdgpu_device *adev)
 	WREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
 		     (u32)((u64)adev->dummy_page_addr >> 44));
 
-	WREG32_FIELD15_PREREG(GC, 0, GCVM_L2_PROTECTION_FAULT_CNTL2,
-		       ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
+	tmp = RREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_CNTL2);
+	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
+				 ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
+	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
+				 ENABLE_RETRY_FAULT_INTERRUPT, !adev->gmc.noretry);
+	WREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp);
 }
 
 
diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c b/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c
index b3b1085c7cd3..1b3c067ab48c 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c
@@ -153,6 +153,7 @@ static void gfxhub_v3_0_3_init_gart_aperture_regs(struct amdgpu_device *adev)
 static void gfxhub_v3_0_3_init_system_aperture_regs(struct amdgpu_device *adev)
 {
 	uint64_t value;
+	u32 tmp;
 
 	if (amdgpu_sriov_vf(adev))
 		return;
@@ -181,8 +182,12 @@ static void gfxhub_v3_0_3_init_system_aperture_regs(struct amdgpu_device *adev)
 	WREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
 		     (u32)((u64)adev->dummy_page_addr >> 44));
 
-	WREG32_FIELD15_PREREG(GC, 0, GCVM_L2_PROTECTION_FAULT_CNTL2,
-		       ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
+	tmp = RREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_CNTL2);
+	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
+				 ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
+	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
+				 ENABLE_RETRY_FAULT_INTERRUPT, !adev->gmc.noretry);
+	WREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp);
 }
 
 
-- 
2.54.0

[PATCH 03/14] drm/amdgpu/ih: Don't perturb HW registers when accessing soft IH ring

[Timur Kristóf]

The soft IH ring is implemented entirely in software.
We shouldn't read (or write) any HW registers when accessing it.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com>
---
 drivers/gpu/drm/amd/amdgpu/ih_v6_0.c   | 7 +++++++
 drivers/gpu/drm/amd/amdgpu/ih_v6_1.c   | 7 +++++++
 drivers/gpu/drm/amd/amdgpu/ih_v7_0.c   | 7 +++++++
 drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 4 ++++
 4 files changed, 25 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
index 333e9c30c091..65e5d21753f9 100644
--- a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
@@ -439,6 +439,10 @@ static u32 ih_v6_0_get_wptr(struct amdgpu_device *adev,
 	struct amdgpu_ih_regs *ih_regs;
 
 	wptr = le32_to_cpu(*ih->wptr_cpu);
+
+	if (ih == &adev->irq.ih_soft)
+		goto out;
+
 	ih_regs = &ih->ih_regs;
 
 	if (!REG_GET_FIELD(wptr, IH_RB_WPTR, RB_OVERFLOW))
@@ -514,6 +518,9 @@ static void ih_v6_0_set_rptr(struct amdgpu_device *adev,
 {
 	struct amdgpu_ih_regs *ih_regs;
 
+	if (ih == &adev->irq.ih_soft)
+		return;
+
 	if (ih->use_doorbell) {
 		/* XXX check if swapping is necessary on BE */
 		*ih->rptr_cpu = ih->rptr;
diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v6_1.c b/drivers/gpu/drm/amd/amdgpu/ih_v6_1.c
index 699c274d357e..9dbc20131410 100644
--- a/drivers/gpu/drm/amd/amdgpu/ih_v6_1.c
+++ b/drivers/gpu/drm/amd/amdgpu/ih_v6_1.c
@@ -410,6 +410,10 @@ static u32 ih_v6_1_get_wptr(struct amdgpu_device *adev,
 	struct amdgpu_ih_regs *ih_regs;
 
 	wptr = le32_to_cpu(*ih->wptr_cpu);
+
+	if (ih == &adev->irq.ih_soft)
+		goto out;
+
 	ih_regs = &ih->ih_regs;
 
 	if (!REG_GET_FIELD(wptr, IH_RB_WPTR, RB_OVERFLOW))
@@ -481,6 +485,9 @@ static void ih_v6_1_irq_rearm(struct amdgpu_device *adev,
 static void ih_v6_1_set_rptr(struct amdgpu_device *adev,
 			       struct amdgpu_ih_ring *ih)
 {
+	if (ih == &adev->irq.ih_soft)
+		return;
+
 	struct amdgpu_ih_regs *ih_regs;
 
 	if (ih->use_doorbell) {
diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
index 6de9e87e04e1..bd332e8cc5bf 100644
--- a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
@@ -457,6 +457,10 @@ static u32 ih_v7_0_get_wptr(struct amdgpu_device *adev,
 	struct amdgpu_ih_regs *ih_regs;
 
 	wptr = le32_to_cpu(*ih->wptr_cpu);
+
+	if (ih == &adev->irq.ih_soft)
+		goto out;
+
 	ih_regs = &ih->ih_regs;
 
 	if (!REG_GET_FIELD(wptr, IH_RB_WPTR, RB_OVERFLOW))
@@ -527,6 +531,9 @@ static void ih_v7_0_set_rptr(struct amdgpu_device *adev,
 {
 	struct amdgpu_ih_regs *ih_regs;
 
+	if (ih == &adev->irq.ih_soft)
+		return;
+
 	if (ih->use_doorbell) {
 		/* XXX check if swapping is necessary on BE */
 		*ih->rptr_cpu = ih->rptr;
diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
index 4cd325149b63..e7ed37bb48e0 100644
--- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
@@ -417,6 +417,10 @@ static u32 navi10_ih_get_wptr(struct amdgpu_device *adev,
 		 */
 		wptr = le32_to_cpu(*ih->wptr_cpu);
 
+		if (ih == &adev->irq.ih_soft)
+			goto out;
+
+
 		if (!REG_GET_FIELD(wptr, IH_RB_WPTR, RB_OVERFLOW))
 			goto out;
 	}
-- 
2.54.0

[PATCH 04/14] drm/amdgpu/ih: Add retry_cam_ack IH function pointer

[Timur Kristóf]

No functional changes.

This is a refactor to allow different filter CAM implementation
in subsequent commits. The actual new implementations are
going to be in subsequent commits.

Instead of writing the doorbell in amdgpu_gmc_handle_retry_fault()
directly, add an IH function pointer which can be defined in
a different way for different IH versions.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 7 ++++++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ih.h  | 1 +
 drivers/gpu/drm/amd/amdgpu/ih_v7_0.c    | 6 ++++++
 drivers/gpu/drm/amd/amdgpu/vega20_ih.c  | 8 +++++++-
 4 files changed, 20 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
index 3f0b1b7a557b..bb278a61dc9e 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
@@ -552,6 +552,11 @@ int amdgpu_gmc_handle_retry_fault(struct amdgpu_device *adev,
 	int ret;
 
 	if (adev->irq.retry_cam_enabled) {
+		if (!adev->irq.ih_funcs->retry_cam_ack) {
+			dev_warn(adev->dev, "retry CAM is enabled, but retry_cam_ack is NULL\n");
+			return -EOPNOTSUPP;
+		}
+
 		/* Delegate it to a different ring if the hardware hasn't
 		 * already done it.
 		 */
@@ -562,7 +567,7 @@ int amdgpu_gmc_handle_retry_fault(struct amdgpu_device *adev,
 
 		ret = amdgpu_vm_handle_fault(adev, entry->pasid, entry->vmid, node_id,
 					     addr, entry->timestamp, write_fault);
-		WDOORBELL32(adev->irq.retry_cam_doorbell_index, cam_index);
+		adev->irq.ih_funcs->retry_cam_ack(adev, cam_index);
 		if (ret)
 			return 1;
 	} else {
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.h
index 444437c30088..e6e34f6e86f4 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.h
@@ -97,6 +97,7 @@ struct amdgpu_ih_funcs {
 	const char *(*node_id_to_die_name)(struct amdgpu_device *adev,
 					   unsigned int node_id,
 					   char *buf, size_t size);
+	void (*retry_cam_ack)(struct amdgpu_device *adev, u32 cam_index);
 };
 
 #define amdgpu_ih_get_wptr(adev, ih) (adev)->irq.ih_funcs->get_wptr((adev), (ih))
diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
index bd332e8cc5bf..24be9d726428 100644
--- a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
@@ -289,6 +289,11 @@ static uint32_t ih_v7_0_setup_retry_doorbell(u32 doorbell_index)
 	return val;
 }
 
+static void ih_v7_0_retry_cam_ack(struct amdgpu_device *adev, u32 cam_index)
+{
+	WDOORBELL32(adev->irq.retry_cam_doorbell_index, cam_index);
+}
+
 #define regIH_RING1_CLIENT_CFG_INDEX_V7_1             0x122
 #define regIH_RING1_CLIENT_CFG_INDEX_V7_1_BASE_IDX    0
 #define regIH_RING1_CLIENT_CFG_DATA_V7_1              0x123
@@ -865,6 +870,7 @@ static const struct amdgpu_ih_funcs ih_v7_0_funcs = {
 	.decode_iv_ts = amdgpu_ih_decode_iv_ts_helper,
 	.set_rptr = ih_v7_0_set_rptr,
 	.node_id_to_die_name = ih_v7_0_node_id_to_die_name,
+	.retry_cam_ack = ih_v7_0_retry_cam_ack,
 };
 
 static void ih_v7_0_set_interrupt_funcs(struct amdgpu_device *adev)
diff --git a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c
index 85846fd08ce4..30a82fff3ff7 100644
--- a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c
@@ -293,6 +293,11 @@ static uint32_t vega20_setup_retry_doorbell(u32 doorbell_index)
 	return val;
 }
 
+static void vega20_retry_cam_ack(struct amdgpu_device *adev, u32 cam_index)
+{
+	WDOORBELL32(adev->irq.retry_cam_doorbell_index, cam_index);
+}
+
 /**
  * vega20_ih_irq_init - init and enable the interrupt ring
  *
@@ -738,7 +743,8 @@ static const struct amdgpu_ih_funcs vega20_ih_funcs = {
 	.get_wptr = vega20_ih_get_wptr,
 	.decode_iv = amdgpu_ih_decode_iv_helper,
 	.decode_iv_ts = amdgpu_ih_decode_iv_ts_helper,
-	.set_rptr = vega20_ih_set_rptr
+	.set_rptr = vega20_ih_set_rptr,
+	.retry_cam_ack = vega20_retry_cam_ack,
 };
 
 static void vega20_ih_set_interrupt_funcs(struct amdgpu_device *adev)
-- 
2.54.0

[PATCH 05/14] drm/amdgpu/ih6.1: Use IH_SW_RING_SIZE for soft IH ring instead of PAGE_SIZE

[Timur Kristóf]

When there are a lot of retry faults happening, the soft IH ring
can fill up really quickly and possibly overflow. PAGE_SIZE was
too small, use IH_SW_RING_SIZE to match what other GPU generations
are doing.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com>
---
 drivers/gpu/drm/amd/amdgpu/ih_v6_1.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v6_1.c b/drivers/gpu/drm/amd/amdgpu/ih_v6_1.c
index 9dbc20131410..f6fdb3a950e0 100644
--- a/drivers/gpu/drm/amd/amdgpu/ih_v6_1.c
+++ b/drivers/gpu/drm/amd/amdgpu/ih_v6_1.c
@@ -590,7 +590,7 @@ static int ih_v6_1_sw_init(struct amdgpu_ip_block *ip_block)
 	/* initialize ih control register offset */
 	ih_v6_1_init_register_offset(adev);
 
-	r = amdgpu_ih_ring_init(adev, &adev->irq.ih_soft, PAGE_SIZE, true);
+	r = amdgpu_ih_ring_init(adev, &adev->irq.ih_soft, IH_SW_RING_SIZE, true);
 	if (r)
 		return r;
 
-- 
2.54.0

[PATCH 06/14] drm/amdgpu/ih7.0: Use IH_SW_RING_SIZE for soft IH ring instead of PAGE_SIZE

[Timur Kristóf]

When there are a lot of retry faults happening, the soft IH ring
can fill up really quickly and possibly overflow. PAGE_SIZE was
too small, use IH_SW_RING_SIZE to match what other GPU generations
are doing.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com>
---
 drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
index 24be9d726428..f1de6450a31d 100644
--- a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
@@ -601,7 +601,6 @@ static int ih_v7_0_sw_init(struct amdgpu_ip_block *ip_block)
 	int r;
 	struct amdgpu_device *adev = ip_block->adev;
 	bool use_bus_addr;
-	unsigned int sw_ring_size;
 
 	r = amdgpu_irq_add_id(adev, SOC21_IH_CLIENTID_IH, 0,
 			      &adev->irq.self_irq);
@@ -633,9 +632,7 @@ static int ih_v7_0_sw_init(struct amdgpu_ip_block *ip_block)
 	/* initialize ih control register offset */
 	ih_v7_0_init_register_offset(adev);
 
-	sw_ring_size = (amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7, 1, 0)) ?
-			IH_SW_RING_SIZE : PAGE_SIZE;
-	r = amdgpu_ih_ring_init(adev, &adev->irq.ih_soft, sw_ring_size, true);
+	r = amdgpu_ih_ring_init(adev, &adev->irq.ih_soft, IH_SW_RING_SIZE, true);
 	if (r)
 		return r;
 
-- 
2.54.0

[PATCH 07/14] drm/amdgpu/gmc11: Pass cam_index to retry fault handler

[Timur Kristóf]

This is necessary if we want to make use of the filter CAM.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
---
 drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c
index c40d9c467204..18a53b9df77f 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c
@@ -108,13 +108,16 @@ static int gmc_v11_0_process_interrupt(struct amdgpu_device *adev,
 	bool write_fault = !!(entry->src_data[1] &
 			      AMDGPU_GMC9_FAULT_SOURCE_DATA_WRITE);
 	uint32_t status = 0;
+	uint32_t cam_index;
 	u64 addr;
 
 	addr = (u64)entry->src_data[0] << 12;
 	addr |= ((u64)entry->src_data[1] & 0xf) << 44;
 
 	if (retry_fault) {
-		int ret = amdgpu_gmc_handle_retry_fault(adev, entry, addr, 0, 0,
+		cam_index = entry->src_data[2] & 0x3ff;
+
+		int ret = amdgpu_gmc_handle_retry_fault(adev, entry, addr, cam_index, 0,
 							write_fault);
 		/* Returning 1 here also prevents sending the IV to the KFD */
 		if (ret == 1)
-- 
2.54.0

[PATCH 08/14] drm/amdgpu/gmc12: Pass cam_index to retry fault handler

[Timur Kristóf]

This is necessary if we want to make use of the filter CAM.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com>
---
 drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c
index 84c93364d220..299fe960b99e 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c
@@ -99,6 +99,7 @@ static int gmc_v12_0_process_interrupt(struct amdgpu_device *adev,
 	bool write_fault = !!(entry->src_data[1] &
 			      AMDGPU_GMC9_FAULT_SOURCE_DATA_WRITE);
 	uint32_t status = 0;
+	uint32_t cam_index;
 	u64 addr;
 
 	addr = (u64)entry->src_data[0] << 12;
@@ -110,7 +111,9 @@ static int gmc_v12_0_process_interrupt(struct amdgpu_device *adev,
 		hub = &adev->vmhub[AMDGPU_GFXHUB(0)];
 
 	if (retry_fault) {
-		int ret = amdgpu_gmc_handle_retry_fault(adev, entry, addr, 0, 0,
+		cam_index = entry->src_data[2] & 0x3ff;
+
+		int ret = amdgpu_gmc_handle_retry_fault(adev, entry, addr, cam_index, 0,
 							write_fault);
 		/* Returning 1 here also prevents sending the IV to the KFD */
 		if (ret == 1)
-- 
2.54.0

[PATCH 09/14] drm/amdgpu/gmc12: Use AMDGPU_PTE_IS_PTE flag for init_pte_flags on GFX12.0

[Timur Kristóf]

This flag seems to work around a "fault priority problem"
and is necessary for handling faults on GFX12.
The kernel seems unable to mitigate retry faults on GFX12
without this flag.

For reference see:
amdgpu_vm_pte_update_flags() that explains the problem
svm_range_get_pte_flags() that uses the flag on GFX12

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
---
 drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c
index 299fe960b99e..c210e2463a27 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c
@@ -644,11 +644,12 @@ static int gmc_v12_0_early_init(struct amdgpu_ip_block *ip_block)
 		adev->gmc.xgmi.connected_to_cpu =
 			adev->smuio.funcs->is_host_gpu_xgmi_supported(adev);
 
+	adev->gmc.init_pte_flags = AMDGPU_PTE_IS_PTE;
+
 	switch (amdgpu_ip_version(adev, GC_HWIP, 0)) {
 	case IP_VERSION(12, 1, 0):
 		gmc_v12_1_set_gmc_funcs(adev);
 		gmc_v12_1_set_irq_funcs(adev);
-		adev->gmc.init_pte_flags = AMDGPU_PTE_IS_PTE;
 		break;
 	default:
 		gmc_v12_0_set_gmc_funcs(adev);
-- 
2.54.0

[PATCH 10/14] drm/amdgpu/vm: Use init PTE flags and NOALLOC in amdgpu_vm_handle_fault()

[Timur Kristóf]

PTE_IS_PTE seems necessary for handling retry faults on GFX12.

For reference see:
amdgpu_vm_pte_update_flags() that explains the problem
svm_range_get_pte_flags() that uses the flag on GFX12

Also add NOALLOC on GFX10.3+ as we don't need to allocate
the fault handling PTE in the infinity cache (MALL).

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index 32719f31b6c9..a915d061085f 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -3044,7 +3044,8 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid,
 	}
 
 	addr /= AMDGPU_GPU_PAGE_SIZE;
-	flags = AMDGPU_PTE_VALID | AMDGPU_PTE_SNOOPED |
+	flags = adev->gmc.init_pte_flags |
+		AMDGPU_PTE_VALID | AMDGPU_PTE_SNOOPED |
 		AMDGPU_PTE_SYSTEM;
 
 	if (is_compute_context) {
@@ -3054,11 +3055,14 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid,
 		flags = AMDGPU_VM_NORETRY_FLAGS;
 		value = 0;
 	} else if (amdgpu_vm_fault_stop == AMDGPU_VM_FAULT_STOP_NEVER) {
+		/* Don't allocate this PTE in the MALL */
+		if (amdgpu_ip_version(adev, GC_HWIP, 0) >= IP_VERSION(10, 3, 0))
+			flags |= AMDGPU_PTE_NOALLOC;
+
 		/* Redirect the access to the dummy page */
 		value = adev->dummy_page_addr;
 		flags |= AMDGPU_PTE_EXECUTABLE | AMDGPU_PTE_READABLE |
 			AMDGPU_PTE_WRITEABLE;
-
 	} else {
 		/* Let the hw retry silently on the PTE */
 		value = 0;
-- 
2.54.0

[PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH 6.0

[Timur Kristóf]

The doorbell is not working on Navi 31.
Use the IH_RETRY_CAM_ACK register.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
---
 drivers/gpu/drm/amd/amdgpu/ih_v6_0.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
index 65e5d21753f9..a97c85b0cb99 100644
--- a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
@@ -307,6 +307,11 @@ static int ih_v6_0_enable_ring(struct amdgpu_device *adev,
 	return 0;
 }
 
+static void ih_v6_0_retry_cam_ack(struct amdgpu_device *adev, u32 cam_index)
+{
+	WREG32_SOC15(OSSSYS, 0, regIH_RETRY_CAM_ACK, cam_index);
+}
+
 /**
  * ih_v6_0_irq_init - init and enable the interrupt ring
  *
@@ -807,7 +812,8 @@ static const struct amdgpu_ih_funcs ih_v6_0_funcs = {
 	.get_wptr = ih_v6_0_get_wptr,
 	.decode_iv = amdgpu_ih_decode_iv_helper,
 	.decode_iv_ts = amdgpu_ih_decode_iv_ts_helper,
-	.set_rptr = ih_v6_0_set_rptr
+	.set_rptr = ih_v6_0_set_rptr,
+	.retry_cam_ack = ih_v6_0_retry_cam_ack,
 };
 
 static void ih_v6_0_set_interrupt_funcs(struct amdgpu_device *adev)
-- 
2.54.0

[PATCH 12/14] drm/amdgpu/ih7.0: Use MMIO ACK instead of doorbell for retry CAM on IH 7.0

[Timur Kristóf]

The doorbell is not working on Navi 48.
Use the IH_RETRY_CAM_ACK register just like on Navi 3x.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
---
 drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 17 +----------------
 1 file changed, 1 insertion(+), 16 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
index f1de6450a31d..291326d2ee8b 100644
--- a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
@@ -279,19 +279,9 @@ static int ih_v7_0_enable_ring(struct amdgpu_device *adev,
 	return 0;
 }
 
-static uint32_t ih_v7_0_setup_retry_doorbell(u32 doorbell_index)
-{
-	u32 val = 0;
-
-	val = REG_SET_FIELD(val, IH_DOORBELL_RPTR, OFFSET, doorbell_index);
-	val = REG_SET_FIELD(val, IH_DOORBELL_RPTR, ENABLE, 1);
-
-	return val;
-}
-
 static void ih_v7_0_retry_cam_ack(struct amdgpu_device *adev, u32 cam_index)
 {
-	WDOORBELL32(adev->irq.retry_cam_doorbell_index, cam_index);
+	WREG32_SOC15(OSSSYS, 0, regIH_RETRY_CAM_ACK, cam_index);
 }
 
 #define regIH_RING1_CLIENT_CFG_INDEX_V7_1             0x122
@@ -401,11 +391,6 @@ static int ih_v7_0_irq_init(struct amdgpu_device *adev)
 	pci_set_master(adev->pdev);
 
 	if (amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7, 1, 0)) {
-		/* Allocate the doorbell for IH Retry CAM */
-		adev->irq.retry_cam_doorbell_index = (adev->doorbell_index.ih + 2) << 1;
-		WREG32_SOC15(OSSSYS, 0, regIH_DOORBELL_RETRY_CAM,
-				ih_v7_0_setup_retry_doorbell(adev->irq.retry_cam_doorbell_index));
-
 		/* Enable IH Retry CAM */
 		tmp = RREG32_SOC15(OSSSYS, 0, regIH_RETRY_INT_CAM_CNTL);
 		tmp = REG_SET_FIELD(tmp, IH_RETRY_INT_CAM_CNTL, ENABLE, 1);
-- 
2.54.0

[PATCH 13/14] drm/amdgpu/ih6.0: Enable retry CAM on Navi 3 dGPUs

[Timur Kristóf]

The retry CAM can filter interrupts which occur repeatedly,
such as page fault interrupts when retry faults are enabled.
This makes processing those interrupts much more efficient,
because the CPU won't have to deal with processing the same
interrupt repeatedly.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
---
 drivers/gpu/drm/amd/amdgpu/ih_v6_0.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
index a97c85b0cb99..bbe399f9e233 100644
--- a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
@@ -397,6 +397,16 @@ static int ih_v6_0_irq_init(struct amdgpu_device *adev)
 
 	pci_set_master(adev->pdev);
 
+	if (!(adev->flags & AMD_IS_APU)) {
+		/* Enable IH Retry CAM */
+		tmp = RREG32_SOC15(OSSSYS, 0, regIH_RETRY_INT_CAM_CNTL);
+		tmp = REG_SET_FIELD(tmp, IH_RETRY_INT_CAM_CNTL, ENABLE, 1);
+		tmp = REG_SET_FIELD(tmp, IH_RETRY_INT_CAM_CNTL, CAM_SIZE, 0xF);
+		WREG32_SOC15(OSSSYS, 0, regIH_RETRY_INT_CAM_CNTL, tmp);
+
+		adev->irq.retry_cam_enabled = true;
+	}
+
 	/* enable interrupts */
 	ret = ih_v6_0_toggle_interrupts(adev, true);
 	if (ret)
-- 
2.54.0

[PATCH 14/14] drm/amdgpu/ih7.0: Enable retry CAM on Navi 4 dGPUs

[Timur Kristóf]

The retry CAM can filter interrupts which occur repeatedly,
such as page fault interrupts when retry faults are enabled.
This makes processing those interrupts much more efficient,
because the CPU won't have to deal with processing the same
interrupt repeatedly.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
---
 drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
index 291326d2ee8b..ec0919fa8254 100644
--- a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
@@ -390,7 +390,8 @@ static int ih_v7_0_irq_init(struct amdgpu_device *adev)
 
 	pci_set_master(adev->pdev);
 
-	if (amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7, 1, 0)) {
+	if (amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7, 1, 0) ||
+	    !(adev->flags & AMD_IS_APU)) {
 		/* Enable IH Retry CAM */
 		tmp = RREG32_SOC15(OSSSYS, 0, regIH_RETRY_INT_CAM_CNTL);
 		tmp = REG_SET_FIELD(tmp, IH_RETRY_INT_CAM_CNTL, ENABLE, 1);
-- 
2.54.0

Re: [PATCH 01/14] drm/amdgpu: Respect noretry flag for retry faults on GFX12.1

[Christian König]

On 7/1/26 18:17, Timur Kristóf wrote:
> When retry faults are disabled (amdgpu.noretry=1),
> the ENABLE_RETRY_FAULT_INTERRUPT bit should be programmed to 0.
> 
> Note that retry faults are enabled by default on GFX12.1
> so this just fixes the case when they are explicitly disabled.
> 
> Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>

Reviewed-by: Christian König <christian.koenig@amd.com>

> ---
>  drivers/gpu/drm/amd/amdgpu/gfxhub_v12_1.c | 4 ++--
>  drivers/gpu/drm/amd/amdgpu/mmhub_v4_2_0.c | 2 +-
>  2 files changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v12_1.c b/drivers/gpu/drm/amd/amdgpu/gfxhub_v12_1.c
> index 4c2fd1e6616e..0cada13d92a4 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v12_1.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v12_1.c
> @@ -243,7 +243,7 @@ static void gfxhub_v12_1_xcc_init_system_aperture_regs(struct amdgpu_device *ade
>  		tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
>  				    ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
>  		tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
> -				    ENABLE_RETRY_FAULT_INTERRUPT, 0x1);
> +				    ENABLE_RETRY_FAULT_INTERRUPT, !adev->gmc.noretry);
>  		WREG32_SOC15(GC, GET_INST(GC, i),
>  			     regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp);
>  	}
> @@ -447,7 +447,7 @@ static void gfxhub_v12_1_xcc_setup_vmid_config(struct amdgpu_device *adev,
>  			/* Send no-retry XNACK on fault to suppress VM fault storm */
>  			tmp = REG_SET_FIELD(tmp, GCVM_CONTEXT1_CNTL,
>  					    RETRY_PERMISSION_OR_INVALID_PAGE_FAULT,
> -					    1);
> +					    !adev->gmc.noretry);
>  			WREG32_SOC15_OFFSET(GC, GET_INST(GC, j), regGCVM_CONTEXT1_CNTL,
>  					    i * hub->ctx_distance, tmp);
>  			WREG32_SOC15_OFFSET(GC, GET_INST(GC, j),
> diff --git a/drivers/gpu/drm/amd/amdgpu/mmhub_v4_2_0.c b/drivers/gpu/drm/amd/amdgpu/mmhub_v4_2_0.c
> index 49b7f16a941f..f2c549737e02 100644
> --- a/drivers/gpu/drm/amd/amdgpu/mmhub_v4_2_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/mmhub_v4_2_0.c
> @@ -320,7 +320,7 @@ static void mmhub_v4_2_0_mid_init_system_aperture_regs(struct amdgpu_device *ade
>  		tmp = REG_SET_FIELD(tmp, MMVM_L2_PROTECTION_FAULT_CNTL2,
>  				    ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
>  		tmp = REG_SET_FIELD(tmp, MMVM_L2_PROTECTION_FAULT_CNTL2,
> -				    ENABLE_RETRY_FAULT_INTERRUPT, 0x1);
> +				    ENABLE_RETRY_FAULT_INTERRUPT, !adev->gmc.noretry);
>  		WREG32_SOC15(MMHUB, GET_INST(MMHUB, i),
>  			     regMMVM_L2_PROTECTION_FAULT_CNTL2, tmp);
>  	}

Re: [PATCH 02/14] drm/amdgpu/gfxhub: Enable retry fault interrupts when needed

[Christian König]

On 7/1/26 18:17, Timur Kristóf wrote:
> Enable retry fault interrupts when initializing the GFXHUB
> system aperture registers according to whether retrying
> page faults is enabled in amdgpu (ie. amdgpu.noretry=0).
> 
> Needs to be done for each GFXHUB version at once,
> because none of them actually enabled this interrupt.

Thinking more about it we are clearly missing something here. The retry fault interrupt itself should be enabled all the time.

IIRC only the RETRY_PERMISSION_OR_INVALID_PAGE_FAULT bit in the VM_CONTEXT0_CNTL register should be set or cleared by the kernel driver or firmware to control if the HW retries the access or not.

Regards,
Christian.

> 
> Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
> ---
>  drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c | 9 +++++++--
>  drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c   | 9 +++++++--
>  drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c    | 9 +++++++--
>  drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c    | 2 ++
>  drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c    | 9 +++++++--
>  drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c    | 9 +++++++--
>  drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c    | 9 +++++++--
>  drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c  | 9 +++++++--
>  8 files changed, 51 insertions(+), 14 deletions(-)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c b/drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c
> index 652eea6eae4a..ef20eafd59ae 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c
> @@ -155,6 +155,7 @@ static void gfxhub_v11_5_0_init_gart_aperture_regs(struct amdgpu_device *adev)
>  static void gfxhub_v11_5_0_init_system_aperture_regs(struct amdgpu_device *adev)
>  {
>  	uint64_t value;
> +	u32 tmp;
>  
>  	WREG32_SOC15(GC, 0, regGCMC_VM_AGP_BASE, 0);
>  	WREG32_SOC15(GC, 0, regGCMC_VM_AGP_BOT, adev->gmc.agp_start >> 24);
> @@ -180,8 +181,12 @@ static void gfxhub_v11_5_0_init_system_aperture_regs(struct amdgpu_device *adev)
>  	WREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
>  		     (u32)((u64)adev->dummy_page_addr >> 44));
>  
> -	WREG32_FIELD15_PREREG(GC, 0, GCVM_L2_PROTECTION_FAULT_CNTL2,
> -		       ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
> +	tmp = RREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_CNTL2);
> +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
> +				 ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
> +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
> +				 ENABLE_RETRY_FAULT_INTERRUPT, !adev->gmc.noretry);
> +	WREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp);
>  }
>  
>  static void gfxhub_v11_5_0_init_tlb_regs(struct amdgpu_device *adev)
> diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c b/drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c
> index 6cbf837d50dd..ec3ff4dec674 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c
> @@ -158,6 +158,7 @@ static void gfxhub_v12_0_init_gart_aperture_regs(struct amdgpu_device *adev)
>  static void gfxhub_v12_0_init_system_aperture_regs(struct amdgpu_device *adev)
>  {
>  	uint64_t value;
> +	u32 tmp;
>  
>  	/* Program the AGP BAR */
>  	WREG32_SOC15(GC, 0, regGCMC_VM_AGP_BASE, 0);
> @@ -184,8 +185,12 @@ static void gfxhub_v12_0_init_system_aperture_regs(struct amdgpu_device *adev)
>  	WREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
>  		     (u32)((u64)adev->dummy_page_addr >> 44));
>  
> -	WREG32_FIELD15_PREREG(GC, 0, GCVM_L2_PROTECTION_FAULT_CNTL2,
> -		       ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
> +	tmp = RREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_CNTL2);
> +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
> +				 ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
> +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
> +				 ENABLE_RETRY_FAULT_INTERRUPT, !adev->gmc.noretry);
> +	WREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp);
>  }
>  
>  
> diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c b/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c
> index bfe247b1a333..27d7f7cb903f 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c
> @@ -91,6 +91,7 @@ static void gfxhub_v1_0_init_gart_aperture_regs(struct amdgpu_device *adev)
>  static void gfxhub_v1_0_init_system_aperture_regs(struct amdgpu_device *adev)
>  {
>  	uint64_t value;
> +	u32 tmp;
>  
>  	if (!amdgpu_sriov_vf(adev) || adev->asic_type <= CHIP_VEGA10) {
>  		/* Program the AGP BAR */
> @@ -134,8 +135,12 @@ static void gfxhub_v1_0_init_system_aperture_regs(struct amdgpu_device *adev)
>  		WREG32_SOC15(GC, 0, mmVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
>  			     (u32)((u64)adev->dummy_page_addr >> 44));
>  
> -		WREG32_FIELD15(GC, 0, VM_L2_PROTECTION_FAULT_CNTL2,
> -			       ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
> +		tmp = RREG32_SOC15(GC, 0, mmVM_L2_PROTECTION_FAULT_CNTL2);
> +		tmp = REG_SET_FIELD(tmp, VM_L2_PROTECTION_FAULT_CNTL2,
> +					 ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
> +		tmp = REG_SET_FIELD(tmp, VM_L2_PROTECTION_FAULT_CNTL2,
> +					 ENABLE_RETRY_FAULT_INTERRUPT, !adev->gmc.noretry);
> +		WREG32_SOC15(GC, 0, mmVM_L2_PROTECTION_FAULT_CNTL2, tmp);
>  	}
>  
>  	/* In the case squeezing vram into GART aperture, we don't use
> diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c b/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c
> index fbdf46070b38..ed9a64bc5aaa 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c
> @@ -176,6 +176,8 @@ gfxhub_v1_2_xcc_init_system_aperture_regs(struct amdgpu_device *adev,
>  			tmp = RREG32_SOC15(GC, GET_INST(GC, i), regVM_L2_PROTECTION_FAULT_CNTL2);
>  			tmp = REG_SET_FIELD(tmp, VM_L2_PROTECTION_FAULT_CNTL2,
>  					    ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
> +			tmp = REG_SET_FIELD(tmp, VM_L2_PROTECTION_FAULT_CNTL2,
> +					    ENABLE_RETRY_FAULT_INTERRUPT, !adev->gmc.noretry);
>  			WREG32_SOC15(GC, GET_INST(GC, i), regVM_L2_PROTECTION_FAULT_CNTL2, tmp);
>  		}
>  
> diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c b/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c
> index 9ea593e2c719..152b2735d360 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c
> @@ -151,6 +151,7 @@ static void gfxhub_v2_0_init_gart_aperture_regs(struct amdgpu_device *adev)
>  static void gfxhub_v2_0_init_system_aperture_regs(struct amdgpu_device *adev)
>  {
>  	uint64_t value;
> +	u32 tmp;
>  
>  	if (!amdgpu_sriov_vf(adev)) {
>  		/* Program the AGP BAR */
> @@ -178,8 +179,12 @@ static void gfxhub_v2_0_init_system_aperture_regs(struct amdgpu_device *adev)
>  	WREG32_SOC15(GC, 0, mmGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
>  		     (u32)((u64)adev->dummy_page_addr >> 44));
>  
> -	WREG32_FIELD15(GC, 0, GCVM_L2_PROTECTION_FAULT_CNTL2,
> -		       ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
> +	tmp = RREG32_SOC15(GC, 0, mmGCVM_L2_PROTECTION_FAULT_CNTL2);
> +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
> +				 ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
> +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
> +				 ENABLE_RETRY_FAULT_INTERRUPT, !adev->gmc.noretry);
> +	WREG32_SOC15(GC, 0, mmGCVM_L2_PROTECTION_FAULT_CNTL2, tmp);
>  }
>  
>  
> diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c b/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c
> index 30b90d35abd0..83c2ddbbd292 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c
> @@ -154,6 +154,7 @@ static void gfxhub_v2_1_init_gart_aperture_regs(struct amdgpu_device *adev)
>  static void gfxhub_v2_1_init_system_aperture_regs(struct amdgpu_device *adev)
>  {
>  	uint64_t value;
> +	u32 tmp;
>  
>  	if (amdgpu_sriov_vf(adev))
>  		return;
> @@ -182,8 +183,12 @@ static void gfxhub_v2_1_init_system_aperture_regs(struct amdgpu_device *adev)
>  	WREG32_SOC15(GC, 0, mmGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
>  		     (u32)((u64)adev->dummy_page_addr >> 44));
>  
> -	WREG32_FIELD15(GC, 0, GCVM_L2_PROTECTION_FAULT_CNTL2,
> -		       ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
> +	tmp = RREG32_SOC15(GC, 0, mmGCVM_L2_PROTECTION_FAULT_CNTL2);
> +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
> +				 ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
> +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
> +				 ENABLE_RETRY_FAULT_INTERRUPT, !adev->gmc.noretry);
> +	WREG32_SOC15(GC, 0, mmGCVM_L2_PROTECTION_FAULT_CNTL2, tmp);
>  }
>  
>  
> diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c b/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c
> index 9e6a6e13dec0..90bbb2fe4884 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c
> @@ -150,6 +150,7 @@ static void gfxhub_v3_0_init_gart_aperture_regs(struct amdgpu_device *adev)
>  static void gfxhub_v3_0_init_system_aperture_regs(struct amdgpu_device *adev)
>  {
>  	uint64_t value;
> +	u32 tmp;
>  
>  	/* Program the AGP BAR */
>  	WREG32_SOC15(GC, 0, regGCMC_VM_AGP_BASE, 0);
> @@ -176,8 +177,12 @@ static void gfxhub_v3_0_init_system_aperture_regs(struct amdgpu_device *adev)
>  	WREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
>  		     (u32)((u64)adev->dummy_page_addr >> 44));
>  
> -	WREG32_FIELD15_PREREG(GC, 0, GCVM_L2_PROTECTION_FAULT_CNTL2,
> -		       ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
> +	tmp = RREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_CNTL2);
> +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
> +				 ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
> +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
> +				 ENABLE_RETRY_FAULT_INTERRUPT, !adev->gmc.noretry);
> +	WREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp);
>  }
>  
>  
> diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c b/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c
> index b3b1085c7cd3..1b3c067ab48c 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c
> @@ -153,6 +153,7 @@ static void gfxhub_v3_0_3_init_gart_aperture_regs(struct amdgpu_device *adev)
>  static void gfxhub_v3_0_3_init_system_aperture_regs(struct amdgpu_device *adev)
>  {
>  	uint64_t value;
> +	u32 tmp;
>  
>  	if (amdgpu_sriov_vf(adev))
>  		return;
> @@ -181,8 +182,12 @@ static void gfxhub_v3_0_3_init_system_aperture_regs(struct amdgpu_device *adev)
>  	WREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
>  		     (u32)((u64)adev->dummy_page_addr >> 44));
>  
> -	WREG32_FIELD15_PREREG(GC, 0, GCVM_L2_PROTECTION_FAULT_CNTL2,
> -		       ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
> +	tmp = RREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_CNTL2);
> +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
> +				 ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
> +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
> +				 ENABLE_RETRY_FAULT_INTERRUPT, !adev->gmc.noretry);
> +	WREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp);
>  }
>  
>  

Re: [PATCH 03/14] drm/amdgpu/ih: Don't perturb HW registers when accessing soft IH ring

[Christian König]

On 7/1/26 18:17, Timur Kristóf wrote:
> The soft IH ring is implemented entirely in software.
> We shouldn't read (or write) any HW registers when accessing it.
> 
> Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
> Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com>

Reviewed-by: Christian König <christian.koenig@amd.com>

> ---
>  drivers/gpu/drm/amd/amdgpu/ih_v6_0.c   | 7 +++++++
>  drivers/gpu/drm/amd/amdgpu/ih_v6_1.c   | 7 +++++++
>  drivers/gpu/drm/amd/amdgpu/ih_v7_0.c   | 7 +++++++
>  drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 4 ++++
>  4 files changed, 25 insertions(+)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
> index 333e9c30c091..65e5d21753f9 100644
> --- a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
> @@ -439,6 +439,10 @@ static u32 ih_v6_0_get_wptr(struct amdgpu_device *adev,
>  	struct amdgpu_ih_regs *ih_regs;
>  
>  	wptr = le32_to_cpu(*ih->wptr_cpu);
> +
> +	if (ih == &adev->irq.ih_soft)
> +		goto out;
> +
>  	ih_regs = &ih->ih_regs;
>  
>  	if (!REG_GET_FIELD(wptr, IH_RB_WPTR, RB_OVERFLOW))
> @@ -514,6 +518,9 @@ static void ih_v6_0_set_rptr(struct amdgpu_device *adev,
>  {
>  	struct amdgpu_ih_regs *ih_regs;
>  
> +	if (ih == &adev->irq.ih_soft)
> +		return;
> +
>  	if (ih->use_doorbell) {
>  		/* XXX check if swapping is necessary on BE */
>  		*ih->rptr_cpu = ih->rptr;
> diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v6_1.c b/drivers/gpu/drm/amd/amdgpu/ih_v6_1.c
> index 699c274d357e..9dbc20131410 100644
> --- a/drivers/gpu/drm/amd/amdgpu/ih_v6_1.c
> +++ b/drivers/gpu/drm/amd/amdgpu/ih_v6_1.c
> @@ -410,6 +410,10 @@ static u32 ih_v6_1_get_wptr(struct amdgpu_device *adev,
>  	struct amdgpu_ih_regs *ih_regs;
>  
>  	wptr = le32_to_cpu(*ih->wptr_cpu);
> +
> +	if (ih == &adev->irq.ih_soft)
> +		goto out;
> +
>  	ih_regs = &ih->ih_regs;
>  
>  	if (!REG_GET_FIELD(wptr, IH_RB_WPTR, RB_OVERFLOW))
> @@ -481,6 +485,9 @@ static void ih_v6_1_irq_rearm(struct amdgpu_device *adev,
>  static void ih_v6_1_set_rptr(struct amdgpu_device *adev,
>  			       struct amdgpu_ih_ring *ih)
>  {
> +	if (ih == &adev->irq.ih_soft)
> +		return;
> +
>  	struct amdgpu_ih_regs *ih_regs;
>  
>  	if (ih->use_doorbell) {
> diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> index 6de9e87e04e1..bd332e8cc5bf 100644
> --- a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> @@ -457,6 +457,10 @@ static u32 ih_v7_0_get_wptr(struct amdgpu_device *adev,
>  	struct amdgpu_ih_regs *ih_regs;
>  
>  	wptr = le32_to_cpu(*ih->wptr_cpu);
> +
> +	if (ih == &adev->irq.ih_soft)
> +		goto out;
> +
>  	ih_regs = &ih->ih_regs;
>  
>  	if (!REG_GET_FIELD(wptr, IH_RB_WPTR, RB_OVERFLOW))
> @@ -527,6 +531,9 @@ static void ih_v7_0_set_rptr(struct amdgpu_device *adev,
>  {
>  	struct amdgpu_ih_regs *ih_regs;
>  
> +	if (ih == &adev->irq.ih_soft)
> +		return;
> +
>  	if (ih->use_doorbell) {
>  		/* XXX check if swapping is necessary on BE */
>  		*ih->rptr_cpu = ih->rptr;
> diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
> index 4cd325149b63..e7ed37bb48e0 100644
> --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
> @@ -417,6 +417,10 @@ static u32 navi10_ih_get_wptr(struct amdgpu_device *adev,
>  		 */
>  		wptr = le32_to_cpu(*ih->wptr_cpu);
>  
> +		if (ih == &adev->irq.ih_soft)
> +			goto out;
> +
> +
>  		if (!REG_GET_FIELD(wptr, IH_RB_WPTR, RB_OVERFLOW))
>  			goto out;
>  	}

Re: [PATCH 04/14] drm/amdgpu/ih: Add retry_cam_ack IH function pointer

[Christian König]

On 7/1/26 18:17, Timur Kristóf wrote:
> No functional changes.
> 
> This is a refactor to allow different filter CAM implementation
> in subsequent commits. The actual new implementations are
> going to be in subsequent commits.
> 
> Instead of writing the doorbell in amdgpu_gmc_handle_retry_fault()
> directly, add an IH function pointer which can be defined in
> a different way for different IH versions.
> 
> Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>

Reviewed-by: Christian König <christian.koenig@amd.com>

> ---
>  drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 7 ++++++-
>  drivers/gpu/drm/amd/amdgpu/amdgpu_ih.h  | 1 +
>  drivers/gpu/drm/amd/amdgpu/ih_v7_0.c    | 6 ++++++
>  drivers/gpu/drm/amd/amdgpu/vega20_ih.c  | 8 +++++++-
>  4 files changed, 20 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
> index 3f0b1b7a557b..bb278a61dc9e 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
> @@ -552,6 +552,11 @@ int amdgpu_gmc_handle_retry_fault(struct amdgpu_device *adev,
>  	int ret;
>  
>  	if (adev->irq.retry_cam_enabled) {
> +		if (!adev->irq.ih_funcs->retry_cam_ack) {
> +			dev_warn(adev->dev, "retry CAM is enabled, but retry_cam_ack is NULL\n");
> +			return -EOPNOTSUPP;
> +		}
> +
>  		/* Delegate it to a different ring if the hardware hasn't
>  		 * already done it.
>  		 */
> @@ -562,7 +567,7 @@ int amdgpu_gmc_handle_retry_fault(struct amdgpu_device *adev,
>  
>  		ret = amdgpu_vm_handle_fault(adev, entry->pasid, entry->vmid, node_id,
>  					     addr, entry->timestamp, write_fault);
> -		WDOORBELL32(adev->irq.retry_cam_doorbell_index, cam_index);
> +		adev->irq.ih_funcs->retry_cam_ack(adev, cam_index);
>  		if (ret)
>  			return 1;
>  	} else {
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.h
> index 444437c30088..e6e34f6e86f4 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.h
> @@ -97,6 +97,7 @@ struct amdgpu_ih_funcs {
>  	const char *(*node_id_to_die_name)(struct amdgpu_device *adev,
>  					   unsigned int node_id,
>  					   char *buf, size_t size);
> +	void (*retry_cam_ack)(struct amdgpu_device *adev, u32 cam_index);
>  };
>  
>  #define amdgpu_ih_get_wptr(adev, ih) (adev)->irq.ih_funcs->get_wptr((adev), (ih))
> diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> index bd332e8cc5bf..24be9d726428 100644
> --- a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> @@ -289,6 +289,11 @@ static uint32_t ih_v7_0_setup_retry_doorbell(u32 doorbell_index)
>  	return val;
>  }
>  
> +static void ih_v7_0_retry_cam_ack(struct amdgpu_device *adev, u32 cam_index)
> +{
> +	WDOORBELL32(adev->irq.retry_cam_doorbell_index, cam_index);
> +}
> +
>  #define regIH_RING1_CLIENT_CFG_INDEX_V7_1             0x122
>  #define regIH_RING1_CLIENT_CFG_INDEX_V7_1_BASE_IDX    0
>  #define regIH_RING1_CLIENT_CFG_DATA_V7_1              0x123
> @@ -865,6 +870,7 @@ static const struct amdgpu_ih_funcs ih_v7_0_funcs = {
>  	.decode_iv_ts = amdgpu_ih_decode_iv_ts_helper,
>  	.set_rptr = ih_v7_0_set_rptr,
>  	.node_id_to_die_name = ih_v7_0_node_id_to_die_name,
> +	.retry_cam_ack = ih_v7_0_retry_cam_ack,
>  };
>  
>  static void ih_v7_0_set_interrupt_funcs(struct amdgpu_device *adev)
> diff --git a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c
> index 85846fd08ce4..30a82fff3ff7 100644
> --- a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c
> @@ -293,6 +293,11 @@ static uint32_t vega20_setup_retry_doorbell(u32 doorbell_index)
>  	return val;
>  }
>  
> +static void vega20_retry_cam_ack(struct amdgpu_device *adev, u32 cam_index)
> +{
> +	WDOORBELL32(adev->irq.retry_cam_doorbell_index, cam_index);
> +}
> +
>  /**
>   * vega20_ih_irq_init - init and enable the interrupt ring
>   *
> @@ -738,7 +743,8 @@ static const struct amdgpu_ih_funcs vega20_ih_funcs = {
>  	.get_wptr = vega20_ih_get_wptr,
>  	.decode_iv = amdgpu_ih_decode_iv_helper,
>  	.decode_iv_ts = amdgpu_ih_decode_iv_ts_helper,
> -	.set_rptr = vega20_ih_set_rptr
> +	.set_rptr = vega20_ih_set_rptr,
> +	.retry_cam_ack = vega20_retry_cam_ack,
>  };
>  
>  static void vega20_ih_set_interrupt_funcs(struct amdgpu_device *adev)

Re: [PATCH 05/14] drm/amdgpu/ih6.1: Use IH_SW_RING_SIZE for soft IH ring instead of PAGE_SIZE

[Christian König]

On 7/1/26 18:17, Timur Kristóf wrote:
> When there are a lot of retry faults happening, the soft IH ring
> can fill up really quickly and possibly overflow. PAGE_SIZE was
> too small, use IH_SW_RING_SIZE to match what other GPU generations
> are doing.
> 
> Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
> Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com>

Reviewed-by: Christian König <christian.koenig@amd.com>

> ---
>  drivers/gpu/drm/amd/amdgpu/ih_v6_1.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v6_1.c b/drivers/gpu/drm/amd/amdgpu/ih_v6_1.c
> index 9dbc20131410..f6fdb3a950e0 100644
> --- a/drivers/gpu/drm/amd/amdgpu/ih_v6_1.c
> +++ b/drivers/gpu/drm/amd/amdgpu/ih_v6_1.c
> @@ -590,7 +590,7 @@ static int ih_v6_1_sw_init(struct amdgpu_ip_block *ip_block)
>  	/* initialize ih control register offset */
>  	ih_v6_1_init_register_offset(adev);
>  
> -	r = amdgpu_ih_ring_init(adev, &adev->irq.ih_soft, PAGE_SIZE, true);
> +	r = amdgpu_ih_ring_init(adev, &adev->irq.ih_soft, IH_SW_RING_SIZE, true);
>  	if (r)
>  		return r;
>  

Re: [PATCH 02/14] drm/amdgpu/gfxhub: Enable retry fault interrupts when needed

[Timur Kristóf]

On 2026. július 2., csütörtök 10:10:10 közép-európai nyári idő Christian König 
wrote:
> On 7/1/26 18:17, Timur Kristóf wrote:
> > Enable retry fault interrupts when initializing the GFXHUB
> > system aperture registers according to whether retrying
> > page faults is enabled in amdgpu (ie. amdgpu.noretry=0).
> > 
> > Needs to be done for each GFXHUB version at once,
> > because none of them actually enabled this interrupt.
> 
> Thinking more about it we are clearly missing something here. The retry
> fault interrupt itself should be enabled all the time.

Why would it be enabled all the time?
I haven't seen any retry faults on neither Navi 3 nor Navi 4 without enabling 
the ENABLE_RETRY_FAULT_INTERRUPT bit.

> 
> IIRC only the RETRY_PERMISSION_OR_INVALID_PAGE_FAULT bit in the
> VM_CONTEXT0_CNTL register should be set or cleared by the kernel driver or
> firmware to control if the HW retries the access or not.

That is clearly not the case on GFX12.1 and I haven't seen any indication that 
it would be different on GFX11.x and 12.0 either.


> > ---
> > 
> >  drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c | 9 +++++++--
> >  drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c   | 9 +++++++--
> >  drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c    | 9 +++++++--
> >  drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c    | 2 ++
> >  drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c    | 9 +++++++--
> >  drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c    | 9 +++++++--
> >  drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c    | 9 +++++++--
> >  drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c  | 9 +++++++--
> >  8 files changed, 51 insertions(+), 14 deletions(-)
> > 
> > diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c
> > b/drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c index
> > 652eea6eae4a..ef20eafd59ae 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c
> > @@ -155,6 +155,7 @@ static void
> > gfxhub_v11_5_0_init_gart_aperture_regs(struct amdgpu_device *adev)> 
> >  static void gfxhub_v11_5_0_init_system_aperture_regs(struct amdgpu_device
> >  *adev) {
> >  
> >  	uint64_t value;
> > 
> > +	u32 tmp;
> > 
> >  	WREG32_SOC15(GC, 0, regGCMC_VM_AGP_BASE, 0);
> >  	WREG32_SOC15(GC, 0, regGCMC_VM_AGP_BOT, adev->gmc.agp_start >> 
24);
> > 
> > @@ -180,8 +181,12 @@ static void
> > gfxhub_v11_5_0_init_system_aperture_regs(struct amdgpu_device *adev)> 
> >  	WREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
> >  	
> >  		     (u32)((u64)adev->dummy_page_addr >> 44));
> > 
> > -	WREG32_FIELD15_PREREG(GC, 0, GCVM_L2_PROTECTION_FAULT_CNTL2,
> > -		       ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
> > +	tmp = RREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_CNTL2);
> > +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
> > +				 
ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
> > +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
> > +				 ENABLE_RETRY_FAULT_INTERRUPT, !
adev->gmc.noretry);
> > +	WREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp);
> > 
> >  }
> >  
> >  static void gfxhub_v11_5_0_init_tlb_regs(struct amdgpu_device *adev)
> > 
> > diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c
> > b/drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c index
> > 6cbf837d50dd..ec3ff4dec674 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c
> > @@ -158,6 +158,7 @@ static void
> > gfxhub_v12_0_init_gart_aperture_regs(struct amdgpu_device *adev)> 
> >  static void gfxhub_v12_0_init_system_aperture_regs(struct amdgpu_device
> >  *adev) {
> >  
> >  	uint64_t value;
> > 
> > +	u32 tmp;
> > 
> >  	/* Program the AGP BAR */
> >  	WREG32_SOC15(GC, 0, regGCMC_VM_AGP_BASE, 0);
> > 
> > @@ -184,8 +185,12 @@ static void
> > gfxhub_v12_0_init_system_aperture_regs(struct amdgpu_device *adev)> 
> >  	WREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
> >  	
> >  		     (u32)((u64)adev->dummy_page_addr >> 44));
> > 
> > -	WREG32_FIELD15_PREREG(GC, 0, GCVM_L2_PROTECTION_FAULT_CNTL2,
> > -		       ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
> > +	tmp = RREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_CNTL2);
> > +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
> > +				 
ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
> > +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
> > +				 ENABLE_RETRY_FAULT_INTERRUPT, !
adev->gmc.noretry);
> > +	WREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp);
> > 
> >  }
> > 
> > diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c
> > b/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c index
> > bfe247b1a333..27d7f7cb903f 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c
> > @@ -91,6 +91,7 @@ static void gfxhub_v1_0_init_gart_aperture_regs(struct
> > amdgpu_device *adev)> 
> >  static void gfxhub_v1_0_init_system_aperture_regs(struct amdgpu_device
> >  *adev) {
> >  
> >  	uint64_t value;
> > 
> > +	u32 tmp;
> > 
> >  	if (!amdgpu_sriov_vf(adev) || adev->asic_type <= CHIP_VEGA10) {
> >  	
> >  		/* Program the AGP BAR */
> > 
> > @@ -134,8 +135,12 @@ static void
> > gfxhub_v1_0_init_system_aperture_regs(struct amdgpu_device *adev)> 
> >  		WREG32_SOC15(GC, 0, 
mmVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
> >  		
> >  			     (u32)((u64)adev->dummy_page_addr >> 
44));
> > 
> > -		WREG32_FIELD15(GC, 0, VM_L2_PROTECTION_FAULT_CNTL2,
> > -			       ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 
1);
> > +		tmp = RREG32_SOC15(GC, 0, 
mmVM_L2_PROTECTION_FAULT_CNTL2);
> > +		tmp = REG_SET_FIELD(tmp, VM_L2_PROTECTION_FAULT_CNTL2,
> > +					 
ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
> > +		tmp = REG_SET_FIELD(tmp, VM_L2_PROTECTION_FAULT_CNTL2,
> > +					 
ENABLE_RETRY_FAULT_INTERRUPT, !adev->gmc.noretry);
> > +		WREG32_SOC15(GC, 0, mmVM_L2_PROTECTION_FAULT_CNTL2, 
tmp);
> > 
> >  	}
> >  	
> >  	/* In the case squeezing vram into GART aperture, we don't use
> > 
> > diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c
> > b/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c index
> > fbdf46070b38..ed9a64bc5aaa 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c
> > @@ -176,6 +176,8 @@ gfxhub_v1_2_xcc_init_system_aperture_regs(struct
> > amdgpu_device *adev,> 
> >  			tmp = RREG32_SOC15(GC, GET_INST(GC, i),
> >  			regVM_L2_PROTECTION_FAULT_CNTL2);
> >  			tmp = REG_SET_FIELD(tmp, 
VM_L2_PROTECTION_FAULT_CNTL2,
> >  			
> >  					    
ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
> > 
> > +			tmp = REG_SET_FIELD(tmp, 
VM_L2_PROTECTION_FAULT_CNTL2,
> > +					    
ENABLE_RETRY_FAULT_INTERRUPT, !adev->gmc.noretry);
> > 
> >  			WREG32_SOC15(GC, GET_INST(GC, i), 
regVM_L2_PROTECTION_FAULT_CNTL2,
> >  			tmp);
> >  		
> >  		}
> > 
> > diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c
> > b/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c index
> > 9ea593e2c719..152b2735d360 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c
> > @@ -151,6 +151,7 @@ static void gfxhub_v2_0_init_gart_aperture_regs(struct
> > amdgpu_device *adev)> 
> >  static void gfxhub_v2_0_init_system_aperture_regs(struct amdgpu_device
> >  *adev) {
> >  
> >  	uint64_t value;
> > 
> > +	u32 tmp;
> > 
> >  	if (!amdgpu_sriov_vf(adev)) {
> >  	
> >  		/* Program the AGP BAR */
> > 
> > @@ -178,8 +179,12 @@ static void
> > gfxhub_v2_0_init_system_aperture_regs(struct amdgpu_device *adev)> 
> >  	WREG32_SOC15(GC, 0, mmGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
> >  	
> >  		     (u32)((u64)adev->dummy_page_addr >> 44));
> > 
> > -	WREG32_FIELD15(GC, 0, GCVM_L2_PROTECTION_FAULT_CNTL2,
> > -		       ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
> > +	tmp = RREG32_SOC15(GC, 0, mmGCVM_L2_PROTECTION_FAULT_CNTL2);
> > +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
> > +				 
ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
> > +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
> > +				 ENABLE_RETRY_FAULT_INTERRUPT, !
adev->gmc.noretry);
> > +	WREG32_SOC15(GC, 0, mmGCVM_L2_PROTECTION_FAULT_CNTL2, tmp);
> > 
> >  }
> > 
> > diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c
> > b/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c index
> > 30b90d35abd0..83c2ddbbd292 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c
> > @@ -154,6 +154,7 @@ static void gfxhub_v2_1_init_gart_aperture_regs(struct
> > amdgpu_device *adev)> 
> >  static void gfxhub_v2_1_init_system_aperture_regs(struct amdgpu_device
> >  *adev) {
> >  
> >  	uint64_t value;
> > 
> > +	u32 tmp;
> > 
> >  	if (amdgpu_sriov_vf(adev))
> >  	
> >  		return;
> > 
> > @@ -182,8 +183,12 @@ static void
> > gfxhub_v2_1_init_system_aperture_regs(struct amdgpu_device *adev)> 
> >  	WREG32_SOC15(GC, 0, mmGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
> >  	
> >  		     (u32)((u64)adev->dummy_page_addr >> 44));
> > 
> > -	WREG32_FIELD15(GC, 0, GCVM_L2_PROTECTION_FAULT_CNTL2,
> > -		       ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
> > +	tmp = RREG32_SOC15(GC, 0, mmGCVM_L2_PROTECTION_FAULT_CNTL2);
> > +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
> > +				 
ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
> > +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
> > +				 ENABLE_RETRY_FAULT_INTERRUPT, !
adev->gmc.noretry);
> > +	WREG32_SOC15(GC, 0, mmGCVM_L2_PROTECTION_FAULT_CNTL2, tmp);
> > 
> >  }
> > 
> > diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c
> > b/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c index
> > 9e6a6e13dec0..90bbb2fe4884 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c
> > @@ -150,6 +150,7 @@ static void gfxhub_v3_0_init_gart_aperture_regs(struct
> > amdgpu_device *adev)> 
> >  static void gfxhub_v3_0_init_system_aperture_regs(struct amdgpu_device
> >  *adev) {
> >  
> >  	uint64_t value;
> > 
> > +	u32 tmp;
> > 
> >  	/* Program the AGP BAR */
> >  	WREG32_SOC15(GC, 0, regGCMC_VM_AGP_BASE, 0);
> > 
> > @@ -176,8 +177,12 @@ static void
> > gfxhub_v3_0_init_system_aperture_regs(struct amdgpu_device *adev)> 
> >  	WREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
> >  	
> >  		     (u32)((u64)adev->dummy_page_addr >> 44));
> > 
> > -	WREG32_FIELD15_PREREG(GC, 0, GCVM_L2_PROTECTION_FAULT_CNTL2,
> > -		       ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
> > +	tmp = RREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_CNTL2);
> > +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
> > +				 
ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
> > +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
> > +				 ENABLE_RETRY_FAULT_INTERRUPT, !
adev->gmc.noretry);
> > +	WREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp);
> > 
> >  }
> > 
> > diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c
> > b/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c index
> > b3b1085c7cd3..1b3c067ab48c 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c
> > @@ -153,6 +153,7 @@ static void
> > gfxhub_v3_0_3_init_gart_aperture_regs(struct amdgpu_device *adev)> 
> >  static void gfxhub_v3_0_3_init_system_aperture_regs(struct amdgpu_device
> >  *adev) {
> >  
> >  	uint64_t value;
> > 
> > +	u32 tmp;
> > 
> >  	if (amdgpu_sriov_vf(adev))
> >  	
> >  		return;
> > 
> > @@ -181,8 +182,12 @@ static void
> > gfxhub_v3_0_3_init_system_aperture_regs(struct amdgpu_device *adev)> 
> >  	WREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
> >  	
> >  		     (u32)((u64)adev->dummy_page_addr >> 44));
> > 
> > -	WREG32_FIELD15_PREREG(GC, 0, GCVM_L2_PROTECTION_FAULT_CNTL2,
> > -		       ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
> > +	tmp = RREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_CNTL2);
> > +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
> > +				 
ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
> > +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
> > +				 ENABLE_RETRY_FAULT_INTERRUPT, !
adev->gmc.noretry);
> > +	WREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp);
> > 
> >  }

Re: [PATCH 06/14] drm/amdgpu/ih7.0: Use IH_SW_RING_SIZE for soft IH ring instead of PAGE_SIZE

[Christian König]

On 7/1/26 18:17, Timur Kristóf wrote:
> When there are a lot of retry faults happening, the soft IH ring
> can fill up really quickly and possibly overflow. PAGE_SIZE was
> too small, use IH_SW_RING_SIZE to match what other GPU generations
> are doing.
> 
> Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
> Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com>

Reviewed-by: Christian König <christian.koenig@amd.com>

> ---
>  drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 5 +----
>  1 file changed, 1 insertion(+), 4 deletions(-)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> index 24be9d726428..f1de6450a31d 100644
> --- a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> @@ -601,7 +601,6 @@ static int ih_v7_0_sw_init(struct amdgpu_ip_block *ip_block)
>  	int r;
>  	struct amdgpu_device *adev = ip_block->adev;
>  	bool use_bus_addr;
> -	unsigned int sw_ring_size;
>  
>  	r = amdgpu_irq_add_id(adev, SOC21_IH_CLIENTID_IH, 0,
>  			      &adev->irq.self_irq);
> @@ -633,9 +632,7 @@ static int ih_v7_0_sw_init(struct amdgpu_ip_block *ip_block)
>  	/* initialize ih control register offset */
>  	ih_v7_0_init_register_offset(adev);
>  
> -	sw_ring_size = (amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7, 1, 0)) ?
> -			IH_SW_RING_SIZE : PAGE_SIZE;
> -	r = amdgpu_ih_ring_init(adev, &adev->irq.ih_soft, sw_ring_size, true);
> +	r = amdgpu_ih_ring_init(adev, &adev->irq.ih_soft, IH_SW_RING_SIZE, true);
>  	if (r)
>  		return r;
>  

Re: [PATCH 07/14] drm/amdgpu/gmc11: Pass cam_index to retry fault handler

[Christian König]

On 7/1/26 18:17, Timur Kristóf wrote:
> This is necessary if we want to make use of the filter CAM.
> 
> Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
> ---
>  drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c
> index c40d9c467204..18a53b9df77f 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v11_0.c
> @@ -108,13 +108,16 @@ static int gmc_v11_0_process_interrupt(struct amdgpu_device *adev,
>  	bool write_fault = !!(entry->src_data[1] &
>  			      AMDGPU_GMC9_FAULT_SOURCE_DATA_WRITE);
>  	uint32_t status = 0;
> +	uint32_t cam_index;
>  	u64 addr;

Please declare "int ret;" here instead.

Apart from that looks good to me,
Christian.

>  
>  	addr = (u64)entry->src_data[0] << 12;
>  	addr |= ((u64)entry->src_data[1] & 0xf) << 44;
>  
>  	if (retry_fault) {
> -		int ret = amdgpu_gmc_handle_retry_fault(adev, entry, addr, 0, 0,
> +		cam_index = entry->src_data[2] & 0x3ff;
> +
> +		int ret = amdgpu_gmc_handle_retry_fault(adev, entry, addr, cam_index, 0,
>  							write_fault);
>  		/* Returning 1 here also prevents sending the IV to the KFD */
>  		if (ret == 1)

Re: [PATCH 08/14] drm/amdgpu/gmc12: Pass cam_index to retry fault handler

[Christian König]

On 7/1/26 18:17, Timur Kristóf wrote:
> This is necessary if we want to make use of the filter CAM.
> 
> Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
> Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com>
> ---
>  drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c
> index 84c93364d220..299fe960b99e 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c
> @@ -99,6 +99,7 @@ static int gmc_v12_0_process_interrupt(struct amdgpu_device *adev,
>  	bool write_fault = !!(entry->src_data[1] &
>  			      AMDGPU_GMC9_FAULT_SOURCE_DATA_WRITE);
>  	uint32_t status = 0;
> +	uint32_t cam_index;
>  	u64 addr;

Same as patch #7, declare "int ret;" here instead.

Christian.

>  
>  	addr = (u64)entry->src_data[0] << 12;
> @@ -110,7 +111,9 @@ static int gmc_v12_0_process_interrupt(struct amdgpu_device *adev,
>  		hub = &adev->vmhub[AMDGPU_GFXHUB(0)];
>  
>  	if (retry_fault) {
> -		int ret = amdgpu_gmc_handle_retry_fault(adev, entry, addr, 0, 0,
> +		cam_index = entry->src_data[2] & 0x3ff;
> +
> +		int ret = amdgpu_gmc_handle_retry_fault(adev, entry, addr, cam_index, 0,
>  							write_fault);
>  		/* Returning 1 here also prevents sending the IV to the KFD */
>  		if (ret == 1)

Re: [PATCH 09/14] drm/amdgpu/gmc12: Use AMDGPU_PTE_IS_PTE flag for init_pte_flags on GFX12.0

[Christian König]

On 7/1/26 18:17, Timur Kristóf wrote:
> This flag seems to work around a "fault priority problem"
> and is necessary for handling faults on GFX12.
> The kernel seems unable to mitigate retry faults on GFX12
> without this flag.
> 
> For reference see:
> amdgpu_vm_pte_update_flags() that explains the problem
> svm_range_get_pte_flags() that uses the flag on GFX12
> 
> Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>

Reviewed-by: Christian König <christian.koenig@amd.com>

> ---
>  drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c
> index 299fe960b99e..c210e2463a27 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c
> @@ -644,11 +644,12 @@ static int gmc_v12_0_early_init(struct amdgpu_ip_block *ip_block)
>  		adev->gmc.xgmi.connected_to_cpu =
>  			adev->smuio.funcs->is_host_gpu_xgmi_supported(adev);
>  
> +	adev->gmc.init_pte_flags = AMDGPU_PTE_IS_PTE;
> +
>  	switch (amdgpu_ip_version(adev, GC_HWIP, 0)) {
>  	case IP_VERSION(12, 1, 0):
>  		gmc_v12_1_set_gmc_funcs(adev);
>  		gmc_v12_1_set_irq_funcs(adev);
> -		adev->gmc.init_pte_flags = AMDGPU_PTE_IS_PTE;
>  		break;
>  	default:
>  		gmc_v12_0_set_gmc_funcs(adev);

Re: [PATCH 10/14] drm/amdgpu/vm: Use init PTE flags and NOALLOC in amdgpu_vm_handle_fault()

[Christian König]

On 7/1/26 18:17, Timur Kristóf wrote:
> PTE_IS_PTE seems necessary for handling retry faults on GFX12.
> 
> For reference see:
> amdgpu_vm_pte_update_flags() that explains the problem
> svm_range_get_pte_flags() that uses the flag on GFX12
> 
> Also add NOALLOC on GFX10.3+ as we don't need to allocate
> the fault handling PTE in the infinity cache (MALL).
> 
> Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
> ---
>  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 8 ++++++--
>  1 file changed, 6 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> index 32719f31b6c9..a915d061085f 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> @@ -3044,7 +3044,8 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid,
>  	}
>  
>  	addr /= AMDGPU_GPU_PAGE_SIZE;
> -	flags = AMDGPU_PTE_VALID | AMDGPU_PTE_SNOOPED |
> +	flags = adev->gmc.init_pte_flags |
> +		AMDGPU_PTE_VALID | AMDGPU_PTE_SNOOPED |
>  		AMDGPU_PTE_SYSTEM;

That doesn't make sense the PTE flags should be adjusted to the device specific flags by the callbacks.

>  
>  	if (is_compute_context) {
> @@ -3054,11 +3055,14 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid,
>  		flags = AMDGPU_VM_NORETRY_FLAGS;
>  		value = 0;
>  	} else if (amdgpu_vm_fault_stop == AMDGPU_VM_FAULT_STOP_NEVER) {
> +		/* Don't allocate this PTE in the MALL */
> +		if (amdgpu_ip_version(adev, GC_HWIP, 0) >= IP_VERSION(10, 3, 0))
> +			flags |= AMDGPU_PTE_NOALLOC;
> +

This doesn't make sense either, mall allocation for the dummy page should be perfectly fine.

Regards,
Christian.

>  		/* Redirect the access to the dummy page */
>  		value = adev->dummy_page_addr;
>  		flags |= AMDGPU_PTE_EXECUTABLE | AMDGPU_PTE_READABLE |
>  			AMDGPU_PTE_WRITEABLE;
> -
>  	} else {
>  		/* Let the hw retry silently on the PTE */
>  		value = 0;

Re: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH 6.0

[Christian König]

On 7/1/26 18:17, Timur Kristóf wrote:
> The doorbell is not working on Navi 31.
> Use the IH_RETRY_CAM_ACK register.
> 
> Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>

Acked-by: Christian König <christian.koenig@amd.com>

I still haven't found any explanation why we use a doorbell for the CAM in the first place? Lijo, Alex, Felix does anybody know that?

Thanks,
Christian.

> ---
>  drivers/gpu/drm/amd/amdgpu/ih_v6_0.c | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
> index 65e5d21753f9..a97c85b0cb99 100644
> --- a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
> @@ -307,6 +307,11 @@ static int ih_v6_0_enable_ring(struct amdgpu_device *adev,
>  	return 0;
>  }
>  
> +static void ih_v6_0_retry_cam_ack(struct amdgpu_device *adev, u32 cam_index)
> +{
> +	WREG32_SOC15(OSSSYS, 0, regIH_RETRY_CAM_ACK, cam_index);
> +}
> +
>  /**
>   * ih_v6_0_irq_init - init and enable the interrupt ring
>   *
> @@ -807,7 +812,8 @@ static const struct amdgpu_ih_funcs ih_v6_0_funcs = {
>  	.get_wptr = ih_v6_0_get_wptr,
>  	.decode_iv = amdgpu_ih_decode_iv_helper,
>  	.decode_iv_ts = amdgpu_ih_decode_iv_ts_helper,
> -	.set_rptr = ih_v6_0_set_rptr
> +	.set_rptr = ih_v6_0_set_rptr,
> +	.retry_cam_ack = ih_v6_0_retry_cam_ack,
>  };
>  
>  static void ih_v6_0_set_interrupt_funcs(struct amdgpu_device *adev)

Re: [PATCH 12/14] drm/amdgpu/ih7.0: Use MMIO ACK instead of doorbell for retry CAM on IH 7.0

[Christian König]

On 7/1/26 18:17, Timur Kristóf wrote:
> The doorbell is not working on Navi 48.
> Use the IH_RETRY_CAM_ACK register just like on Navi 3x.
> 
> Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>

Acked-by: Christian König <christian.koenig@amd.com>

> ---
>  drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 17 +----------------
>  1 file changed, 1 insertion(+), 16 deletions(-)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> index f1de6450a31d..291326d2ee8b 100644
> --- a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> @@ -279,19 +279,9 @@ static int ih_v7_0_enable_ring(struct amdgpu_device *adev,
>  	return 0;
>  }
>  
> -static uint32_t ih_v7_0_setup_retry_doorbell(u32 doorbell_index)
> -{
> -	u32 val = 0;
> -
> -	val = REG_SET_FIELD(val, IH_DOORBELL_RPTR, OFFSET, doorbell_index);
> -	val = REG_SET_FIELD(val, IH_DOORBELL_RPTR, ENABLE, 1);
> -
> -	return val;
> -}
> -
>  static void ih_v7_0_retry_cam_ack(struct amdgpu_device *adev, u32 cam_index)
>  {
> -	WDOORBELL32(adev->irq.retry_cam_doorbell_index, cam_index);
> +	WREG32_SOC15(OSSSYS, 0, regIH_RETRY_CAM_ACK, cam_index);
>  }
>  
>  #define regIH_RING1_CLIENT_CFG_INDEX_V7_1             0x122
> @@ -401,11 +391,6 @@ static int ih_v7_0_irq_init(struct amdgpu_device *adev)
>  	pci_set_master(adev->pdev);
>  
>  	if (amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7, 1, 0)) {
> -		/* Allocate the doorbell for IH Retry CAM */
> -		adev->irq.retry_cam_doorbell_index = (adev->doorbell_index.ih + 2) << 1;
> -		WREG32_SOC15(OSSSYS, 0, regIH_DOORBELL_RETRY_CAM,
> -				ih_v7_0_setup_retry_doorbell(adev->irq.retry_cam_doorbell_index));
> -
>  		/* Enable IH Retry CAM */
>  		tmp = RREG32_SOC15(OSSSYS, 0, regIH_RETRY_INT_CAM_CNTL);
>  		tmp = REG_SET_FIELD(tmp, IH_RETRY_INT_CAM_CNTL, ENABLE, 1);

Re: [PATCH 13/14] drm/amdgpu/ih6.0: Enable retry CAM on Navi 3 dGPUs

[Christian König]

On 7/1/26 18:17, Timur Kristóf wrote:
> The retry CAM can filter interrupts which occur repeatedly,
> such as page fault interrupts when retry faults are enabled.
> This makes processing those interrupts much more efficient,
> because the CPU won't have to deal with processing the same
> interrupt repeatedly.
> 
> Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>

Reviewed-by: Christian König <christian.koenig@amd.com>

> ---
>  drivers/gpu/drm/amd/amdgpu/ih_v6_0.c | 10 ++++++++++
>  1 file changed, 10 insertions(+)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
> index a97c85b0cb99..bbe399f9e233 100644
> --- a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
> @@ -397,6 +397,16 @@ static int ih_v6_0_irq_init(struct amdgpu_device *adev)
>  
>  	pci_set_master(adev->pdev);
>  
> +	if (!(adev->flags & AMD_IS_APU)) {
> +		/* Enable IH Retry CAM */
> +		tmp = RREG32_SOC15(OSSSYS, 0, regIH_RETRY_INT_CAM_CNTL);
> +		tmp = REG_SET_FIELD(tmp, IH_RETRY_INT_CAM_CNTL, ENABLE, 1);
> +		tmp = REG_SET_FIELD(tmp, IH_RETRY_INT_CAM_CNTL, CAM_SIZE, 0xF);
> +		WREG32_SOC15(OSSSYS, 0, regIH_RETRY_INT_CAM_CNTL, tmp);
> +
> +		adev->irq.retry_cam_enabled = true;
> +	}
> +
>  	/* enable interrupts */
>  	ret = ih_v6_0_toggle_interrupts(adev, true);
>  	if (ret)

Re: [PATCH 14/14] drm/amdgpu/ih7.0: Enable retry CAM on Navi 4 dGPUs

[Christian König]

On 7/1/26 18:17, Timur Kristóf wrote:
> The retry CAM can filter interrupts which occur repeatedly,
> such as page fault interrupts when retry faults are enabled.
> This makes processing those interrupts much more efficient,
> because the CPU won't have to deal with processing the same
> interrupt repeatedly.
> 
> Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
> ---
>  drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> index 291326d2ee8b..ec0919fa8254 100644
> --- a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> @@ -390,7 +390,8 @@ static int ih_v7_0_irq_init(struct amdgpu_device *adev)
>  
>  	pci_set_master(adev->pdev);
>  
> -	if (amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7, 1, 0)) {
> +	if (amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7, 1, 0) ||
> +	    !(adev->flags & AMD_IS_APU)) {

I think the check should be the other way around.

In other words we can enable the cam on dGPU or APUs with IP version 7.1 because that one has the CAM as well.

Apart from that looks good to me,
Christian.

>  		/* Enable IH Retry CAM */
>  		tmp = RREG32_SOC15(OSSSYS, 0, regIH_RETRY_INT_CAM_CNTL);
>  		tmp = REG_SET_FIELD(tmp, IH_RETRY_INT_CAM_CNTL, ENABLE, 1);

Re: [PATCH 10/14] drm/amdgpu/vm: Use init PTE flags and NOALLOC in amdgpu_vm_handle_fault()

[Timur Kristóf]

On 2026. július 2., csütörtök 12:22:23 közép-európai nyári idő Christian König 
wrote:
> On 7/1/26 18:17, Timur Kristóf wrote:
> > PTE_IS_PTE seems necessary for handling retry faults on GFX12.
> > 
> > For reference see:
> > amdgpu_vm_pte_update_flags() that explains the problem
> > svm_range_get_pte_flags() that uses the flag on GFX12
> > 
> > Also add NOALLOC on GFX10.3+ as we don't need to allocate
> > the fault handling PTE in the infinity cache (MALL).
> > 
> > Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
> > ---
> > 
> >  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 8 ++++++--
> >  1 file changed, 6 insertions(+), 2 deletions(-)
> > 
> > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> > b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c index 32719f31b6c9..a915d061085f
> > 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> > @@ -3044,7 +3044,8 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device
> > *adev, u32 pasid,> 
> >  	}
> >  	
> >  	addr /= AMDGPU_GPU_PAGE_SIZE;
> > 
> > -	flags = AMDGPU_PTE_VALID | AMDGPU_PTE_SNOOPED |
> > +	flags = adev->gmc.init_pte_flags |
> > +		AMDGPU_PTE_VALID | AMDGPU_PTE_SNOOPED |
> > 
> >  		AMDGPU_PTE_SYSTEM;
> 
> That doesn't make sense the PTE flags should be adjusted to the device
> specific flags by the callbacks.

Can you please elaborate on what is it that doesn't make sense here and why, 
and what you recommend to do instead?

> >  	if (is_compute_context) {
> > 
> > @@ -3054,11 +3055,14 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device
> > *adev, u32 pasid,> 
> >  		flags = AMDGPU_VM_NORETRY_FLAGS;
> >  		value = 0;
> >  	
> >  	} else if (amdgpu_vm_fault_stop == AMDGPU_VM_FAULT_STOP_NEVER) {
> > 
> > +		/* Don't allocate this PTE in the MALL */
> > +		if (amdgpu_ip_version(adev, GC_HWIP, 0) >= 
IP_VERSION(10, 3, 0))
> > +			flags |= AMDGPU_PTE_NOALLOC;
> > +
> 
> This doesn't make sense either, mall allocation for the dummy page should be
> perfectly fine.

I think we shouldn't waste space for this PTE in the MALL.

> 
> >  		/* Redirect the access to the dummy page */
> >  		value = adev->dummy_page_addr;
> >  		flags |= AMDGPU_PTE_EXECUTABLE | AMDGPU_PTE_READABLE |
> >  		
> >  			AMDGPU_PTE_WRITEABLE;
> > 
> > -
> > 
> >  	} else {
> >  	
> >  		/* Let the hw retry silently on the PTE */
> >  		value = 0;

Re: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH 6.0

[Timur Kristóf]

On 2026. július 2., csütörtök 12:23:42 közép-európai nyári idő Christian König 
wrote:
> On 7/1/26 18:17, Timur Kristóf wrote:
> > The doorbell is not working on Navi 31.
> > Use the IH_RETRY_CAM_ACK register.
> > 
> > Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
> 
> Acked-by: Christian König <christian.koenig@amd.com>
> 
> I still haven't found any explanation why we use a doorbell for the CAM in
> the first place? Lijo, Alex, Felix does anybody know that?

The first mention of the retry CAM in the kernel is:
commit 318e431b306e - drm/amdgpu: Enable IH retry CAM on GFX9
which was written by Mukul (also on CC for this series).
Mukul - can you please give us some information on that?

It seems that the IH 7.1 code was just copied from there.

As far as I see, Vega 20 already has the mmIH_RETRY_CAM_ACK register, though I 
don't know why that wasn't used. I guess you guys would have heard from your 
datacenter customers if the current code didn't work on Vega 20 and CDNA.
I can say with certainty that I couldn't get the doorbell working on Navi 31 
or Navi 48.

Unfortunately I don't have a Vega 20 card (or any datacenter GPU) so I can't 
test what actually works there. I have a Vega 10, but it's unclear if that has 
a retry CAM or not.

Thanks & best regards,
Timur

> 
> > ---
> > 
> >  drivers/gpu/drm/amd/amdgpu/ih_v6_0.c | 8 +++++++-
> >  1 file changed, 7 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
> > b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c index 65e5d21753f9..a97c85b0cb99
> > 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
> > @@ -307,6 +307,11 @@ static int ih_v6_0_enable_ring(struct amdgpu_device
> > *adev,> 
> >  	return 0;
> >  
> >  }
> > 
> > +static void ih_v6_0_retry_cam_ack(struct amdgpu_device *adev, u32
> > cam_index) +{
> > +	WREG32_SOC15(OSSSYS, 0, regIH_RETRY_CAM_ACK, cam_index);
> > +}
> > +
> > 
> >  /**
> >  
> >   * ih_v6_0_irq_init - init and enable the interrupt ring
> >   *
> > 
> > @@ -807,7 +812,8 @@ static const struct amdgpu_ih_funcs ih_v6_0_funcs = {
> > 
> >  	.get_wptr = ih_v6_0_get_wptr,
> >  	.decode_iv = amdgpu_ih_decode_iv_helper,
> >  	.decode_iv_ts = amdgpu_ih_decode_iv_ts_helper,
> > 
> > -	.set_rptr = ih_v6_0_set_rptr
> > +	.set_rptr = ih_v6_0_set_rptr,
> > +	.retry_cam_ack = ih_v6_0_retry_cam_ack,
> > 
> >  };
> >  
> >  static void ih_v6_0_set_interrupt_funcs(struct amdgpu_device *adev)

Re: [PATCH 14/14] drm/amdgpu/ih7.0: Enable retry CAM on Navi 4 dGPUs

[Timur Kristóf]

On 2026. július 2., csütörtök 12:38:12 közép-európai nyári idő Christian König 
wrote:
> On 7/1/26 18:17, Timur Kristóf wrote:
> > The retry CAM can filter interrupts which occur repeatedly,
> > such as page fault interrupts when retry faults are enabled.
> > This makes processing those interrupts much more efficient,
> > because the CPU won't have to deal with processing the same
> > interrupt repeatedly.
> > 
> > Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
> > ---
> > 
> >  drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> > b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c index 291326d2ee8b..ec0919fa8254
> > 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> > @@ -390,7 +390,8 @@ static int ih_v7_0_irq_init(struct amdgpu_device
> > *adev)
> > 
> >  	pci_set_master(adev->pdev);
> > 
> > -	if (amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7, 1, 
0)) {
> > +	if (amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7, 1, 0) 
||
> > +	    !(adev->flags & AMD_IS_APU)) {
> 
> I think the check should be the other way around.
> 
> In other words we can enable the cam on dGPU or APUs with IP version 7.1
> because that one has the CAM as well.

What do you mean by the other way around?

The patch keeps pre-existing behaviour on IH 7.1 and additionally enables the 
retry CAM on dGPUs. The patch makes no functional changes to IH 7.1, on that 
version the retry CAM is always enabled regardless of whether it's an APU or 
dGPU.

> 
> >  		/* Enable IH Retry CAM */
> >  		tmp = RREG32_SOC15(OSSSYS, 0, 
regIH_RETRY_INT_CAM_CNTL);
> >  		tmp = REG_SET_FIELD(tmp, IH_RETRY_INT_CAM_CNTL, ENABLE, 
1);

Re: [PATCH 02/14] drm/amdgpu/gfxhub: Enable retry fault interrupts when needed

[Christian König]

On 7/2/26 11:14, Timur Kristóf wrote:
> On 2026. július 2., csütörtök 10:10:10 közép-európai nyári idő Christian König 
> wrote:
>> On 7/1/26 18:17, Timur Kristóf wrote:
>>> Enable retry fault interrupts when initializing the GFXHUB
>>> system aperture registers according to whether retrying
>>> page faults is enabled in amdgpu (ie. amdgpu.noretry=0).
>>>
>>> Needs to be done for each GFXHUB version at once,
>>> because none of them actually enabled this interrupt.
>>
>> Thinking more about it we are clearly missing something here. The retry
>> fault interrupt itself should be enabled all the time.
> 
> Why would it be enabled all the time?
> I haven't seen any retry faults on neither Navi 3 nor Navi 4 without enabling 
> the ENABLE_RETRY_FAULT_INTERRUPT bit.

As far as I understand it the GCVM_L2_PROTECTION_FAULT_CNTL2 is actually a debug register for the fault behavior of the GC block.

In other words you can for example turn on interrupts for PRT accesses as well to debug those.

According to the register spec for Navi 44 the ENABLE_RETRY_FAULT_INTERRUPT should be default 1 and always be set under normal cricumstances.

>>
>> IIRC only the RETRY_PERMISSION_OR_INVALID_PAGE_FAULT bit in the
>> VM_CONTEXT0_CNTL register should be set or cleared by the kernel driver or
>> firmware to control if the HW retries the access or not.
> 
> That is clearly not the case on GFX12.1 and I haven't seen any indication that 
> it would be different on GFX11.x and 12.0 either.

Mhm, then either the FW or golden register settings for that register is not correct.

That we set ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY to one is rather strange as well since we don't use that feature in the Linux driver.

My suggestion is to always set the ENABLE_RETRY_FAULT_INTERRUPT in GCVM_L2_PROTECTION_FAULT_CNTL2 no matter what the noretry flag says, but I'm going to ask Alex and our ROCm people about that again when I have time.

Regards,
Christian.

> 
> 
>>> ---
>>>
>>>  drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c | 9 +++++++--
>>>  drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c   | 9 +++++++--
>>>  drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c    | 9 +++++++--
>>>  drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c    | 2 ++
>>>  drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c    | 9 +++++++--
>>>  drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c    | 9 +++++++--
>>>  drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c    | 9 +++++++--
>>>  drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c  | 9 +++++++--
>>>  8 files changed, 51 insertions(+), 14 deletions(-)
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c
>>> b/drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c index
>>> 652eea6eae4a..ef20eafd59ae 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c
>>> @@ -155,6 +155,7 @@ static void
>>> gfxhub_v11_5_0_init_gart_aperture_regs(struct amdgpu_device *adev)> 
>>>  static void gfxhub_v11_5_0_init_system_aperture_regs(struct amdgpu_device
>>>  *adev) {
>>>  
>>>  	uint64_t value;
>>>
>>> +	u32 tmp;
>>>
>>>  	WREG32_SOC15(GC, 0, regGCMC_VM_AGP_BASE, 0);
>>>  	WREG32_SOC15(GC, 0, regGCMC_VM_AGP_BOT, adev->gmc.agp_start >> 
> 24);
>>>
>>> @@ -180,8 +181,12 @@ static void
>>> gfxhub_v11_5_0_init_system_aperture_regs(struct amdgpu_device *adev)> 
>>>  	WREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
>>>  	
>>>  		     (u32)((u64)adev->dummy_page_addr >> 44));
>>>
>>> -	WREG32_FIELD15_PREREG(GC, 0, GCVM_L2_PROTECTION_FAULT_CNTL2,
>>> -		       ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
>>> +	tmp = RREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_CNTL2);
>>> +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
>>> +				 
> ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
>>> +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
>>> +				 ENABLE_RETRY_FAULT_INTERRUPT, !
> adev->gmc.noretry);
>>> +	WREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp);
>>>
>>>  }
>>>  
>>>  static void gfxhub_v11_5_0_init_tlb_regs(struct amdgpu_device *adev)
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c
>>> b/drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c index
>>> 6cbf837d50dd..ec3ff4dec674 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c
>>> @@ -158,6 +158,7 @@ static void
>>> gfxhub_v12_0_init_gart_aperture_regs(struct amdgpu_device *adev)> 
>>>  static void gfxhub_v12_0_init_system_aperture_regs(struct amdgpu_device
>>>  *adev) {
>>>  
>>>  	uint64_t value;
>>>
>>> +	u32 tmp;
>>>
>>>  	/* Program the AGP BAR */
>>>  	WREG32_SOC15(GC, 0, regGCMC_VM_AGP_BASE, 0);
>>>
>>> @@ -184,8 +185,12 @@ static void
>>> gfxhub_v12_0_init_system_aperture_regs(struct amdgpu_device *adev)> 
>>>  	WREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
>>>  	
>>>  		     (u32)((u64)adev->dummy_page_addr >> 44));
>>>
>>> -	WREG32_FIELD15_PREREG(GC, 0, GCVM_L2_PROTECTION_FAULT_CNTL2,
>>> -		       ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
>>> +	tmp = RREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_CNTL2);
>>> +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
>>> +				 
> ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
>>> +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
>>> +				 ENABLE_RETRY_FAULT_INTERRUPT, !
> adev->gmc.noretry);
>>> +	WREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp);
>>>
>>>  }
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c
>>> b/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c index
>>> bfe247b1a333..27d7f7cb903f 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c
>>> @@ -91,6 +91,7 @@ static void gfxhub_v1_0_init_gart_aperture_regs(struct
>>> amdgpu_device *adev)> 
>>>  static void gfxhub_v1_0_init_system_aperture_regs(struct amdgpu_device
>>>  *adev) {
>>>  
>>>  	uint64_t value;
>>>
>>> +	u32 tmp;
>>>
>>>  	if (!amdgpu_sriov_vf(adev) || adev->asic_type <= CHIP_VEGA10) {
>>>  	
>>>  		/* Program the AGP BAR */
>>>
>>> @@ -134,8 +135,12 @@ static void
>>> gfxhub_v1_0_init_system_aperture_regs(struct amdgpu_device *adev)> 
>>>  		WREG32_SOC15(GC, 0, 
> mmVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
>>>  		
>>>  			     (u32)((u64)adev->dummy_page_addr >> 
> 44));
>>>
>>> -		WREG32_FIELD15(GC, 0, VM_L2_PROTECTION_FAULT_CNTL2,
>>> -			       ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 
> 1);
>>> +		tmp = RREG32_SOC15(GC, 0, 
> mmVM_L2_PROTECTION_FAULT_CNTL2);
>>> +		tmp = REG_SET_FIELD(tmp, VM_L2_PROTECTION_FAULT_CNTL2,
>>> +					 
> ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
>>> +		tmp = REG_SET_FIELD(tmp, VM_L2_PROTECTION_FAULT_CNTL2,
>>> +					 
> ENABLE_RETRY_FAULT_INTERRUPT, !adev->gmc.noretry);
>>> +		WREG32_SOC15(GC, 0, mmVM_L2_PROTECTION_FAULT_CNTL2, 
> tmp);
>>>
>>>  	}
>>>  	
>>>  	/* In the case squeezing vram into GART aperture, we don't use
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c
>>> b/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c index
>>> fbdf46070b38..ed9a64bc5aaa 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_2.c
>>> @@ -176,6 +176,8 @@ gfxhub_v1_2_xcc_init_system_aperture_regs(struct
>>> amdgpu_device *adev,> 
>>>  			tmp = RREG32_SOC15(GC, GET_INST(GC, i),
>>>  			regVM_L2_PROTECTION_FAULT_CNTL2);
>>>  			tmp = REG_SET_FIELD(tmp, 
> VM_L2_PROTECTION_FAULT_CNTL2,
>>>  			
>>>  					    
> ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
>>>
>>> +			tmp = REG_SET_FIELD(tmp, 
> VM_L2_PROTECTION_FAULT_CNTL2,
>>> +					    
> ENABLE_RETRY_FAULT_INTERRUPT, !adev->gmc.noretry);
>>>
>>>  			WREG32_SOC15(GC, GET_INST(GC, i), 
> regVM_L2_PROTECTION_FAULT_CNTL2,
>>>  			tmp);
>>>  		
>>>  		}
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c
>>> b/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c index
>>> 9ea593e2c719..152b2735d360 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c
>>> @@ -151,6 +151,7 @@ static void gfxhub_v2_0_init_gart_aperture_regs(struct
>>> amdgpu_device *adev)> 
>>>  static void gfxhub_v2_0_init_system_aperture_regs(struct amdgpu_device
>>>  *adev) {
>>>  
>>>  	uint64_t value;
>>>
>>> +	u32 tmp;
>>>
>>>  	if (!amdgpu_sriov_vf(adev)) {
>>>  	
>>>  		/* Program the AGP BAR */
>>>
>>> @@ -178,8 +179,12 @@ static void
>>> gfxhub_v2_0_init_system_aperture_regs(struct amdgpu_device *adev)> 
>>>  	WREG32_SOC15(GC, 0, mmGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
>>>  	
>>>  		     (u32)((u64)adev->dummy_page_addr >> 44));
>>>
>>> -	WREG32_FIELD15(GC, 0, GCVM_L2_PROTECTION_FAULT_CNTL2,
>>> -		       ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
>>> +	tmp = RREG32_SOC15(GC, 0, mmGCVM_L2_PROTECTION_FAULT_CNTL2);
>>> +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
>>> +				 
> ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
>>> +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
>>> +				 ENABLE_RETRY_FAULT_INTERRUPT, !
> adev->gmc.noretry);
>>> +	WREG32_SOC15(GC, 0, mmGCVM_L2_PROTECTION_FAULT_CNTL2, tmp);
>>>
>>>  }
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c
>>> b/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c index
>>> 30b90d35abd0..83c2ddbbd292 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c
>>> @@ -154,6 +154,7 @@ static void gfxhub_v2_1_init_gart_aperture_regs(struct
>>> amdgpu_device *adev)> 
>>>  static void gfxhub_v2_1_init_system_aperture_regs(struct amdgpu_device
>>>  *adev) {
>>>  
>>>  	uint64_t value;
>>>
>>> +	u32 tmp;
>>>
>>>  	if (amdgpu_sriov_vf(adev))
>>>  	
>>>  		return;
>>>
>>> @@ -182,8 +183,12 @@ static void
>>> gfxhub_v2_1_init_system_aperture_regs(struct amdgpu_device *adev)> 
>>>  	WREG32_SOC15(GC, 0, mmGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
>>>  	
>>>  		     (u32)((u64)adev->dummy_page_addr >> 44));
>>>
>>> -	WREG32_FIELD15(GC, 0, GCVM_L2_PROTECTION_FAULT_CNTL2,
>>> -		       ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
>>> +	tmp = RREG32_SOC15(GC, 0, mmGCVM_L2_PROTECTION_FAULT_CNTL2);
>>> +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
>>> +				 
> ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
>>> +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
>>> +				 ENABLE_RETRY_FAULT_INTERRUPT, !
> adev->gmc.noretry);
>>> +	WREG32_SOC15(GC, 0, mmGCVM_L2_PROTECTION_FAULT_CNTL2, tmp);
>>>
>>>  }
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c
>>> b/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c index
>>> 9e6a6e13dec0..90bbb2fe4884 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c
>>> @@ -150,6 +150,7 @@ static void gfxhub_v3_0_init_gart_aperture_regs(struct
>>> amdgpu_device *adev)> 
>>>  static void gfxhub_v3_0_init_system_aperture_regs(struct amdgpu_device
>>>  *adev) {
>>>  
>>>  	uint64_t value;
>>>
>>> +	u32 tmp;
>>>
>>>  	/* Program the AGP BAR */
>>>  	WREG32_SOC15(GC, 0, regGCMC_VM_AGP_BASE, 0);
>>>
>>> @@ -176,8 +177,12 @@ static void
>>> gfxhub_v3_0_init_system_aperture_regs(struct amdgpu_device *adev)> 
>>>  	WREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
>>>  	
>>>  		     (u32)((u64)adev->dummy_page_addr >> 44));
>>>
>>> -	WREG32_FIELD15_PREREG(GC, 0, GCVM_L2_PROTECTION_FAULT_CNTL2,
>>> -		       ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
>>> +	tmp = RREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_CNTL2);
>>> +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
>>> +				 
> ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
>>> +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
>>> +				 ENABLE_RETRY_FAULT_INTERRUPT, !
> adev->gmc.noretry);
>>> +	WREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp);
>>>
>>>  }
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c
>>> b/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c index
>>> b3b1085c7cd3..1b3c067ab48c 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.c
>>> @@ -153,6 +153,7 @@ static void
>>> gfxhub_v3_0_3_init_gart_aperture_regs(struct amdgpu_device *adev)> 
>>>  static void gfxhub_v3_0_3_init_system_aperture_regs(struct amdgpu_device
>>>  *adev) {
>>>  
>>>  	uint64_t value;
>>>
>>> +	u32 tmp;
>>>
>>>  	if (amdgpu_sriov_vf(adev))
>>>  	
>>>  		return;
>>>
>>> @@ -181,8 +182,12 @@ static void
>>> gfxhub_v3_0_3_init_system_aperture_regs(struct amdgpu_device *adev)> 
>>>  	WREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
>>>  	
>>>  		     (u32)((u64)adev->dummy_page_addr >> 44));
>>>
>>> -	WREG32_FIELD15_PREREG(GC, 0, GCVM_L2_PROTECTION_FAULT_CNTL2,
>>> -		       ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
>>> +	tmp = RREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_CNTL2);
>>> +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
>>> +				 
> ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
>>> +	tmp = REG_SET_FIELD(tmp, GCVM_L2_PROTECTION_FAULT_CNTL2,
>>> +				 ENABLE_RETRY_FAULT_INTERRUPT, !
> adev->gmc.noretry);
>>> +	WREG32_SOC15(GC, 0, regGCVM_L2_PROTECTION_FAULT_CNTL2, tmp);
>>>
>>>  }
> 
> 
> 
> 

Re: [PATCH 10/14] drm/amdgpu/vm: Use init PTE flags and NOALLOC in amdgpu_vm_handle_fault()

[Christian König]

On 7/2/26 13:28, Timur Kristóf wrote:
> On 2026. július 2., csütörtök 12:22:23 közép-európai nyári idő Christian König 
> wrote:
>> On 7/1/26 18:17, Timur Kristóf wrote:
>>> PTE_IS_PTE seems necessary for handling retry faults on GFX12.
>>>
>>> For reference see:
>>> amdgpu_vm_pte_update_flags() that explains the problem
>>> svm_range_get_pte_flags() that uses the flag on GFX12
>>>
>>> Also add NOALLOC on GFX10.3+ as we don't need to allocate
>>> the fault handling PTE in the infinity cache (MALL).
>>>
>>> Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
>>> ---
>>>
>>>  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 8 ++++++--
>>>  1 file changed, 6 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c index 32719f31b6c9..a915d061085f
>>> 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
>>> @@ -3044,7 +3044,8 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device
>>> *adev, u32 pasid,> 
>>>  	}
>>>  	
>>>  	addr /= AMDGPU_GPU_PAGE_SIZE;
>>>
>>> -	flags = AMDGPU_PTE_VALID | AMDGPU_PTE_SNOOPED |
>>> +	flags = adev->gmc.init_pte_flags |
>>> +		AMDGPU_PTE_VALID | AMDGPU_PTE_SNOOPED |
>>>
>>>  		AMDGPU_PTE_SYSTEM;
>>
>> That doesn't make sense the PTE flags should be adjusted to the device
>> specific flags by the callbacks.
> 
> Can you please elaborate on what is it that doesn't make sense here and why, 
> and what you recommend to do instead?

I had to dig up what the problem here is as well. We use the ASIC specific AMDGPU_PTE_* flags directly.

Instead we should use the AMDGPU_VM_PAGE_* flags and then call amdgpu_gmc_get_vm_pte(adev, vm, NULL, in_flags, &out_flags) to translate the AMDGPU_VM_PAGE_* flags into the ASIC specific ones.

This will automatically add flags like AMDGPU_PTE_TF and AMDGPU_PTE_IS_PTE etc...

>>>  	if (is_compute_context) {
>>>
>>> @@ -3054,11 +3055,14 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device
>>> *adev, u32 pasid,> 
>>>  		flags = AMDGPU_VM_NORETRY_FLAGS;
>>>  		value = 0;
>>>  	
>>>  	} else if (amdgpu_vm_fault_stop == AMDGPU_VM_FAULT_STOP_NEVER) {
>>>
>>> +		/* Don't allocate this PTE in the MALL */
>>> +		if (amdgpu_ip_version(adev, GC_HWIP, 0) >= 
> IP_VERSION(10, 3, 0))
>>> +			flags |= AMDGPU_PTE_NOALLOC;
>>> +
>>
>> This doesn't make sense either, mall allocation for the dummy page should be
>> perfectly fine.
> 
> I think we shouldn't waste space for this PTE in the MALL.

Hui? The PTE doesn't enter the MALL, it is the dummy page which enters the MALL and since it is only 4k it basically doesn't waste much space.

Regards,
Christian.

> 
>>
>>>  		/* Redirect the access to the dummy page */
>>>  		value = adev->dummy_page_addr;
>>>  		flags |= AMDGPU_PTE_EXECUTABLE | AMDGPU_PTE_READABLE |
>>>  		
>>>  			AMDGPU_PTE_WRITEABLE;
>>>
>>> -
>>>
>>>  	} else {
>>>  	
>>>  		/* Let the hw retry silently on the PTE */
>>>  		value = 0;
> 
> 
> 
> 

Re: [PATCH 14/14] drm/amdgpu/ih7.0: Enable retry CAM on Navi 4 dGPUs

[Christian König]

On 7/2/26 13:53, Timur Kristóf wrote:
> On 2026. július 2., csütörtök 12:38:12 közép-európai nyári idő Christian König 
> wrote:
>> On 7/1/26 18:17, Timur Kristóf wrote:
>>> The retry CAM can filter interrupts which occur repeatedly,
>>> such as page fault interrupts when retry faults are enabled.
>>> This makes processing those interrupts much more efficient,
>>> because the CPU won't have to deal with processing the same
>>> interrupt repeatedly.
>>>
>>> Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
>>> ---
>>>
>>>  drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 3 ++-
>>>  1 file changed, 2 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
>>> b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c index 291326d2ee8b..ec0919fa8254
>>> 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
>>> @@ -390,7 +390,8 @@ static int ih_v7_0_irq_init(struct amdgpu_device
>>> *adev)
>>>
>>>  	pci_set_master(adev->pdev);
>>>
>>> -	if (amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7, 1, 
> 0)) {
>>> +	if (amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7, 1, 0) 
> ||
>>> +	    !(adev->flags & AMD_IS_APU)) {
>>
>> I think the check should be the other way around.
>>
>> In other words we can enable the cam on dGPU or APUs with IP version 7.1
>> because that one has the CAM as well.
> 
> What do you mean by the other way around?
> 
> The patch keeps pre-existing behaviour on IH 7.1 and additionally enables the 
> retry CAM on dGPUs. The patch makes no functional changes to IH 7.1, on that 
> version the retry CAM is always enabled regardless of whether it's an APU or 
> dGPU.

I just wanted to make the logic more readable. In other words:

if (!(adev->flags & AMD_IS_APU) ||
    amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7, 1, 0))
...

I still need to double check why we have the exception for OSSSYS 7.1.

Could be that this is for some MI* product, but I'm not 100% sure yet.

Regards,
Christian.

> 
>>
>>>  		/* Enable IH Retry CAM */
>>>  		tmp = RREG32_SOC15(OSSSYS, 0, 
> regIH_RETRY_INT_CAM_CNTL);
>>>  		tmp = REG_SET_FIELD(tmp, IH_RETRY_INT_CAM_CNTL, ENABLE, 
> 1);
> 
> 
> 
> 

Re: [PATCH 14/14] drm/amdgpu/ih7.0: Enable retry CAM on Navi 4 dGPUs

[Timur Kristóf]

On 2026. július 2., csütörtök 14:30:20 közép-európai nyári idő Christian König 
wrote:
> On 7/2/26 13:53, Timur Kristóf wrote:
> > On 2026. július 2., csütörtök 12:38:12 közép-európai nyári idő Christian
> > König> 
> > wrote:
> >> On 7/1/26 18:17, Timur Kristóf wrote:
> >>> The retry CAM can filter interrupts which occur repeatedly,
> >>> such as page fault interrupts when retry faults are enabled.
> >>> This makes processing those interrupts much more efficient,
> >>> because the CPU won't have to deal with processing the same
> >>> interrupt repeatedly.
> >>> 
> >>> Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
> >>> ---
> >>> 
> >>>  drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 3 ++-
> >>>  1 file changed, 2 insertions(+), 1 deletion(-)
> >>> 
> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> >>> b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c index 291326d2ee8b..ec0919fa8254
> >>> 100644
> >>> --- a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> >>> +++ b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> >>> @@ -390,7 +390,8 @@ static int ih_v7_0_irq_init(struct amdgpu_device
> >>> *adev)
> >>> 
> >>>  	pci_set_master(adev->pdev);
> >>> 
> >>> -	if (amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7, 1,
> > 
> > 0)) {
> > 
> >>> +	if (amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7, 1, 0)
> >>> 
> >>> +	    !(adev->flags & AMD_IS_APU)) {
> >> 
> >> I think the check should be the other way around.
> >> 
> >> In other words we can enable the cam on dGPU or APUs with IP version 7.1
> >> because that one has the CAM as well.
> > 
> > What do you mean by the other way around?
> > 
> > The patch keeps pre-existing behaviour on IH 7.1 and additionally enables
> > the retry CAM on dGPUs. The patch makes no functional changes to IH 7.1,
> > on that version the retry CAM is always enabled regardless of whether
> > it's an APU or dGPU.
> 
> I just wanted to make the logic more readable. In other words:
> 
> if (!(adev->flags & AMD_IS_APU) ||
>     amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7, 1, 0))

Thanks, and sorry for the confusion.
I'll apply this suggestion to the next version of the series.

> 
> I still need to double check why we have the exception for OSSSYS 7.1.
> 
> Could be that this is for some MI* product, but I'm not 100% sure yet.

It was added in commit e06d19420118
I don't know which products have IH 7.1 because the IH block versions are not 
mentioned in the ASIC info table. I guess IH 7.1 goes with GFX 12.1 but I 
don't have any way to verify that guess.


> 
> >>>  		/* Enable IH Retry CAM */
> >>>  		tmp = RREG32_SOC15(OSSSYS, 0,
> > 
> > regIH_RETRY_INT_CAM_CNTL);
> > 
> >>>  		tmp = REG_SET_FIELD(tmp, IH_RETRY_INT_CAM_CNTL, ENABLE,
> > 
> > 1);

Re: [PATCH 10/14] drm/amdgpu/vm: Use init PTE flags and NOALLOC in amdgpu_vm_handle_fault()

[Timur Kristóf]

On 2026. július 2., csütörtök 14:18:45 közép-európai nyári idő Christian König 
wrote:
> On 7/2/26 13:28, Timur Kristóf wrote:
> > On 2026. július 2., csütörtök 12:22:23 közép-európai nyári idő Christian
> > König> 
> > wrote:
> >> On 7/1/26 18:17, Timur Kristóf wrote:
> >>> PTE_IS_PTE seems necessary for handling retry faults on GFX12.
> >>> 
> >>> For reference see:
> >>> amdgpu_vm_pte_update_flags() that explains the problem
> >>> svm_range_get_pte_flags() that uses the flag on GFX12
> >>> 
> >>> Also add NOALLOC on GFX10.3+ as we don't need to allocate
> >>> the fault handling PTE in the infinity cache (MALL).
> >>> 
> >>> Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
> >>> ---
> >>> 
> >>>  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 8 ++++++--
> >>>  1 file changed, 6 insertions(+), 2 deletions(-)
> >>> 
> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> >>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c index
> >>> 32719f31b6c9..a915d061085f
> >>> 100644
> >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> >>> @@ -3044,7 +3044,8 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device
> >>> *adev, u32 pasid,>
> >>> 
> >>>  	}
> >>>  	
> >>>  	addr /= AMDGPU_GPU_PAGE_SIZE;
> >>> 
> >>> -	flags = AMDGPU_PTE_VALID | AMDGPU_PTE_SNOOPED |
> >>> +	flags = adev->gmc.init_pte_flags |
> >>> +		AMDGPU_PTE_VALID | AMDGPU_PTE_SNOOPED |
> >>> 
> >>>  		AMDGPU_PTE_SYSTEM;
> >> 
> >> That doesn't make sense the PTE flags should be adjusted to the device
> >> specific flags by the callbacks.
> > 
> > Can you please elaborate on what is it that doesn't make sense here and
> > why, and what you recommend to do instead?
> 
> I had to dig up what the problem here is as well. We use the ASIC specific
> AMDGPU_PTE_* flags directly.
> 
> Instead we should use the AMDGPU_VM_PAGE_* flags and then call
> amdgpu_gmc_get_vm_pte(adev, vm, NULL, in_flags, &out_flags) to translate
> the AMDGPU_VM_PAGE_* flags into the ASIC specific ones.
> 
> This will automatically add flags like AMDGPU_PTE_TF and AMDGPU_PTE_IS_PTE
> etc...

As far as I understand, the init_pte_flags was added specifically for this 
purpose, and that's what SVM uses too.

Are you suggesting to call to amdgpu_gmc_get_vm_pte() here?
As far as I see the implementation of that function for Navi 4 is 
gmc_v12_0_get_vm_pte() which doesn't set the PTE_IS_PTE flag so it wouldn't 
resolve the issue with the fault that this commit is fixing.

> >>>  	if (is_compute_context) {
> >>> 
> >>> @@ -3054,11 +3055,14 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device
> >>> *adev, u32 pasid,>
> >>> 
> >>>  		flags = AMDGPU_VM_NORETRY_FLAGS;
> >>>  		value = 0;
> >>>  	
> >>>  	} else if (amdgpu_vm_fault_stop == AMDGPU_VM_FAULT_STOP_NEVER) {
> >>> 
> >>> +		/* Don't allocate this PTE in the MALL */
> >>> +		if (amdgpu_ip_version(adev, GC_HWIP, 0) >=
> > 
> > IP_VERSION(10, 3, 0))
> > 
> >>> +			flags |= AMDGPU_PTE_NOALLOC;
> >>> +
> >> 
> >> This doesn't make sense either, mall allocation for the dummy page should
> >> be perfectly fine.
> > 
> > I think we shouldn't waste space for this PTE in the MALL.
> 
> Hui? The PTE doesn't enter the MALL, it is the dummy page which enters the
> MALL and since it is only 4k it basically doesn't waste much space.
> 

Sure I can remove this line if you feel that strongly about it.
Do we gain any benefit from allowing this page in the MALL?


> 
> >>>  		/* Redirect the access to the dummy page */
> >>>  		value = adev->dummy_page_addr;
> >>>  		flags |= AMDGPU_PTE_EXECUTABLE | AMDGPU_PTE_READABLE |
> >>>  		
> >>>  			AMDGPU_PTE_WRITEABLE;
> >>> 
> >>> -
> >>> 
> >>>  	} else {
> >>>  	
> >>>  		/* Let the hw retry silently on the PTE */
> >>>  		value = 0;

Re: [PATCH 14/14] drm/amdgpu/ih7.0: Enable retry CAM on Navi 4 dGPUs

[Alex Deucher]

On Thu, Jul 2, 2026 at 9:09 AM Christian König <christian.koenig@amd.com> wrote:
>
>
>
> On 7/2/26 13:53, Timur Kristóf wrote:
> > On 2026. július 2., csütörtök 12:38:12 közép-európai nyári idő Christian König
> > wrote:
> >> On 7/1/26 18:17, Timur Kristóf wrote:
> >>> The retry CAM can filter interrupts which occur repeatedly,
> >>> such as page fault interrupts when retry faults are enabled.
> >>> This makes processing those interrupts much more efficient,
> >>> because the CPU won't have to deal with processing the same
> >>> interrupt repeatedly.
> >>>
> >>> Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
> >>> ---
> >>>
> >>>  drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 3 ++-
> >>>  1 file changed, 2 insertions(+), 1 deletion(-)
> >>>
> >>> diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> >>> b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c index 291326d2ee8b..ec0919fa8254
> >>> 100644
> >>> --- a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> >>> +++ b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> >>> @@ -390,7 +390,8 @@ static int ih_v7_0_irq_init(struct amdgpu_device
> >>> *adev)
> >>>
> >>>     pci_set_master(adev->pdev);
> >>>
> >>> -   if (amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7, 1,
> > 0)) {
> >>> +   if (amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7, 1, 0)
> > ||
> >>> +       !(adev->flags & AMD_IS_APU)) {
> >>
> >> I think the check should be the other way around.
> >>
> >> In other words we can enable the cam on dGPU or APUs with IP version 7.1
> >> because that one has the CAM as well.
> >
> > What do you mean by the other way around?
> >
> > The patch keeps pre-existing behaviour on IH 7.1 and additionally enables the
> > retry CAM on dGPUs. The patch makes no functional changes to IH 7.1, on that
> > version the retry CAM is always enabled regardless of whether it's an APU or
> > dGPU.
>
> I just wanted to make the logic more readable. In other words:
>
> if (!(adev->flags & AMD_IS_APU) ||
>     amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7, 1, 0))
> ...
>
> I still need to double check why we have the exception for OSSSYS 7.1.
>
> Could be that this is for some MI* product, but I'm not 100% sure yet.

oss 7.1 is for an datacenter part.

Alex

>
> Regards,
> Christian.
>
> >
> >>
> >>>             /* Enable IH Retry CAM */
> >>>             tmp = RREG32_SOC15(OSSSYS, 0,
> > regIH_RETRY_INT_CAM_CNTL);
> >>>             tmp = REG_SET_FIELD(tmp, IH_RETRY_INT_CAM_CNTL, ENABLE,
> > 1);
> >
> >
> >
> >
>

RE: [PATCH 12/14] drm/amdgpu/ih7.0: Use MMIO ACK instead of doorbell for retry CAM on IH 7.0

[Joshi, Mukul]

AMD General

> -----Original Message-----
> From: amd-gfx <amd-gfx-bounces@lists.freedesktop.org> On Behalf Of Timur
> Kristóf
> Sent: Wednesday, July 1, 2026 12:17 PM
> To: amd-gfx@lists.freedesktop.org; Deucher, Alexander
> <Alexander.Deucher@amd.com>; Koenig, Christian
> <Christian.Koenig@amd.com>; Natalie Vock <natalie.vock@gmx.de>; Shetaia,
> Amir <Amir.Shetaia@amd.com>; Marek Olšák <maraeo@gmail.com>;
> Limonciello, Mario <Mario.Limonciello@amd.com>; Tvrtko Ursulin
> <tursulin@ursulin.net>; Kuehling, Felix <Felix.Kuehling@amd.com>; Lazar, Lijo
> <Lijo.Lazar@amd.com>; He, Siwei <Siwei.He@amd.com>; Yang, Philip
> <Philip.Yang@amd.com>; Joshi, Mukul <Mukul.Joshi@amd.com>
> Cc: Timur Kristóf <timur.kristof@gmail.com>
> Subject: [PATCH 12/14] drm/amdgpu/ih7.0: Use MMIO ACK instead of
> doorbell for retry CAM on IH 7.0
>
> The doorbell is not working on Navi 48.
> Use the IH_RETRY_CAM_ACK register just like on Navi 3x.
>
> Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
> ---
>  drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 17 +----------------
>  1 file changed, 1 insertion(+), 16 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> index f1de6450a31d..291326d2ee8b 100644
> --- a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> @@ -279,19 +279,9 @@ static int ih_v7_0_enable_ring(struct amdgpu_device
> *adev,
>       return 0;
>  }
>
> -static uint32_t ih_v7_0_setup_retry_doorbell(u32 doorbell_index) -{
> -     u32 val = 0;
> -
> -     val = REG_SET_FIELD(val, IH_DOORBELL_RPTR, OFFSET,
> doorbell_index);
> -     val = REG_SET_FIELD(val, IH_DOORBELL_RPTR, ENABLE, 1);
> -
> -     return val;
> -}
> -
>  static void ih_v7_0_retry_cam_ack(struct amdgpu_device *adev, u32
> cam_index)  {
> -     WDOORBELL32(adev->irq.retry_cam_doorbell_index, cam_index);
> +     WREG32_SOC15(OSSSYS, 0, regIH_RETRY_CAM_ACK, cam_index);
>  }
>
>  #define regIH_RING1_CLIENT_CFG_INDEX_V7_1             0x122
> @@ -401,11 +391,6 @@ static int ih_v7_0_irq_init(struct amdgpu_device
> *adev)
>       pci_set_master(adev->pdev);
>
>       if (amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7, 1, 0))
> {

We are re-purposing the IH 7.0 file for IH v7.1 since most of it is same.
The code being removed below is for IH 7.1 and not 7.0.
Using doorbells for acking INT CAM works completely fine for GFX 12.1 and IH 7.1.
I would recommend to drop this patch.

Regards,
Mukul

> -             /* Allocate the doorbell for IH Retry CAM */
> -             adev->irq.retry_cam_doorbell_index = (adev-
> >doorbell_index.ih + 2) << 1;
> -             WREG32_SOC15(OSSSYS, 0, regIH_DOORBELL_RETRY_CAM,
> -                             ih_v7_0_setup_retry_doorbell(adev-
> >irq.retry_cam_doorbell_index));
> -
>               /* Enable IH Retry CAM */
>               tmp = RREG32_SOC15(OSSSYS, 0,
> regIH_RETRY_INT_CAM_CNTL);
>               tmp = REG_SET_FIELD(tmp, IH_RETRY_INT_CAM_CNTL,
> ENABLE, 1);
> --
> 2.54.0

RE: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH 6.0

[Joshi, Mukul]

AMD General

> -----Original Message-----
> From: Timur Kristóf <timur.kristof@gmail.com>
> Sent: Wednesday, July 1, 2026 12:17 PM
> To: amd-gfx@lists.freedesktop.org; Deucher, Alexander
> <Alexander.Deucher@amd.com>; Koenig, Christian
> <Christian.Koenig@amd.com>; Natalie Vock <natalie.vock@gmx.de>; Shetaia,
> Amir <Amir.Shetaia@amd.com>; Marek Olšák <maraeo@gmail.com>;
> Limonciello, Mario <Mario.Limonciello@amd.com>; Tvrtko Ursulin
> <tursulin@ursulin.net>; Kuehling, Felix <Felix.Kuehling@amd.com>; Lazar, Lijo
> <Lijo.Lazar@amd.com>; He, Siwei <Siwei.He@amd.com>; Yang, Philip
> <Philip.Yang@amd.com>; Joshi, Mukul <Mukul.Joshi@amd.com>
> Cc: Timur Kristóf <timur.kristof@gmail.com>
> Subject: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH
> 6.0
>
> The doorbell is not working on Navi 31.
> Use the IH_RETRY_CAM_ACK register.

I think the doorbell path is not enabled for IH CAM so that’s probably the reason its not working.
I think we need an update to the NBIO ih_doorbell_range() function to enable the doorbell path.

Having said that, we intend to use doorbells on all ASICs to ack IH CAMs that support retry faults.

Regards,
Mukul

>
> Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
> ---
>  drivers/gpu/drm/amd/amdgpu/ih_v6_0.c | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
> b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
> index 65e5d21753f9..a97c85b0cb99 100644
> --- a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
> @@ -307,6 +307,11 @@ static int ih_v6_0_enable_ring(struct amdgpu_device
> *adev,
>       return 0;
>  }
>
> +static void ih_v6_0_retry_cam_ack(struct amdgpu_device *adev, u32
> +cam_index) {
> +     WREG32_SOC15(OSSSYS, 0, regIH_RETRY_CAM_ACK, cam_index); }
> +
>  /**
>   * ih_v6_0_irq_init - init and enable the interrupt ring
>   *
> @@ -807,7 +812,8 @@ static const struct amdgpu_ih_funcs ih_v6_0_funcs = {
>       .get_wptr = ih_v6_0_get_wptr,
>       .decode_iv = amdgpu_ih_decode_iv_helper,
>       .decode_iv_ts = amdgpu_ih_decode_iv_ts_helper,
> -     .set_rptr = ih_v6_0_set_rptr
> +     .set_rptr = ih_v6_0_set_rptr,
> +     .retry_cam_ack = ih_v6_0_retry_cam_ack,
>  };
>
>  static void ih_v6_0_set_interrupt_funcs(struct amdgpu_device *adev)
> --
> 2.54.0

RE: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH 6.0

[Joshi, Mukul]

AMD General

> -----Original Message-----
> From: Koenig, Christian <Christian.Koenig@amd.com>
> Sent: Thursday, July 2, 2026 6:24 AM
> To: Timur Kristóf <timur.kristof@gmail.com>; amd-gfx@lists.freedesktop.org;
> Deucher, Alexander <Alexander.Deucher@amd.com>; Natalie Vock
> <natalie.vock@gmx.de>; Shetaia, Amir <Amir.Shetaia@amd.com>; Marek Olšák
> <maraeo@gmail.com>; Limonciello, Mario <Mario.Limonciello@amd.com>;
> Tvrtko Ursulin <tursulin@ursulin.net>; Kuehling, Felix
> <Felix.Kuehling@amd.com>; Lazar, Lijo <Lijo.Lazar@amd.com>; He, Siwei
> <Siwei.He@amd.com>; Yang, Philip <Philip.Yang@amd.com>; Joshi, Mukul
> <Mukul.Joshi@amd.com>
> Subject: Re: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on
> IH 6.0
>
> On 7/1/26 18:17, Timur Kristóf wrote:
> > The doorbell is not working on Navi 31.
> > Use the IH_RETRY_CAM_ACK register.
> >
> > Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
>
> Acked-by: Christian König <christian.koenig@amd.com>
>
> I still haven't found any explanation why we use a doorbell for the CAM in the first
> place? Lijo, Alex, Felix does anybody know that?
>

If I remember correctly, we weren't sure if the IH_RETRY_CAM_ACK register would be available under
virtualization. And we were sure doorbells would work for both bare-metal and virtualization.

Regards,
Mukul

> Thanks,
> Christian.
>
> > ---
> >  drivers/gpu/drm/amd/amdgpu/ih_v6_0.c | 8 +++++++-
> >  1 file changed, 7 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
> > b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
> > index 65e5d21753f9..a97c85b0cb99 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
> > @@ -307,6 +307,11 @@ static int ih_v6_0_enable_ring(struct amdgpu_device
> *adev,
> >     return 0;
> >  }
> >
> > +static void ih_v6_0_retry_cam_ack(struct amdgpu_device *adev, u32
> > +cam_index) {
> > +   WREG32_SOC15(OSSSYS, 0, regIH_RETRY_CAM_ACK, cam_index); }
> > +
> >  /**
> >   * ih_v6_0_irq_init - init and enable the interrupt ring
> >   *
> > @@ -807,7 +812,8 @@ static const struct amdgpu_ih_funcs ih_v6_0_funcs = {
> >     .get_wptr = ih_v6_0_get_wptr,
> >     .decode_iv = amdgpu_ih_decode_iv_helper,
> >     .decode_iv_ts = amdgpu_ih_decode_iv_ts_helper,
> > -   .set_rptr = ih_v6_0_set_rptr
> > +   .set_rptr = ih_v6_0_set_rptr,
> > +   .retry_cam_ack = ih_v6_0_retry_cam_ack,
> >  };
> >
> >  static void ih_v6_0_set_interrupt_funcs(struct amdgpu_device *adev)

Re: [PATCH 12/14] drm/amdgpu/ih7.0: Use MMIO ACK instead of doorbell for retry CAM on IH 7.0

[Timur Kristóf]

On 2026. július 3., péntek 18:31:12 közép-európai nyári idő Joshi, Mukul 
wrote:
> AMD General
> 
> 
> > -----Original Message-----
> > From: amd-gfx <amd-gfx-bounces@lists.freedesktop.org> On Behalf Of Timur
> > Kristóf
> > Sent: Wednesday, July 1, 2026 12:17 PM
> > To: amd-gfx@lists.freedesktop.org; Deucher, Alexander
> > <Alexander.Deucher@amd.com>; Koenig, Christian
> > <Christian.Koenig@amd.com>; Natalie Vock <natalie.vock@gmx.de>; Shetaia,
> > Amir <Amir.Shetaia@amd.com>; Marek Olšák <maraeo@gmail.com>;
> > Limonciello, Mario <Mario.Limonciello@amd.com>; Tvrtko Ursulin
> > <tursulin@ursulin.net>; Kuehling, Felix <Felix.Kuehling@amd.com>; Lazar,
> > Lijo
 <Lijo.Lazar@amd.com>; He, Siwei <Siwei.He@amd.com>; Yang, Philip
> > <Philip.Yang@amd.com>; Joshi, Mukul <Mukul.Joshi@amd.com>
> > Cc: Timur Kristóf <timur.kristof@gmail.com>
> > Subject: [PATCH 12/14] drm/amdgpu/ih7.0: Use MMIO ACK instead of
> > doorbell for retry CAM on IH 7.0
> >
> >
> >
> > The doorbell is not working on Navi 48.
> > Use the IH_RETRY_CAM_ACK register just like on Navi 3x.
> >
> >
> >
> > Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
> > ---
> > 
> >  drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 17 +----------------
> >  1 file changed, 1 insertion(+), 16 deletions(-)
> >
> >
> >
> > diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> > b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> > index f1de6450a31d..291326d2ee8b 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> > @@ -279,19 +279,9 @@ static int ih_v7_0_enable_ring(struct amdgpu_device
> > *adev,
> > 
> >       return 0;
> >  
> >  }
> >
> >
> >
> > -static uint32_t ih_v7_0_setup_retry_doorbell(u32 doorbell_index) -{
> > -     u32 val = 0;
> > -
> > -     val = REG_SET_FIELD(val, IH_DOORBELL_RPTR, OFFSET,
> > doorbell_index);
> > -     val = REG_SET_FIELD(val, IH_DOORBELL_RPTR, ENABLE, 1);
> > -
> > -     return val;
> > -}
> > -
> > 
> >  static void ih_v7_0_retry_cam_ack(struct amdgpu_device *adev, u32
> > 
> > cam_index)  {
> > -     WDOORBELL32(adev->irq.retry_cam_doorbell_index, cam_index);
> > +     WREG32_SOC15(OSSSYS, 0, regIH_RETRY_CAM_ACK, cam_index);
> > 
> >  }
> >
> >
> >
> >  #define regIH_RING1_CLIENT_CFG_INDEX_V7_1             0x122
> > 
> > @@ -401,11 +391,6 @@ static int ih_v7_0_irq_init(struct amdgpu_device
> > *adev)
> > 
> >       pci_set_master(adev->pdev);
> >
> >
> >
> >       if (amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7, 1, 0))
> > 
> > {
> 
> 
> We are re-purposing the IH 7.0 file for IH v7.1 since most of it is same.
> The code being removed below is for IH 7.1 and not 7.0.
> Using doorbells for acking INT CAM works completely fine for GFX 12.1 and IH
> 7.1.
> I would recommend to drop this patch.
> 

Hello Mukul,

Unfortunately I couldn't figure out how to get the doorbell working on Navi 48 
and in the previous thread Amir said that it also didn't work for him, and 
that's why I decided to try the MMIO register.

If you have an idea how to get the doorbell to work on Navi 48, I'm happy to 
try your suggestion.

Ideally I'd prefer a solution here that works on both Navi 48 and GFX12.1 to 
keep the code maintainable. If it's not possible then we can also have a 
slightly different code path for the two different IH versions here.

Thanks & best regards,
Timur

> 
> 
> > -             /* Allocate the doorbell for IH Retry CAM */
> > -             adev->irq.retry_cam_doorbell_index = (adev-
> > 
> > >doorbell_index.ih + 2) << 1;
> > 
> > -             WREG32_SOC15(OSSSYS, 0, regIH_DOORBELL_RETRY_CAM,
> > -                             ih_v7_0_setup_retry_doorbell(adev-
> > 
> > >irq.retry_cam_doorbell_index));
> > 
> > -
> > 
> >               /* Enable IH Retry CAM */
> >               tmp = RREG32_SOC15(OSSSYS, 0,
> > 
> > regIH_RETRY_INT_CAM_CNTL);
> > 
> >               tmp = REG_SET_FIELD(tmp, IH_RETRY_INT_CAM_CNTL,
> > 
> > ENABLE, 1);
> > --
> > 2.54.0
> 
> 

Re: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH 6.0

[Kuehling, Felix]

On 2026-07-02 06:23, Christian König wrote:
> On 7/1/26 18:17, Timur Kristóf wrote:
>> The doorbell is not working on Navi 31.
>> Use the IH_RETRY_CAM_ACK register.
>>
>> Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
> Acked-by: Christian König <christian.koenig@amd.com>
>
> I still haven't found any explanation why we use a doorbell for the CAM in the first place? Lijo, Alex, Felix does anybody know that?

Why are we using doorbells to update WPTRs instead of writing to the 
registers directly? I was assuming that using doorbells for the IH CAM 
had similar reasons. I think register writes can be problematic with 
certain power saving modes and virtualization. I'm not sure if there is 
a difference in terms of CPU overhead between writing a doorbell or 
writing an MMIO register.

If doorbells aren't working, it's probably some configuration issue. I'd 
rather fix those issues than throw away use of doorbells altogether.

Regards,
   Felix


>
> Thanks,
> Christian.
>
>> ---
>>   drivers/gpu/drm/amd/amdgpu/ih_v6_0.c | 8 +++++++-
>>   1 file changed, 7 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
>> index 65e5d21753f9..a97c85b0cb99 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
>> @@ -307,6 +307,11 @@ static int ih_v6_0_enable_ring(struct amdgpu_device *adev,
>>   	return 0;
>>   }
>>   
>> +static void ih_v6_0_retry_cam_ack(struct amdgpu_device *adev, u32 cam_index)
>> +{
>> +	WREG32_SOC15(OSSSYS, 0, regIH_RETRY_CAM_ACK, cam_index);
>> +}
>> +
>>   /**
>>    * ih_v6_0_irq_init - init and enable the interrupt ring
>>    *
>> @@ -807,7 +812,8 @@ static const struct amdgpu_ih_funcs ih_v6_0_funcs = {
>>   	.get_wptr = ih_v6_0_get_wptr,
>>   	.decode_iv = amdgpu_ih_decode_iv_helper,
>>   	.decode_iv_ts = amdgpu_ih_decode_iv_ts_helper,
>> -	.set_rptr = ih_v6_0_set_rptr
>> +	.set_rptr = ih_v6_0_set_rptr,
>> +	.retry_cam_ack = ih_v6_0_retry_cam_ack,
>>   };
>>   
>>   static void ih_v6_0_set_interrupt_funcs(struct amdgpu_device *adev)

Re: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH 6.0

[Timur Kristóf]

On 2026. július 3., péntek 19:46:03 közép-európai nyári idő Joshi, Mukul 
wrote:
> AMD General
> 
> 
> > -----Original Message-----
> > From: Timur Kristóf <timur.kristof@gmail.com>
> > Sent: Wednesday, July 1, 2026 12:17 PM
> > To: amd-gfx@lists.freedesktop.org; Deucher, Alexander
> > <Alexander.Deucher@amd.com>; Koenig, Christian
> > <Christian.Koenig@amd.com>; Natalie Vock <natalie.vock@gmx.de>; Shetaia,
> > Amir <Amir.Shetaia@amd.com>; Marek Olšák <maraeo@gmail.com>;
> > Limonciello, Mario <Mario.Limonciello@amd.com>; Tvrtko Ursulin
> > <tursulin@ursulin.net>; Kuehling, Felix <Felix.Kuehling@amd.com>; Lazar,
> > Lijo
> <Lijo.Lazar@amd.com>; He, Siwei <Siwei.He@amd.com>; Yang, Philip
> > <Philip.Yang@amd.com>; Joshi, Mukul <Mukul.Joshi@amd.com>
> > Cc: Timur Kristóf <timur.kristof@gmail.com>
> > Subject: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH
> > 6.0
> >
> >
> >
> > The doorbell is not working on Navi 31.
> > Use the IH_RETRY_CAM_ACK register.
> 

Hi Mukul,

> I think the doorbell path is not enabled for IH CAM so that’s probably the
> reason its not working.
> I think we need an update to the NBIO
> ih_doorbell_range() function to enable the doorbell path.

Can you please be more specific and elaborate on what we need to do on the NBIO 
ih_doorbell_range() function? Also, can you say which NBIO version is used for 
for Navi 3 and Navi 4 GPUs?

> Having said that, we intend to use doorbells on all ASICs to ack IH CAMs
> that support retry faults.
 
I'm OK with that, if the doorbell really works on these GPUs.

Thanks & best regards,
Timur

> 
> 
> >
> >
> > Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
> > ---
> > 
> >  drivers/gpu/drm/amd/amdgpu/ih_v6_0.c | 8 +++++++-
> >  1 file changed, 7 insertions(+), 1 deletion(-)
> >
> >
> >
> > diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
> > b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
> > index 65e5d21753f9..a97c85b0cb99 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
> > @@ -307,6 +307,11 @@ static int ih_v6_0_enable_ring(struct amdgpu_device
> > *adev,
> > 
> >       return 0;
> >  
> >  }
> >
> >
> >
> > +static void ih_v6_0_retry_cam_ack(struct amdgpu_device *adev, u32
> > +cam_index) {
> > +     WREG32_SOC15(OSSSYS, 0, regIH_RETRY_CAM_ACK, cam_index); }
> > +
> > 
> >  /**
> >  
> >   * ih_v6_0_irq_init - init and enable the interrupt ring
> >   *
> > 
> > @@ -807,7 +812,8 @@ static const struct amdgpu_ih_funcs ih_v6_0_funcs = {
> > 
> >       .get_wptr = ih_v6_0_get_wptr,
> >       .decode_iv = amdgpu_ih_decode_iv_helper,
> >       .decode_iv_ts = amdgpu_ih_decode_iv_ts_helper,
> > 
> > -     .set_rptr = ih_v6_0_set_rptr
> > +     .set_rptr = ih_v6_0_set_rptr,
> > +     .retry_cam_ack = ih_v6_0_retry_cam_ack,
> > 
> >  };
> >
> >
> >
> >  static void ih_v6_0_set_interrupt_funcs(struct amdgpu_device *adev)
> > 
> > --
> > 2.54.0
> 
> 

Re: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH 6.0

[Timur Kristóf]

On 2026. július 3., péntek 20:11:59 közép-európai nyári idő Joshi, Mukul 
wrote:
> > I still haven't found any explanation why we use a doorbell for the CAM in
> > the first
> place? Lijo, Alex, Felix does anybody know that?
> >
> 
> If I remember correctly, we weren't sure if the IH_RETRY_CAM_ACK register
> would be available under
> virtualization. And we were sure doorbells would
> work for both bare-metal and virtualization. 

Hi Mukul,

Can you clarify that please?

How are retry faults supposed to work under virtualization? Is the interrupt 
processed by the guest or the host? Why is the IH_RETRY_CAM_ACK register not 
available under virtualization?

Thanks & best regards,
Timur

Re: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH 6.0

[Alex Deucher]

On Fri, Jul 3, 2026 at 2:46 PM Timur Kristóf <timur.kristof@gmail.com> wrote:
>
> On 2026. július 3., péntek 19:46:03 közép-európai nyári idő Joshi, Mukul
> wrote:
> > AMD General
> >
> >
> > > -----Original Message-----
> > > From: Timur Kristóf <timur.kristof@gmail.com>
> > > Sent: Wednesday, July 1, 2026 12:17 PM
> > > To: amd-gfx@lists.freedesktop.org; Deucher, Alexander
> > > <Alexander.Deucher@amd.com>; Koenig, Christian
> > > <Christian.Koenig@amd.com>; Natalie Vock <natalie.vock@gmx.de>; Shetaia,
> > > Amir <Amir.Shetaia@amd.com>; Marek Olšák <maraeo@gmail.com>;
> > > Limonciello, Mario <Mario.Limonciello@amd.com>; Tvrtko Ursulin
> > > <tursulin@ursulin.net>; Kuehling, Felix <Felix.Kuehling@amd.com>; Lazar,
> > > Lijo
> > <Lijo.Lazar@amd.com>; He, Siwei <Siwei.He@amd.com>; Yang, Philip
> > > <Philip.Yang@amd.com>; Joshi, Mukul <Mukul.Joshi@amd.com>
> > > Cc: Timur Kristóf <timur.kristof@gmail.com>
> > > Subject: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH
> > > 6.0
> > >
> > >
> > >
> > > The doorbell is not working on Navi 31.
> > > Use the IH_RETRY_CAM_ACK register.
> >
>
> Hi Mukul,
>
> > I think the doorbell path is not enabled for IH CAM so that’s probably the
> > reason its not working.
> > I think we need an update to the NBIO
> > ih_doorbell_range() function to enable the doorbell path.
>
> Can you please be more specific and elaborate on what we need to do on the NBIO
> ih_doorbell_range() function? Also, can you say which NBIO version is used for
> for Navi 3 and Navi 4 GPUs?

navi3 dGPUs use nbio_v4_3.c.  navi3 APUs use nbio_v7_11.c (strix) and
nbio_v7_7.c (phoenix).  navi4 uses nbif_v6_3_1.c.

Alex

>
> > Having said that, we intend to use doorbells on all ASICs to ack IH CAMs
> > that support retry faults.
>
> I'm OK with that, if the doorbell really works on these GPUs.
>
> Thanks & best regards,
> Timur
>
> >
> >
> > >
> > >
> > > Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
> > > ---
> > >
> > >  drivers/gpu/drm/amd/amdgpu/ih_v6_0.c | 8 +++++++-
> > >  1 file changed, 7 insertions(+), 1 deletion(-)
> > >
> > >
> > >
> > > diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
> > > b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
> > > index 65e5d21753f9..a97c85b0cb99 100644
> > > --- a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
> > > +++ b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
> > > @@ -307,6 +307,11 @@ static int ih_v6_0_enable_ring(struct amdgpu_device
> > > *adev,
> > >
> > >       return 0;
> > >
> > >  }
> > >
> > >
> > >
> > > +static void ih_v6_0_retry_cam_ack(struct amdgpu_device *adev, u32
> > > +cam_index) {
> > > +     WREG32_SOC15(OSSSYS, 0, regIH_RETRY_CAM_ACK, cam_index); }
> > > +
> > >
> > >  /**
> > >
> > >   * ih_v6_0_irq_init - init and enable the interrupt ring
> > >   *
> > >
> > > @@ -807,7 +812,8 @@ static const struct amdgpu_ih_funcs ih_v6_0_funcs = {
> > >
> > >       .get_wptr = ih_v6_0_get_wptr,
> > >       .decode_iv = amdgpu_ih_decode_iv_helper,
> > >       .decode_iv_ts = amdgpu_ih_decode_iv_ts_helper,
> > >
> > > -     .set_rptr = ih_v6_0_set_rptr
> > > +     .set_rptr = ih_v6_0_set_rptr,
> > > +     .retry_cam_ack = ih_v6_0_retry_cam_ack,
> > >
> > >  };
> > >
> > >
> > >
> > >  static void ih_v6_0_set_interrupt_funcs(struct amdgpu_device *adev)
> > >
> > > --
> > > 2.54.0
> >
> >
>
>
>
>

RE: [PATCH 12/14] drm/amdgpu/ih7.0: Use MMIO ACK instead of doorbell for retry CAM on IH 7.0

[Joshi, Mukul]

AMD General

> -----Original Message-----
> From: Timur Kristóf <timur.kristof@gmail.com>
> Sent: Friday, July 3, 2026 2:41 PM
> To: amd-gfx@lists.freedesktop.org; Deucher, Alexander
> <Alexander.Deucher@amd.com>; Koenig, Christian
> <Christian.Koenig@amd.com>; Natalie Vock <natalie.vock@gmx.de>; Shetaia,
> Amir <Amir.Shetaia@amd.com>; Marek Olšák <maraeo@gmail.com>;
> Limonciello, Mario <Mario.Limonciello@amd.com>; Tvrtko Ursulin
> <tursulin@ursulin.net>; Kuehling, Felix <Felix.Kuehling@amd.com>; Lazar, Lijo
> <Lijo.Lazar@amd.com>; He, Siwei <Siwei.He@amd.com>; Yang, Philip
> <Philip.Yang@amd.com>; Joshi, Mukul <Mukul.Joshi@amd.com>
> Subject: Re: [PATCH 12/14] drm/amdgpu/ih7.0: Use MMIO ACK instead of
> doorbell for retry CAM on IH 7.0
>
> On 2026. július 3., péntek 18:31:12 közép-európai nyári idő Joshi, Mukul
> wrote:
> > AMD General
> >
> >
> > > -----Original Message-----
> > > From: amd-gfx <amd-gfx-bounces@lists.freedesktop.org> On Behalf Of
> > > Timur Kristóf
> > > Sent: Wednesday, July 1, 2026 12:17 PM
> > > To: amd-gfx@lists.freedesktop.org; Deucher, Alexander
> > > <Alexander.Deucher@amd.com>; Koenig, Christian
> > > <Christian.Koenig@amd.com>; Natalie Vock <natalie.vock@gmx.de>;
> > > Shetaia, Amir <Amir.Shetaia@amd.com>; Marek Olšák
> > > <maraeo@gmail.com>; Limonciello, Mario <Mario.Limonciello@amd.com>;
> > > Tvrtko Ursulin <tursulin@ursulin.net>; Kuehling, Felix
> > > <Felix.Kuehling@amd.com>; Lazar, Lijo
>  <Lijo.Lazar@amd.com>; He, Siwei <Siwei.He@amd.com>; Yang, Philip
> > > <Philip.Yang@amd.com>; Joshi, Mukul <Mukul.Joshi@amd.com>
> > > Cc: Timur Kristóf <timur.kristof@gmail.com>
> > > Subject: [PATCH 12/14] drm/amdgpu/ih7.0: Use MMIO ACK instead of
> > > doorbell for retry CAM on IH 7.0
> > >
> > >
> > >
> > > The doorbell is not working on Navi 48.
> > > Use the IH_RETRY_CAM_ACK register just like on Navi 3x.
> > >
> > >
> > >
> > > Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
> > > ---
> > >
> > >  drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 17 +----------------
> > >  1 file changed, 1 insertion(+), 16 deletions(-)
> > >
> > >
> > >
> > > diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> > > b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> > > index f1de6450a31d..291326d2ee8b 100644
> > > --- a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> > > +++ b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
> > > @@ -279,19 +279,9 @@ static int ih_v7_0_enable_ring(struct
> > > amdgpu_device *adev,
> > >
> > >       return 0;
> > >
> > >  }
> > >
> > >
> > >
> > > -static uint32_t ih_v7_0_setup_retry_doorbell(u32 doorbell_index) -{
> > > -     u32 val = 0;
> > > -
> > > -     val = REG_SET_FIELD(val, IH_DOORBELL_RPTR, OFFSET,
> > > doorbell_index);
> > > -     val = REG_SET_FIELD(val, IH_DOORBELL_RPTR, ENABLE, 1);
> > > -
> > > -     return val;
> > > -}
> > > -
> > >
> > >  static void ih_v7_0_retry_cam_ack(struct amdgpu_device *adev, u32
> > >
> > > cam_index)  {
> > > -     WDOORBELL32(adev->irq.retry_cam_doorbell_index, cam_index);
> > > +     WREG32_SOC15(OSSSYS, 0, regIH_RETRY_CAM_ACK, cam_index);
> > >
> > >  }
> > >
> > >
> > >
> > >  #define regIH_RING1_CLIENT_CFG_INDEX_V7_1             0x122
> > >
> > > @@ -401,11 +391,6 @@ static int ih_v7_0_irq_init(struct
> > > amdgpu_device
> > > *adev)
> > >
> > >       pci_set_master(adev->pdev);
> > >
> > >
> > >
> > >       if (amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7,
> > > 1, 0))
> > >
> > > {
> >
> >
> > We are re-purposing the IH 7.0 file for IH v7.1 since most of it is same.
> > The code being removed below is for IH 7.1 and not 7.0.
> > Using doorbells for acking INT CAM works completely fine for GFX 12.1
> > and IH 7.1.
> > I would recommend to drop this patch.
> >
>
> Hello Mukul,
>
> Unfortunately I couldn't figure out how to get the doorbell working on Navi 48
> and in the previous thread Amir said that it also didn't work for him, and that's
> why I decided to try the MMIO register.
>
> If you have an idea how to get the doorbell to work on Navi 48, I'm happy to
> try your suggestion.
>
> Ideally I'd prefer a solution here that works on both Navi 48 and GFX12.1 to
> keep the code maintainable. If it's not possible then we can also have a slightly
> different code path for the two different IH versions here.
>

Hi Timur,

As I had mentioned on the other patch, we need to enable the NBIO path to route
the doorbell correctly. That’s the reason wy your doorbells are not working on
Navi3x and Navi4x.

I am working on the change for Navi4x first and will send it out to you soon one I have verified it on
my end.

Regards,
Mukul


> Thanks & best regards,
> Timur
>
> >
> >
> > > -             /* Allocate the doorbell for IH Retry CAM */
> > > -             adev->irq.retry_cam_doorbell_index = (adev-
> > >
> > > >doorbell_index.ih + 2) << 1;
> > >
> > > -             WREG32_SOC15(OSSSYS, 0, regIH_DOORBELL_RETRY_CAM,
> > > -                             ih_v7_0_setup_retry_doorbell(adev-
> > >
> > > >irq.retry_cam_doorbell_index));
> > >
> > > -
> > >
> > >               /* Enable IH Retry CAM */
> > >               tmp = RREG32_SOC15(OSSSYS, 0,
> > >
> > > regIH_RETRY_INT_CAM_CNTL);
> > >
> > >               tmp = REG_SET_FIELD(tmp, IH_RETRY_INT_CAM_CNTL,
> > >
> > > ENABLE, 1);
> > > --
> > > 2.54.0
> >
> >
>
>
>

RE: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH 6.0

[Joshi, Mukul]

AMD General

> -----Original Message-----
> From: Timur Kristóf <timur.kristof@gmail.com>
> Sent: Friday, July 3, 2026 2:47 PM
> To: amd-gfx@lists.freedesktop.org; Deucher, Alexander
> <Alexander.Deucher@amd.com>; Koenig, Christian
> <Christian.Koenig@amd.com>; Natalie Vock <natalie.vock@gmx.de>; Shetaia,
> Amir <Amir.Shetaia@amd.com>; Marek Olšák <maraeo@gmail.com>;
> Limonciello, Mario <Mario.Limonciello@amd.com>; Tvrtko Ursulin
> <tursulin@ursulin.net>; Kuehling, Felix <Felix.Kuehling@amd.com>; Lazar, Lijo
> <Lijo.Lazar@amd.com>; He, Siwei <Siwei.He@amd.com>; Yang, Philip
> <Philip.Yang@amd.com>; Joshi, Mukul <Mukul.Joshi@amd.com>
> Subject: Re: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry
> CAM on IH 6.0
>
> On 2026. július 3., péntek 19:46:03 közép-európai nyári idő Joshi, Mukul
> wrote:
> > AMD General
> >
> >
> > > -----Original Message-----
> > > From: Timur Kristóf <timur.kristof@gmail.com>
> > > Sent: Wednesday, July 1, 2026 12:17 PM
> > > To: amd-gfx@lists.freedesktop.org; Deucher, Alexander
> > > <Alexander.Deucher@amd.com>; Koenig, Christian
> > > <Christian.Koenig@amd.com>; Natalie Vock <natalie.vock@gmx.de>;
> > > Shetaia, Amir <Amir.Shetaia@amd.com>; Marek Olšák
> > > <maraeo@gmail.com>; Limonciello, Mario <Mario.Limonciello@amd.com>;
> > > Tvrtko Ursulin <tursulin@ursulin.net>; Kuehling, Felix
> > > <Felix.Kuehling@amd.com>; Lazar, Lijo
> > <Lijo.Lazar@amd.com>; He, Siwei <Siwei.He@amd.com>; Yang, Philip
> > > <Philip.Yang@amd.com>; Joshi, Mukul <Mukul.Joshi@amd.com>
> > > Cc: Timur Kristóf <timur.kristof@gmail.com>
> > > Subject: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM
> > > on IH
> > > 6.0
> > >
> > >
> > >
> > > The doorbell is not working on Navi 31.
> > > Use the IH_RETRY_CAM_ACK register.
> >
>
> Hi Mukul,
>
> > I think the doorbell path is not enabled for IH CAM so that’s probably
> > the reason its not working.
> > I think we need an update to the NBIO
> > ih_doorbell_range() function to enable the doorbell path.
>
Hi Timur,

> Can you please be more specific and elaborate on what we need to do on the
> NBIO
> ih_doorbell_range() function?

Through NBIO, we route the doorbell to the correct HW block.
That is currently not enabled/done for the IH CAM doorbell.
So, you basically need to enable it inside ih_doorbell_range() function.

>Also, can you say which NBIO version is used
> for for Navi 3 and Navi 4 GPUs?
>
Navi3 --> NBIO v4.3 (nbio_v4_3.c)
Navi4 --> NBIO v6.3.1 (nbif_v6_3_1.c)

I am working on the change that you need to enable the doorbell path for Navi4x first.
Once that is working, I can send you a test patch for enabling it on Navi3x.

Regards,
Mukul


> > Having said that, we intend to use doorbells on all ASICs to ack IH
> > CAMs that support retry faults.
>
> I'm OK with that, if the doorbell really works on these GPUs.
>
> Thanks & best regards,
> Timur
>
> >
> >
> > >
> > >
> > > Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
> > > ---
> > >
> > >  drivers/gpu/drm/amd/amdgpu/ih_v6_0.c | 8 +++++++-
> > >  1 file changed, 7 insertions(+), 1 deletion(-)
> > >
> > >
> > >
> > > diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
> > > b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
> > > index 65e5d21753f9..a97c85b0cb99 100644
> > > --- a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
> > > +++ b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
> > > @@ -307,6 +307,11 @@ static int ih_v6_0_enable_ring(struct
> > > amdgpu_device *adev,
> > >
> > >       return 0;
> > >
> > >  }
> > >
> > >
> > >
> > > +static void ih_v6_0_retry_cam_ack(struct amdgpu_device *adev, u32
> > > +cam_index) {
> > > +     WREG32_SOC15(OSSSYS, 0, regIH_RETRY_CAM_ACK, cam_index); }
> > > +
> > >
> > >  /**
> > >
> > >   * ih_v6_0_irq_init - init and enable the interrupt ring
> > >   *
> > >
> > > @@ -807,7 +812,8 @@ static const struct amdgpu_ih_funcs
> > > ih_v6_0_funcs = {
> > >
> > >       .get_wptr = ih_v6_0_get_wptr,
> > >       .decode_iv = amdgpu_ih_decode_iv_helper,
> > >       .decode_iv_ts = amdgpu_ih_decode_iv_ts_helper,
> > >
> > > -     .set_rptr = ih_v6_0_set_rptr
> > > +     .set_rptr = ih_v6_0_set_rptr,
> > > +     .retry_cam_ack = ih_v6_0_retry_cam_ack,
> > >
> > >  };
> > >
> > >
> > >
> > >  static void ih_v6_0_set_interrupt_funcs(struct amdgpu_device *adev)
> > >
> > > --
> > > 2.54.0
> >
> >
>
>
>

Re: [PATCH 12/14] drm/amdgpu/ih7.0: Use MMIO ACK instead of doorbell for retry CAM on IH 7.0

[Mukul Joshi]

[-- Attachment #1: Type: text/plain, Size: 6113 bytes --]

Hi Timur,
For Navi4x, can you please give this change to enable the doorbell path 
a try in
addition to your changes and use doorbells then to clear the CAM entries:


diff --git a/drivers/gpu/drm/amd/amdgpu/nbif_v6_3_1.c 
b/drivers/gpu/drm/amd/amdgpu/nbif_v6_3_1.c
index b6f832c53860..8aefe64fcd41 100644
--- a/drivers/gpu/drm/amd/amdgpu/nbif_v6_3_1.c
+++ b/drivers/gpu/drm/amd/amdgpu/nbif_v6_3_1.c
@@ -251,7 +251,7 @@ static void nbif_v6_3_1_ih_doorbell_range(struct 
amdgpu_device *adev,
                 ih_doorbell_range = REG_SET_FIELD(ih_doorbell_range,
GDC_S2A0_S2A_DOORBELL_ENTRY_1_CTRL,
S2A_DOORBELL_PORT1_RANGE_SIZE,
-                                                 2);
+                                                 8);


Please let us know how it goes.


Thanks and regards,
Mukul

On 7/6/2026 1:48 PM, Joshi, Mukul wrote:
> AMD General
>
>> -----Original Message-----
>> From: Timur Kristóf<timur.kristof@gmail.com>
>> Sent: Friday, July 3, 2026 2:41 PM
>> To:amd-gfx@lists.freedesktop.org; Deucher, Alexander
>> <Alexander.Deucher@amd.com>; Koenig, Christian
>> <Christian.Koenig@amd.com>; Natalie Vock<natalie.vock@gmx.de>; Shetaia,
>> Amir<Amir.Shetaia@amd.com>; Marek Olšák<maraeo@gmail.com>;
>> Limonciello, Mario<Mario.Limonciello@amd.com>; Tvrtko Ursulin
>> <tursulin@ursulin.net>; Kuehling, Felix<Felix.Kuehling@amd.com>; Lazar, Lijo
>> <Lijo.Lazar@amd.com>; He, Siwei<Siwei.He@amd.com>; Yang, Philip
>> <Philip.Yang@amd.com>; Joshi, Mukul<Mukul.Joshi@amd.com>
>> Subject: Re: [PATCH 12/14] drm/amdgpu/ih7.0: Use MMIO ACK instead of
>> doorbell for retry CAM on IH 7.0
>>
>> On 2026. július 3., péntek 18:31:12 közép-európai nyári idő Joshi, Mukul
>> wrote:
>>> AMD General
>>>
>>>
>>>> -----Original Message-----
>>>> From: amd-gfx<amd-gfx-bounces@lists.freedesktop.org> On Behalf Of
>>>> Timur Kristóf
>>>> Sent: Wednesday, July 1, 2026 12:17 PM
>>>> To:amd-gfx@lists.freedesktop.org; Deucher, Alexander
>>>> <Alexander.Deucher@amd.com>; Koenig, Christian
>>>> <Christian.Koenig@amd.com>; Natalie Vock<natalie.vock@gmx.de>;
>>>> Shetaia, Amir<Amir.Shetaia@amd.com>; Marek Olšák
>>>> <maraeo@gmail.com>; Limonciello, Mario<Mario.Limonciello@amd.com>;
>>>> Tvrtko Ursulin<tursulin@ursulin.net>; Kuehling, Felix
>>>> <Felix.Kuehling@amd.com>; Lazar, Lijo
>>   <Lijo.Lazar@amd.com>; He, Siwei<Siwei.He@amd.com>; Yang, Philip
>>>> <Philip.Yang@amd.com>; Joshi, Mukul<Mukul.Joshi@amd.com>
>>>> Cc: Timur Kristóf<timur.kristof@gmail.com>
>>>> Subject: [PATCH 12/14] drm/amdgpu/ih7.0: Use MMIO ACK instead of
>>>> doorbell for retry CAM on IH 7.0
>>>>
>>>>
>>>>
>>>> The doorbell is not working on Navi 48.
>>>> Use the IH_RETRY_CAM_ACK register just like on Navi 3x.
>>>>
>>>>
>>>>
>>>> Signed-off-by: Timur Kristóf<timur.kristof@gmail.com>
>>>> ---
>>>>
>>>>   drivers/gpu/drm/amd/amdgpu/ih_v7_0.c | 17 +----------------
>>>>   1 file changed, 1 insertion(+), 16 deletions(-)
>>>>
>>>>
>>>>
>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
>>>> b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
>>>> index f1de6450a31d..291326d2ee8b 100644
>>>> --- a/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
>>>> +++ b/drivers/gpu/drm/amd/amdgpu/ih_v7_0.c
>>>> @@ -279,19 +279,9 @@ static int ih_v7_0_enable_ring(struct
>>>> amdgpu_device *adev,
>>>>
>>>>        return 0;
>>>>
>>>>   }
>>>>
>>>>
>>>>
>>>> -static uint32_t ih_v7_0_setup_retry_doorbell(u32 doorbell_index) -{
>>>> -     u32 val = 0;
>>>> -
>>>> -     val = REG_SET_FIELD(val, IH_DOORBELL_RPTR, OFFSET,
>>>> doorbell_index);
>>>> -     val = REG_SET_FIELD(val, IH_DOORBELL_RPTR, ENABLE, 1);
>>>> -
>>>> -     return val;
>>>> -}
>>>> -
>>>>
>>>>   static void ih_v7_0_retry_cam_ack(struct amdgpu_device *adev, u32
>>>>
>>>> cam_index)  {
>>>> -     WDOORBELL32(adev->irq.retry_cam_doorbell_index, cam_index);
>>>> +     WREG32_SOC15(OSSSYS, 0, regIH_RETRY_CAM_ACK, cam_index);
>>>>
>>>>   }
>>>>
>>>>
>>>>
>>>>   #define regIH_RING1_CLIENT_CFG_INDEX_V7_1             0x122
>>>>
>>>> @@ -401,11 +391,6 @@ static int ih_v7_0_irq_init(struct
>>>> amdgpu_device
>>>> *adev)
>>>>
>>>>        pci_set_master(adev->pdev);
>>>>
>>>>
>>>>
>>>>        if (amdgpu_ip_version(adev, OSSSYS_HWIP, 0) == IP_VERSION(7,
>>>> 1, 0))
>>>>
>>>> {
>>>
>>> We are re-purposing the IH 7.0 file for IH v7.1 since most of it is same.
>>> The code being removed below is for IH 7.1 and not 7.0.
>>> Using doorbells for acking INT CAM works completely fine for GFX 12.1
>>> and IH 7.1.
>>> I would recommend to drop this patch.
>>>
>> Hello Mukul,
>>
>> Unfortunately I couldn't figure out how to get the doorbell working on Navi 48
>> and in the previous thread Amir said that it also didn't work for him, and that's
>> why I decided to try the MMIO register.
>>
>> If you have an idea how to get the doorbell to work on Navi 48, I'm happy to
>> try your suggestion.
>>
>> Ideally I'd prefer a solution here that works on both Navi 48 and GFX12.1 to
>> keep the code maintainable. If it's not possible then we can also have a slightly
>> different code path for the two different IH versions here.
>>
> Hi Timur,
>
> As I had mentioned on the other patch, we need to enable the NBIO path to route
> the doorbell correctly. That’s the reason wy your doorbells are not working on
> Navi3x and Navi4x.
>
> I am working on the change for Navi4x first and will send it out to you soon one I have verified it on
> my end.
>
> Regards,
> Mukul
>
>
>> Thanks & best regards,
>> Timur
>>
>>>
>>>> -             /* Allocate the doorbell for IH Retry CAM */
>>>> -             adev->irq.retry_cam_doorbell_index = (adev-
>>>>
>>>>> doorbell_index.ih + 2) << 1;
>>>> -             WREG32_SOC15(OSSSYS, 0, regIH_DOORBELL_RETRY_CAM,
>>>> -                             ih_v7_0_setup_retry_doorbell(adev-
>>>>
>>>>> irq.retry_cam_doorbell_index));
>>>> -
>>>>
>>>>                /* Enable IH Retry CAM */
>>>>                tmp = RREG32_SOC15(OSSSYS, 0,
>>>>
>>>> regIH_RETRY_INT_CAM_CNTL);
>>>>
>>>>                tmp = REG_SET_FIELD(tmp, IH_RETRY_INT_CAM_CNTL,
>>>>
>>>> ENABLE, 1);
>>>> --
>>>> 2.54.0
>>>
>>
>>

[-- Attachment #2: Type: text/html, Size: 10948 bytes --]

Re: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM on IH 6.0

[Mukul Joshi]

[-- Attachment #1: Type: text/plain, Size: 5257 bytes --]

Hi Timur,

For Navi3x, please make the similar change as I suggested for Navi4x to 
enable the doorbell path along with your
existing changes and use doorbell to clear the CAM entries.

diff --git a/drivers/gpu/drm/amd/amdgpu/nbio_v4_3.c 
b/drivers/gpu/drm/amd/amdgpu/nbio_v4_3.c
index f89e5f40e1a5..a66e3073aed8 100644
--- a/drivers/gpu/drm/amd/amdgpu/nbio_v4_3.c
+++ b/drivers/gpu/drm/amd/amdgpu/nbio_v4_3.c
@@ -199,7 +199,7 @@ static void nbio_v4_3_ih_doorbell_range(struct 
amdgpu_device *adev,
                 ih_doorbell_range = REG_SET_FIELD(ih_doorbell_range,
S2A_DOORBELL_ENTRY_1_CTRL,
S2A_DOORBELL_PORT1_RANGE_SIZE,
-                                                 2);
+                                                 8);
                 ih_doorbell_range = REG_SET_FIELD(ih_doorbell_range,

Please let us know how it goes.

Thanks and regards,

Mukul


On 7/6/2026 2:04 PM, Joshi, Mukul wrote:
> AMD General
>
>> -----Original Message-----
>> From: Timur Kristóf<timur.kristof@gmail.com>
>> Sent: Friday, July 3, 2026 2:47 PM
>> To:amd-gfx@lists.freedesktop.org; Deucher, Alexander
>> <Alexander.Deucher@amd.com>; Koenig, Christian
>> <Christian.Koenig@amd.com>; Natalie Vock<natalie.vock@gmx.de>; Shetaia,
>> Amir<Amir.Shetaia@amd.com>; Marek Olšák<maraeo@gmail.com>;
>> Limonciello, Mario<Mario.Limonciello@amd.com>; Tvrtko Ursulin
>> <tursulin@ursulin.net>; Kuehling, Felix<Felix.Kuehling@amd.com>; Lazar, Lijo
>> <Lijo.Lazar@amd.com>; He, Siwei<Siwei.He@amd.com>; Yang, Philip
>> <Philip.Yang@amd.com>; Joshi, Mukul<Mukul.Joshi@amd.com>
>> Subject: Re: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry
>> CAM on IH 6.0
>>
>> On 2026. július 3., péntek 19:46:03 közép-európai nyári idő Joshi, Mukul
>> wrote:
>>> AMD General
>>>
>>>
>>>> -----Original Message-----
>>>> From: Timur Kristóf<timur.kristof@gmail.com>
>>>> Sent: Wednesday, July 1, 2026 12:17 PM
>>>> To:amd-gfx@lists.freedesktop.org; Deucher, Alexander
>>>> <Alexander.Deucher@amd.com>; Koenig, Christian
>>>> <Christian.Koenig@amd.com>; Natalie Vock<natalie.vock@gmx.de>;
>>>> Shetaia, Amir<Amir.Shetaia@amd.com>; Marek Olšák
>>>> <maraeo@gmail.com>; Limonciello, Mario<Mario.Limonciello@amd.com>;
>>>> Tvrtko Ursulin<tursulin@ursulin.net>; Kuehling, Felix
>>>> <Felix.Kuehling@amd.com>; Lazar, Lijo
>>> <Lijo.Lazar@amd.com>; He, Siwei<Siwei.He@amd.com>; Yang, Philip
>>>> <Philip.Yang@amd.com>; Joshi, Mukul<Mukul.Joshi@amd.com>
>>>> Cc: Timur Kristóf<timur.kristof@gmail.com>
>>>> Subject: [PATCH 11/14] drm/amdgpu/ih6.0: Use MMIO ACK for retry CAM
>>>> on IH
>>>> 6.0
>>>>
>>>>
>>>>
>>>> The doorbell is not working on Navi 31.
>>>> Use the IH_RETRY_CAM_ACK register.
>> Hi Mukul,
>>
>>> I think the doorbell path is not enabled for IH CAM so that’s probably
>>> the reason its not working.
>>> I think we need an update to the NBIO
>>> ih_doorbell_range() function to enable the doorbell path.
> Hi Timur,
>
>> Can you please be more specific and elaborate on what we need to do on the
>> NBIO
>> ih_doorbell_range() function?
> Through NBIO, we route the doorbell to the correct HW block.
> That is currently not enabled/done for the IH CAM doorbell.
> So, you basically need to enable it inside ih_doorbell_range() function.
>
>> Also, can you say which NBIO version is used
>> for for Navi 3 and Navi 4 GPUs?
>>
> Navi3 --> NBIO v4.3 (nbio_v4_3.c)
> Navi4 --> NBIO v6.3.1 (nbif_v6_3_1.c)
>
> I am working on the change that you need to enable the doorbell path for Navi4x first.
> Once that is working, I can send you a test patch for enabling it on Navi3x.
>
> Regards,
> Mukul
>
>
>>> Having said that, we intend to use doorbells on all ASICs to ack IH
>>> CAMs that support retry faults.
>> I'm OK with that, if the doorbell really works on these GPUs.
>>
>> Thanks & best regards,
>> Timur
>>
>>>
>>>>
>>>> Signed-off-by: Timur Kristóf<timur.kristof@gmail.com>
>>>> ---
>>>>
>>>>   drivers/gpu/drm/amd/amdgpu/ih_v6_0.c | 8 +++++++-
>>>>   1 file changed, 7 insertions(+), 1 deletion(-)
>>>>
>>>>
>>>>
>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
>>>> b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
>>>> index 65e5d21753f9..a97c85b0cb99 100644
>>>> --- a/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
>>>> +++ b/drivers/gpu/drm/amd/amdgpu/ih_v6_0.c
>>>> @@ -307,6 +307,11 @@ static int ih_v6_0_enable_ring(struct
>>>> amdgpu_device *adev,
>>>>
>>>>        return 0;
>>>>
>>>>   }
>>>>
>>>>
>>>>
>>>> +static void ih_v6_0_retry_cam_ack(struct amdgpu_device *adev, u32
>>>> +cam_index) {
>>>> +     WREG32_SOC15(OSSSYS, 0, regIH_RETRY_CAM_ACK, cam_index); }
>>>> +
>>>>
>>>>   /**
>>>>
>>>>    * ih_v6_0_irq_init - init and enable the interrupt ring
>>>>    *
>>>>
>>>> @@ -807,7 +812,8 @@ static const struct amdgpu_ih_funcs
>>>> ih_v6_0_funcs = {
>>>>
>>>>        .get_wptr = ih_v6_0_get_wptr,
>>>>        .decode_iv = amdgpu_ih_decode_iv_helper,
>>>>        .decode_iv_ts = amdgpu_ih_decode_iv_ts_helper,
>>>>
>>>> -     .set_rptr = ih_v6_0_set_rptr
>>>> +     .set_rptr = ih_v6_0_set_rptr,
>>>> +     .retry_cam_ack = ih_v6_0_retry_cam_ack,
>>>>
>>>>   };
>>>>
>>>>
>>>>
>>>>   static void ih_v6_0_set_interrupt_funcs(struct amdgpu_device *adev)
>>>>
>>>> --
>>>> 2.54.0
>>>
>>
>>

[-- Attachment #2: Type: text/html, Size: 10344 bytes --]