* [PATCH 1/2] drm/amdgpu: enable Navi 48-bit IH timestamp counter @ 2021-11-23 16:03 Philip Yang 2021-11-23 16:03 ` [PATCH 2/2] drm/amdgpu: enable Navi retry fault wptr overflow Philip Yang 2021-11-24 7:08 ` [PATCH 1/2] drm/amdgpu: enable Navi 48-bit IH timestamp counter Christian König 0 siblings, 2 replies; 4+ messages in thread From: Philip Yang @ 2021-11-23 16:03 UTC (permalink / raw) To: amd-gfx; +Cc: alex.sierra, Philip Yang, Felix.Kuehling By default this timestamp is 32 bit counter. It gets overflowed in around 10 minutes. Signed-off-by: Philip Yang <Philip.Yang@amd.com> --- drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c index 1d8414c3fadb..dafad6030947 100644 --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c @@ -160,6 +160,7 @@ static int navi10_ih_toggle_ring_interrupts(struct amdgpu_device *adev, tmp = RREG32(ih_regs->ih_rb_cntl); tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, RB_ENABLE, (enable ? 1 : 0)); + tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, RB_GPU_TS_ENABLE, 1); /* enable_intr field is only valid in ring0 */ if (ih == &adev->irq.ih) tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, ENABLE_INTR, (enable ? 1 : 0)); -- 2.17.1 ^ permalink raw reply related [flat|nested] 4+ messages in thread
* [PATCH 2/2] drm/amdgpu: enable Navi retry fault wptr overflow 2021-11-23 16:03 [PATCH 1/2] drm/amdgpu: enable Navi 48-bit IH timestamp counter Philip Yang @ 2021-11-23 16:03 ` Philip Yang 2021-11-23 16:58 ` Felix Kuehling 2021-11-24 7:08 ` [PATCH 1/2] drm/amdgpu: enable Navi 48-bit IH timestamp counter Christian König 1 sibling, 1 reply; 4+ messages in thread From: Philip Yang @ 2021-11-23 16:03 UTC (permalink / raw) To: amd-gfx; +Cc: alex.sierra, Philip Yang, Felix.Kuehling If xnack is on, VM retry fault interrupt send to IH ring1, and ring1 will be full quickly. IH cannot receive other interrupts, this causes deadlock if migrating buffer using sdma and waiting for sdma done while handling retry fault. Remove VMC from IH storm client, enable ring1 write pointer overflow, then IH will drop retry fault interrupts and be able to receive other interrupts while driver is handling retry fault. IH ring1 write pointer doesn't writeback to memory by IH, and ring1 write pointer recorded by self-irq is not updated, so always read the latest ring1 write pointer from register. Signed-off-by: Philip Yang <Philip.Yang@amd.com> --- drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 33 ++++++++++---------------- 1 file changed, 12 insertions(+), 21 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c index dafad6030947..38241cf0e1f1 100644 --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c @@ -276,10 +276,8 @@ static int navi10_ih_enable_ring(struct amdgpu_device *adev, tmp = navi10_ih_rb_cntl(ih, tmp); if (ih == &adev->irq.ih) tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, RPTR_REARM, !!adev->irq.msi_enabled); - if (ih == &adev->irq.ih1) { - tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, WPTR_OVERFLOW_ENABLE, 0); + if (ih == &adev->irq.ih1) tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, RB_FULL_DRAIN_ENABLE, 1); - } if (amdgpu_sriov_vf(adev) && amdgpu_sriov_reg_indirect_ih(adev)) { if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp)) { @@ -320,7 +318,6 @@ static int navi10_ih_irq_init(struct amdgpu_device *adev) { struct amdgpu_ih_ring *ih[] = {&adev->irq.ih, &adev->irq.ih1, &adev->irq.ih2}; u32 ih_chicken; - u32 tmp; int ret; int i; @@ -364,15 +361,6 @@ static int navi10_ih_irq_init(struct amdgpu_device *adev) adev->nbio.funcs->ih_doorbell_range(adev, ih[0]->use_doorbell, ih[0]->doorbell_index); - tmp = RREG32_SOC15(OSSSYS, 0, mmIH_STORM_CLIENT_LIST_CNTL); - tmp = REG_SET_FIELD(tmp, IH_STORM_CLIENT_LIST_CNTL, - CLIENT18_IS_STORM_CLIENT, 1); - WREG32_SOC15(OSSSYS, 0, mmIH_STORM_CLIENT_LIST_CNTL, tmp); - - tmp = RREG32_SOC15(OSSSYS, 0, mmIH_INT_FLOOD_CNTL); - tmp = REG_SET_FIELD(tmp, IH_INT_FLOOD_CNTL, FLOOD_CNTL_ENABLE, 1); - WREG32_SOC15(OSSSYS, 0, mmIH_INT_FLOOD_CNTL, tmp); - pci_set_master(adev->pdev); /* enable interrupts */ @@ -421,12 +409,19 @@ static u32 navi10_ih_get_wptr(struct amdgpu_device *adev, u32 wptr, tmp; struct amdgpu_ih_regs *ih_regs; - wptr = le32_to_cpu(*ih->wptr_cpu); - ih_regs = &ih->ih_regs; + if (ih == &adev->irq.ih) { + /* Only ring0 supports writeback. On other rings fall back + * to register-based code with overflow checking below. + */ + wptr = le32_to_cpu(*ih->wptr_cpu); - if (!REG_GET_FIELD(wptr, IH_RB_WPTR, RB_OVERFLOW)) - goto out; + if (!REG_GET_FIELD(wptr, IH_RB_WPTR, RB_OVERFLOW)) + goto out; + } + ih_regs = &ih->ih_regs; + + /* Double check that the overflow wasn't already cleared. */ wptr = RREG32_NO_KIQ(ih_regs->ih_rb_wptr); if (!REG_GET_FIELD(wptr, IH_RB_WPTR, RB_OVERFLOW)) goto out; @@ -514,15 +509,11 @@ static int navi10_ih_self_irq(struct amdgpu_device *adev, struct amdgpu_irq_src *source, struct amdgpu_iv_entry *entry) { - uint32_t wptr = cpu_to_le32(entry->src_data[0]); - switch (entry->ring_id) { case 1: - *adev->irq.ih1.wptr_cpu = wptr; schedule_work(&adev->irq.ih1_work); break; case 2: - *adev->irq.ih2.wptr_cpu = wptr; schedule_work(&adev->irq.ih2_work); break; default: break; -- 2.17.1 ^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH 2/2] drm/amdgpu: enable Navi retry fault wptr overflow 2021-11-23 16:03 ` [PATCH 2/2] drm/amdgpu: enable Navi retry fault wptr overflow Philip Yang @ 2021-11-23 16:58 ` Felix Kuehling 0 siblings, 0 replies; 4+ messages in thread From: Felix Kuehling @ 2021-11-23 16:58 UTC (permalink / raw) To: Philip Yang, amd-gfx; +Cc: alex.sierra Am 2021-11-23 um 11:03 a.m. schrieb Philip Yang: > If xnack is on, VM retry fault interrupt send to IH ring1, and ring1 > will be full quickly. IH cannot receive other interrupts, this causes > deadlock if migrating buffer using sdma and waiting for sdma done > while handling retry fault. > > Remove VMC from IH storm client, enable ring1 write pointer > overflow, then IH will drop retry fault interrupts and be able to receive > other interrupts while driver is handling retry fault. > > IH ring1 write pointer doesn't writeback to memory by IH, and ring1 > write pointer recorded by self-irq is not updated, so always read > the latest ring1 write pointer from register. > > Signed-off-by: Philip Yang <Philip.Yang@amd.com> Looks like the same thing you did in this commit for Vega: b672cb1eee59 drm/amdgpu: enable retry fault wptr overflow This series is Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> > --- > drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 33 ++++++++++---------------- > 1 file changed, 12 insertions(+), 21 deletions(-) > > diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c > index dafad6030947..38241cf0e1f1 100644 > --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c > +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c > @@ -276,10 +276,8 @@ static int navi10_ih_enable_ring(struct amdgpu_device *adev, > tmp = navi10_ih_rb_cntl(ih, tmp); > if (ih == &adev->irq.ih) > tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, RPTR_REARM, !!adev->irq.msi_enabled); > - if (ih == &adev->irq.ih1) { > - tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, WPTR_OVERFLOW_ENABLE, 0); > + if (ih == &adev->irq.ih1) > tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, RB_FULL_DRAIN_ENABLE, 1); > - } > > if (amdgpu_sriov_vf(adev) && amdgpu_sriov_reg_indirect_ih(adev)) { > if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp)) { > @@ -320,7 +318,6 @@ static int navi10_ih_irq_init(struct amdgpu_device *adev) > { > struct amdgpu_ih_ring *ih[] = {&adev->irq.ih, &adev->irq.ih1, &adev->irq.ih2}; > u32 ih_chicken; > - u32 tmp; > int ret; > int i; > > @@ -364,15 +361,6 @@ static int navi10_ih_irq_init(struct amdgpu_device *adev) > adev->nbio.funcs->ih_doorbell_range(adev, ih[0]->use_doorbell, > ih[0]->doorbell_index); > > - tmp = RREG32_SOC15(OSSSYS, 0, mmIH_STORM_CLIENT_LIST_CNTL); > - tmp = REG_SET_FIELD(tmp, IH_STORM_CLIENT_LIST_CNTL, > - CLIENT18_IS_STORM_CLIENT, 1); > - WREG32_SOC15(OSSSYS, 0, mmIH_STORM_CLIENT_LIST_CNTL, tmp); > - > - tmp = RREG32_SOC15(OSSSYS, 0, mmIH_INT_FLOOD_CNTL); > - tmp = REG_SET_FIELD(tmp, IH_INT_FLOOD_CNTL, FLOOD_CNTL_ENABLE, 1); > - WREG32_SOC15(OSSSYS, 0, mmIH_INT_FLOOD_CNTL, tmp); > - > pci_set_master(adev->pdev); > > /* enable interrupts */ > @@ -421,12 +409,19 @@ static u32 navi10_ih_get_wptr(struct amdgpu_device *adev, > u32 wptr, tmp; > struct amdgpu_ih_regs *ih_regs; > > - wptr = le32_to_cpu(*ih->wptr_cpu); > - ih_regs = &ih->ih_regs; > + if (ih == &adev->irq.ih) { > + /* Only ring0 supports writeback. On other rings fall back > + * to register-based code with overflow checking below. > + */ > + wptr = le32_to_cpu(*ih->wptr_cpu); > > - if (!REG_GET_FIELD(wptr, IH_RB_WPTR, RB_OVERFLOW)) > - goto out; > + if (!REG_GET_FIELD(wptr, IH_RB_WPTR, RB_OVERFLOW)) > + goto out; > + } > > + ih_regs = &ih->ih_regs; > + > + /* Double check that the overflow wasn't already cleared. */ > wptr = RREG32_NO_KIQ(ih_regs->ih_rb_wptr); > if (!REG_GET_FIELD(wptr, IH_RB_WPTR, RB_OVERFLOW)) > goto out; > @@ -514,15 +509,11 @@ static int navi10_ih_self_irq(struct amdgpu_device *adev, > struct amdgpu_irq_src *source, > struct amdgpu_iv_entry *entry) > { > - uint32_t wptr = cpu_to_le32(entry->src_data[0]); > - > switch (entry->ring_id) { > case 1: > - *adev->irq.ih1.wptr_cpu = wptr; > schedule_work(&adev->irq.ih1_work); > break; > case 2: > - *adev->irq.ih2.wptr_cpu = wptr; > schedule_work(&adev->irq.ih2_work); > break; > default: break; ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH 1/2] drm/amdgpu: enable Navi 48-bit IH timestamp counter 2021-11-23 16:03 [PATCH 1/2] drm/amdgpu: enable Navi 48-bit IH timestamp counter Philip Yang 2021-11-23 16:03 ` [PATCH 2/2] drm/amdgpu: enable Navi retry fault wptr overflow Philip Yang @ 2021-11-24 7:08 ` Christian König 1 sibling, 0 replies; 4+ messages in thread From: Christian König @ 2021-11-24 7:08 UTC (permalink / raw) To: Philip Yang, amd-gfx; +Cc: alex.sierra, Felix.Kuehling Am 23.11.21 um 17:03 schrieb Philip Yang: > By default this timestamp is 32 bit counter. It gets overflowed in > around 10 minutes. > > Signed-off-by: Philip Yang <Philip.Yang@amd.com> Reviewed-by: Christian König <christian.koenig@amd.com> for the series. > --- > drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c > index 1d8414c3fadb..dafad6030947 100644 > --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c > +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c > @@ -160,6 +160,7 @@ static int navi10_ih_toggle_ring_interrupts(struct amdgpu_device *adev, > > tmp = RREG32(ih_regs->ih_rb_cntl); > tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, RB_ENABLE, (enable ? 1 : 0)); > + tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, RB_GPU_TS_ENABLE, 1); > /* enable_intr field is only valid in ring0 */ > if (ih == &adev->irq.ih) > tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, ENABLE_INTR, (enable ? 1 : 0)); ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2021-11-24 7:08 UTC | newest] Thread overview: 4+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2021-11-23 16:03 [PATCH 1/2] drm/amdgpu: enable Navi 48-bit IH timestamp counter Philip Yang 2021-11-23 16:03 ` [PATCH 2/2] drm/amdgpu: enable Navi retry fault wptr overflow Philip Yang 2021-11-23 16:58 ` Felix Kuehling 2021-11-24 7:08 ` [PATCH 1/2] drm/amdgpu: enable Navi 48-bit IH timestamp counter Christian König
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox