[PATCH] drm/amdgpu: Assign correct bits for SDMA HDP flush

[PATCH] drm/amdgpu: Assign correct bits for SDMA HDP flush

[Lijo Lazar]

HDP Flush request bit can be kept unique per AID, and doesn't need to be
unique SOC-wide. Assign only bits 10-13 for SDMA v4.4.2.

Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.c b/drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.c
index f8e2cd514493..09e45ef16c0d 100644
--- a/drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.c
+++ b/drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.c
@@ -368,7 +368,8 @@ static void sdma_v4_4_2_ring_emit_hdp_flush(struct amdgpu_ring *ring)
 	u32 ref_and_mask = 0;
 	const struct nbio_hdp_flush_reg *nbio_hf_reg = adev->nbio.hdp_flush_reg;
 
-	ref_and_mask = nbio_hf_reg->ref_and_mask_sdma0 << ring->me;
+	ref_and_mask = nbio_hf_reg->ref_and_mask_sdma0
+		       << (ring->me % adev->sdma.num_inst_per_aid);
 
 	sdma_v4_4_2_wait_reg_mem(ring, 0, 1,
 			       adev->nbio.funcs->get_hdp_flush_done_offset(adev),
-- 
2.25.1

Re: [PATCH] drm/amdgpu: Assign correct bits for SDMA HDP flush

[Alex Deucher]

On Wed, Apr 17, 2024 at 8:07 AM Lijo Lazar <lijo.lazar@amd.com> wrote:
>
> HDP Flush request bit can be kept unique per AID, and doesn't need to be
> unique SOC-wide. Assign only bits 10-13 for SDMA v4.4.2.
>
> Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>

Acked-by: Alex Deucher <alexander.deucher@amd.com>

> ---
>  drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.c b/drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.c
> index f8e2cd514493..09e45ef16c0d 100644
> --- a/drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.c
> +++ b/drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.c
> @@ -368,7 +368,8 @@ static void sdma_v4_4_2_ring_emit_hdp_flush(struct amdgpu_ring *ring)
>         u32 ref_and_mask = 0;
>         const struct nbio_hdp_flush_reg *nbio_hf_reg = adev->nbio.hdp_flush_reg;
>
> -       ref_and_mask = nbio_hf_reg->ref_and_mask_sdma0 << ring->me;
> +       ref_and_mask = nbio_hf_reg->ref_and_mask_sdma0
> +                      << (ring->me % adev->sdma.num_inst_per_aid);
>
>         sdma_v4_4_2_wait_reg_mem(ring, 0, 1,
>                                adev->nbio.funcs->get_hdp_flush_done_offset(adev),
> --
> 2.25.1
>

Re: [PATCH] drm/amdgpu: Assign correct bits for SDMA HDP flush

[Harry Wentland]

This patch is causing crashes of Manor Lords on my Navi 21 on the 6.8.9 
stable kernel. It leads to an assertion failure in wine:

File: ../src-wine/dlls/winevulkan/loader_thunks.c
Line: 3621

Expression "!status && vkEndCommandBuffer""

This happens both with radv and amdvlk. It starts happening on v6.8.8 
with this patch. The previous patch (drm/amdgpu/sdma5.2: use legacy HDP 
flush for SDMA2/3) is fine.

Harry

On 2024-04-17 09:19, Alex Deucher wrote:
> On Wed, Apr 17, 2024 at 8:07 AM Lijo Lazar <lijo.lazar@amd.com> wrote:
>>
>> HDP Flush request bit can be kept unique per AID, and doesn't need to be
>> unique SOC-wide. Assign only bits 10-13 for SDMA v4.4.2.
>>
>> Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
> 
> Acked-by: Alex Deucher <alexander.deucher@amd.com>
> 
>> ---
>>   drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.c | 3 ++-
>>   1 file changed, 2 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.c b/drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.c
>> index f8e2cd514493..09e45ef16c0d 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.c
>> @@ -368,7 +368,8 @@ static void sdma_v4_4_2_ring_emit_hdp_flush(struct amdgpu_ring *ring)
>>          u32 ref_and_mask = 0;
>>          const struct nbio_hdp_flush_reg *nbio_hf_reg = adev->nbio.hdp_flush_reg;
>>
>> -       ref_and_mask = nbio_hf_reg->ref_and_mask_sdma0 << ring->me;
>> +       ref_and_mask = nbio_hf_reg->ref_and_mask_sdma0
>> +                      << (ring->me % adev->sdma.num_inst_per_aid);
>>
>>          sdma_v4_4_2_wait_reg_mem(ring, 0, 1,
>>                                 adev->nbio.funcs->get_hdp_flush_done_offset(adev),
>> --
>> 2.25.1
>>

Re: [PATCH] drm/amdgpu: Assign correct bits for SDMA HDP flush

[Lazar, Lijo]

On 5/7/2024 6:00 AM, Harry Wentland wrote:
> This patch is causing crashes of Manor Lords on my Navi 21 on the 6.8.9
> stable kernel. It leads to an assertion failure in wine:
> 
> File: ../src-wine/dlls/winevulkan/loader_thunks.c
> Line: 3621
> 
> Expression "!status && vkEndCommandBuffer""
> 
> This happens both with radv and amdvlk. It starts happening on v6.8.8
> with this patch. The previous patch (drm/amdgpu/sdma5.2: use legacy HDP
> flush for SDMA2/3) is fine.
> 

This patch will affect only aquavanjaram SOCs and shouldn't affect any
NV series (unless there is something going totally wrong and coming to
this path).

.emit_hdp_flush = sdma_v4_4_2_ring_emit_hdp_flush,

Thanks,
Lijo

> Harry
> 
> On 2024-04-17 09:19, Alex Deucher wrote:
>> On Wed, Apr 17, 2024 at 8:07 AM Lijo Lazar <lijo.lazar@amd.com> wrote:
>>>
>>> HDP Flush request bit can be kept unique per AID, and doesn't need to be
>>> unique SOC-wide. Assign only bits 10-13 for SDMA v4.4.2.
>>>
>>> Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
>>
>> Acked-by: Alex Deucher <alexander.deucher@amd.com>
>>
>>> ---
>>>   drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.c | 3 ++-
>>>   1 file changed, 2 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.c
>>> b/drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.c
>>> index f8e2cd514493..09e45ef16c0d 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.c
>>> @@ -368,7 +368,8 @@ static void
>>> sdma_v4_4_2_ring_emit_hdp_flush(struct amdgpu_ring *ring)
>>>          u32 ref_and_mask = 0;
>>>          const struct nbio_hdp_flush_reg *nbio_hf_reg =
>>> adev->nbio.hdp_flush_reg;
>>>
>>> -       ref_and_mask = nbio_hf_reg->ref_and_mask_sdma0 << ring->me;
>>> +       ref_and_mask = nbio_hf_reg->ref_and_mask_sdma0
>>> +                      << (ring->me % adev->sdma.num_inst_per_aid);
>>>
>>>          sdma_v4_4_2_wait_reg_mem(ring, 0, 1,
>>>                                
>>> adev->nbio.funcs->get_hdp_flush_done_offset(adev),
>>> -- 
>>> 2.25.1
>>>

Re: [PATCH] drm/amdgpu: Assign correct bits for SDMA HDP flush

[Harry Wentland]

On 2024-05-06 22:50, Lazar, Lijo wrote:
> 
> 
> On 5/7/2024 6:00 AM, Harry Wentland wrote:
>> This patch is causing crashes of Manor Lords on my Navi 21 on the 6.8.9
>> stable kernel. It leads to an assertion failure in wine:
>>
>> File: ../src-wine/dlls/winevulkan/loader_thunks.c
>> Line: 3621
>>
>> Expression "!status && vkEndCommandBuffer""
>>
>> This happens both with radv and amdvlk. It starts happening on v6.8.8
>> with this patch. The previous patch (drm/amdgpu/sdma5.2: use legacy HDP
>> flush for SDMA2/3) is fine.
>>
> 
> This patch will affect only aquavanjaram SOCs and shouldn't affect any
> NV series (unless there is something going totally wrong and coming to
> this path).
> 
> .emit_hdp_flush = sdma_v4_4_2_ring_emit_hdp_flush,
> 

That makes sense. I am pretty sure when I regressed the problem it lead
to this patch but will double-check when I have time. Will probably have to
wait a couple weeks as I'll be travelling starting tomorrow.

Harry

> Thanks,
> Lijo
> 
>> Harry
>>
>> On 2024-04-17 09:19, Alex Deucher wrote:
>>> On Wed, Apr 17, 2024 at 8:07 AM Lijo Lazar <lijo.lazar@amd.com> wrote:
>>>>
>>>> HDP Flush request bit can be kept unique per AID, and doesn't need to be
>>>> unique SOC-wide. Assign only bits 10-13 for SDMA v4.4.2.
>>>>
>>>> Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
>>>
>>> Acked-by: Alex Deucher <alexander.deucher@amd.com>
>>>
>>>> ---
>>>>   drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.c | 3 ++-
>>>>   1 file changed, 2 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.c
>>>> b/drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.c
>>>> index f8e2cd514493..09e45ef16c0d 100644
>>>> --- a/drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.c
>>>> +++ b/drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.c
>>>> @@ -368,7 +368,8 @@ static void
>>>> sdma_v4_4_2_ring_emit_hdp_flush(struct amdgpu_ring *ring)
>>>>          u32 ref_and_mask = 0;
>>>>          const struct nbio_hdp_flush_reg *nbio_hf_reg =
>>>> adev->nbio.hdp_flush_reg;
>>>>
>>>> -       ref_and_mask = nbio_hf_reg->ref_and_mask_sdma0 << ring->me;
>>>> +       ref_and_mask = nbio_hf_reg->ref_and_mask_sdma0
>>>> +                      << (ring->me % adev->sdma.num_inst_per_aid);
>>>>
>>>>          sdma_v4_4_2_wait_reg_mem(ring, 0, 1,
>>>>                                
>>>> adev->nbio.funcs->get_hdp_flush_done_offset(adev),
>>>> -- 
>>>> 2.25.1
>>>>