[PATCH 1/2] drm/amdgpu: add utcl2_harvest to gc 10.3.1

[PATCH 1/2] drm/amdgpu: add utcl2_harvest to gc 10.3.1

[Aaron Liu]

Confirmed with hardware team, there is harvesting for gc 10.3.1.

Signed-off-by: Aaron Liu <aaron.liu@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c b/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c
index b4eddf6e98a6..ff738e9725ee 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c
@@ -543,7 +543,9 @@ static void gfxhub_v2_1_utcl2_harvest(struct amdgpu_device *adev)
 		adev->gfx.config.max_sh_per_se *
 		adev->gfx.config.max_shader_engines);
 
-	if (adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 3, 3)) {
+	switch (adev->ip_versions[GC_HWIP][0]) {
+	case IP_VERSION(10, 3, 1):
+	case IP_VERSION(10, 3, 3):
 		/* Get SA disabled bitmap from eFuse setting */
 		efuse_setting = RREG32_SOC15(GC, 0, mmCC_GC_SA_UNIT_DISABLE);
 		efuse_setting &= CC_GC_SA_UNIT_DISABLE__SA_DISABLE_MASK;
@@ -566,6 +568,9 @@ static void gfxhub_v2_1_utcl2_harvest(struct amdgpu_device *adev)
 		disabled_sa = tmp;
 
 		WREG32_SOC15(GC, 0, mmGCUTCL2_HARVEST_BYPASS_GROUPS_YELLOW_CARP, disabled_sa);
+		break;
+	default:
+		break;
 	}
 }
 
-- 
2.25.1

[PATCH 2/2] drm/amdgpu: check the GART table before invalidating TLB

[Aaron Liu]

Bypass group programming (utcl2_harvest) aims to forbid UTCL2 to send
invalidation command to harvested SE/SA. Once invalidation command comes
into harvested SE/SA, SE/SA has no response and system hang.

This patch is to add checking if the GART table is already allocated before
invalidating TLB. The new procedure is as following:
1. Calling amdgpu_gtt_mgr_init() in amdgpu_ttm_init(). After this step GTT
   BOs can be allocated, but GART mappings are still ignored.
2. Calling amdgpu_gart_table_vram_alloc() from the GMC code. This allocates
   the GART backing store.
3. Initializing the hardware, and programming the backing store into VMID0
   for all VMHUBs.
4. Calling amdgpu_gtt_mgr_recover() to make sure the table is updated with
   the GTT allocations done before it was allocated.

Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Aaron Liu <aaron.liu@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
index 91d8207336c1..01cb89ffbd56 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
@@ -259,6 +259,9 @@ void amdgpu_gart_invalidate_tlb(struct amdgpu_device *adev)
 {
 	int i;
 
+	if (!adev->gart.ptr)
+		return;
+
 	mb();
 	amdgpu_device_flush_hdp(adev, NULL);
 	for (i = 0; i < adev->num_vmhubs; i++)
-- 
2.25.1

Re: [PATCH 2/2] drm/amdgpu: check the GART table before invalidating TLB

[Huang Rui]

On Mon, Feb 07, 2022 at 10:41:55AM +0800, Liu, Aaron wrote:
> Bypass group programming (utcl2_harvest) aims to forbid UTCL2 to send
> invalidation command to harvested SE/SA. Once invalidation command comes
> into harvested SE/SA, SE/SA has no response and system hang.
> 
> This patch is to add checking if the GART table is already allocated before
> invalidating TLB. The new procedure is as following:
> 1. Calling amdgpu_gtt_mgr_init() in amdgpu_ttm_init(). After this step GTT
>    BOs can be allocated, but GART mappings are still ignored.
> 2. Calling amdgpu_gart_table_vram_alloc() from the GMC code. This allocates
>    the GART backing store.
> 3. Initializing the hardware, and programming the backing store into VMID0
>    for all VMHUBs.
> 4. Calling amdgpu_gtt_mgr_recover() to make sure the table is updated with
>    the GTT allocations done before it was allocated.
> 
> Signed-off-by: Christian König <christian.koenig@amd.com>
> Signed-off-by: Aaron Liu <aaron.liu@amd.com>

Acked-by: Huang Rui <ray.huang@amd.com>

> ---
>  drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
> index 91d8207336c1..01cb89ffbd56 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
> @@ -259,6 +259,9 @@ void amdgpu_gart_invalidate_tlb(struct amdgpu_device *adev)
>  {
>  	int i;
>  
> +	if (!adev->gart.ptr)
> +		return;
> +
>  	mb();
>  	amdgpu_device_flush_hdp(adev, NULL);
>  	for (i = 0; i < adev->num_vmhubs; i++)
> -- 
> 2.25.1
> 

Re: [PATCH 1/2] drm/amdgpu: add utcl2_harvest to gc 10.3.1

[Huang Rui]

On Mon, Feb 07, 2022 at 10:41:54AM +0800, Liu, Aaron wrote:
> Confirmed with hardware team, there is harvesting for gc 10.3.1.
> 
> Signed-off-by: Aaron Liu <aaron.liu@amd.com>

Reviewed-by: Huang Rui <ray.huang@amd.com>

> ---
>  drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c b/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c
> index b4eddf6e98a6..ff738e9725ee 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c
> @@ -543,7 +543,9 @@ static void gfxhub_v2_1_utcl2_harvest(struct amdgpu_device *adev)
>  		adev->gfx.config.max_sh_per_se *
>  		adev->gfx.config.max_shader_engines);
>  
> -	if (adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 3, 3)) {
> +	switch (adev->ip_versions[GC_HWIP][0]) {
> +	case IP_VERSION(10, 3, 1):
> +	case IP_VERSION(10, 3, 3):
>  		/* Get SA disabled bitmap from eFuse setting */
>  		efuse_setting = RREG32_SOC15(GC, 0, mmCC_GC_SA_UNIT_DISABLE);
>  		efuse_setting &= CC_GC_SA_UNIT_DISABLE__SA_DISABLE_MASK;
> @@ -566,6 +568,9 @@ static void gfxhub_v2_1_utcl2_harvest(struct amdgpu_device *adev)
>  		disabled_sa = tmp;
>  
>  		WREG32_SOC15(GC, 0, mmGCUTCL2_HARVEST_BYPASS_GROUPS_YELLOW_CARP, disabled_sa);
> +		break;
> +	default:
> +		break;
>  	}
>  }
>  
> -- 
> 2.25.1
> 

Re: [PATCH 2/2] drm/amdgpu: check the GART table before invalidating TLB

[Christian König]

Am 07.02.22 um 07:30 schrieb Huang Rui:
> On Mon, Feb 07, 2022 at 10:41:55AM +0800, Liu, Aaron wrote:
>> Bypass group programming (utcl2_harvest) aims to forbid UTCL2 to send
>> invalidation command to harvested SE/SA. Once invalidation command comes
>> into harvested SE/SA, SE/SA has no response and system hang.
>>
>> This patch is to add checking if the GART table is already allocated before
>> invalidating TLB. The new procedure is as following:
>> 1. Calling amdgpu_gtt_mgr_init() in amdgpu_ttm_init(). After this step GTT
>>     BOs can be allocated, but GART mappings are still ignored.
>> 2. Calling amdgpu_gart_table_vram_alloc() from the GMC code. This allocates
>>     the GART backing store.
>> 3. Initializing the hardware, and programming the backing store into VMID0
>>     for all VMHUBs.
>> 4. Calling amdgpu_gtt_mgr_recover() to make sure the table is updated with
>>     the GTT allocations done before it was allocated.
>>
>> Signed-off-by: Christian König <christian.koenig@amd.com>
>> Signed-off-by: Aaron Liu <aaron.liu@amd.com>
> Acked-by: Huang Rui <ray.huang@amd.com>

Reviewed-by: Christian König <christian.koenig@amd.com>

>
>> ---
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 3 +++
>>   1 file changed, 3 insertions(+)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
>> index 91d8207336c1..01cb89ffbd56 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
>> @@ -259,6 +259,9 @@ void amdgpu_gart_invalidate_tlb(struct amdgpu_device *adev)
>>   {
>>   	int i;
>>   
>> +	if (!adev->gart.ptr)
>> +		return;
>> +
>>   	mb();
>>   	amdgpu_device_flush_hdp(adev, NULL);
>>   	for (i = 0; i < adev->num_vmhubs; i++)
>> -- 
>> 2.25.1
>>