Re: [3/3] drm/amdgpu: add a retry for IP discovery init

[Mario Limonciello <mario.limonciello@amd.com>]

On 10/27/2023 10:42, Alex Deucher wrote:
> AMD dGPUs have integrated FW that runs as soon as the
> device gets power and initializes the board (determines
> the amount of memory, provides configuration details to
> the driver, etc.).  For direct PCIe attached cards this
> happens as soon as power is applied and normally completes
> well before the OS has even started loading.  However, with
> hotpluggable ports like USB4, the driver needs to wait for
> this to complete before initializing the device.
> 
> This normally takes 60-100ms, but could take longer on
> some older boards periodically due to memory training.
> 
> Retry for up to a second.  In the non-hotplug case, there
> should be no change in behavior and this should complete
> on the first try.
> 
> v2: adjust test criteria
> v3: adjust checks for the masks, only enable on removable devices
> v4: skip bif_fb_en check
> 
> Link: https://gitlab.freedesktop.org/drm/amd/-/issues/2925
> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
> ---

Reviewed-by: Mario Limonciello <mario.limonciello@amd.com>

>   drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.c | 23 +++++++++++++++++--
>   1 file changed, 21 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.c
> index 5f9d75900bfa..9ca4d89352d4 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.c
> @@ -99,6 +99,7 @@
>   MODULE_FIRMWARE(FIRMWARE_IP_DISCOVERY);
>   
>   #define mmRCC_CONFIG_MEMSIZE	0xde3
> +#define mmMP0_SMN_C2PMSG_33	0x16061
>   #define mmMM_INDEX		0x0
>   #define mmMM_INDEX_HI		0x6
>   #define mmMM_DATA		0x1
> @@ -239,8 +240,26 @@ static int amdgpu_discovery_read_binary_from_sysmem(struct amdgpu_device *adev,
>   static int amdgpu_discovery_read_binary_from_mem(struct amdgpu_device *adev,
>   						 uint8_t *binary)
>   {
> -	uint64_t vram_size = (uint64_t)RREG32(mmRCC_CONFIG_MEMSIZE) << 20;
> -	int ret = 0;
> +	uint64_t vram_size;
> +	u32 msg;
> +	int i, ret = 0;
> +
> +	/* It can take up to a second for IFWI init to complete on some dGPUs,
> +	 * but generally it should be in the 60-100ms range.  Normally this starts
> +	 * as soon as the device gets power so by the time the OS loads this has long
> +	 * completed.  However, when a card is hotplugged via e.g., USB4, we need to
> +	 * wait for this to complete.  Once the C2PMSG is updated, we can
> +	 * continue.
> +	 */
> +	if (dev_is_removable(&adev->pdev->dev)) {
> +		for (i = 0; i < 1000; i++) {
> +			msg = RREG32(mmMP0_SMN_C2PMSG_33);
> +			if (msg & 0x80000000)
> +				break;
> +			msleep(1);
> +		}
> +	}
> +	vram_size = (uint64_t)RREG32(mmRCC_CONFIG_MEMSIZE) << 20;
>   
>   	if (vram_size) {
>   		uint64_t pos = vram_size - DISCOVERY_TMR_OFFSET;