[PATCH v2] drm/amdgpu: Check extended configuration space register when system uses large bar

[PATCH v2] drm/amdgpu: Check extended configuration space register when system uses large bar

[Ma Jun]

Some customer platforms do not enable mmconfig for various reasons,
such as bios bug, and therefore cannot access the GPU extend configuration
space through mmio.

When the system enters the d3cold state and resumes, the amdgpu driver
fails to resume because the extend configuration space registers of
GPU can't be restored. At this point, Usually we only see some failure
dmesg log printed by amdgpu driver, it is difficult to find the root
cause.

Therefor print a warnning message if the system can't access the
extended configuration space register when using large bar.

Signed-off-by: Ma Jun <Jun.Ma2@amd.com>
---
v2:
- Check the register at 0x100 but not resize bar register(Christian)
- Modify the commit message
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 4b694696930e..0d7d55f13b8b 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -1417,6 +1417,13 @@ void amdgpu_device_wb_free(struct amdgpu_device *adev, u32 wb)
 		__clear_bit(wb, adev->wb.used);
 }
 
+static inline void amdgpu_device_check_extended_capability(struct amdgpu_device *adev)
+{
+	/* PCI_EXT_CAP_ID_VNDR extended capability is located at 0x100 */
+	if (!pci_find_ext_capability(adev->pdev, PCI_EXT_CAP_ID_VNDR))
+		DRM_WARN("System can't access extended configuration space,please check!!\n");
+}
+
 /**
  * amdgpu_device_resize_fb_bar - try to resize FB BAR
  *
@@ -1444,8 +1451,10 @@ int amdgpu_device_resize_fb_bar(struct amdgpu_device *adev)
 
 	/* skip if the bios has already enabled large BAR */
 	if (adev->gmc.real_vram_size &&
-	    (pci_resource_len(adev->pdev, 0) >= adev->gmc.real_vram_size))
+	    (pci_resource_len(adev->pdev, 0) >= adev->gmc.real_vram_size)) {
+		amdgpu_device_check_extended_capability(adev);
 		return 0;
+	}
 
 	/* Check if the root BUS has 64bit memory resources */
 	root = adev->pdev->bus;
-- 
2.34.1

Re: [PATCH v2] drm/amdgpu: Check extended configuration space register when system uses large bar

[Christian König]

Am 09.01.24 um 11:31 schrieb Ma Jun:
> Some customer platforms do not enable mmconfig for various reasons,
> such as bios bug, and therefore cannot access the GPU extend configuration
> space through mmio.
>
> When the system enters the d3cold state and resumes, the amdgpu driver
> fails to resume because the extend configuration space registers of
> GPU can't be restored. At this point, Usually we only see some failure
> dmesg log printed by amdgpu driver, it is difficult to find the root
> cause.
>
> Therefor print a warnning message if the system can't access the
> extended configuration space register when using large bar.
>
> Signed-off-by: Ma Jun <Jun.Ma2@amd.com>
> ---
> v2:
> - Check the register at 0x100 but not resize bar register(Christian)
> - Modify the commit message
> ---
>   drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 11 ++++++++++-
>   1 file changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> index 4b694696930e..0d7d55f13b8b 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> @@ -1417,6 +1417,13 @@ void amdgpu_device_wb_free(struct amdgpu_device *adev, u32 wb)
>   		__clear_bit(wb, adev->wb.used);
>   }
>   
> +static inline void amdgpu_device_check_extended_capability(struct amdgpu_device *adev)

Please don't use inline when it's not necessary, see the coding style 
guide as well.

> +{
> +	/* PCI_EXT_CAP_ID_VNDR extended capability is located at 0x100 */
> +	if (!pci_find_ext_capability(adev->pdev, PCI_EXT_CAP_ID_VNDR))
> +		DRM_WARN("System can't access extended configuration space,please check!!\n");
> +}
> +
>   /**
>    * amdgpu_device_resize_fb_bar - try to resize FB BAR
>    *
> @@ -1444,8 +1451,10 @@ int amdgpu_device_resize_fb_bar(struct amdgpu_device *adev)
>   
>   	/* skip if the bios has already enabled large BAR */
>   	if (adev->gmc.real_vram_size &&
> -	    (pci_resource_len(adev->pdev, 0) >= adev->gmc.real_vram_size))
> +	    (pci_resource_len(adev->pdev, 0) >= adev->gmc.real_vram_size)) {
> +		amdgpu_device_check_extended_capability(adev);

We should probably check this case independent of the BAR size since we 
will most likely run into trouble with error reporting as well.

Christian.

>   		return 0;
> +	}
>   
>   	/* Check if the root BUS has 64bit memory resources */
>   	root = adev->pdev->bus;