Re: [PATCH v5 5/5] drm/amdgpu: do not resume device in thaw for normal hibernation

["Zhang, GuoQing (Sam)" <guoqzhan@amd.com>]

On 2025/7/10 12:18, Lazar, Lijo wrote:
>
> On 7/10/2025 1:20 AM, Mario Limonciello wrote:
>> On 7/9/2025 6:05 AM, Samuel Zhang wrote:
>>> For normal hibernation, GPU do not need to be resumed in thaw since it is
>>> not involved in writing the hibernation image. Skip resume in this case
>>> can reduce the hibernation time.
>>>
>>> On VM with 8 * 192GB VRAM dGPUs, 98% VRAM usage and 1.7TB system memory,
>>> this can save 50 minutes.
>>>
>>> Signed-off-by: Samuel Zhang <guoqing.zhang@amd.com>
>> I hand modified the patches for other changes missing from linux-next in
>> your base.
>>
>> I checked on an APU with an eDP display connected and from a VT
>> hibernate does keep the display off now so this is definitely an
>> improvement there too.
>>
>> Tested-by: Mario Limonciello <mario.limonciello@amd.com>


Thank you, Mario!


>>
>>> ---
>>>    drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 17 +++++++++++++++++
>>>    1 file changed, 17 insertions(+)
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/
>>> drm/amd/amdgpu/amdgpu_drv.c
>>> index 4f8632737574..b24c420983ef 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
>>> @@ -2541,6 +2541,10 @@ amdgpu_pci_shutdown(struct pci_dev *pdev)
>>>        if (amdgpu_ras_intr_triggered())
>>>            return;
>>>    +    /* device maybe not resumed here, return immediately in this
>>> case */
>>> +    if (adev->in_s4 && adev->in_suspend)
>>> +        return;
>>> +
>>>        /* if we are running in a VM, make sure the device
>>>         * torn down properly on reboot/shutdown.
>>>         * unfortunately we can't detect certain
>>> @@ -2557,6 +2561,10 @@ static int amdgpu_pmops_prepare(struct device
>>> *dev)
>>>        struct drm_device *drm_dev = dev_get_drvdata(dev);
>>>        struct amdgpu_device *adev = drm_to_adev(drm_dev);
>>>    +    /* device maybe not resumed here, return immediately in this
>>> case */
>>> +    if (adev->in_s4 && adev->in_suspend)
>>> +        return 0;
>>> +
>> Is this one right?  Don't we still want to call prepare() for all the HW
>> IP blocks?  The eviction call that happens in prepare() is a no-op but
>> there are other IP blocks with an prepare_suspend() callback like DCN.
>>
>> That is I think you're destroying the optimization from commit
>> 50e0bae34fa6b ("drm/amd/display: Add and use new dm_prepare_suspend()
>> callback") by adding this code here.
>>
> I guess this takes care of the prepare() before a power_off(). For the
> hibernate prepare() call, in_suspend flag will remain false and it
> should get executed. If the device is runtime-suspended already, then
> the path won't be taken. Assuming that's fine.
>
> Thanks,
> Lijo


That's right. Thank you for the clarification, Lijo.

Rafael reminded me yesterday that there are 2 hibernation mode, controlled by `/sys/power/disk`.
The 2 mode will call different callbacks after saving the image:

shutdown mode, hibernate will go through following steps:
1. amdgpu_pmops_prepare
2. amdgpu_pmops_freeze
3. create image
4. amdgpu_pmops_thaw
5. amdgpu_pmops_complete
6. save image

7. amdgpu_pci_shutdown


platform mode, hibernate will go through following steps:
1. amdgpu_pmops_prepare
2. amdgpu_pmops_freeze
3. create image
4. amdgpu_pmops_thaw
5. amdgpu_pmops_complete
6. save image

7. amdgpu_pmops_prepare
8. amdgpu_pmops_poweroff

This code is just to skip the step 7 of platform mode hibernation, 
prepare to power off.

adev->in_suspend is false in step 2, so it will not return in new the check.
adev->in_suspend is true in step 7, it will return in new the check.

Regards
Sam


>
>>>        /* Return a positive number here so
>>>         * DPM_FLAG_SMART_SUSPEND works properly
>>>         */
>>> @@ -2655,12 +2663,21 @@ static int amdgpu_pmops_thaw(struct device *dev)
>>>    {
>>>        struct drm_device *drm_dev = dev_get_drvdata(dev);
>>>    +    /* do not resume device if it's normal hibernation */
>>> +    if (!pm_hibernate_is_recovering())
>>> +        return 0;
>>> +
>>>        return amdgpu_device_resume(drm_dev, true);
>>>    }
>>>      static int amdgpu_pmops_poweroff(struct device *dev)
>>>    {
>>>        struct drm_device *drm_dev = dev_get_drvdata(dev);
>>> +    struct amdgpu_device *adev = drm_to_adev(drm_dev);
>>> +
>>> +    /* device maybe not resumed here, return immediately in this case */
>>> +    if (adev->in_s4 && adev->in_suspend)
>>> +        return 0;
>>>          return amdgpu_device_suspend(drm_dev, true);
>>>    }