6.12.30: black screen after waking up from hibernate; bisected

6.12.30: black screen after waking up from hibernate; bisected

[Rainer Fiebig]

With kernel 6.12.30 waking up from hibernate fails in a Ryzen 3 5600G
system with the latest BIOS. At the end of the wake-up procedure the
screen goes black instead of showing the log-in screen and the system
becomes unresponsive.  A hard reset is necessary.

Seeing messages like the following in the system log, I suspected an
amdgpu problem:

May 23 19:09:30 LUX kernel: [16885.524496] amdgpu 0000:30:00.0: [drm]
*ERROR* flip_done timed out
May 23 19:09:30 LUX kernel: [16885.524501] amdgpu 0000:30:00.0: [drm]
*ERROR* [CRTC:73:crtc-0] commit wait timed out

I don't know whether those messages and the problem are really related
but I bisected in 'drivers/gpu/drm/amd' anyway and the result was:

> git bisect bad
25e07c8403f4daad35cffc18d96e32a80a2a3222 is the first bad commit
commit 25e07c8403f4daad35cffc18d96e32a80a2a3222 (HEAD)
Author: Alex Deucher <alexander.deucher@amd.com>
Date:   Thu May 1 13:46:46 2025 -0400

    drm/amdgpu: fix pm notifier handling

    commit 4aaffc85751da5722e858e4333e8cf0aa4b6c78f upstream.

    Set the s3/s0ix and s4 flags in the pm notifier so that we can skip
    the resource evictions properly in pm prepare based on whether
    we are suspending or hibernating.  Drop the eviction as processes
    are not frozen at this time, we we can end up getting stuck trying
    to evict VRAM while applications continue to submit work which
    causes the buffers to get pulled back into VRAM.

HTH.  Thanks.

Rainer

-- 
The truth always turns out to be simpler than you thought.
Richard Feynman

RE: 6.12.30: black screen after waking up from hibernate; bisected

[Deucher, Alexander]

[Public]

> -----Original Message-----
> From: Rainer Fiebig <jrf@mailbox.org>
> Sent: Friday, May 23, 2025 3:54 PM
> To: stable@vger.kernel.org; Deucher, Alexander <Alexander.Deucher@amd.com>
> Subject: 6.12.30: black screen after waking up from hibernate; bisected
>
> With kernel 6.12.30 waking up from hibernate fails in a Ryzen 3 5600G system with
> the latest BIOS. At the end of the wake-up procedure the screen goes black instead
> of showing the log-in screen and the system becomes unresponsive.  A hard reset
> is necessary.
>
> Seeing messages like the following in the system log, I suspected an amdgpu
> problem:
>
> May 23 19:09:30 LUX kernel: [16885.524496] amdgpu 0000:30:00.0: [drm]
> *ERROR* flip_done timed out
> May 23 19:09:30 LUX kernel: [16885.524501] amdgpu 0000:30:00.0: [drm]
> *ERROR* [CRTC:73:crtc-0] commit wait timed out
>
> I don't know whether those messages and the problem are really related but I
> bisected in 'drivers/gpu/drm/amd' anyway and the result was:
>
> > git bisect bad
> 25e07c8403f4daad35cffc18d96e32a80a2a3222 is the first bad commit commit
> 25e07c8403f4daad35cffc18d96e32a80a2a3222 (HEAD)
> Author: Alex Deucher <alexander.deucher@amd.com>
> Date:   Thu May 1 13:46:46 2025 -0400
>
>     drm/amdgpu: fix pm notifier handling
>
>     commit 4aaffc85751da5722e858e4333e8cf0aa4b6c78f upstream.
>
>     Set the s3/s0ix and s4 flags in the pm notifier so that we can skip
>     the resource evictions properly in pm prepare based on whether
>     we are suspending or hibernating.  Drop the eviction as processes
>     are not frozen at this time, we we can end up getting stuck trying
>     to evict VRAM while applications continue to submit work which
>     causes the buffers to get pulled back into VRAM.
>
> HTH.  Thanks.
>

Fixed in:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=7e7cb7a13c81073d38a10fa7b450d23712281ec4
and on it's way to stable.

Alex

> Rainer
>
> --
> The truth always turns out to be simpler than you thought.
> Richard Feynman

Re: 6.12.30: black screen after waking up from hibernate; bisected

[Rainer Fiebig]

Am 28.05.25 um 23:09 schrieb Deucher, Alexander:
> [Public]
> 
>> -----Original Message-----
>> From: Rainer Fiebig <jrf@mailbox.org>
>> Sent: Friday, May 23, 2025 3:54 PM
>> To: stable@vger.kernel.org; Deucher, Alexander <Alexander.Deucher@amd.com>
>> Subject: 6.12.30: black screen after waking up from hibernate; bisected
>>
>> With kernel 6.12.30 waking up from hibernate fails in a Ryzen 3 5600G system with
>> the latest BIOS. At the end of the wake-up procedure the screen goes black instead
>> of showing the log-in screen and the system becomes unresponsive.  A hard reset
>> is necessary.
>>
>> Seeing messages like the following in the system log, I suspected an amdgpu
>> problem:
>>
>> May 23 19:09:30 LUX kernel: [16885.524496] amdgpu 0000:30:00.0: [drm]
>> *ERROR* flip_done timed out
>> May 23 19:09:30 LUX kernel: [16885.524501] amdgpu 0000:30:00.0: [drm]
>> *ERROR* [CRTC:73:crtc-0] commit wait timed out
>>
>> I don't know whether those messages and the problem are really related but I
>> bisected in 'drivers/gpu/drm/amd' anyway and the result was:
>>
>>> git bisect bad
>> 25e07c8403f4daad35cffc18d96e32a80a2a3222 is the first bad commit commit
>> 25e07c8403f4daad35cffc18d96e32a80a2a3222 (HEAD)
>> Author: Alex Deucher <alexander.deucher@amd.com>
>> Date:   Thu May 1 13:46:46 2025 -0400
>>
>>     drm/amdgpu: fix pm notifier handling
>>
>>     commit 4aaffc85751da5722e858e4333e8cf0aa4b6c78f upstream.
>>
>>     Set the s3/s0ix and s4 flags in the pm notifier so that we can skip
>>     the resource evictions properly in pm prepare based on whether
>>     we are suspending or hibernating.  Drop the eviction as processes
>>     are not frozen at this time, we we can end up getting stuck trying
>>     to evict VRAM while applications continue to submit work which
>>     causes the buffers to get pulled back into VRAM.
>>
>> HTH.  Thanks.
>>
> 
> Fixed in:
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=7e7cb7a13c81073d38a10fa7b450d23712281ec4
> and on it's way to stable.
Great, thanks!  I had already reverted your commit in an experimental
branch and that solved the problem - so either your commit was bad or
something that it somehow depended on.

The problem that now reverted commit 68bfdc8dc0a1a tried to solve is
indeed irritating/confusing and hopefully you'll find an other way to
solve it.  The whole procedure is suboptimal insofar as there is no
feedback as to what is going on and whether the process has finally
concluded and it is safe to switch off the box.

My - perhaps naive - suggestion would be to provide at least some
feedback by leaving the monitor _on_ until the image has been written to
disk and the box can be switched off.

Rainer

Re: 6.12.30: black screen after waking up from hibernate; bisected

[Rainer Fiebig]

Am 29.05.25 um 09:17 schrieb Rainer Fiebig:
> Am 28.05.25 um 23:09 schrieb Deucher, Alexander:
>> [Public]
>>
>>> -----Original Message-----
>>> From: Rainer Fiebig <jrf@mailbox.org>
>>> Sent: Friday, May 23, 2025 3:54 PM
>>> To: stable@vger.kernel.org; Deucher, Alexander <Alexander.Deucher@amd.com>
>>> Subject: 6.12.30: black screen after waking up from hibernate; bisected
>>>
>>> With kernel 6.12.30 waking up from hibernate fails in a Ryzen 3 5600G system with
>>> the latest BIOS. At the end of the wake-up procedure the screen goes black instead
>>> of showing the log-in screen and the system becomes unresponsive.  A hard reset
>>> is necessary.
>>>
>>> Seeing messages like the following in the system log, I suspected an amdgpu
>>> problem:
>>>
>>> May 23 19:09:30 LUX kernel: [16885.524496] amdgpu 0000:30:00.0: [drm]
>>> *ERROR* flip_done timed out
>>> May 23 19:09:30 LUX kernel: [16885.524501] amdgpu 0000:30:00.0: [drm]
>>> *ERROR* [CRTC:73:crtc-0] commit wait timed out
>>>
>>> I don't know whether those messages and the problem are really related but I
>>> bisected in 'drivers/gpu/drm/amd' anyway and the result was:
>>>
>>>> git bisect bad
>>> 25e07c8403f4daad35cffc18d96e32a80a2a3222 is the first bad commit commit
>>> 25e07c8403f4daad35cffc18d96e32a80a2a3222 (HEAD)
>>> Author: Alex Deucher <alexander.deucher@amd.com>
>>> Date:   Thu May 1 13:46:46 2025 -0400
>>>
>>>     drm/amdgpu: fix pm notifier handling
>>>
>>>     commit 4aaffc85751da5722e858e4333e8cf0aa4b6c78f upstream.
>>>
>>>     Set the s3/s0ix and s4 flags in the pm notifier so that we can skip
>>>     the resource evictions properly in pm prepare based on whether
>>>     we are suspending or hibernating.  Drop the eviction as processes
>>>     are not frozen at this time, we we can end up getting stuck trying
>>>     to evict VRAM while applications continue to submit work which
>>>     causes the buffers to get pulled back into VRAM.
>>>
>>> HTH.  Thanks.
>>>
>>
>> Fixed in:
>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=7e7cb7a13c81073d38a10fa7b450d23712281ec4
>> and on it's way to stable.
> Great, thanks!  I had already reverted your commit in an experimental
> branch and that solved the problem - so either your commit was bad or
> something that it somehow depended on.
> 
> The problem that now reverted commit 68bfdc8dc0a1a tried to solve is
> indeed irritating/confusing and hopefully you'll find an other way to
> solve it.  The whole procedure is suboptimal insofar as there is no
> feedback as to what is going on and whether the process has finally
> concluded and it is safe to switch off the box.
> 
> My - perhaps naive - suggestion would be to provide at least some
> feedback by leaving the monitor _on_ until the image has been written to
> disk and the box can be switched off.
To clarify a bit what I mean: if possible, the display should stay "on"
_all the while_ from initiating hibernate until the image has been
written to disk and shutdown is complete, so that the user can tell from
the status of the monitor's power-LED whether it's safe to switch the
computer off.  Neither an "off-on, off-on..." nor an "off" during that
phase is helpful.

Rainer

Re: 6.12.30: black screen after waking up from hibernate; bisected

[Limonciello, Mario]

On 5/29/25 6:02 AM, Rainer Fiebig wrote:
> Am 29.05.25 um 09:17 schrieb Rainer Fiebig:
>> Am 28.05.25 um 23:09 schrieb Deucher, Alexander:
>>> [Public]
>>>
>>>> -----Original Message-----
>>>> From: Rainer Fiebig <jrf@mailbox.org>
>>>> Sent: Friday, May 23, 2025 3:54 PM
>>>> To: stable@vger.kernel.org; Deucher, Alexander <Alexander.Deucher@amd.com>
>>>> Subject: 6.12.30: black screen after waking up from hibernate; bisected
>>>>
>>>> With kernel 6.12.30 waking up from hibernate fails in a Ryzen 3 5600G system with
>>>> the latest BIOS. At the end of the wake-up procedure the screen goes black instead
>>>> of showing the log-in screen and the system becomes unresponsive.  A hard reset
>>>> is necessary.
>>>>
>>>> Seeing messages like the following in the system log, I suspected an amdgpu
>>>> problem:
>>>>
>>>> May 23 19:09:30 LUX kernel: [16885.524496] amdgpu 0000:30:00.0: [drm]
>>>> *ERROR* flip_done timed out
>>>> May 23 19:09:30 LUX kernel: [16885.524501] amdgpu 0000:30:00.0: [drm]
>>>> *ERROR* [CRTC:73:crtc-0] commit wait timed out
>>>>
>>>> I don't know whether those messages and the problem are really related but I
>>>> bisected in 'drivers/gpu/drm/amd' anyway and the result was:
>>>>
>>>>> git bisect bad
>>>> 25e07c8403f4daad35cffc18d96e32a80a2a3222 is the first bad commit commit
>>>> 25e07c8403f4daad35cffc18d96e32a80a2a3222 (HEAD)
>>>> Author: Alex Deucher <alexander.deucher@amd.com>
>>>> Date:   Thu May 1 13:46:46 2025 -0400
>>>>
>>>>      drm/amdgpu: fix pm notifier handling
>>>>
>>>>      commit 4aaffc85751da5722e858e4333e8cf0aa4b6c78f upstream.
>>>>
>>>>      Set the s3/s0ix and s4 flags in the pm notifier so that we can skip
>>>>      the resource evictions properly in pm prepare based on whether
>>>>      we are suspending or hibernating.  Drop the eviction as processes
>>>>      are not frozen at this time, we we can end up getting stuck trying
>>>>      to evict VRAM while applications continue to submit work which
>>>>      causes the buffers to get pulled back into VRAM.
>>>>
>>>> HTH.  Thanks.
>>>>
>>>
>>> Fixed in:
>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=7e7cb7a13c81073d38a10fa7b450d23712281ec4
>>> and on it's way to stable.
>> Great, thanks!  I had already reverted your commit in an experimental
>> branch and that solved the problem - so either your commit was bad or
>> something that it somehow depended on.
>>
>> The problem that now reverted commit 68bfdc8dc0a1a tried to solve is
>> indeed irritating/confusing and hopefully you'll find an other way to
>> solve it.  The whole procedure is suboptimal insofar as there is no
>> feedback as to what is going on and whether the process has finally
>> concluded and it is safe to switch off the box.
>>
>> My - perhaps naive - suggestion would be to provide at least some
>> feedback by leaving the monitor _on_ until the image has been written to
>> disk and the box can be switched off.
> To clarify a bit what I mean: if possible, the display should stay "on"
> _all the while_ from initiating hibernate until the image has been
> written to disk and shutdown is complete, so that the user can tell from
> the status of the monitor's power-LED whether it's safe to switch the
> computer off.  Neither an "off-on, off-on..." nor an "off" during that
> phase is helpful.
> 
> Rainer
> 

Before the hibernate image is created the GPU is supposed to be put into 
a state that it will be when resuming from the hibernate image.

That is why all the IP blocks are reset before creating the hibernate 
image.  This will turn off displays because DCN is reset.

Re: 6.12.30: black screen after waking up from hibernate; bisected

[Rainer Fiebig]

Am 29.05.25 um 15:07 schrieb Limonciello, Mario:
> On 5/29/25 6:02 AM, Rainer Fiebig wrote:
>> Am 29.05.25 um 09:17 schrieb Rainer Fiebig:
>>> Am 28.05.25 um 23:09 schrieb Deucher, Alexander:
>>>> [Public]
>>>>
>>>>> -----Original Message-----
>>>>> From: Rainer Fiebig <jrf@mailbox.org>
>>>>> Sent: Friday, May 23, 2025 3:54 PM
>>>>> To: stable@vger.kernel.org; Deucher, Alexander <Alexander.Deucher@amd.com>
>>>>> Subject: 6.12.30: black screen after waking up from hibernate; bisected
>>>>>
>>>>> With kernel 6.12.30 waking up from hibernate fails in a Ryzen 3 5600G system with
>>>>> the latest BIOS. At the end of the wake-up procedure the screen goes black instead
>>>>> of showing the log-in screen and the system becomes unresponsive.  A hard reset
>>>>> is necessary.
>>>>>
>>>>> Seeing messages like the following in the system log, I suspected an amdgpu
>>>>> problem:
>>>>>
>>>>> May 23 19:09:30 LUX kernel: [16885.524496] amdgpu 0000:30:00.0: [drm]
>>>>> *ERROR* flip_done timed out
>>>>> May 23 19:09:30 LUX kernel: [16885.524501] amdgpu 0000:30:00.0: [drm]
>>>>> *ERROR* [CRTC:73:crtc-0] commit wait timed out
>>>>>
>>>>> I don't know whether those messages and the problem are really related but I
>>>>> bisected in 'drivers/gpu/drm/amd' anyway and the result was:
>>>>>
>>>>>> git bisect bad
>>>>> 25e07c8403f4daad35cffc18d96e32a80a2a3222 is the first bad commit commit
>>>>> 25e07c8403f4daad35cffc18d96e32a80a2a3222 (HEAD)
>>>>> Author: Alex Deucher <alexander.deucher@amd.com>
>>>>> Date:   Thu May 1 13:46:46 2025 -0400
>>>>>
>>>>>      drm/amdgpu: fix pm notifier handling
>>>>>
>>>>>      commit 4aaffc85751da5722e858e4333e8cf0aa4b6c78f upstream.
>>>>>
>>>>>      Set the s3/s0ix and s4 flags in the pm notifier so that we can skip
>>>>>      the resource evictions properly in pm prepare based on whether
>>>>>      we are suspending or hibernating.  Drop the eviction as processes
>>>>>      are not frozen at this time, we we can end up getting stuck trying
>>>>>      to evict VRAM while applications continue to submit work which
>>>>>      causes the buffers to get pulled back into VRAM.
>>>>>
>>>>> HTH.  Thanks.
>>>>>
>>>>
>>>> Fixed in:
>>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=7e7cb7a13c81073d38a10fa7b450d23712281ec4
>>>> and on it's way to stable.
>>> Great, thanks!  I had already reverted your commit in an experimental
>>> branch and that solved the problem - so either your commit was bad or
>>> something that it somehow depended on.
>>>
>>> The problem that now reverted commit 68bfdc8dc0a1a tried to solve is
>>> indeed irritating/confusing and hopefully you'll find an other way to
>>> solve it.  The whole procedure is suboptimal insofar as there is no
>>> feedback as to what is going on and whether the process has finally
>>> concluded and it is safe to switch off the box.
>>>
>>> My - perhaps naive - suggestion would be to provide at least some
>>> feedback by leaving the monitor _on_ until the image has been written to
>>> disk and the box can be switched off.
>> To clarify a bit what I mean: if possible, the display should stay "on"
>> _all the while_ from initiating hibernate until the image has been
>> written to disk and shutdown is complete, so that the user can tell from
>> the status of the monitor's power-LED whether it's safe to switch the
>> computer off.  Neither an "off-on, off-on..." nor an "off" during that
>> phase is helpful.
>>
>> Rainer
>>
> 
> Before the hibernate image is created the GPU is supposed to be put into 
> a state that it will be when resuming from the hibernate image.
> 
> That is why all the IP blocks are reset before creating the hibernate 
> image.  This will turn off displays because DCN is reset.
I see, thanks for the info.

Just built 6.12.31 and ran one hibernate/resume cycle and that was OK.
Hibernating the system showed the old behaviour (the one before your
"always off" patch): screen goes black, monitor stays "on" for a while,
then goes "off", after a while "on" again with a black screen and the
mouse-pointer visible.  The monitor then goes "off" which ususally marks
the end of the procedure.  Sometimes there's more than one "off-on"
cycle which can be rather confusing at first.  But one gets used to it.

Rainer

>