[REGRESSION][BISECTED] Dell Precision 7780 wakes up on its own from suspend

[REGRESSION][BISECTED] Dell Precision 7780 wakes up on its own from suspend

[Askar Safin]

Hi. This commit:

#regzbot introduced: 1796f808e4bb2c074824dc32258ed1e719370cb3

caused a regression. Dell Precision 7780 often wakes up on its own from suspend.
Sometimes wake up happens immediately (i. e. within 7 seconds), sometimes it happens after, say, 30 minutes.

(The laptop comes with Ubuntu preinstalled, but I didn't test this bug on preinstalled Ubuntu)

That commit already caused a lot of other regressions on other laptops:

#regzbot link: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4cb786180dfb5258ff3111181b5e4ecb1d4a297b
#regzbot link: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d63f11c02b8d3e54bdb65d8c309f73b7f474aec4
#regzbot link: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=a69982c37cd0586e6832268155349301b87f2e35
#regzbot link: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=782eea0c89f7d071d6b56ecfa1b8b0c81164b9be

I will be available for testing in coming days, then I will switch to other things, and so will not be available for testing.
If you want more time, then, please, ask for it, i. e. say me something like "Please, be available for testing in more 10 days".

Here is config I used for testing:

https://zerobin.net/?5c4e4b8f6e7a6a0d#W35/IXYAxzjFyp7yP71V0EZ6jcF83HOWB/52ldl5jig=

In other words, this is config I used for bisecting. In other words, the bug reproduces
with this config on 1796f808, but doesn't reproduce on d86388c9.

This config was created using localmodconfig, so it is quite possible it will not boot on other computers.

Also, the bug reproduces on pretty standard Debian kernels, for example, on

http://deb.debian.org/debian/pool/main/l/linux-signed-amd64/linux-image-6.12.32-amd64_6.12.32-1_amd64.deb

with standard Debian config.

Here is "bad" dmesg:
https://zerobin.net/?525f36df8d698fba#e3+DHsvv3Lsqmi91d1TqHx19mrilYNDR0bL8DbeNxw0=

I. e. this is dmesg generated during attempts to reproduce the bug on "bad" kernel, i. e. on 1796f808
with minimized config provided above.

And here is "good" dmesg:
https://zerobin.net/?09b26bfcfa523eff#zDMKsIGmfXBLgpml3qDjwW0NnclErstolYEjCMzmmsA=

I. e. it was generated the same way, but with d86388c9.

This is script I used for bisecting:

https://zerobin.net/?a2acfc92e5a33cc3#L7/Z0jY47FFFp7UJ1GPmbhmfrcMTr0zBZst82ubyDs8=

The script uses "rtcwake" to automatize bisecting. Note that the bug reproduces
not only with "rtcwake", but also with normal suspend initiated from GUI without any timer.

This bug causes "rtcwake" to wake up BEFORE specified time.

Bug sometimes reproduces and sometimes not, so script above makes multiple attempts.

The script writes some info to /dev/kmsg , and you may see its output in dmesg above.

If you need more info, ask me. But again: I will be available within some days and then not.
If you need more time, ask me for it.

Reverting commit 1796f808 on top of v6.12.32 causes the bug to disappear.

When I connect external wireless mouse via USB, then the bug becomes harder to reproduce,
so I did testing without anything connected to laptop (except for power source).

--
Askar Safin
https://types.pl/@safinaskar

Re: [REGRESSION][BISECTED] Dell Precision 7780 wakes up on its own from suspend

[Mario Limonciello]

On 6/26/25 6:32 PM, Askar Safin wrote:
> Hi. This commit:
> 
> #regzbot introduced: 1796f808e4bb2c074824dc32258ed1e719370cb3
> 
> caused a regression. Dell Precision 7780 often wakes up on its own from suspend.
> Sometimes wake up happens immediately (i. e. within 7 seconds), sometimes it happens after, say, 30 minutes.
> 
> (The laptop comes with Ubuntu preinstalled, but I didn't test this bug on preinstalled Ubuntu)
> 
> That commit already caused a lot of other regressions on other laptops:
> 
> #regzbot link: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4cb786180dfb5258ff3111181b5e4ecb1d4a297b
> #regzbot link: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d63f11c02b8d3e54bdb65d8c309f73b7f474aec4
> #regzbot link: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=a69982c37cd0586e6832268155349301b87f2e35
> #regzbot link: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=782eea0c89f7d071d6b56ecfa1b8b0c81164b9be
> 
> I will be available for testing in coming days, then I will switch to other things, and so will not be available for testing.
> If you want more time, then, please, ask for it, i. e. say me something like "Please, be available for testing in more 10 days".
> 
> Here is config I used for testing:
> 
> https://zerobin.net/?5c4e4b8f6e7a6a0d#W35/IXYAxzjFyp7yP71V0EZ6jcF83HOWB/52ldl5jig=
> 
> In other words, this is config I used for bisecting. In other words, the bug reproduces
> with this config on 1796f808, but doesn't reproduce on d86388c9.
> 
> This config was created using localmodconfig, so it is quite possible it will not boot on other computers.
> 
> Also, the bug reproduces on pretty standard Debian kernels, for example, on
> 
> http://deb.debian.org/debian/pool/main/l/linux-signed-amd64/linux-image-6.12.32-amd64_6.12.32-1_amd64.deb
> 
> with standard Debian config.
> 
> Here is "bad" dmesg:
> https://zerobin.net/?525f36df8d698fba#e3+DHsvv3Lsqmi91d1TqHx19mrilYNDR0bL8DbeNxw0=
> 
> I. e. this is dmesg generated during attempts to reproduce the bug on "bad" kernel, i. e. on 1796f808
> with minimized config provided above.
> 
> And here is "good" dmesg:
> https://zerobin.net/?09b26bfcfa523eff#zDMKsIGmfXBLgpml3qDjwW0NnclErstolYEjCMzmmsA=
> 
> I. e. it was generated the same way, but with d86388c9.
> 
> This is script I used for bisecting:
> 
> https://zerobin.net/?a2acfc92e5a33cc3#L7/Z0jY47FFFp7UJ1GPmbhmfrcMTr0zBZst82ubyDs8=
> 
> The script uses "rtcwake" to automatize bisecting. Note that the bug reproduces
> not only with "rtcwake", but also with normal suspend initiated from GUI without any timer.
> 
> This bug causes "rtcwake" to wake up BEFORE specified time.
> 
> Bug sometimes reproduces and sometimes not, so script above makes multiple attempts.
> 
> The script writes some info to /dev/kmsg , and you may see its output in dmesg above.
> 
> If you need more info, ask me. But again: I will be available within some days and then not.
> If you need more time, ask me for it.
> 
> Reverting commit 1796f808 on top of v6.12.32 causes the bug to disappear.
> 
> When I connect external wireless mouse via USB, then the bug becomes harder to reproduce,
> so I did testing without anything connected to laptop (except for power source).
> 
> --
> Askar Safin
> https://types.pl/@safinaskar
> 
> 

In order to debug this a few helpful things to add to your script:
1) Set /sys/power/pm_debug_messages before starting
2) Capture the value of /sys/power/suspend_stats/last_hw_sleep

BTW - When a wireless mouse is connected via USB I /suspect/ you aren't 
getting into hardware sleep state (or if your it's for not as long of a 
duration).  I'm not sure if that's a hint for this issue or not right 
now.  That's why I think capturing it might be helpful for this issue.

Re: [REGRESSION][BISECTED] Dell Precision 7780 wakes up on its own from suspend

[Askar Safin]

Okay, so here are results of new expirements.

Now I write 1 to /sys/power/pm_debug_messages in the beginning.

I was unable to find "last_hw_sleep" in /sys/power/suspend_stats on these particular kernel configurations.
But I print all other files in /sys/power/suspend_stats .

I used same minimized config as in previous expirements.
I used modified script:

https://zerobin.net/?bccdd94f5f07bf85#8jop5IXDG32p0I7SBfEc29ng0/MdAFZgrqI9x+4A/Fo=

Here is dmesg for "good" commit "d86388c902b45fe2" without external mouse:
https://zerobin.net/?90d41139996f1cd8#Xuy0UjyxtXrOyLxq4YMP/IzUB559v7ESEIqsd/4Fqo0=

Here is dmesg for "bad" commit "1796f808e4bb2c07" without external mouse:
https://zerobin.net/?181b4d79c16fd9b3#nu0HyJX/r9cFtKitVxJB+3CZsGfwKHc6JLYX4OyZHR4=

Here is dmesg for "good" commit "d86388c902b45fe2" with external mouse:
https://zerobin.net/?a13c513b12549f62#LPLtc3MA1iSEPUjaizDk6xwBgEitH16hZE/WUWdgr6c=

Here is dmesg for "bad" commit "1796f808e4bb2c07" with external mouse:
https://zerobin.net/?cda0d7dd920ad66a#VHWKTp8motWKNJawyFC+v+qNP55an/ZR7HTlhmLJ7gA=

As we can see, external USB wireless mouse doesn't really change anything.
No wonder, I have CONFIG_USB_SUPPORT disabled in my minimized config. :)

--
Askar Safin
https://types.pl/@safinaskar

Re: [REGRESSION][BISECTED] Dell Precision 7780 wakes up on its own from suspend

[Mario Limonciello]

On 6/26/25 10:51 PM, Askar Safin wrote:
> Okay, so here are results of new expirements.
> 
> Now I write 1 to /sys/power/pm_debug_messages in the beginning.
> 
> I was unable to find "last_hw_sleep" in /sys/power/suspend_stats on these particular kernel configurations.
> But I print all other files in /sys/power/suspend_stats .

That's odd.  It should be made when the PMC core driver binds.  Maybe 
others will know what's missing here.

> 
> I used same minimized config as in previous expirements.
> I used modified script:
> 
> https://zerobin.net/?bccdd94f5f07bf85#8jop5IXDG32p0I7SBfEc29ng0/MdAFZgrqI9x+4A/Fo=
> 
> Here is dmesg for "good" commit "d86388c902b45fe2" without external mouse:
> https://zerobin.net/?90d41139996f1cd8#Xuy0UjyxtXrOyLxq4YMP/IzUB559v7ESEIqsd/4Fqo0=
> 
> Here is dmesg for "bad" commit "1796f808e4bb2c07" without external mouse:
> https://zerobin.net/?181b4d79c16fd9b3#nu0HyJX/r9cFtKitVxJB+3CZsGfwKHc6JLYX4OyZHR4=
> 
> Here is dmesg for "good" commit "d86388c902b45fe2" with external mouse:
> https://zerobin.net/?a13c513b12549f62#LPLtc3MA1iSEPUjaizDk6xwBgEitH16hZE/WUWdgr6c=
> 
> Here is dmesg for "bad" commit "1796f808e4bb2c07" with external mouse:
> https://zerobin.net/?cda0d7dd920ad66a#VHWKTp8motWKNJawyFC+v+qNP55an/ZR7HTlhmLJ7gA=
> 
> As we can see, external USB wireless mouse doesn't really change anything.
> No wonder, I have CONFIG_USB_SUPPORT disabled in my minimized config. :)
> 
I see in your bad config interrupt 14 is waking the system.  In the good 
config interrupt 8 is waking it.

What is in /proc/interrupts?

Re: [REGRESSION][BISECTED] Dell Precision 7780 wakes up on its own from suspend

[Askar Safin]

 ---- On Fri, 27 Jun 2025 07:58:15 +0400  Mario Limonciello <superm1@kernel.org> wrote ---
 > That's odd.  It should be made when the PMC core driver binds.  Maybe
 > others will know what's missing here.

Command "grep -r -E -I last_hw_sleep ." in culpit kernel (1796f808e4bb2c074824d)
shows nothing. (This is somewhere around 6.1).

So, culpit commit is too old.

If you want, I can retest this thing on current master and on current master with
revert 1796f808e4bb2c074824d.

 > I see in your bad config interrupt 14 is waking the system.  In the good
 > config interrupt 8 is waking it.
 >
 > What is in /proc/interrupts?

/proc/interrupts from culpit kernel: https://paste.debian.net/1382819/

--
Askar Safin
https://types.pl/@safinaskar

Re: [REGRESSION][BISECTED] Dell Precision 7780 wakes up on its own from suspend

[Mario Limonciello]

On 6/30/2025 3:14 AM, Askar Safin wrote:
>   ---- On Fri, 27 Jun 2025 07:58:15 +0400  Mario Limonciello <superm1@kernel.org> wrote ---
>   > That's odd.  It should be made when the PMC core driver binds.  Maybe
>   > others will know what's missing here.
> 
> Command "grep -r -E -I last_hw_sleep ." in culpit kernel (1796f808e4bb2c074824d)
> shows nothing. (This is somewhere around 6.1).
> 
> So, culpit commit is too old.
> 
> If you want, I can retest this thing on current master and on current master with
> revert 1796f808e4bb2c074824d.
> 
>   > I see in your bad config interrupt 14 is waking the system.  In the good
>   > config interrupt 8 is waking it.
>   >
>   > What is in /proc/interrupts?
> 
> /proc/interrupts from culpit kernel: https://paste.debian.net/1382819/
> 
> --
> Askar Safin
> https://types.pl/@safinaskar
> 

Looks like your interrupt 14 is ACPI device INTC1085:00.

Some quick searches this seems to be an Intel GPIO controller.

Andy,

Any ideas how to debug next?

Re: [REGRESSION][BISECTED] Dell Precision 7780 wakes up on its own from suspend

[Askar Safin]

 ---- On Mon, 30 Jun 2025 22:40:28 +0400  Mario Limonciello <superm1@kernel.org> wrote --- 
 > Looks like your interrupt 14 is ACPI device INTC1085:00.
 > 
 > Some quick searches this seems to be an Intel GPIO controller.
 > 
 > Andy,
 > 
 > Any ideas how to debug next?

So what?

I will repeat: similar bugs were already fixed here:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=782eea0c89f7d071d6b56ecfa1b8b0c81164b9be
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=a69982c37cd0586e6832268155349301b87f2e35
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4cb786180dfb5258ff3111181b5e4ecb1d4a297b

So we just need to add similar patch. I. e. something similar to this:

+		.matches = {
+			DMI_MATCH(DMI_BOARD_NAME, "NL5xRU"),
+		},
+		.driver_data = &(struct acpi_gpiolib_dmi_quirk) {
+			.ignore_wake = "ELAN0415:00@9",
+		},

Remaining question: how should I get these strings on my system? "NL5xRU" and "ELAN0415:00@9"?
--
Askar Safin
https://types.pl/@safinaskar

Re: [REGRESSION][BISECTED] Dell Precision 7780 wakes up on its own from suspend

[Werner Sembach]

Hi,

Am 18.07.25 um 01:29 schrieb Askar Safin:
>   ---- On Mon, 30 Jun 2025 22:40:28 +0400  Mario Limonciello<superm1@kernel.org> wrote ---
>   > Looks like your interrupt 14 is ACPI device INTC1085:00.
>   >
>   > Some quick searches this seems to be an Intel GPIO controller.
>   >
>   > Andy,
>   >
>   > Any ideas how to debug next?
>
> So what?
>
> I will repeat: similar bugs were already fixed here:
>
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=782eea0c89f7d071d6b56ecfa1b8b0c81164b9be
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=a69982c37cd0586e6832268155349301b87f2e35
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4cb786180dfb5258ff3111181b5e4ecb1d4a297b
>
> So we just need to add similar patch. I. e. something similar to this:
>
> +		.matches = {
> +			DMI_MATCH(DMI_BOARD_NAME, "NL5xRU"),
> +		},
> +		.driver_data = &(struct acpi_gpiolib_dmi_quirk) {
> +			.ignore_wake = "ELAN0415:00@9",
> +		},
>
> Remaining question: how should I get these strings on my system? "NL5xRU" and "ELAN0415:00@9"?

DMI_MATCH(DMI_BOARD_NAME, *) will match to /sys/class/dmi/id/board_name 
(/sys/class/dmi/id/ also contains a lot of other strings to identify your device 
e.g. /sys/class/dmi/id/sys_vendor which can be matched with 
DMI_MATCH(DMI_SYS_VENDOR, *))

How to get "ELAN0415:00@9" is somewhere buried in here 
https://gitlab.freedesktop.org/drm/amd/-/issues/1722#note_1725104

Best regards,

Werner

> --
> Askar Safin
> https://types.pl/@safinaskar
>

Re: [REGRESSION][BISECTED] Dell Precision 7780 wakes up on its own from suspend

[Mario Limonciello]

On 7/18/2025 10:13 AM, Werner Sembach wrote:
> Hi,
> 
> Am 18.07.25 um 01:29 schrieb Askar Safin:
>>   ---- On Mon, 30 Jun 2025 22:40:28 +0400  Mario 
>> Limonciello<superm1@kernel.org> wrote ---
>>   > Looks like your interrupt 14 is ACPI device INTC1085:00.
>>   >
>>   > Some quick searches this seems to be an Intel GPIO controller.
>>   >
>>   > Andy,
>>   >
>>   > Any ideas how to debug next?
>>
>> So what?
>>
>> I will repeat: similar bugs were already fixed here:
>>
>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/ 
>> commit/?id=782eea0c89f7d071d6b56ecfa1b8b0c81164b9be
>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/ 
>> commit/?id=a69982c37cd0586e6832268155349301b87f2e35
>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/ 
>> commit/?id=4cb786180dfb5258ff3111181b5e4ecb1d4a297b
>>
>> So we just need to add similar patch. I. e. something similar to this:
>>
>> +        .matches = {
>> +            DMI_MATCH(DMI_BOARD_NAME, "NL5xRU"),
>> +        },
>> +        .driver_data = &(struct acpi_gpiolib_dmi_quirk) {
>> +            .ignore_wake = "ELAN0415:00@9",
>> +        },
>>
>> Remaining question: how should I get these strings on my system? 
>> "NL5xRU" and "ELAN0415:00@9"?
> 
> DMI_MATCH(DMI_BOARD_NAME, *) will match to /sys/class/dmi/id/board_name 
> (/sys/class/dmi/id/ also contains a lot of other strings to identify 
> your device e.g. /sys/class/dmi/id/sys_vendor which can be matched with 
> DMI_MATCH(DMI_SYS_VENDOR, *))
> 
> How to get "ELAN0415:00@9" is somewhere buried in here https:// 
> gitlab.freedesktop.org/drm/amd/-/issues/1722#note_1725104
> 
> Best regards,
> 
> Werner
> 
>> -- 
>> Askar Safin
>> https://types.pl/@safinaskar
>>

Before jumping into quirk development I think we need to know exactly 
what GPIO is being tripped and that needs to be matched against the Dell 
design (or Intel specs if it's an "internal" GPIO).

That's why I was hoping Andy could help advise.

On the issue that was linked there we knew the GPIO was tied to the 
attention interrupt for touchpad/touchscreen causing the issue so it was 
easy to look at an acpidump and figure out what it's supposed to be.

Re: [REGRESSION][BISECTED] Dell Precision 7780 wakes up on its own from suspend

[Andy Shevchenko]

On Mon, Jun 30, 2025 at 02:40:28PM -0400, Mario Limonciello wrote:
> On 6/30/2025 3:14 AM, Askar Safin wrote:
> >   ---- On Fri, 27 Jun 2025 07:58:15 +0400  Mario Limonciello <superm1@kernel.org> wrote ---
> >   > That's odd.  It should be made when the PMC core driver binds.  Maybe
> >   > others will know what's missing here.
> > 
> > Command "grep -r -E -I last_hw_sleep ." in culpit kernel (1796f808e4bb2c074824d)
> > shows nothing. (This is somewhere around 6.1).
> > 
> > So, culpit commit is too old.
> > 
> > If you want, I can retest this thing on current master and on current master with
> > revert 1796f808e4bb2c074824d.
> > 
> >   > I see in your bad config interrupt 14 is waking the system.  In the good
> >   > config interrupt 8 is waking it.
> >   >
> >   > What is in /proc/interrupts?
> > 
> > /proc/interrupts from culpit kernel: https://paste.debian.net/1382819/
> > 
> > --
> > Askar Safin
> > https://types.pl/@safinaskar
> > 
> 
> Looks like your interrupt 14 is ACPI device INTC1085:00.
> 
> Some quick searches this seems to be an Intel GPIO controller.
> 
> Andy,
> 
> Any ideas how to debug next?

I believe it's related to the touchpad (can you check that wake happens due to
actually IRQ on pin 355 of the GPIO controller?


-- 
With Best Regards,
Andy Shevchenko

Re: [REGRESSION][BISECTED] Dell Precision 7780 wakes up on its own from suspend

[Andy Shevchenko]

On Mon, Aug 25, 2025 at 06:34:48PM +0300, Andy Shevchenko wrote:
> On Mon, Jun 30, 2025 at 02:40:28PM -0400, Mario Limonciello wrote:
> > On 6/30/2025 3:14 AM, Askar Safin wrote:
> > >   ---- On Fri, 27 Jun 2025 07:58:15 +0400  Mario Limonciello <superm1@kernel.org> wrote ---
> > >   > That's odd.  It should be made when the PMC core driver binds.  Maybe
> > >   > others will know what's missing here.
> > > 
> > > Command "grep -r -E -I last_hw_sleep ." in culpit kernel (1796f808e4bb2c074824d)
> > > shows nothing. (This is somewhere around 6.1).
> > > 
> > > So, culpit commit is too old.
> > > 
> > > If you want, I can retest this thing on current master and on current master with
> > > revert 1796f808e4bb2c074824d.
> > > 
> > >   > I see in your bad config interrupt 14 is waking the system.  In the good
> > >   > config interrupt 8 is waking it.
> > >   >
> > >   > What is in /proc/interrupts?
> > > 
> > > /proc/interrupts from culpit kernel: https://paste.debian.net/1382819/
> > > 
> > > --
> > > Askar Safin
> > > https://types.pl/@safinaskar
> > > 
> > 
> > Looks like your interrupt 14 is ACPI device INTC1085:00.
> > 
> > Some quick searches this seems to be an Intel GPIO controller.
> > 
> > Andy,
> > 
> > Any ideas how to debug next?
> 
> I believe it's related to the touchpad (can you check that wake happens due to
> actually IRQ on pin 355 of the GPIO controller?

In other words we need to enable debug of the pin control subsystem and see
what it will print in dmesg.

-- 
With Best Regards,
Andy Shevchenko