[PATCH v2] PCI/PM: Put devices to low power state on shutdown

[PATCH v2] PCI/PM: Put devices to low power state on shutdown

[Kai-Heng Feng]

Some laptops wake up after poweroff when HP Thunderbolt Dock G4 is
connected.

The following error message can be found during shutdown:
pcieport 0000:00:1d.0: AER: Correctable error message received from 0000:09:04.0
pcieport 0000:09:04.0: PCIe Bus Error: severity=Correctable, type=Data Link Layer, (Receiver ID)
pcieport 0000:09:04.0:   device [8086:0b26] error status/mask=00000080/00002000
pcieport 0000:09:04.0:    [ 7] BadDLLP

Calling aer_remove() during shutdown can quiesce the error message,
however the spurious wakeup still happens.

The issue won't happen if the device is in D3 before system shutdown, so
putting device to low power state before shutdown to solve the issue.

ACPI Spec 6.5, "7.4.2.5 System \_S4 State" says "Devices states are
compatible with the current Power Resource states. In other words, all
devices are in the D3 state when the system state is S4."

The following "7.4.2.6 System \_S5 State (Soft Off)" states "The S5
state is similar to the S4 state except that OSPM does not save any
context." so it's safe to assume devices should be at D3 for S5.

Link: https://bugzilla.kernel.org/show_bug.cgi?id=219036
Cc: AceLan Kao <acelan.kao@canonical.com>
Reviewed-by: Mario Limonciello <mario.limonciello@amd.com>
Tested-by: Mario Limonciello <mario.limonciello@amd.com>
Signed-off-by: Kai-Heng Feng <kaihengf@nvidia.com>
---
 drivers/pci/pci-driver.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c
index 35270172c833..248e0c9fd161 100644
--- a/drivers/pci/pci-driver.c
+++ b/drivers/pci/pci-driver.c
@@ -510,6 +510,14 @@ static void pci_device_shutdown(struct device *dev)
 	if (drv && drv->shutdown)
 		drv->shutdown(pci_dev);
 
+	/*
+	 * If driver already changed device's power state, it can mean the
+	 * wakeup setting is in place, or a workaround is used. Hence keep it
+	 * as is.
+	 */
+	if (!kexec_in_progress && pci_dev->current_state == PCI_D0)
+		pci_prepare_to_sleep(pci_dev);
+
 	/*
 	 * If this is a kexec reboot, turn off Bus Master bit on the
 	 * device to tell it to not continue to do DMA. Don't touch
-- 
2.47.0

Re: [PATCH v2] PCI/PM: Put devices to low power state on shutdown

[Mark Pearson]

Hi,

On Sun, Dec 8, 2024, at 2:41 AM, Kai-Heng Feng wrote:
> Some laptops wake up after poweroff when HP Thunderbolt Dock G4 is
> connected.
>
> The following error message can be found during shutdown:
> pcieport 0000:00:1d.0: AER: Correctable error message received from 
> 0000:09:04.0
> pcieport 0000:09:04.0: PCIe Bus Error: severity=Correctable, type=Data 
> Link Layer, (Receiver ID)
> pcieport 0000:09:04.0:   device [8086:0b26] error 
> status/mask=00000080/00002000
> pcieport 0000:09:04.0:    [ 7] BadDLLP
>
> Calling aer_remove() during shutdown can quiesce the error message,
> however the spurious wakeup still happens.
>
> The issue won't happen if the device is in D3 before system shutdown, so
> putting device to low power state before shutdown to solve the issue.
>
> ACPI Spec 6.5, "7.4.2.5 System \_S4 State" says "Devices states are
> compatible with the current Power Resource states. In other words, all
> devices are in the D3 state when the system state is S4."
>
> The following "7.4.2.6 System \_S5 State (Soft Off)" states "The S5
> state is similar to the S4 state except that OSPM does not save any
> context." so it's safe to assume devices should be at D3 for S5.
>
> Link: https://bugzilla.kernel.org/show_bug.cgi?id=219036
> Cc: AceLan Kao <acelan.kao@canonical.com>
> Reviewed-by: Mario Limonciello <mario.limonciello@amd.com>
> Tested-by: Mario Limonciello <mario.limonciello@amd.com>
> Signed-off-by: Kai-Heng Feng <kaihengf@nvidia.com>
> ---
>  drivers/pci/pci-driver.c | 8 ++++++++
>  1 file changed, 8 insertions(+)
>
> diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c
> index 35270172c833..248e0c9fd161 100644
> --- a/drivers/pci/pci-driver.c
> +++ b/drivers/pci/pci-driver.c
> @@ -510,6 +510,14 @@ static void pci_device_shutdown(struct device *dev)
>  	if (drv && drv->shutdown)
>  		drv->shutdown(pci_dev);
> 
> +	/*
> +	 * If driver already changed device's power state, it can mean the
> +	 * wakeup setting is in place, or a workaround is used. Hence keep it
> +	 * as is.
> +	 */
> +	if (!kexec_in_progress && pci_dev->current_state == PCI_D0)
> +		pci_prepare_to_sleep(pci_dev);
> +
>  	/*
>  	 * If this is a kexec reboot, turn off Bus Master bit on the
>  	 * device to tell it to not continue to do DMA. Don't touch
> -- 
> 2.47.0

Just a note that we've tested this in the Lenovo Linux team and confirmed that it reduces the power draw on a powered off Z16 G2 by 0.6W.
This is enough to bring Linux inline with Windows, and more importantly allow the platform to pass e-star energy certification (which it otherwise fails). We suspect other platforms will show similar benefits.

Let me know if there's anything we can do to help get this patch moving along - I think it's important.

Tested-by: Mark Pearson <mpearson-lenovo@squebb.ca>

Mark

Re: [PATCH v2] PCI/PM: Put devices to low power state on shutdown

[Mario Limonciello]

On 1/17/25 14:31, Mark Pearson wrote:
> Hi,
> 
> On Sun, Dec 8, 2024, at 2:41 AM, Kai-Heng Feng wrote:
>> Some laptops wake up after poweroff when HP Thunderbolt Dock G4 is
>> connected.
>>
>> The following error message can be found during shutdown:
>> pcieport 0000:00:1d.0: AER: Correctable error message received from
>> 0000:09:04.0
>> pcieport 0000:09:04.0: PCIe Bus Error: severity=Correctable, type=Data
>> Link Layer, (Receiver ID)
>> pcieport 0000:09:04.0:   device [8086:0b26] error
>> status/mask=00000080/00002000
>> pcieport 0000:09:04.0:    [ 7] BadDLLP
>>
>> Calling aer_remove() during shutdown can quiesce the error message,
>> however the spurious wakeup still happens.
>>
>> The issue won't happen if the device is in D3 before system shutdown, so
>> putting device to low power state before shutdown to solve the issue.
>>
>> ACPI Spec 6.5, "7.4.2.5 System \_S4 State" says "Devices states are
>> compatible with the current Power Resource states. In other words, all
>> devices are in the D3 state when the system state is S4."
>>
>> The following "7.4.2.6 System \_S5 State (Soft Off)" states "The S5
>> state is similar to the S4 state except that OSPM does not save any
>> context." so it's safe to assume devices should be at D3 for S5.
>>
>> Link: https://bugzilla.kernel.org/show_bug.cgi?id=219036
>> Cc: AceLan Kao <acelan.kao@canonical.com>
>> Reviewed-by: Mario Limonciello <mario.limonciello@amd.com>
>> Tested-by: Mario Limonciello <mario.limonciello@amd.com>
>> Signed-off-by: Kai-Heng Feng <kaihengf@nvidia.com>
>> ---
>>   drivers/pci/pci-driver.c | 8 ++++++++
>>   1 file changed, 8 insertions(+)
>>
>> diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c
>> index 35270172c833..248e0c9fd161 100644
>> --- a/drivers/pci/pci-driver.c
>> +++ b/drivers/pci/pci-driver.c
>> @@ -510,6 +510,14 @@ static void pci_device_shutdown(struct device *dev)
>>   	if (drv && drv->shutdown)
>>   		drv->shutdown(pci_dev);
>>
>> +	/*
>> +	 * If driver already changed device's power state, it can mean the
>> +	 * wakeup setting is in place, or a workaround is used. Hence keep it
>> +	 * as is.
>> +	 */
>> +	if (!kexec_in_progress && pci_dev->current_state == PCI_D0)
>> +		pci_prepare_to_sleep(pci_dev);
>> +
>>   	/*
>>   	 * If this is a kexec reboot, turn off Bus Master bit on the
>>   	 * device to tell it to not continue to do DMA. Don't touch
>> -- 
>> 2.47.0
> 
> Just a note that we've tested this in the Lenovo Linux team and confirmed that it reduces the power draw on a powered off Z16 G2 by 0.6W.
> This is enough to bring Linux inline with Windows, and more importantly allow the platform to pass e-star energy certification (which it otherwise fails). We suspect other platforms will show similar benefits.
> 
> Let me know if there's anything we can do to help get this patch moving along - I think it's important.
> 
> Tested-by: Mark Pearson <mpearson-lenovo@squebb.ca>
> 
> Mark
> 

Bjorn,

Anything else that you would like to see for this patch?
Or different direction you would like to see it go?

Thanks,

Re: [PATCH v2] PCI/PM: Put devices to low power state on shutdown

[Denis Benato]

On 17/01/25 21:31, Mark Pearson wrote:
> Hi,
> 
> On Sun, Dec 8, 2024, at 2:41 AM, Kai-Heng Feng wrote:
>> Some laptops wake up after poweroff when HP Thunderbolt Dock G4 is
>> connected.
>>
>> The following error message can be found during shutdown:
>> pcieport 0000:00:1d.0: AER: Correctable error message received from 
>> 0000:09:04.0
>> pcieport 0000:09:04.0: PCIe Bus Error: severity=Correctable, type=Data 
>> Link Layer, (Receiver ID)
>> pcieport 0000:09:04.0:   device [8086:0b26] error 
>> status/mask=00000080/00002000
>> pcieport 0000:09:04.0:    [ 7] BadDLLP
>>
>> Calling aer_remove() during shutdown can quiesce the error message,
>> however the spurious wakeup still happens.
>>
>> The issue won't happen if the device is in D3 before system shutdown, so
>> putting device to low power state before shutdown to solve the issue.
>>
>> ACPI Spec 6.5, "7.4.2.5 System \_S4 State" says "Devices states are
>> compatible with the current Power Resource states. In other words, all
>> devices are in the D3 state when the system state is S4."
>>
>> The following "7.4.2.6 System \_S5 State (Soft Off)" states "The S5
>> state is similar to the S4 state except that OSPM does not save any
>> context." so it's safe to assume devices should be at D3 for S5.
>>
>> Link: https://bugzilla.kernel.org/show_bug.cgi?id=219036
>> Cc: AceLan Kao <acelan.kao@canonical.com>
>> Reviewed-by: Mario Limonciello <mario.limonciello@amd.com>
>> Tested-by: Mario Limonciello <mario.limonciello@amd.com>
>> Signed-off-by: Kai-Heng Feng <kaihengf@nvidia.com>
>> ---
>>  drivers/pci/pci-driver.c | 8 ++++++++
>>  1 file changed, 8 insertions(+)
>>
>> diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c
>> index 35270172c833..248e0c9fd161 100644
>> --- a/drivers/pci/pci-driver.c
>> +++ b/drivers/pci/pci-driver.c
>> @@ -510,6 +510,14 @@ static void pci_device_shutdown(struct device *dev)
>>  	if (drv && drv->shutdown)
>>  		drv->shutdown(pci_dev);
>>
>> +	/*
>> +	 * If driver already changed device's power state, it can mean the
>> +	 * wakeup setting is in place, or a workaround is used. Hence keep it
>> +	 * as is.
>> +	 */
>> +	if (!kexec_in_progress && pci_dev->current_state == PCI_D0)
>> +		pci_prepare_to_sleep(pci_dev);
>> +
>>  	/*
>>  	 * If this is a kexec reboot, turn off Bus Master bit on the
>>  	 * device to tell it to not continue to do DMA. Don't touch
>> -- 
>> 2.47.0
> 
> Just a note that we've tested this in the Lenovo Linux team and confirmed that it reduces the power draw on a powered off Z16 G2 by 0.6W.
> This is enough to bring Linux inline with Windows, and more importantly allow the platform to pass e-star energy certification (which it otherwise fails). We suspect other platforms will show similar benefits.
> 
> Let me know if there's anything we can do to help get this patch moving along - I think it's important.
> 
> Tested-by: Mark Pearson <mpearson-lenovo@squebb.ca>
> 
> Mark

Hello,

I have helped an user affected by the issue this patch solves (FA507NV).

The issue has been solved by this patch.

I also tested this patch with two unaffected laptops: my own (G634JZ) and my brother's (FX507ZE): neither showed sign of any regressions.

Tested-by: Merthan Karakaş <m3rthn.k@gmail.com>
Tested-by: Denis Benato <benato.denis96@gmail.com>

Re: [PATCH v2] PCI/PM: Put devices to low power state on shutdown'

[Bjorn Helgaas]

[+to Rafael, +cc linux-pm]

On Sun, Dec 08, 2024 at 03:41:47PM +0800, Kai-Heng Feng wrote:
> Some laptops wake up after poweroff when HP Thunderbolt Dock G4 is
> connected.
> 
> The following error message can be found during shutdown:
> pcieport 0000:00:1d.0: AER: Correctable error message received from 0000:09:04.0
> pcieport 0000:09:04.0: PCIe Bus Error: severity=Correctable, type=Data Link Layer, (Receiver ID)
> pcieport 0000:09:04.0:   device [8086:0b26] error status/mask=00000080/00002000
> pcieport 0000:09:04.0:    [ 7] BadDLLP
> 
> Calling aer_remove() during shutdown can quiesce the error message,
> however the spurious wakeup still happens.

aer_remove() disables AER interrupts, so I guess there must be a
non-AER interrupt being generated during shutdown?

If so, AER is a red herring and including the AER details above is a
distraction from whatever the real interrupt cause is.

> The issue won't happen if the device is in D3 before system shutdown, so
> putting device to low power state before shutdown to solve the issue.
> 
> ACPI Spec 6.5, "7.4.2.5 System \_S4 State" says "Devices states are
> compatible with the current Power Resource states. In other words, all
> devices are in the D3 state when the system state is S4."
> 
> The following "7.4.2.6 System \_S5 State (Soft Off)" states "The S5
> state is similar to the S4 state except that OSPM does not save any
> context." so it's safe to assume devices should be at D3 for S5.
> 
> Link: https://bugzilla.kernel.org/show_bug.cgi?id=219036
> Cc: AceLan Kao <acelan.kao@canonical.com>
> Reviewed-by: Mario Limonciello <mario.limonciello@amd.com>
> Tested-by: Mario Limonciello <mario.limonciello@amd.com>
> Signed-off-by: Kai-Heng Feng <kaihengf@nvidia.com>
> ---
>  drivers/pci/pci-driver.c | 8 ++++++++
>  1 file changed, 8 insertions(+)
> 
> diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c
> index 35270172c833..248e0c9fd161 100644
> --- a/drivers/pci/pci-driver.c
> +++ b/drivers/pci/pci-driver.c
> @@ -510,6 +510,14 @@ static void pci_device_shutdown(struct device *dev)
>  	if (drv && drv->shutdown)
>  		drv->shutdown(pci_dev);
>  
> +	/*
> +	 * If driver already changed device's power state, it can mean the
> +	 * wakeup setting is in place, or a workaround is used. Hence keep it
> +	 * as is.
> +	 */
> +	if (!kexec_in_progress && pci_dev->current_state == PCI_D0)
> +		pci_prepare_to_sleep(pci_dev);


I don't know enough to draw inferences about PCI_D0 meaning a wakeup
setting is in place or a workaround being used.  That doesn't seem
like enough to be useful for me to maintain this in the future.  But
my power management understanding is pretty meager.

Would like an ack from Rafael for this.

>  	/*
>  	 * If this is a kexec reboot, turn off Bus Master bit on the
>  	 * device to tell it to not continue to do DMA. Don't touch
> -- 
> 2.47.0
> 

Re: [PATCH v2] PCI/PM: Put devices to low power state on shutdown'

[Mario Limonciello]

On 2/19/2025 12:46 PM, Bjorn Helgaas wrote:
> [+to Rafael, +cc linux-pm]
> 
> On Sun, Dec 08, 2024 at 03:41:47PM +0800, Kai-Heng Feng wrote:
>> Some laptops wake up after poweroff when HP Thunderbolt Dock G4 is
>> connected.
>>
>> The following error message can be found during shutdown:
>> pcieport 0000:00:1d.0: AER: Correctable error message received from 0000:09:04.0
>> pcieport 0000:09:04.0: PCIe Bus Error: severity=Correctable, type=Data Link Layer, (Receiver ID)
>> pcieport 0000:09:04.0:   device [8086:0b26] error status/mask=00000080/00002000
>> pcieport 0000:09:04.0:    [ 7] BadDLLP
>>
>> Calling aer_remove() during shutdown can quiesce the error message,
>> however the spurious wakeup still happens.
> 
> aer_remove() disables AER interrupts, so I guess there must be a
> non-AER interrupt being generated during shutdown?
> 
> If so, AER is a red herring and including the AER details above is a
> distraction from whatever the real interrupt cause is.
> 
>> The issue won't happen if the device is in D3 before system shutdown, so
>> putting device to low power state before shutdown to solve the issue.
>>
>> ACPI Spec 6.5, "7.4.2.5 System \_S4 State" says "Devices states are
>> compatible with the current Power Resource states. In other words, all
>> devices are in the D3 state when the system state is S4."
>>
>> The following "7.4.2.6 System \_S5 State (Soft Off)" states "The S5
>> state is similar to the S4 state except that OSPM does not save any
>> context." so it's safe to assume devices should be at D3 for S5.
>>
>> Link: https://bugzilla.kernel.org/show_bug.cgi?id=219036
>> Cc: AceLan Kao <acelan.kao@canonical.com>
>> Reviewed-by: Mario Limonciello <mario.limonciello@amd.com>
>> Tested-by: Mario Limonciello <mario.limonciello@amd.com>
>> Signed-off-by: Kai-Heng Feng <kaihengf@nvidia.com>
>> ---
>>   drivers/pci/pci-driver.c | 8 ++++++++
>>   1 file changed, 8 insertions(+)
>>
>> diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c
>> index 35270172c833..248e0c9fd161 100644
>> --- a/drivers/pci/pci-driver.c
>> +++ b/drivers/pci/pci-driver.c
>> @@ -510,6 +510,14 @@ static void pci_device_shutdown(struct device *dev)
>>   	if (drv && drv->shutdown)
>>   		drv->shutdown(pci_dev);
>>   
>> +	/*
>> +	 * If driver already changed device's power state, it can mean the
>> +	 * wakeup setting is in place, or a workaround is used. Hence keep it
>> +	 * as is.
>> +	 */
>> +	if (!kexec_in_progress && pci_dev->current_state == PCI_D0)
>> +		pci_prepare_to_sleep(pci_dev);
> 
> 
> I don't know enough to draw inferences about PCI_D0 meaning a wakeup
> setting is in place or a workaround being used.  That doesn't seem
> like enough to be useful for me to maintain this in the future.  But
> my power management understanding is pretty meager.
> 
> Would like an ack from Rafael for this.

Rafael,

Can you take a look at this patch and provide your thoughts?

Thanks!
> 
>>   	/*
>>   	 * If this is a kexec reboot, turn off Bus Master bit on the
>>   	 * device to tell it to not continue to do DMA. Don't touch
>> -- 
>> 2.47.0
>>
>