[PATCH] PCI/ASPM: Enable L0s/L1 for removable devices when BIOS didn't configure ASPM

[PATCH] PCI/ASPM: Enable L0s/L1 for removable devices when BIOS didn't configure ASPM

[Mario Limonciello]

When comparing lspci output between Windows and Linux for hotplugged
Thunderbolt 5 eGPU devices, Windows enables ASPM L1 but Linux doesn't:

  Windows: LnkCtl: ASPM L1 Enabled
  Linux:   LnkCtl: ASPM Disabled

This difference in ASPM configuration can cause behavioral differences
between the two operating systems for the same hardware.

The root cause is that Linux's default ASPM policy (POLICY_DEFAULT) relies
on firmware/BIOS configuration. For hotplugged devices like Thunderbolt/USB4
eGPUs, the BIOS may not have configured ASPM since the device wasn't present
at boot time. As a result, link->aspm_enabled is 0, link->aspm_default is
set to 0, and Linux never enables ASPM for these devices.

Devicetree platforms already have special handling to enable L0s/L1 by
default regardless of firmware configuration. Extend this same logic to
removable devices when firmware hasn't configured any ASPM states.

This makes Linux behavior more consistent with Windows for hotplugged
Thunderbolt/USB4 devices.

Link: https://bugzilla.kernel.org/show_bug.cgi?id=221319
Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
---
 drivers/pci/pcie/aspm.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c
index 925373b98dff0..77497d90be0b7 100644
--- a/drivers/pci/pcie/aspm.c
+++ b/drivers/pci/pcie/aspm.c
@@ -804,8 +804,15 @@ static void pcie_aspm_override_default_link_state(struct pcie_link_state *link)
 	struct pci_dev *pdev = link->downstream;
 	u32 override;
 
-	/* For devicetree platforms, enable L0s and L1 by default */
-	if (of_have_populated_dt()) {
+	/*
+	 * For devicetree platforms, enable L0s and L1 by default.
+	 *
+	 * For removable devices (e.g., Thunderbolt/USB4), enable L0s and L1
+	 * by default if BIOS didn't configure any ASPM states. This handles
+	 * hotplugged devices where firmware may not have configured ASPM.
+	 */
+	if (of_have_populated_dt() ||
+	    (dev_is_removable(&pdev->dev) && !link->aspm_enabled)) {
 		if (link->aspm_support & PCIE_LINK_STATE_L0S)
 			link->aspm_default |= PCIE_LINK_STATE_L0S;
 		if (link->aspm_support & PCIE_LINK_STATE_L1)
-- 
2.43.0

Re: [PATCH] PCI/ASPM: Enable L0s/L1 for removable devices when BIOS didn't configure ASPM

[Bjorn Helgaas]

On Mon, May 04, 2026 at 05:52:46PM -0500, Mario Limonciello wrote:
> When comparing lspci output between Windows and Linux for hotplugged
> Thunderbolt 5 eGPU devices, Windows enables ASPM L1 but Linux doesn't:
> 
>   Windows: LnkCtl: ASPM L1 Enabled
>   Linux:   LnkCtl: ASPM Disabled
> 
> This difference in ASPM configuration can cause behavioral differences
> between the two operating systems for the same hardware.

A tangent, not a comment on the patch itself, but what sort of
behavioral differences are these?  If ASPM is working correctly, the
only differences *should* be in power consumption and performance.

Re: [PATCH] PCI/ASPM: Enable L0s/L1 for removable devices when BIOS didn't configure ASPM

[Mario Limonciello]

On 5/5/26 11:05, Bjorn Helgaas wrote:
> On Mon, May 04, 2026 at 05:52:46PM -0500, Mario Limonciello wrote:
>> When comparing lspci output between Windows and Linux for hotplugged
>> Thunderbolt 5 eGPU devices, Windows enables ASPM L1 but Linux doesn't:
>>
>>    Windows: LnkCtl: ASPM L1 Enabled
>>    Linux:   LnkCtl: ASPM Disabled
>>
>> This difference in ASPM configuration can cause behavioral differences
>> between the two operating systems for the same hardware.
> 
> A tangent, not a comment on the patch itself, but what sort of
> behavioral differences are these?  If ASPM is working correctly, the
> only differences *should* be in power consumption and performance.

This originally stemmed from a significant performance difference that 
was observed between Windows and Linux with eGPUs.  The link in the 
patch points at that bug if you want to look more closely at it.

I was hopeful that aligning ASPM would align the behavior, but alas this 
didn't.

It was still a difference that I figured we should discuss whether it 
should be changed to be consistent.

Re: [PATCH] PCI/ASPM: Enable L0s/L1 for removable devices when BIOS didn't configure ASPM

[Rafael J. Wysocki]

+Mika Westerberg

On Tue, May 5, 2026 at 12:53 AM Mario Limonciello
<mario.limonciello@amd.com> wrote:
>
> When comparing lspci output between Windows and Linux for hotplugged
> Thunderbolt 5 eGPU devices, Windows enables ASPM L1 but Linux doesn't:
>
>   Windows: LnkCtl: ASPM L1 Enabled
>   Linux:   LnkCtl: ASPM Disabled
>
> This difference in ASPM configuration can cause behavioral differences
> between the two operating systems for the same hardware.
>
> The root cause is that Linux's default ASPM policy (POLICY_DEFAULT) relies
> on firmware/BIOS configuration. For hotplugged devices like Thunderbolt/USB4
> eGPUs, the BIOS may not have configured ASPM since the device wasn't present
> at boot time. As a result, link->aspm_enabled is 0, link->aspm_default is
> set to 0, and Linux never enables ASPM for these devices.
>
> Devicetree platforms already have special handling to enable L0s/L1 by
> default regardless of firmware configuration. Extend this same logic to
> removable devices when firmware hasn't configured any ASPM states.
>
> This makes Linux behavior more consistent with Windows for hotplugged
> Thunderbolt/USB4 devices.
>
> Link: https://bugzilla.kernel.org/show_bug.cgi?id=221319
> Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>
> Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
> ---
>  drivers/pci/pcie/aspm.c | 11 +++++++++--
>  1 file changed, 9 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c
> index 925373b98dff0..77497d90be0b7 100644
> --- a/drivers/pci/pcie/aspm.c
> +++ b/drivers/pci/pcie/aspm.c
> @@ -804,8 +804,15 @@ static void pcie_aspm_override_default_link_state(struct pcie_link_state *link)
>         struct pci_dev *pdev = link->downstream;
>         u32 override;
>
> -       /* For devicetree platforms, enable L0s and L1 by default */
> -       if (of_have_populated_dt()) {
> +       /*
> +        * For devicetree platforms, enable L0s and L1 by default.
> +        *
> +        * For removable devices (e.g., Thunderbolt/USB4), enable L0s and L1
> +        * by default if BIOS didn't configure any ASPM states. This handles
> +        * hotplugged devices where firmware may not have configured ASPM.
> +        */
> +       if (of_have_populated_dt() ||
> +           (dev_is_removable(&pdev->dev) && !link->aspm_enabled)) {
>                 if (link->aspm_support & PCIE_LINK_STATE_L0S)
>                         link->aspm_default |= PCIE_LINK_STATE_L0S;
>                 if (link->aspm_support & PCIE_LINK_STATE_L1)
> --
> 2.43.0
>

Re: [PATCH] PCI/ASPM: Enable L0s/L1 for removable devices when BIOS didn't configure ASPM

[Bjorn Helgaas]

[+cc Mika]

On Tue, May 05, 2026 at 11:08:14AM -0500, Mario Limonciello wrote:
> On 5/5/26 11:05, Bjorn Helgaas wrote:
> > On Mon, May 04, 2026 at 05:52:46PM -0500, Mario Limonciello wrote:
> > > When comparing lspci output between Windows and Linux for hotplugged
> > > Thunderbolt 5 eGPU devices, Windows enables ASPM L1 but Linux doesn't:
> > > 
> > >    Windows: LnkCtl: ASPM L1 Enabled
> > >    Linux:   LnkCtl: ASPM Disabled
> > > 
> > > This difference in ASPM configuration can cause behavioral differences
> > > between the two operating systems for the same hardware.
> > 
> > A tangent, not a comment on the patch itself, but what sort of
> > behavioral differences are these?  If ASPM is working correctly, the
> > only differences *should* be in power consumption and performance.
> 
> This originally stemmed from a significant performance difference that was
> observed between Windows and Linux with eGPUs.  The link in the patch points
> at that bug if you want to look more closely at it.

Hmm.  The bug (https://bugzilla.kernel.org/show_bug.cgi?id=221319)
reports "instant reboot", which is definitely a behavioral difference.
But AFAICS this patch would just fix something noticed along the way
but not the reboot itself.

To avoid confusion, I would use "performance difference" or "power
difference" when describing this patch.

> I was hopeful that aligning ASPM would align the behavior, but alas this
> didn't.
> 
> It was still a difference that I figured we should discuss whether it should
> be changed to be consistent.

Definitely.  I hope we can at least enable L1.1.  L1.2 is a whole
'nother issue.

Re: [PATCH] PCI/ASPM: Enable L0s/L1 for removable devices when BIOS didn't configure ASPM

[Mario Limonciello]

On 5/5/26 16:42, Bjorn Helgaas wrote:
> [+cc Mika]
> 
> On Tue, May 05, 2026 at 11:08:14AM -0500, Mario Limonciello wrote:
>> On 5/5/26 11:05, Bjorn Helgaas wrote:
>>> On Mon, May 04, 2026 at 05:52:46PM -0500, Mario Limonciello wrote:
>>>> When comparing lspci output between Windows and Linux for hotplugged
>>>> Thunderbolt 5 eGPU devices, Windows enables ASPM L1 but Linux doesn't:
>>>>
>>>>     Windows: LnkCtl: ASPM L1 Enabled
>>>>     Linux:   LnkCtl: ASPM Disabled
>>>>
>>>> This difference in ASPM configuration can cause behavioral differences
>>>> between the two operating systems for the same hardware.
>>>
>>> A tangent, not a comment on the patch itself, but what sort of
>>> behavioral differences are these?  If ASPM is working correctly, the
>>> only differences *should* be in power consumption and performance.
>>
>> This originally stemmed from a significant performance difference that was
>> observed between Windows and Linux with eGPUs.  The link in the patch points
>> at that bug if you want to look more closely at it.
> 
> Hmm.  The bug (https://bugzilla.kernel.org/show_bug.cgi?id=221319)
> reports "instant reboot", which is definitely a behavioral difference.
> But AFAICS this patch would just fix something noticed along the way
> but not the reboot itself.
> 
> To avoid confusion, I would use "performance difference" or "power
> difference" when describing this patch.

There is a lot of traffic in that bug and similar eGPU bugs; but some 
people have narrowed down that using NVIDIA's GSP "causes the instant 
reboot" but the performance difference is tangential to the reboot (or 
maybe it's part of the cause - I don't actually know).

The reboots /seem/ to be caused by sync floods which I originally 
hypothesized to be caused by Linux using AER and Windows not using it 
(potentially leading to a flood of errors in Linux), but turning off AER 
from kernel command line didn't change that.

> 
>> I was hopeful that aligning ASPM would align the behavior, but alas this
>> didn't.
>>
>> It was still a difference that I figured we should discuss whether it should
>> be changed to be consistent.
> 
> Definitely.  I hope we can at least enable L1.1.  L1.2 is a whole
> 'nother issue.

Yup.

Re: [PATCH] PCI/ASPM: Enable L0s/L1 for removable devices when BIOS didn't configure ASPM

[Mika Westerberg]

Hi,

On Tue, May 05, 2026 at 08:09:22PM +0200, Rafael J. Wysocki wrote:
> +Mika Westerberg
> 
> On Tue, May 5, 2026 at 12:53 AM Mario Limonciello
> <mario.limonciello@amd.com> wrote:
> >
> > When comparing lspci output between Windows and Linux for hotplugged
> > Thunderbolt 5 eGPU devices, Windows enables ASPM L1 but Linux doesn't:
> >
> >   Windows: LnkCtl: ASPM L1 Enabled
> >   Linux:   LnkCtl: ASPM Disabled
> >
> > This difference in ASPM configuration can cause behavioral differences
> > between the two operating systems for the same hardware.
> >
> > The root cause is that Linux's default ASPM policy (POLICY_DEFAULT) relies
> > on firmware/BIOS configuration. For hotplugged devices like Thunderbolt/USB4
> > eGPUs, the BIOS may not have configured ASPM since the device wasn't present
> > at boot time. As a result, link->aspm_enabled is 0, link->aspm_default is
> > set to 0, and Linux never enables ASPM for these devices.
> >
> > Devicetree platforms already have special handling to enable L0s/L1 by
> > default regardless of firmware configuration. Extend this same logic to
> > removable devices when firmware hasn't configured any ASPM states.
> >
> > This makes Linux behavior more consistent with Windows for hotplugged
> > Thunderbolt/USB4 devices.
> >
> > Link: https://bugzilla.kernel.org/show_bug.cgi?id=221319
> > Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>
> > Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
> > ---
> >  drivers/pci/pcie/aspm.c | 11 +++++++++--
> >  1 file changed, 9 insertions(+), 2 deletions(-)
> >
> > diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c
> > index 925373b98dff0..77497d90be0b7 100644
> > --- a/drivers/pci/pcie/aspm.c
> > +++ b/drivers/pci/pcie/aspm.c
> > @@ -804,8 +804,15 @@ static void pcie_aspm_override_default_link_state(struct pcie_link_state *link)
> >         struct pci_dev *pdev = link->downstream;
> >         u32 override;
> >
> > -       /* For devicetree platforms, enable L0s and L1 by default */
> > -       if (of_have_populated_dt()) {
> > +       /*
> > +        * For devicetree platforms, enable L0s and L1 by default.
> > +        *
> > +        * For removable devices (e.g., Thunderbolt/USB4), enable L0s and L1
> > +        * by default if BIOS didn't configure any ASPM states. This handles
> > +        * hotplugged devices where firmware may not have configured ASPM.
> > +        */

Only L1 is supported over TB/USB4 tunnel (no L0s, no L1 substates). The
PCIe endpoint and the downstream port it connects to of course can support
the full range as that's a real PCIe link.

> > +       if (of_have_populated_dt() ||
> > +           (dev_is_removable(&pdev->dev) && !link->aspm_enabled)) {
> >                 if (link->aspm_support & PCIE_LINK_STATE_L0S)
> >                         link->aspm_default |= PCIE_LINK_STATE_L0S;
> >                 if (link->aspm_support & PCIE_LINK_STATE_L1)
> > --
> > 2.43.0
> >

Re: [PATCH] PCI/ASPM: Enable L0s/L1 for removable devices when BIOS didn't configure ASPM

[Mario Limonciello]

On 5/5/26 23:53, Mika Westerberg wrote:
> Hi,
> 
> On Tue, May 05, 2026 at 08:09:22PM +0200, Rafael J. Wysocki wrote:
>> +Mika Westerberg
>>
>> On Tue, May 5, 2026 at 12:53 AM Mario Limonciello
>> <mario.limonciello@amd.com> wrote:
>>>
>>> When comparing lspci output between Windows and Linux for hotplugged
>>> Thunderbolt 5 eGPU devices, Windows enables ASPM L1 but Linux doesn't:
>>>
>>>    Windows: LnkCtl: ASPM L1 Enabled
>>>    Linux:   LnkCtl: ASPM Disabled
>>>
>>> This difference in ASPM configuration can cause behavioral differences
>>> between the two operating systems for the same hardware.
>>>
>>> The root cause is that Linux's default ASPM policy (POLICY_DEFAULT) relies
>>> on firmware/BIOS configuration. For hotplugged devices like Thunderbolt/USB4
>>> eGPUs, the BIOS may not have configured ASPM since the device wasn't present
>>> at boot time. As a result, link->aspm_enabled is 0, link->aspm_default is
>>> set to 0, and Linux never enables ASPM for these devices.
>>>
>>> Devicetree platforms already have special handling to enable L0s/L1 by
>>> default regardless of firmware configuration. Extend this same logic to
>>> removable devices when firmware hasn't configured any ASPM states.
>>>
>>> This makes Linux behavior more consistent with Windows for hotplugged
>>> Thunderbolt/USB4 devices.
>>>
>>> Link: https://bugzilla.kernel.org/show_bug.cgi?id=221319
>>> Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>
>>> Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
>>> ---
>>>   drivers/pci/pcie/aspm.c | 11 +++++++++--
>>>   1 file changed, 9 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/drivers/pci/pcie/aspm.c b/drivers/pci/pcie/aspm.c
>>> index 925373b98dff0..77497d90be0b7 100644
>>> --- a/drivers/pci/pcie/aspm.c
>>> +++ b/drivers/pci/pcie/aspm.c
>>> @@ -804,8 +804,15 @@ static void pcie_aspm_override_default_link_state(struct pcie_link_state *link)
>>>          struct pci_dev *pdev = link->downstream;
>>>          u32 override;
>>>
>>> -       /* For devicetree platforms, enable L0s and L1 by default */
>>> -       if (of_have_populated_dt()) {
>>> +       /*
>>> +        * For devicetree platforms, enable L0s and L1 by default.
>>> +        *
>>> +        * For removable devices (e.g., Thunderbolt/USB4), enable L0s and L1
>>> +        * by default if BIOS didn't configure any ASPM states. This handles
>>> +        * hotplugged devices where firmware may not have configured ASPM.
>>> +        */
> 
> Only L1 is supported over TB/USB4 tunnel (no L0s, no L1 substates). The
> PCIe endpoint and the downstream port it connects to of course can support
> the full range as that's a real PCIe link.
> 

OK - the comment should be updated but I do expect that below code 
(link->aspm_support) should remain OK.

>>> +       if (of_have_populated_dt() ||
>>> +           (dev_is_removable(&pdev->dev) && !link->aspm_enabled)) {
>>>                  if (link->aspm_support & PCIE_LINK_STATE_L0S)
>>>                          link->aspm_default |= PCIE_LINK_STATE_L0S;
>>>                  if (link->aspm_support & PCIE_LINK_STATE_L1)
>>> --
>>> 2.43.0
>>>

Re: [PATCH] PCI/ASPM: Enable L0s/L1 for removable devices when BIOS didn't configure ASPM

[Bjorn Helgaas]

On Wed, May 06, 2026 at 10:10:47AM -0500, Mario Limonciello wrote:
> On 5/5/26 23:53, Mika Westerberg wrote:
> > On Tue, May 05, 2026 at 08:09:22PM +0200, Rafael J. Wysocki wrote:
> > > On Tue, May 5, 2026 at 12:53 AM Mario Limonciello
> > > <mario.limonciello@amd.com> wrote:
> ...

> > > > +        * For devicetree platforms, enable L0s and L1 by default.
> > > > +        *
> > > > +        * For removable devices (e.g., Thunderbolt/USB4), enable L0s and L1
> > > > +        * by default if BIOS didn't configure any ASPM states. This handles
> > > > +        * hotplugged devices where firmware may not have configured ASPM.
> > > > +        */
> > 
> > Only L1 is supported over TB/USB4 tunnel (no L0s, no L1 substates). The
> > PCIe endpoint and the downstream port it connects to of course can support
> > the full range as that's a real PCIe link.
> 
> OK - the comment should be updated but I do expect that below code
> (link->aspm_support) should remain OK.

TB/USB4 are examples of removable devices but they're not the only
ones, so I think it's OK for the comment to mention L0s.  In fact, it
*should* mention L0s since the code below includes L0s, and mentioning
only L1 would just be confusing.

> > > > +       if (of_have_populated_dt() ||
> > > > +           (dev_is_removable(&pdev->dev) && !link->aspm_enabled)) {
> > > >                  if (link->aspm_support & PCIE_LINK_STATE_L0S)
> > > >                          link->aspm_default |= PCIE_LINK_STATE_L0S;
> > > >                  if (link->aspm_support & PCIE_LINK_STATE_L1)
> > > > --
> > > > 2.43.0
> > > > 
>