[PATCH] p2sb: Don't init until unassigned resources have been assigned.

[PATCH] p2sb: Don't init until unassigned resources have been assigned.

[bcfradella]

From: Ben Fradella <bfradell@netapp.com>

The P2SB could get an invalid BAR from the BIOS, and that won't be fixed
up until pcibios_assign_resources(), which is an fs_initcall().

- Move p2sb_fs_init() to an fs_initcall_sync(). This is still early
  enough to avoid a race with any dependent drivers.

- Add a check for IORESOURCE_UNSET in p2sb_valid_resource() to catch
  unset BARs going forward.

- Return error values from p2sb_fs_init() so that the 'initcall_debug'
  cmdline arg provides useful data.

Signed-off-by: Ben Fradella <bfradell@netapp.com>
---
 drivers/platform/x86/p2sb.c | 29 +++++++++++++++--------------
 1 file changed, 15 insertions(+), 14 deletions(-)

diff --git a/drivers/platform/x86/p2sb.c b/drivers/platform/x86/p2sb.c
index 3d66e1d4eb1f..1938a3ef9480 100644
--- a/drivers/platform/x86/p2sb.c
+++ b/drivers/platform/x86/p2sb.c
@@ -56,12 +56,9 @@ static int p2sb_get_devfn(unsigned int *devfn)
 	return 0;
 }
 
-static bool p2sb_valid_resource(struct resource *res)
+static bool p2sb_valid_resource(const struct resource *res)
 {
-	if (res->flags)
-		return true;
-
-	return false;
+	return res->flags & ~IORESOURCE_UNSET;
 }
 
 /* Copy resource from the first BAR of the device in question */
@@ -220,16 +217,20 @@ EXPORT_SYMBOL_GPL(p2sb_bar);
 
 static int __init p2sb_fs_init(void)
 {
-	p2sb_cache_resources();
-	return 0;
+	return p2sb_cache_resources();
 }
 
 /*
- * pci_rescan_remove_lock to avoid access to unhidden P2SB devices can
- * not be locked in sysfs pci bus rescan path because of deadlock. To
- * avoid the deadlock, access to P2SB devices with the lock at an early
- * step in kernel initialization and cache required resources. This
- * should happen after subsys_initcall which initializes PCI subsystem
- * and before device_initcall which requires P2SB resources.
+ * pci_rescan_remove_lock() can not be locked in sysfs pci bus rescan path
+ * because of deadlock. To avoid the deadlock, access P2SB devices with the lock
+ * at an early step in kernel initialization and cache required resources.
+ *
+ * We want to run as early as possible. If the P2SB was assigned a bad BAR,
+ * we'll need to wait on pcibios_assign_resources() to fix it. So, our list of
+ * initcall dependencies looks something like this:
+ *
+ * ...
+ * subsys_initcall (pci_subsys_init)
+ * fs_initcall     (pcibios_assign_resources)
  */
-fs_initcall(p2sb_fs_init);
+fs_initcall_sync(p2sb_fs_init);
-- 
2.43.0

Re: [PATCH] p2sb: Don't init until unassigned resources have been assigned.

[Lukas Wunner]

[cc += Shin'ichiro, Klara, Andy, Danil]

On Thu, May 09, 2024 at 04:49:34PM +0000, bcfradella@proton.me wrote:
> From: Ben Fradella <bfradell@netapp.com>
> 
> The P2SB could get an invalid BAR from the BIOS, and that won't be fixed
> up until pcibios_assign_resources(), which is an fs_initcall().
> 
> - Move p2sb_fs_init() to an fs_initcall_sync(). This is still early
>   enough to avoid a race with any dependent drivers.
> 
> - Add a check for IORESOURCE_UNSET in p2sb_valid_resource() to catch
>   unset BARs going forward.
> 
> - Return error values from p2sb_fs_init() so that the 'initcall_debug'
>   cmdline arg provides useful data.
> 
> Signed-off-by: Ben Fradella <bfradell@netapp.com>
> ---
>  drivers/platform/x86/p2sb.c | 29 +++++++++++++++--------------
>  1 file changed, 15 insertions(+), 14 deletions(-)
> 
> diff --git a/drivers/platform/x86/p2sb.c b/drivers/platform/x86/p2sb.c
> index 3d66e1d4eb1f..1938a3ef9480 100644
> --- a/drivers/platform/x86/p2sb.c
> +++ b/drivers/platform/x86/p2sb.c
> @@ -56,12 +56,9 @@ static int p2sb_get_devfn(unsigned int *devfn)
>  	return 0;
>  }
>  
> -static bool p2sb_valid_resource(struct resource *res)
> +static bool p2sb_valid_resource(const struct resource *res)
>  {
> -	if (res->flags)
> -		return true;
> -
> -	return false;
> +	return res->flags & ~IORESOURCE_UNSET;
>  }
>  
>  /* Copy resource from the first BAR of the device in question */
> @@ -220,16 +217,20 @@ EXPORT_SYMBOL_GPL(p2sb_bar);
>  
>  static int __init p2sb_fs_init(void)
>  {
> -	p2sb_cache_resources();
> -	return 0;
> +	return p2sb_cache_resources();
>  }
>  
>  /*
> - * pci_rescan_remove_lock to avoid access to unhidden P2SB devices can
> - * not be locked in sysfs pci bus rescan path because of deadlock. To
> - * avoid the deadlock, access to P2SB devices with the lock at an early
> - * step in kernel initialization and cache required resources. This
> - * should happen after subsys_initcall which initializes PCI subsystem
> - * and before device_initcall which requires P2SB resources.
> + * pci_rescan_remove_lock() can not be locked in sysfs pci bus rescan path
> + * because of deadlock. To avoid the deadlock, access P2SB devices with the lock
> + * at an early step in kernel initialization and cache required resources.
> + *
> + * We want to run as early as possible. If the P2SB was assigned a bad BAR,
> + * we'll need to wait on pcibios_assign_resources() to fix it. So, our list of
> + * initcall dependencies looks something like this:
> + *
> + * ...
> + * subsys_initcall (pci_subsys_init)
> + * fs_initcall     (pcibios_assign_resources)
>   */
> -fs_initcall(p2sb_fs_init);
> +fs_initcall_sync(p2sb_fs_init);
> -- 
> 2.43.0

Re: [PATCH] p2sb: Don't init until unassigned resources have been assigned.

[Andy Shevchenko]

On Thu, May 09, 2024 at 07:04:32PM +0200, Lukas Wunner wrote:
> [cc += Shin'ichiro, Klara, Andy, Danil]

Thank you!

> On Thu, May 09, 2024 at 04:49:34PM +0000, bcfradella@proton.me wrote:
> > From: Ben Fradella <bfradell@netapp.com>
> > 
> > The P2SB could get an invalid BAR from the BIOS, and that won't be fixed
> > up until pcibios_assign_resources(), which is an fs_initcall().
> > 
> > - Move p2sb_fs_init() to an fs_initcall_sync(). This is still early
> >   enough to avoid a race with any dependent drivers.
> > 
> > - Add a check for IORESOURCE_UNSET in p2sb_valid_resource() to catch
> >   unset BARs going forward.
> > 
> > - Return error values from p2sb_fs_init() so that the 'initcall_debug'
> >   cmdline arg provides useful data.

...

> >  /*
> > - * pci_rescan_remove_lock to avoid access to unhidden P2SB devices can
> > - * not be locked in sysfs pci bus rescan path because of deadlock. To
> > - * avoid the deadlock, access to P2SB devices with the lock at an early
> > - * step in kernel initialization and cache required resources. This
> > - * should happen after subsys_initcall which initializes PCI subsystem
> > - * and before device_initcall which requires P2SB resources.
> > + * pci_rescan_remove_lock() can not be locked in sysfs pci bus rescan path

PCI bus

> > + * because of deadlock. To avoid the deadlock, access P2SB devices with the lock
> > + * at an early step in kernel initialization and cache required resources.
> > + *
> > + * We want to run as early as possible. If the P2SB was assigned a bad BAR,
> > + * we'll need to wait on pcibios_assign_resources() to fix it. So, our list of
> > + * initcall dependencies looks something like this:
> > + *
> > + * ...
> > + * subsys_initcall (pci_subsys_init)
> > + * fs_initcall     (pcibios_assign_resources)
> >   */

This makes sense to me
Acked-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>

I assume others will conduct the proper review and testing.

-- 
With Best Regards,
Andy Shevchenko

Re: [PATCH] p2sb: Don't init until unassigned resources have been assigned.

[Klara Modin]

On 2024-05-10 16:50, Andy Shevchenko wrote:
> On Thu, May 09, 2024 at 07:04:32PM +0200, Lukas Wunner wrote:
>> [cc += Shin'ichiro, Klara, Andy, Danil]
> 
> Thank you!
> 
>> On Thu, May 09, 2024 at 04:49:34PM +0000, bcfradella@proton.me wrote:
>>> From: Ben Fradella <bfradell@netapp.com>
>>>
>>> The P2SB could get an invalid BAR from the BIOS, and that won't be fixed
>>> up until pcibios_assign_resources(), which is an fs_initcall().
>>>
>>> - Move p2sb_fs_init() to an fs_initcall_sync(). This is still early
>>>    enough to avoid a race with any dependent drivers.
>>>
>>> - Add a check for IORESOURCE_UNSET in p2sb_valid_resource() to catch
>>>    unset BARs going forward.
>>>
>>> - Return error values from p2sb_fs_init() so that the 'initcall_debug'
>>>    cmdline arg provides useful data.
> 
> ...
> 
>>>   /*
>>> - * pci_rescan_remove_lock to avoid access to unhidden P2SB devices can
>>> - * not be locked in sysfs pci bus rescan path because of deadlock. To
>>> - * avoid the deadlock, access to P2SB devices with the lock at an early
>>> - * step in kernel initialization and cache required resources. This
>>> - * should happen after subsys_initcall which initializes PCI subsystem
>>> - * and before device_initcall which requires P2SB resources.
>>> + * pci_rescan_remove_lock() can not be locked in sysfs pci bus rescan path
> 
> PCI bus
> 
>>> + * because of deadlock. To avoid the deadlock, access P2SB devices with the lock
>>> + * at an early step in kernel initialization and cache required resources.
>>> + *
>>> + * We want to run as early as possible. If the P2SB was assigned a bad BAR,
>>> + * we'll need to wait on pcibios_assign_resources() to fix it. So, our list of
>>> + * initcall dependencies looks something like this:
>>> + *
>>> + * ...
>>> + * subsys_initcall (pci_subsys_init)
>>> + * fs_initcall     (pcibios_assign_resources)
>>>    */
> 
> This makes sense to me
> Acked-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
> 
> I assume others will conduct the proper review and testing.
> 

This patch did not introduce any new issues on my previously problematic 
system.

Tested-by: Klara Modin <klarasmodin@gmail.com>

Re: [PATCH] p2sb: Don't init until unassigned resources have been assigned.

[Shinichiro Kawasaki]

On May 10, 2024 / 21:22, Klara Modin wrote:
> On 2024-05-10 16:50, Andy Shevchenko wrote:
> > On Thu, May 09, 2024 at 07:04:32PM +0200, Lukas Wunner wrote:
> > > [cc += Shin'ichiro, Klara, Andy, Danil]
> > 
> > Thank you!
> > 
> > > On Thu, May 09, 2024 at 04:49:34PM +0000, bcfradella@proton.me wrote:
> > > > From: Ben Fradella <bfradell@netapp.com>
> > > > 
> > > > The P2SB could get an invalid BAR from the BIOS, and that won't be fixed
> > > > up until pcibios_assign_resources(), which is an fs_initcall().
> > > > 
> > > > - Move p2sb_fs_init() to an fs_initcall_sync(). This is still early
> > > >    enough to avoid a race with any dependent drivers.
> > > > 
> > > > - Add a check for IORESOURCE_UNSET in p2sb_valid_resource() to catch
> > > >    unset BARs going forward.
> > > > 
> > > > - Return error values from p2sb_fs_init() so that the 'initcall_debug'
> > > >    cmdline arg provides useful data.
> > 
> > ...
> > 
> > > >   /*
> > > > - * pci_rescan_remove_lock to avoid access to unhidden P2SB devices can
> > > > - * not be locked in sysfs pci bus rescan path because of deadlock. To
> > > > - * avoid the deadlock, access to P2SB devices with the lock at an early
> > > > - * step in kernel initialization and cache required resources. This
> > > > - * should happen after subsys_initcall which initializes PCI subsystem
> > > > - * and before device_initcall which requires P2SB resources.
> > > > + * pci_rescan_remove_lock() can not be locked in sysfs pci bus rescan path
> > 
> > PCI bus
> > 
> > > > + * because of deadlock. To avoid the deadlock, access P2SB devices with the lock
> > > > + * at an early step in kernel initialization and cache required resources.
> > > > + *
> > > > + * We want to run as early as possible. If the P2SB was assigned a bad BAR,
> > > > + * we'll need to wait on pcibios_assign_resources() to fix it. So, our list of
> > > > + * initcall dependencies looks something like this:
> > > > + *
> > > > + * ...
> > > > + * subsys_initcall (pci_subsys_init)
> > > > + * fs_initcall     (pcibios_assign_resources)
> > > >    */
> > 
> > This makes sense to me
> > Acked-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
> > 
> > I assume others will conduct the proper review and testing.
> > 
> 
> This patch did not introduce any new issues on my previously problematic
> system.
> 
> Tested-by: Klara Modin <klarasmodin@gmail.com>

The changes look reasonable and good to me. I also confirmed that the patch
does not affect on my use case using two my test machines.

Reviewed-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>

Re: [PATCH] p2sb: Don't init until unassigned resources have been assigned.

[Hans de Goede]

Hi,

On 5/9/24 6:49 PM, bcfradella@proton.me wrote:
> From: Ben Fradella <bfradell@netapp.com>
> 
> The P2SB could get an invalid BAR from the BIOS, and that won't be fixed
> up until pcibios_assign_resources(), which is an fs_initcall().
> 
> - Move p2sb_fs_init() to an fs_initcall_sync(). This is still early
>   enough to avoid a race with any dependent drivers.
> 
> - Add a check for IORESOURCE_UNSET in p2sb_valid_resource() to catch
>   unset BARs going forward.
> 
> - Return error values from p2sb_fs_init() so that the 'initcall_debug'
>   cmdline arg provides useful data.
> 
> Signed-off-by: Ben Fradella <bfradell@netapp.com>

Thank you for your patch, I've applied this patch to my review-hans 
branch:
https://git.kernel.org/pub/scm/linux/kernel/git/pdx86/platform-drivers-x86.git/log/?h=review-hans

Note it will show up in my review-hans branch once I've pushed my
local branch there, which might take a while.

Once I've run some tests on this branch the patches there will be
added to the platform-drivers-x86/for-next branch and eventually
will be included in the pdx86 pull-request to Linus for the next
merge-window.

Regards,

Hans



> ---
>  drivers/platform/x86/p2sb.c | 29 +++++++++++++++--------------
>  1 file changed, 15 insertions(+), 14 deletions(-)
> 
> diff --git a/drivers/platform/x86/p2sb.c b/drivers/platform/x86/p2sb.c
> index 3d66e1d4eb1f..1938a3ef9480 100644
> --- a/drivers/platform/x86/p2sb.c
> +++ b/drivers/platform/x86/p2sb.c
> @@ -56,12 +56,9 @@ static int p2sb_get_devfn(unsigned int *devfn)
>  	return 0;
>  }
>  
> -static bool p2sb_valid_resource(struct resource *res)
> +static bool p2sb_valid_resource(const struct resource *res)
>  {
> -	if (res->flags)
> -		return true;
> -
> -	return false;
> +	return res->flags & ~IORESOURCE_UNSET;
>  }
>  
>  /* Copy resource from the first BAR of the device in question */
> @@ -220,16 +217,20 @@ EXPORT_SYMBOL_GPL(p2sb_bar);
>  
>  static int __init p2sb_fs_init(void)
>  {
> -	p2sb_cache_resources();
> -	return 0;
> +	return p2sb_cache_resources();
>  }
>  
>  /*
> - * pci_rescan_remove_lock to avoid access to unhidden P2SB devices can
> - * not be locked in sysfs pci bus rescan path because of deadlock. To
> - * avoid the deadlock, access to P2SB devices with the lock at an early
> - * step in kernel initialization and cache required resources. This
> - * should happen after subsys_initcall which initializes PCI subsystem
> - * and before device_initcall which requires P2SB resources.
> + * pci_rescan_remove_lock() can not be locked in sysfs pci bus rescan path
> + * because of deadlock. To avoid the deadlock, access P2SB devices with the lock
> + * at an early step in kernel initialization and cache required resources.
> + *
> + * We want to run as early as possible. If the P2SB was assigned a bad BAR,
> + * we'll need to wait on pcibios_assign_resources() to fix it. So, our list of
> + * initcall dependencies looks something like this:
> + *
> + * ...
> + * subsys_initcall (pci_subsys_init)
> + * fs_initcall     (pcibios_assign_resources)
>   */
> -fs_initcall(p2sb_fs_init);
> +fs_initcall_sync(p2sb_fs_init);

Re: [PATCH] p2sb: Don't init until unassigned resources have been assigned.

[Fradella, Ben]

> On 2024-05-13 07:03, hdegoede@redhat.com wrote:
>
> Thank you for your patch, I've applied this patch to my review-hans branch: https://git.kernel.org/pub/scm/linux/kernel/git/pdx86/platform-drivers-x86.git/log/?h=review-hans
>
> Note it will show up in my review-hans branch once I've pushed my local branch there, which might take a while.
>
> Once I've run some tests on this branch the patches there will be added to the platform-drivers-x86/for-next branch and eventually will be included in the pdx86 pull-request to Linus for the next merge-window.
>
>Regards,
>
>Hans

Thank you!