[PATCH] PCI: vmd: guard device addition and removal

[PATCH] PCI: vmd: guard device addition and removal

[You-Sheng Yang (vicamo)]

VMD may fail to create sysfs entries while `pci_rescan_bus()` called in
some other drivers like t7xx wwan driver:

  sysfs: cannot create duplicate filename '/devices/.../resource0'
  Call Trace:
   <TASK>
   sysfs_warn_dup.cold+0x17/0x34
   sysfs_add_bin_file_mode_ns+0xc0/0xf0
   sysfs_create_bin_file+0x6d/0xb0
   pci_create_attr+0x117/0x260
   pci_create_resource_files+0x6b/0x150
   pci_create_sysfs_dev_files+0x18/0x30
   pci_bus_add_device+0x30/0x80
   pci_bus_add_devices+0x31/0x80
   pci_bus_add_devices+0x5b/0x80
   vmd_enable_domain.constprop.0+0x6b7/0x880 [vmd]
   vmd_probe+0x16d/0x193 [vmd]

Fixes: 185a383ada2e ("x86/PCI: Add driver for Intel Volume Management Device (VMD)")
Signed-off-by: You-Sheng Yang (vicamo) <vicamo.yang@canonical.com>
---
 drivers/pci/controller/vmd.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c
index 769eedeb8802..f050991bd1e9 100644
--- a/drivers/pci/controller/vmd.c
+++ b/drivers/pci/controller/vmd.c
@@ -838,9 +838,13 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)
 	pci_add_resource_offset(&resources, &vmd->resources[1], offset[0]);
 	pci_add_resource_offset(&resources, &vmd->resources[2], offset[1]);
 
+	pci_lock_rescan_remove();
+
 	vmd->bus = pci_create_root_bus(&vmd->dev->dev, vmd->busn_start,
 				       &vmd_ops, sd, &resources);
 	if (!vmd->bus) {
+		pci_unlock_rescan_remove();
+
 		pci_free_resource_list(&resources);
 		vmd_remove_irq_domain(vmd);
 		return -ENODEV;
@@ -893,6 +897,8 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)
 
 	vmd_acpi_end();
 
+	pci_unlock_rescan_remove();
+
 	WARN(sysfs_create_link(&vmd->dev->dev.kobj, &vmd->bus->dev.kobj,
 			       "domain"), "Can't create symlink to domain\n");
 	return 0;
-- 
2.39.2

Re: [PATCH] PCI: vmd: guard device addition and removal

[Patel, Nirmal]

On 3/13/2023 10:37 AM, You-Sheng Yang (vicamo) wrote:
> VMD may fail to create sysfs entries while `pci_rescan_bus()` called in
> some other drivers like t7xx wwan driver:
>
>   sysfs: cannot create duplicate filename '/devices/.../resource0'
>   Call Trace:
>    <TASK>
>    sysfs_warn_dup.cold+0x17/0x34
>    sysfs_add_bin_file_mode_ns+0xc0/0xf0
>    sysfs_create_bin_file+0x6d/0xb0
>    pci_create_attr+0x117/0x260
>    pci_create_resource_files+0x6b/0x150
>    pci_create_sysfs_dev_files+0x18/0x30
>    pci_bus_add_device+0x30/0x80
>    pci_bus_add_devices+0x31/0x80
>    pci_bus_add_devices+0x5b/0x80
>    vmd_enable_domain.constprop.0+0x6b7/0x880 [vmd]
>    vmd_probe+0x16d/0x193 [vmd]
>
> Fixes: 185a383ada2e ("x86/PCI: Add driver for Intel Volume Management Device (VMD)")
> Signed-off-by: You-Sheng Yang (vicamo) <vicamo.yang@canonical.com>
> ---
>  drivers/pci/controller/vmd.c | 6 ++++++
>  1 file changed, 6 insertions(+)
>
> diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c
> index 769eedeb8802..f050991bd1e9 100644
> --- a/drivers/pci/controller/vmd.c
> +++ b/drivers/pci/controller/vmd.c
> @@ -838,9 +838,13 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)
>  	pci_add_resource_offset(&resources, &vmd->resources[1], offset[0]);
>  	pci_add_resource_offset(&resources, &vmd->resources[2], offset[1]);
>  
> +	pci_lock_rescan_remove();
> +
>  	vmd->bus = pci_create_root_bus(&vmd->dev->dev, vmd->busn_start,
>  				       &vmd_ops, sd, &resources);
>  	if (!vmd->bus) {
> +		pci_unlock_rescan_remove();
> +
>  		pci_free_resource_list(&resources);
>  		vmd_remove_irq_domain(vmd);
>  		return -ENODEV;
> @@ -893,6 +897,8 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)
>  
>  	vmd_acpi_end();
>  
> +	pci_unlock_rescan_remove();
> +
>  	WARN(sysfs_create_link(&vmd->dev->dev.kobj, &vmd->bus->dev.kobj,
>  			       "domain"), "Can't create symlink to domain\n");
>  	return 0;

Reviewed-by: Nirmal Patel <nirmal.patel@linux.intel.com>

Thanks

Re: [PATCH] PCI: vmd: guard device addition and removal

[Bjorn Helgaas]

[+cc Korneliusz, Alexander]

On Tue, Mar 14, 2023 at 01:37:33AM +0800, You-Sheng Yang (vicamo) wrote:
> VMD may fail to create sysfs entries while `pci_rescan_bus()` called in
> some other drivers like t7xx wwan driver:
> 
>   sysfs: cannot create duplicate filename '/devices/.../resource0'
>   Call Trace:
>    <TASK>
>    sysfs_warn_dup.cold+0x17/0x34
>    sysfs_add_bin_file_mode_ns+0xc0/0xf0
>    sysfs_create_bin_file+0x6d/0xb0
>    pci_create_attr+0x117/0x260
>    pci_create_resource_files+0x6b/0x150
>    pci_create_sysfs_dev_files+0x18/0x30
>    pci_bus_add_device+0x30/0x80
>    pci_bus_add_devices+0x31/0x80
>    pci_bus_add_devices+0x5b/0x80
>    vmd_enable_domain.constprop.0+0x6b7/0x880 [vmd]
>    vmd_probe+0x16d/0x193 [vmd]

This is a long-standing issue, and I would *love* to nail it, but this
doesn't feel like the right solution to me.  What's unique about vmd
here?

I guess maybe it's similar to the situation Korneliusz and Alexander
ran into at [1]?

And why is t7xx called out specifically here?  This is a pretty
generic sysfs attribute issue and it doesn't *seem* like t7xx should
be special in that respect.  Oooh, maybe it's the fact that there's a
t7xx patch [2] coming that adds pci_rescan_bus() there?

Krzysztof has converted a lot of the sysfs files to static attributes,
e.g.,

  506140f9c06b ("PCI/sysfs: Convert "index", "acpi_index", "label" to static attributes")
  d93f8399053d ("PCI/sysfs: Convert "vpd" to static attribute")
  f42c35ea3b13 ("PCI/sysfs: Convert "reset" to static attribute")
  527139d738d7 ("PCI/sysfs: Convert "rom" to static attribute")
  e1d3f3268b0e ("PCI/sysfs: Convert "config" to static attribute")

I think that's the *best* approach since the sysfs infrastructure
already prevents races here.  But these last few files are more
difficult to convert, so we've been kind of stalled on them.

Bjorn

[1] https://lore.kernel.org/r/20230316091540.494366-1-alexander.stein@ew.tq-group.com
[2] https://lore.kernel.org/linux-pci/20230124204543.550d88e3@kernel.org/

> Fixes: 185a383ada2e ("x86/PCI: Add driver for Intel Volume Management Device (VMD)")
> Signed-off-by: You-Sheng Yang (vicamo) <vicamo.yang@canonical.com>
> ---
>  drivers/pci/controller/vmd.c | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c
> index 769eedeb8802..f050991bd1e9 100644
> --- a/drivers/pci/controller/vmd.c
> +++ b/drivers/pci/controller/vmd.c
> @@ -838,9 +838,13 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)
>  	pci_add_resource_offset(&resources, &vmd->resources[1], offset[0]);
>  	pci_add_resource_offset(&resources, &vmd->resources[2], offset[1]);
>  
> +	pci_lock_rescan_remove();
> +
>  	vmd->bus = pci_create_root_bus(&vmd->dev->dev, vmd->busn_start,
>  				       &vmd_ops, sd, &resources);
>  	if (!vmd->bus) {
> +		pci_unlock_rescan_remove();
> +
>  		pci_free_resource_list(&resources);
>  		vmd_remove_irq_domain(vmd);
>  		return -ENODEV;
> @@ -893,6 +897,8 @@ static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)
>  
>  	vmd_acpi_end();
>  
> +	pci_unlock_rescan_remove();
> +
>  	WARN(sysfs_create_link(&vmd->dev->dev.kobj, &vmd->bus->dev.kobj,
>  			       "domain"), "Can't create symlink to domain\n");
>  	return 0;
> -- 
> 2.39.2
> 

Re: [PATCH] PCI: vmd: guard device addition and removal

[Alexander Stein]

Am Samstag, 8. April 2023, 01:58:29 CEST schrieb Bjorn Helgaas:
> [+cc Korneliusz, Alexander]
> 
> On Tue, Mar 14, 2023 at 01:37:33AM +0800, You-Sheng Yang (vicamo) wrote:
> > VMD may fail to create sysfs entries while `pci_rescan_bus()` called in
> > 
> > some other drivers like t7xx wwan driver:
> >   sysfs: cannot create duplicate filename '/devices/.../resource0'
> >   
> >   Call Trace:
> >    <TASK>
> >    sysfs_warn_dup.cold+0x17/0x34
> >    sysfs_add_bin_file_mode_ns+0xc0/0xf0
> >    sysfs_create_bin_file+0x6d/0xb0
> >    pci_create_attr+0x117/0x260
> >    pci_create_resource_files+0x6b/0x150
> >    pci_create_sysfs_dev_files+0x18/0x30
> >    pci_bus_add_device+0x30/0x80
> >    pci_bus_add_devices+0x31/0x80
> >    pci_bus_add_devices+0x5b/0x80
> >    vmd_enable_domain.constprop.0+0x6b7/0x880 [vmd]
> >    vmd_probe+0x16d/0x193 [vmd]
> 
> This is a long-standing issue, and I would *love* to nail it, but this
> doesn't feel like the right solution to me.  What's unique about vmd
> here?
> 
> I guess maybe it's similar to the situation Korneliusz and Alexander
> ran into at [1]?

What's the other path creating those sysfs files? Also vmd seems to be built 
as a module in this case. So that's a different issue. pci-imx6 is racing 
against an initcall when being built into the kernel. The probe_type 
PROBE_PREFER_ASYNCHRONOUS is also important.
Being built as a module this race is essentially impossible.

Best regards,
Alexander

> And why is t7xx called out specifically here?  This is a pretty
> generic sysfs attribute issue and it doesn't *seem* like t7xx should
> be special in that respect.  Oooh, maybe it's the fact that there's a
> t7xx patch [2] coming that adds pci_rescan_bus() there?
> 
> Krzysztof has converted a lot of the sysfs files to static attributes,
> e.g.,
> 
>   506140f9c06b ("PCI/sysfs: Convert "index", "acpi_index", "label" to static
> attributes") d93f8399053d ("PCI/sysfs: Convert "vpd" to static attribute")
>   f42c35ea3b13 ("PCI/sysfs: Convert "reset" to static attribute")
>   527139d738d7 ("PCI/sysfs: Convert "rom" to static attribute")
>   e1d3f3268b0e ("PCI/sysfs: Convert "config" to static attribute")
> 
> I think that's the *best* approach since the sysfs infrastructure
> already prevents races here.  But these last few files are more
> difficult to convert, so we've been kind of stalled on them.
> 
> Bjorn
> 
> [1]
> https://lore.kernel.org/r/20230316091540.494366-1-alexander.stein@ew.tq-gro
> up.com [2]
> https://lore.kernel.org/linux-pci/20230124204543.550d88e3@kernel.org/
> > Fixes: 185a383ada2e ("x86/PCI: Add driver for Intel Volume Management
> > Device (VMD)") Signed-off-by: You-Sheng Yang (vicamo)
> > <vicamo.yang@canonical.com> ---
> > 
> >  drivers/pci/controller/vmd.c | 6 ++++++
> >  1 file changed, 6 insertions(+)
> > 
> > diff --git a/drivers/pci/controller/vmd.c b/drivers/pci/controller/vmd.c
> > index 769eedeb8802..f050991bd1e9 100644
> > --- a/drivers/pci/controller/vmd.c
> > +++ b/drivers/pci/controller/vmd.c
> > @@ -838,9 +838,13 @@ static int vmd_enable_domain(struct vmd_dev *vmd,
> > unsigned long features)> 
> >  	pci_add_resource_offset(&resources, &vmd->resources[1], offset[0]);
> >  	pci_add_resource_offset(&resources, &vmd->resources[2], offset[1]);
> > 
> > +	pci_lock_rescan_remove();
> > +
> > 
> >  	vmd->bus = pci_create_root_bus(&vmd->dev->dev, vmd->busn_start,
> >  	
> >  				       &vmd_ops, sd, &resources);
> >  	
> >  	if (!vmd->bus) {
> > 
> > +		pci_unlock_rescan_remove();
> > +
> > 
> >  		pci_free_resource_list(&resources);
> >  		vmd_remove_irq_domain(vmd);
> >  		return -ENODEV;
> > 
> > @@ -893,6 +897,8 @@ static int vmd_enable_domain(struct vmd_dev *vmd,
> > unsigned long features)> 
> >  	vmd_acpi_end();
> > 
> > +	pci_unlock_rescan_remove();
> > +
> > 
> >  	WARN(sysfs_create_link(&vmd->dev->dev.kobj, &vmd->bus->dev.kobj,
> >  	
> >  			       "domain"), "Can't create symlink to 
domain\n");
> >  	
> >  	return 0;


-- 
TQ-Systems GmbH | Mühlstraße 2, Gut Delling | 82229 Seefeld, Germany
Amtsgericht München, HRB 105018
Geschäftsführer: Detlef Schneider, Rüdiger Stahl, Stefan Schneider
http://www.tq-group.com/