PCI Rescan Question

PCI Rescan Question

[Kyle Gale]

Hi,



Can someone help me understand why I need to remove my stale PCI
device files from the tree if I want to properly get my device
re-initialized after unplugging and reinserting?



For example, I turn on my system with my PCI device plugged in. All is
fine. I unplug it, plug it back in, and rescan (echo 1 >
/sys/bus/pci/devices/rescan) and end up in this state where my BAR0 is
not set, and other things are not entirely correct.



But if I do a remove (echo 1 > /sys/bus/pci/devices/xxxx/xx.x/remove)
of the port, it then properly re-initializes my device when I do a
rescan.



Why does it not suffice to simply do a rescan? Does the rescan skip
some step if it finds that the device is already in the PCI tree? It's
not as if the rescan completely skips the device if I don't do a
remove, it just doesn't do all of the initialization.



Thanks,

Kyle Gale

Re: PCI Rescan Question

[Lukas Wunner]

On Wed, Apr 26, 2017 at 03:56:09PM -0700, Kyle Gale wrote:
> Can someone help me understand why I need to remove my stale PCI
> device files from the tree if I want to properly get my device
> re-initialized after unplugging and reinserting?
> 
> For example, I turn on my system with my PCI device plugged in. All is
> fine. I unplug it, plug it back in, and rescan (echo 1 >
> /sys/bus/pci/devices/rescan) and end up in this state where my BAR0 is
> not set, and other things are not entirely correct.
> 
> But if I do a remove (echo 1 > /sys/bus/pci/devices/xxxx/xx.x/remove)
> of the port, it then properly re-initializes my device when I do a
> rescan.
> 
> Why does it not suffice to simply do a rescan? Does the rescan skip
> some step if it finds that the device is already in the PCI tree? It's
> not as if the rescan completely skips the device if I don't do a
> remove, it just doesn't do all of the initialization.

Initiating a rescan via sysfs results in a call to pci_rescan_bus(),
which only adds devices but doesn't remove any.  I guess it can be
argued that this behavior is counter-intuitive and should be changed.

Thanks,

Lukas

Re: PCI Rescan Question

[Bjorn Helgaas]

Hi  Kyle,

On Wed, Apr 26, 2017 at 5:56 PM, Kyle Gale <kylewgale@gmail.com> wrote:
> Hi,
>
>
>
> Can someone help me understand why I need to remove my stale PCI
> device files from the tree if I want to properly get my device
> re-initialized after unplugging and reinserting?
>
>
>
> For example, I turn on my system with my PCI device plugged in. All is
> fine. I unplug it, plug it back in, and rescan (echo 1 >
> /sys/bus/pci/devices/rescan) and end up in this state where my BAR0 is
> not set, and other things are not entirely correct.

Ideally, the PCI hotplug code would automatically remove the PCI
device files when you remove the physical device, and it would also
automatically scan the device when you plug it back in.

You shouldn't need to do either the remove or the rescan manually via sysfs.

Apparently your system does actually support hotplug.  Do you have it
enabled in the kernel?  The "lspci -vv" output, dmesg output, and your
.config would help answer these questions.

Bjorn

Re: PCI Rescan Question

[Yinghai Lu]

On Fri, Apr 28, 2017 at 7:56 AM, Bjorn Helgaas <bhelgaas@google.com> wrote:
> Hi  Kyle,
>
> On Wed, Apr 26, 2017 at 5:56 PM, Kyle Gale <kylewgale@gmail.com> wrote:
>> Hi,
>>
>>
>>
>> Can someone help me understand why I need to remove my stale PCI
>> device files from the tree if I want to properly get my device
>> re-initialized after unplugging and reinserting?
>>
>>
>>
>> For example, I turn on my system with my PCI device plugged in. All is
>> fine. I unplug it, plug it back in, and rescan (echo 1 >
>> /sys/bus/pci/devices/rescan) and end up in this state where my BAR0 is
>> not set, and other things are not entirely correct.
>
> Ideally, the PCI hotplug code would automatically remove the PCI
> device files when you remove the physical device, and it would also
> automatically scan the device when you plug it back in.
>
> You shouldn't need to do either the remove or the rescan manually via sysfs.
>
> Apparently your system does actually support hotplug.  Do you have it
> enabled in the kernel?  The "lspci -vv" output, dmesg output, and your
> .config would help answer these questions.

could be BIOS does set correct padding when card is not present.

but if the slot really support pcie hotplug aka pcie slot cap/hp is present,
current code should handle that case, and increase root port bridge resource
directly.

Yinghai

Re: PCI Rescan Question

[Kyle Gale]

Thank you for your responses,

It was not so much a problem with the pciehp driver - it worked fine
when I eventually enabled some of the slot's configuration registers.
I was just doing some things manually via sysfs during my bring-up,
and was surprised that a rescan wasn't enough to fully refresh the
tree. I can see now that a rescan will skip some code if a pci_dev
with the same devfn already exists from an previous enumeration of the
pci_bus, but intuitively I wouldn't have expected this to be the case.

Thanks,
Kyle