Re: PCI: hotplug_event: PCIe PLDA Device BAR Reset

[Bjorn Helgaas <helgaas@kernel.org>]

On Wed, Feb 26, 2025 at 06:28:33PM +0530, Naveen Kumar P wrote:
> On Wed, Feb 26, 2025 at 2:08 AM Bjorn Helgaas <helgaas@kernel.org> wrote:
> > On Tue, Feb 25, 2025 at 06:46:02PM +0530, Naveen Kumar P wrote:
> > > On Tue, Feb 25, 2025 at 1:24 AM Bjorn Helgaas <helgaas@kernel.org> wrote:
> > > > On Tue, Feb 25, 2025 at 12:29:00AM +0530, Naveen Kumar P wrote:
> > > > > On Mon, Feb 24, 2025 at 11:03 PM Bjorn Helgaas <helgaas@kernel.org> wrote:
> > > > > > On Mon, Feb 24, 2025 at 05:45:35PM +0530, Naveen Kumar P wrote:
> > > > > > > On Wed, Feb 19, 2025 at 10:36 PM Bjorn Helgaas <helgaas@kernel.org> wrote:
> > > > > > > > On Wed, Feb 19, 2025 at 05:52:47PM +0530, Naveen Kumar P wrote:
> > > > > > > > > Hi all,
> > > > > > > > >
> > > > > > > > > I am writing to seek assistance with an issue we are
> > > > > > > > > experiencing with a PCIe device (PLDA Device 5555)
> > > > > > > > > connected through PCI Express Root Port 1 to the
> > > > > > > > > host bridge.
> > > > > > > > >
> > > > > > > > > We have observed that after booting the system, the
> > > > > > > > > Base Address Register (BAR0) memory of this device
> > > > > > > > > gets reset to 0x0 after approximately one hour or
> > > > > > > > > more (the timing is inconsistent). This was verified
> > > > > > > > > using the lspci output and the setpci -s 01:00.0
> > > > > > > > > BASE_ADDRESS_0 command.
> > > > > ...

> I have downloaded the 6.13 kernel source and added additional debug
> logs in hotplug_event(), then built the kernel. After that rebooted
> with the new kernel using the following parameters:
> BOOT_IMAGE=/vmlinuz-6.13.0+ root=/dev/mapper/vg00-rootvol ro quiet
> libata.force=noncq pci=nomsi pcie_aspm=off pcie_ports=on "dyndbg=file
> drivers/pci/* +p; file drivers/acpi/* +p"

Why "pci=nomsi"?  I don't think that should make a difference.  Also,
it contributes to the fact that Linux doesn't request OS control of
several features that it ordinarily does, so you end up in a somewhat
unusual state (which *should* still work, of course):

  acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig Segments HPX-Type3]
  acpi PNP0A08:00: _OSC: not requesting OS control; OS requires [ExtendedConfig ASPM ClockPM MSI]

Same for "pcie_aspm=off".

Why "pcie_ports=on"?  That's not a valid parameter:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/pci/pcie/portdrv.c?id=v6.13#n619

> Complete dmesg log and the patch(to get additional debug information)
> are attached to this email.
> 
> Any further guidance on these observations?

I'm out of ideas.  I would instrument the PCI config accessors to log
all the reads and writes to your device (01:00.0) to see what we do to
the device.  Maybe there's some hint:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/pci/access.c?id=v6.13#n35

> Additionally, I noticed that the initial bootup logs with the
> "0.000000" timestamp are missing in the dmesg log with this new
> kernel. I'm unsure what might be causing this issue.

Probably overflowed the message buffer.  You can try increasing the
buffer size:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/admin-guide/kernel-parameters.txt?id=v6.13#n3190

You can also experiment with the dyndbg parameter to be more selective
about the ACPI messages if some aren't useful.

Bjorn