Re: [Bug 215525] New: HotPlug does not work on upstream kernel 5.17.0-rc1

[Bjorn Helgaas <helgaas@kernel.org>]

On Thu, Feb 03, 2022 at 11:47:09AM +0100, Blazej Kucman wrote:
> > > On Wed, Feb 02, 2022 at 04:48:01PM +0100, Blazej Kucman wrote:  
> > >> On Fri, 28 Jan 2022 08:03:28 -0600
> > >> Bjorn Helgaas <helgaas@kernel.org> wrote:  
> > >>>> On Fri, Jan 28, 2022 at 9:08 PM Bjorn Helgaas
> > >>>> <helgaas@kernel.org> wrote:    

> > >>>>> 04b12ef163d1 ("PCI: vmd: Honor ACPI _OSC on PCIe features")
> > >>>>> looks like a it could be related.  Try reverting that commit
> > >>>>> and see whether it makes a difference.    
> > >>>>
> > >>>> The affected NVMe is indeed behind VMD domain, so I think the
> > >>>> commit can make a difference.
> > >>>>
> > >>>> Does VMD behave differently on laptops and servers?
> > >>>> Anyway, I agree that the issue really lies in "pci=nommconf".    
> > >>>
> > >>> Oh, I have a guess:
> > >>>
> > >>>   - With "pci=nommconf", prior to v5.17-rc1, pciehp did not work
> > >>> in general, but *did* work for NVMe behind a VMD.  As of
> > >>> v5.17-rc1, pciehp no longer works for NVMe behind VMD.
> > >>>
> > >>>   - Without "pci=nommconf", pciehp works as expected for all
> > >>> devices including NVMe behind VMD, both before and after
> > >>> v5.17-rc1.
> > >>>
> > >>> Is that what you're observing?
> > >>>
> > >>> If so, I doubt there's anything to fix other than getting rid of
> > >>> "pci=nommconf".  
> > >>
> > >> I haven't tested with VMD disabled earlier. I verified it and my
> > >> observations are as follows:
> > >>
> > >> OS: RHEL 8.4
> > >> NO - hotplug not working
> > >> YES - hotplug working
> > >>
> > >> pci=nommconf added:
> > >> +--------------+-------------------+---------------------+--------------+
> > >> |              | pci-v5.17-changes | revert-04b12ef163d1 | inbox
> > >> kernel
> > >> +--------------+-------------------+---------------------+--------------+
> > >> | VMD enabled  | NO                | YES                 | YES
> > >> +--------------+-------------------+---------------------+--------------+
> > >> | VMD disabled | NO                | NO                  | NO
> > >> +--------------+-------------------+---------------------+--------------+

OK, so the only possible problem case is that booting with VMD enabled
and "pci=nommconf".  In that case, hotplug for devices below VMD
worked before 04b12ef163d1 and doesn't work after.

Your table doesn't show it, but hotplug for devices *not* behind VMD
should not work either before or after 04b12ef163d1 because Linux
doesn't request PCIe hotplug control when booting with "pci=nommconf".

Why were you testing with "pci=nommconf"?  Do you think anybody uses
that with VMD and NVMe?

Bjorn