Re: [PATCH] PCI: Align MPS to upstream bridge for SAFE and PERFORMANCE mode

[Bjorn Helgaas <helgaas@kernel.org>]

On Thu, Oct 27, 2022 at 05:51:49PM -0500, Bjorn Helgaas wrote:
> On Tue, Oct 25, 2022 at 12:07:47AM -0500, Tyler Hicks wrote:
> > On 2022-10-20 15:24:37, Bjorn Helgaas wrote:

> > I've been talking to the firmware folks on why SAFE mode was selected,
> > based on Keith's question from Wednesday. From what I've been told,
> > U-Boot doesn't seed the RP MPS with a value so the kernel sees a value
> > of 128 for p_mps in pci_configure_mps() when using the DEFAULT mode.
> > Apparently UEFI does seed the RP MPS but we don't get that with U-Boot.
> > Therefore, SAFE mode was selected to get an initial, tuned RP MPS value
> > set to 256.
> 
> Are there any devices below the RP at the time we set MPS=256?
> 
> > > A subsequent hot-add will do nothing in pci_configure_mps(), and
> > > pcie_bus_configure_settings() looks like it would set the RP and EP
> > > MPS to the minimum of the RP and EP MPS_Supported.
> > > 
> > > Is that what you see?  What would you like to see instead?
> > 
> > No, not quite. After hot-add, we see the EP MPS set to 128 with SAFE
> > mode even though the RP MPS is 256.
> 
> Can you add the relevant topology here so we can work through the
> concrete details?  Is the endpoint hot-added directly below a Root
> Port?  Or is there a switch involved?  What are the MPS_Supported
> values for all the devices?  If there's a switch in the picture, it
> looks like we currently restrict to 128, I think because it's possible
> an endpoint that can only do 128 may be added.

Ping?  I'd like to talk about a concrete scenario.  It's too hard for
me to imagine all the possible things that could go wrong.

I guess part of what's interesting here is that things work better
when firmware has already configured MPS?  It doesn't seem like we
should *depend* on firmware having done that.

Bjorn