Re: [PATCH] pci: Add pci quirk to turnoff Nosnoop and Relaxed Ordering

[Bjorn Helgaas <helgaas@kernel.org>]

On Wed, Sep 01, 2021 at 04:22:12PM -0700, Keith Busch wrote:
> On Wed, Sep 01, 2021 at 05:23:53PM -0500, Bjorn Helgaas wrote:
> > On Sun, Oct 18, 2015 at 07:55:04PM +0530, Hariprasad Shenai wrote:
> > > Some devices violate the PCI Specification regarding echoing the Root
> > > Port Transaction Layer Packet Request (TLP) No Snoop and Relaxed
> > > Ordering Attributes into the TLP Response. The PCI Specification
> > > "encourages" compliant Root Port implementation to drop such
> > > malformed TLP Responses leading to device access timeouts. Many Root Port
> > > implementations accept such malformed TLP Responses and a few more
> > > strict implementations do drop them.
> > > 
> > > For devices which fail this part of the PCI Specification, we need to
> > > traverse up the PCI Chain to the Root Port and turn off the Enable No
> > > Snoop and Enable Relaxed Ordering bits in the Root Port's PCI-Express
> > > Device Control register. This does affect all other devices which
> > > are downstream of that Root Port.
> > 
> > While researching another thread about RO [1], I got concerned about
> > setting RO for root ports.
> > 
> > Setting RO for *endpoints* makes sense: that allows (but does not
> > require) the endpoint to issue writes that don't require strong
> > ordering.
> > 
> > Setting RO for *root ports* seems more problematic.  It allows the
> > root port to issue PCIe writes that don't require strong ordering.
> > These would be CPU MMIO writes to devices.  But Linux currently does
> > not have a way for drivers to indicate that some MMIO writes need to
> > be ordered while others do not, and I think drivers assume that all
> > MMIO writes are performed in order.
> 
> Is that not what writel_relaxed() is for? While it appears that most
> archs just have that call the generic writel(), it does let drivers
> indicate which writes are not strongly ordered.

Sheesh, I hate when I spend a whole afternoon making a fool of myself.

I was thinking just from the PCI core perspective, expecting a
separate window or something for relaxed ordering, but I'm sure
writel_relaxed() could be implemented using different CPU bus
transactions that the Root Complex knows how to interpret, and that
would be essentially invisible to the PCI core.

Sorry for the noise and thanks for the hint :)