Re: Adding support for Relaxed ordering on a non-root device

[Bjorn Helgaas <helgaas@kernel.org>]

On Thu, Jun 04, 2020 at 03:35:48PM +0300, Aya Levin wrote:
> Hi
> 
> I am writing to you regarding your commits titled: 
> 1. a99b646afa8a PCI: Disable PCIe Relaxed Ordering if unsupported 
> 2. 87e09cdec4da PCI: Disable Relaxed Ordering for some Intel processors
> 
> While adding support to relaxed ordering on Mellanox Ethernet driver I
> tried to avoid the Intel's bug with pcie_relaxed_ordering_enabled.
> Expecting this to return False on Haswell and Broadwell CPU. I run
> this API on: Intel(R) Xeon(R) CPU E5-2650 v3 @ 2.30GHz (I think it is
> a Haswell, right?) and the API returned True. What am I missing?

The quirk is based on Vendor/Device ID, which of course is a problem
as new devices are released.  What does "lspci -vn" show on your
system?

> In addition, I saw your comment in pci_configure_relaxed_ordering
> (pasted below) the non-root ports are not handled since Peer-to-Peer
> DMA is another can of warms. Could you elaborate on the complexity?
> What is the effort to extend this to non-root ports?

There's some discussion about this here and in other parts of the same
thread:

  https://lore.kernel.org/linux-arm-kernel/20170809032503.GB7191@bhelgaas-glaptop.roam.corp.google.com/

> ---------------------------------------------------------------------
> static void pci_configure_relaxed_ordering(struct pci_dev *dev)
> {
> 	struct pci_dev *root;
> 
> 	/* PCI_EXP_DEVICE_RELAX_EN is RsvdP in VFs */
> 	if (dev->is_virtfn)
> 		return;
> 
> 	if (!pcie_relaxed_ordering_enabled(dev))
> 		return;
> 
> 	/*
> 	 * For now, we only deal with Relaxed Ordering issues with Root
> 	 * Ports. Peer-to-Peer DMA is another can of worms.
> 	 */
> 	root = pci_find_pcie_root_port(dev);
> 	if (!root)
> 		return;
> 
> 	if (root->dev_flags & PCI_DEV_FLAGS_NO_RELAXED_ORDERING) {
> 		pcie_capability_clear_word(dev, PCI_EXP_DEVCTL,
> 					   PCI_EXP_DEVCTL_RELAX_EN);
> 		pci_info(dev, "Relaxed Ordering disabled because the Root Port didn't support it\n");
> 	}
> }
> 
>