Linux PCI subsystem development
 help / color / mirror / Atom feed
From: Bjorn Helgaas <helgaas@kernel.org>
To: "Grumbach, Emmanuel" <emmanuel.grumbach@intel.com>
Cc: "thierry.reding@kernel.org" <thierry.reding@kernel.org>,
	"mmaddireddy@nvidia.com" <mmaddireddy@nvidia.com>,
	"lukas@wunner.de" <lukas@wunner.de>,
	"sashiko-reviews@lists.linux.dev"
	<sashiko-reviews@lists.linux.dev>,
	"Korenblit, Miriam Rachel" <miriam.rachel.korenblit@intel.com>,
	"treding@nvidia.com" <treding@nvidia.com>,
	"linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>
Subject: Re: [PATCH] PCI: Disable NoSnoop and Relaxed ordering for Intel wireless BE200
Date: Tue, 4 Aug 2026 12:39:27 -0500	[thread overview]
Message-ID: <20260804173927.GA1845060@bhelgaas> (raw)
In-Reply-To: <6c067883cd68a835a4ab5432e2b509b3686257b5.camel@intel.com>

On Tue, Aug 04, 2026 at 03:57:51PM +0000, Grumbach, Emmanuel wrote:
> On Fri, 2026-07-31 at 10:24 -0500, Bjorn Helgaas wrote:
> > On Thu, Jul 30, 2026 at 08:16:33PM +0000, Grumbach, Emmanuel wrote:
> > > On Thu, 2026-07-30 at 15:08 -0500, Bjorn Helgaas wrote:
> > > ...
> > 
> > > > I wonder if the PCI core should disable RO and NoSnoop on Root
> > > > Ports across the board?
> > > > 
> > > > I think this would mainly affect MMIO accesses performed by
> > > > drivers, and I suspect drivers expect those writes to be
> > > > performed in order, e.g., if the driver writes DMA descriptors
> > > > to the device followed by a write to a doorbell, those should
> > > > stay in order.  If the RP has RO enabled, those writes may be
> > > > reordered.
> > > 
> > > As a device writer I'd say we'd be required to add a memory
> > > barrier in that case, but we use writel usually.  Does that
> > > include a write vs write ordering? Hm...
> > 
> > Not being an ordering expert, I'm not sure I can make a convincing
> > argument for whether RO should be enabled or disabled for Root
> > Ports.
> > 
> > Maybe it's easier to argue that whatever the RO setting is, it
> > should be the same across all platforms because drivers shouldn't
> > have to adapt to platform differences.
> 
> That's playing safe, which is never a bad option if the performance
> hit is not significant, but I can't tell I can claim that have any
> expertise here.

I'm not a PCIe protocol expert, but it's hard for me to see how No
Snoop would be useful for outbound transactions initiated by the Root
Port.

For inbound DMA transactions without No Snoop, the RC can enforce
cache coherency by snooping CPU caches before accessing main memory.
If a driver knows that the main memory target is uncached or doesn't
need coherency, it can set No Snoop on its device to bypass the CPU
cache snoop.

But for outbound reads or writes from the RP, the completer is the
PCIe device that claims the transaction address, and I don't think any
other device or switch is able to snoop caches and affect the
response.

I suspect there *is* potential benefit for Relaxed Ordering on
outbound transactions, but I think drivers would need to be aware of
this.  Maybe the Linux driver model already comprehends this, and we
should enable RO on RPs.  Either way, I think we should configure RO
explicitly so drivers know what to expect and the behavior doesn't
depend on whatever firmware did.

> > Root Port settings potentially affect several endpoints, and I
> > think it would be impractical for drivers to request RO settings
> > that might conflict with each other.
> > 
> > Do you have any visibility into whether and how other OSes
> > actively configure Root Port RO and NoSnoop across the board,
> > independent of individual drivers?
> 
> I asked internally and we don't really know what Windows does when
> the PCI root port has those bits sets. We mostly test with Intel /
> AMD systems and those systems have those bits clear.
> 
> BTW - can you please point to the relevant paragraph in the PCI spec
> that states RelaxedOrder and NoSnoop should be set by default on the
> RP?

This is in PCIe r7.0, sec 7.5.3.4:

  Enable Relaxed Ordering - If this bit is Set, the Function is
  permitted to set the Relaxed Ordering bit in the Attributes field of
  transactions it initiates that do not require strong write ordering
  (see § Section 2.2.6.4 and § Section 2.4).

  A Function is permitted to hardwire this bit to 0b if it never sets
  the Relaxed Ordering attribute in transactions it initiates as a
  Requester.

  When not hardwired to 0b, the default value of this bit is 1b.


  Enable No Snoop - If this bit is Set, the Function is permitted to
  Set the No Snoop bit in the Requester Attributes of transactions it
  initiates that do not require hardware enforced cache coherency (see
  § Section 2.2.6.5). Note that setting this bit to 1b should not
  cause a Function to Set the No Snoop attribute on all transactions
  that it initiates. Even when this bit is Set, a Function is only
  permitted to Set the No Snoop attribute on a transaction when it can
  guarantee that the address of the transaction is not stored in any
  cache in the system.

  This bit is permitted to be hardwired to 0b if a Function would
  never Set the No Snoop attribute in transactions it initiates.

  Default value of this bit is 1b.

  reply	other threads:[~2026-08-04 17:39 UTC|newest]

Thread overview: 28+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-06-21  6:54 [PATCH] PCI: Disable NoSnoop and Relaxed ordering for Intel wireless BE200 Emmanuel Grumbach
2026-06-21  7:12 ` sashiko-bot
2026-06-21  7:30   ` Grumbach, Emmanuel
2026-07-22 14:56     ` Bjorn Helgaas
2026-07-22 15:14       ` Grumbach, Emmanuel
2026-07-22 17:04         ` Bjorn Helgaas
     [not found]           ` <204d9dab-96de-45fb-867d-20defc0b79e6@nvidia.com>
     [not found]             ` <534363f87b37ba6f1449d0524400bcfd7fdf5b3e.camel@intel.com>
2026-07-24 10:54               ` Manikanta Maddireddy
2026-07-28 18:03               ` Bjorn Helgaas
2026-07-28 18:21                 ` Grumbach, Emmanuel
2026-07-28 19:11                   ` Bjorn Helgaas
2026-07-28 19:16                     ` Grumbach, Emmanuel
2026-07-28 19:24                   ` Bjorn Helgaas
2026-07-28 19:47                     ` Grumbach, Emmanuel
2026-07-28 20:18                       ` Bjorn Helgaas
2026-07-29  6:49                         ` Grumbach, Emmanuel
2026-07-30  4:51                           ` Manikanta Maddireddy
2026-07-30 17:28                             ` Bjorn Helgaas
2026-07-30 18:45                               ` Grumbach, Emmanuel
2026-07-30 20:08                                 ` Bjorn Helgaas
2026-07-30 20:16                                   ` Grumbach, Emmanuel
2026-07-31 15:24                                     ` Bjorn Helgaas
2026-08-04 15:57                                       ` Grumbach, Emmanuel
2026-08-04 17:39                                         ` Bjorn Helgaas [this message]
2026-07-23  6:47   ` Lukas Wunner
2026-06-21  8:10 ` Lukas Wunner
2026-06-21  8:28   ` Grumbach, Emmanuel
2026-07-22 11:01   ` Grumbach, Emmanuel
2026-07-22 14:36     ` Lukas Wunner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260804173927.GA1845060@bhelgaas \
    --to=helgaas@kernel.org \
    --cc=emmanuel.grumbach@intel.com \
    --cc=linux-pci@vger.kernel.org \
    --cc=lukas@wunner.de \
    --cc=miriam.rachel.korenblit@intel.com \
    --cc=mmaddireddy@nvidia.com \
    --cc=sashiko-reviews@lists.linux.dev \
    --cc=thierry.reding@kernel.org \
    --cc=treding@nvidia.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox