Linux PCI subsystem development
 help / color / mirror / Atom feed
From: "Grumbach, Emmanuel" <emmanuel.grumbach@intel.com>
To: "helgaas@kernel.org" <helgaas@kernel.org>
Cc: "thierry.reding@kernel.org" <thierry.reding@kernel.org>,
	"lukas@wunner.de" <lukas@wunner.de>,
	"mmaddireddy@nvidia.com" <mmaddireddy@nvidia.com>,
	"sashiko-reviews@lists.linux.dev"
	<sashiko-reviews@lists.linux.dev>,
	"Korenblit, Miriam Rachel" <miriam.rachel.korenblit@intel.com>,
	"treding@nvidia.com" <treding@nvidia.com>,
	"linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>
Subject: Re: [PATCH] PCI: Disable NoSnoop and Relaxed ordering for Intel wireless BE200
Date: Tue, 28 Jul 2026 18:21:23 +0000	[thread overview]
Message-ID: <a1934c51592fc56c65232b1379680c52c6d0cea3.camel@intel.com> (raw)
In-Reply-To: <20260728180315.GA1380884@bhelgaas>

On Tue, 2026-07-28 at 13:03 -0500, Bjorn Helgaas wrote:
> On Thu, Jul 23, 2026 at 11:35:41AM +0000, Grumbach, Emmanuel wrote:
> > On Thu, 2026-07-23 at 15:19 +0530, Manikanta Maddireddy wrote:
> > > On 22/07/26 10:34 pm, Bjorn Helgaas wrote:
> > > > > The problem was that we didn't show up at all in the
> > > > > enumeration.
> > > > > We
> > > > > send a malformed TLP. I'm not quite an expert at this, but
> > > > > our
> > > > > PCIe
> > > > > experts run a PCI analyzer on the enumeration on that
> > > > > specific
> > > > > platform and they saw that platform's PCI controller sets the
> > > > > NoSnoop and Relaxed ordering bit in the TLP. According to the
> > > > > spec
> > > > > (which I ignore), the BE200 is supposed to return the TLP as
> > > > > received, but we reply with 0 Attributes and the enumeration
> > > > > doesn't
> > > > > complete successfully.
> > > > I'm curious about the details of this enumeration failure.  Do
> > > > you
> > > > know which TLPs had No Snoop and Relaxed Ordering set?  Per the
> > > > PCIe
> > > > spec, they shouldn't be set for the config requests used for
> > > > PCI
> > > > core
> > > > enumeration.
> > > > 
> > > > At least*some* config reads to the BE200 must work; otherwise,
> > > > we
> > > > wouldn't know the Vendor or Device ID, which we need to apply
> > > > the
> > > > quirk.  So I think you should see something like this in dmesg,
> > > > and
> > > > BE200 would probably appear in lspci output:
> > > > 
> > > >    pci 0000:04:00.0: [8086:272b] type 00 class ...
> > > > 
> > > > Is the failure that iwl_pci_probe() itself fails somehow?
> > > > 
> > > > You mentioned that this happens on Jetson Thor, but I'm not
> > > > sure
> > > > what PCIe controller that is.  My guess is it might be Tegra264
> > > > [1], which doesn't look like it's merged yet.
> > > > 
> > > > Thierry, Manikanta, do you have any insight into this?  Does
> > > > this PCIe controller set No Snoop and Relaxed Ordering for some
> > > > reason?  I don't think endpoint drivers are expecting that.
> > > > 
> > > > [1]https://lore.kernel.org/linux-pci/20260716-tegra264-pcie- 
> > > > v8-0-23e51589229b@nvidia.com/
> > > 
> > > Hi,
> > > 
> > > Jetson Thor(Tegra264) is not setting NoSnoop and RlxdOrd bits in
> > > config 
> > > read TLP.
> > > 
> > > I made sure that both these bits are set in RP's DevCtl
> > > 
> > >                  DevCtl: CorrErr+ NonFatalErr+ FatalErr+
> > > UnsupReq+
> > >                          RlxdOrd+ ExtTag+ PhantFunc- AuxPwr-
> > > NoSnoop+
> > > 
> > > and dumped TLP header for a config read towards BDF 0x100 with
> > > offset
> > > 0x24.
> > >     0x04000001      0x0000000f      0x01000024      0x00000000
> > > 
> > > I think dmesg and AER log with header information might help with
> > > this 
> > > particular issue.
> > 
> > Ok, so I checked again the logs and I was wrong.
> > We do see the device in the enumeration, it does show up in lspci.
> > Problems start when we want to access our registers.
> > I attached the full dmesg output. In that log we try to load
> > iwlwifi
> > twice.
> > 
> > I'm adding here the snippet of the first load:
> > 
> > [   15.275887] iwlwifi 0001:01:00.0: Adding to iommu group 53
> > [   15.279737] iwlwifi 0001:01:00.0: enabling device (0100 -> 0102)
> > [   15.280047] iwlwifi 0001:01:00.0: HW_REV=0xFFFFFFFF, PCI
> > issues?  \\ **
> 
> The "enabling device" message is from pci_enable_resources(), called
> in the pci_enable_device() path.  The 0100 is from a config read of
> PCI_COMMAND, and the 0102 is from adding PCI_COMMAND_MEMORY to enable
> memory BARs.
> 
> The "HW_REV=" is from iwl_pci_probe(), which looks like the very
> first
> MMIO read to a BE200 BAR.

Indeed

> 
> So I guess the theory is that Tegra264 set NoSnoop and/or RlxdOrd in
> the MMIO read, BE200 didn't copy the attributes from the Request to
> the Completion as required by PCIe r7.0, sec 2.2.9.1, and Tegra264
> logged a Malformed TLP?

That's the assumption based on the AER log, yes.

> 
> AFAICS we still don't know why Tegra264 would set NoSnoop and/or
> RlxdOrd in the MMIO read.

Me neither but... I can't comment on that. And de-facto, once it does
that, the BE200 replies with the NoSnoop and RlxOrd clear in the TLP
which is then considered as a malformed TLP.

> 
> Unless iwlwifi asked for NoSnoop and/or RlxdOrd to be set, I think
> it's a potential problem for drivers if Tegra264 sets them.

I ... don't think we would do that. The driver would certainly not do
that... Regarding the hardware itself, I can't comment, but I can check
internally.

> 
> > [   15.280070] iwlwifi: probe of 0001:01:00.0 failed with error -5 
> > \\ **
> > [   15.280087] pcieport 0001:00:00.0: AER: Correctable error
> > message received from 0001:00:00.0
> > [   15.280104] pcieport 0001:00:00.0: DPC: containment event,
> > status:0x3f01 source:0x0000
> > [   15.280110] pcieport 0001:00:00.0: DPC: unmasked uncorrectable
> > error detected
> > [   15.280127] pcieport 0001:00:00.0: AER: found no error details
> > for 0001:00:00.0
> > [   15.280154] pcieport 0001:00:00.0: PCIe Bus Error:
> > severity=Uncorrectable (Fatal), type=Transaction Layer, (Receiver
> > ID)
> > [   15.280157] pcieport 0001:00:00.0:   device [10de:22d8] error
> > status/mask=00040000/04400000
> > [   15.280160] pcieport 0001:00:00.0:    [18]
> > MalfTLP                (First)
> > [   15.280163] pcieport 0001:00:00.0: AER:   TLP Header: 4a008001
> > 01000004 00000028 72040000
> > [   15.280274] pci 0001:01:00.0: AER: can't recover (no
> > error_detected callback)
> > 
> > Does that help?
> > 
> > > 
> > > Thanks,
> > > Manikanta
> 

  reply	other threads:[~2026-07-28 18:21 UTC|newest]

Thread overview: 28+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-06-21  6:54 [PATCH] PCI: Disable NoSnoop and Relaxed ordering for Intel wireless BE200 Emmanuel Grumbach
2026-06-21  7:12 ` sashiko-bot
2026-06-21  7:30   ` Grumbach, Emmanuel
2026-07-22 14:56     ` Bjorn Helgaas
2026-07-22 15:14       ` Grumbach, Emmanuel
2026-07-22 17:04         ` Bjorn Helgaas
     [not found]           ` <204d9dab-96de-45fb-867d-20defc0b79e6@nvidia.com>
     [not found]             ` <534363f87b37ba6f1449d0524400bcfd7fdf5b3e.camel@intel.com>
2026-07-24 10:54               ` Manikanta Maddireddy
2026-07-28 18:03               ` Bjorn Helgaas
2026-07-28 18:21                 ` Grumbach, Emmanuel [this message]
2026-07-28 19:11                   ` Bjorn Helgaas
2026-07-28 19:16                     ` Grumbach, Emmanuel
2026-07-28 19:24                   ` Bjorn Helgaas
2026-07-28 19:47                     ` Grumbach, Emmanuel
2026-07-28 20:18                       ` Bjorn Helgaas
2026-07-29  6:49                         ` Grumbach, Emmanuel
2026-07-30  4:51                           ` Manikanta Maddireddy
2026-07-30 17:28                             ` Bjorn Helgaas
2026-07-30 18:45                               ` Grumbach, Emmanuel
2026-07-30 20:08                                 ` Bjorn Helgaas
2026-07-30 20:16                                   ` Grumbach, Emmanuel
2026-07-31 15:24                                     ` Bjorn Helgaas
2026-08-04 15:57                                       ` Grumbach, Emmanuel
2026-08-04 17:39                                         ` Bjorn Helgaas
2026-07-23  6:47   ` Lukas Wunner
2026-06-21  8:10 ` Lukas Wunner
2026-06-21  8:28   ` Grumbach, Emmanuel
2026-07-22 11:01   ` Grumbach, Emmanuel
2026-07-22 14:36     ` Lukas Wunner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=a1934c51592fc56c65232b1379680c52c6d0cea3.camel@intel.com \
    --to=emmanuel.grumbach@intel.com \
    --cc=helgaas@kernel.org \
    --cc=linux-pci@vger.kernel.org \
    --cc=lukas@wunner.de \
    --cc=miriam.rachel.korenblit@intel.com \
    --cc=mmaddireddy@nvidia.com \
    --cc=sashiko-reviews@lists.linux.dev \
    --cc=thierry.reding@kernel.org \
    --cc=treding@nvidia.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox