Re: [PATCH] Documentation: PCI: Document decoding of TLP Header in AER messages

Ilpo Järvinen
2026-03-24T11:18:07Z

[-- Attachment #1: Type: text/plain, Size: 7508 bytes --]

On Tue, 24 Mar 2026, mx2pg@pm.me wrote:

> One thing worth calling out: starting with PCIe 6.0, Flit Mode is
> mandatory at 64.0 GT/s and supported at all PCIe link speeds, so a
> Flit-capable PCIe 6.x link may operate below 64.0 GT/s and still be
> in Flit Mode. The raw TLP Header bytes do not encode the framing —
> the same four bytes decode to entirely different packet types in
> non-Flit vs Flit framing. The negotiated mode can be read from the
> Flit Mode Status bit in Link Status 2, or via lspci -vv on a recent
> pciutils build.

There's one caveat in using Link Status 2 Flit Mode Status bit, it can
only be used as the indicator when the Link is Up, which may come into
picture in troubleshooting scenarios.

The kernel code tries to hide that by indicating the Flit mode explicitly
in the log message it prints out.

Sadly, TLP Logging on DPC side was botched in the PCIe spec so it doesn't
indicate the Flit/non-Flit mode information explicitly (in contrast to AER
that has a flag that tells in which mode the TLP Log was captured). To
workaround that limitation, kernel has to save of the Link Status 2
contents and hope the information is not stale when DPC has brought the
Link Down (it seems relatively likely to remain valid but it's still
fundamentally racy way to get the Flit/non-Flit information).

--
i.

> tlp-tool defaults to non-Flit, which is correct for the vast majority
> of hardware deployed today. That will change: as PCIe 6.x adoption
> grows, a significant share of TLP debugging will involve Flit Mode
> links, and this is already a concern among switch and device vendors
> working through the transition. Users on Flit Mode links must pass
> --flit:
>
> # non-Flit link (default, most common today)
> curl -L https://git.kernel.org/linus/2ca1c94ce0b6 | rtlp-tool --aer
>
> # Flit Mode link
> curl -L https://git.kernel.org/linus/2ca1c94ce0b6 | rtlp-tool --aer --flit
>
> It may be worth a one-liner in the Documentation patch:
>
> For PCIe 6.x links with Flit Mode negotiated (check Flit Mode Status
> in Link Status 2, or lspci -vv), pass --flit to rtlp-tool.
>
> Maciej
>
>
>
> On Monday, March 23rd, 2026 at 9:50 AM, Bjorn Helgaas <helgaas@kernel.org> wrote:
>
> > On Mon, Mar 23, 2026 at 07:52:39AM +0100, Lukas Wunner wrote:
> > > The prefix/header of the TLP that caused an error is recorded by the Root
> > > Complex and emitted to the kernel log in raw hex format. Document the
> > > existence and usage of tlp-tool, which allows decoding the TLP Header
> > > into human-readable form.
> > >
> > > The TLP Header hints at the root cause of an error, yet is often ignored
> > > because of its seeming opaqueness. Instead, PCIe errors are frequently
> > > worked around by a change in the kernel without fully understanding the
> > > actual source of the problem. With more documentation on available tools
> > > we'll hopefully come up with better solutions.
> > >
> > > There are also wireshark dissectors for TLPs, but it seems they expect a
> > > complete TLP, not just the header, and they cannot grok the hex format
> > > emitted by the kernel directly. tlp-tool appears to be the most cut and
> > > dried solution out there.
> > >
> > > Signed-off-by: Lukas Wunner <lukas@wunner.de>
> > > Cc: Maciej Grochowski <mx2pg@pm.me>
> >
> > Applied to pci/for-linus for v7.0, thanks!
> >
> > I tweaked the commit log to note that the Header Log is in the AER
> > Capability, which may be in any PCIe function.
> >
> > > ---
> > > We could also go one step further and point users to this tool
> > > in a printk_once() message when the first error occurs.
> > > For now, just amending the documentation is probably sufficient.
> > >
> > > Documentation/PCI/pcieaer-howto.rst | 10 ++++++++++
> > > 1 file changed, 10 insertions(+)
> > >
> > > diff --git a/Documentation/PCI/pcieaer-howto.rst b/Documentation/PCI/pcieaer-howto.rst
> > > index 3210c47..90fdfdd 100644
> > > --- a/Documentation/PCI/pcieaer-howto.rst
> > > +++ b/Documentation/PCI/pcieaer-howto.rst
> > > @@ -85,6 +85,16 @@ In the example, 'Requester ID' means the ID of the device that sent
> > > the error message to the Root Port. Please refer to PCIe specs for other
> > > fields.
> > >
> > > +The 'TLP Header' is the prefix/header of the TLP that caused the error
> > > +in raw hex format. To decode the TLP Header into human-readable form
> > > +one may use tlp-tool:
> > > +
> > > +https://github.com/mmpg-x86/tlp-tool
> > > +
> > > +Example usage::
> > > +
> > > + curl -L https://git.kernel.org/linus/2ca1c94ce0b6 | rtlp-tool --aer
> > > +
> > > AER Ratelimits
> > > --------------
> > >
> > > --
> > > 2.51.0
> > >
> >
>