Intel-XE Archive on lore.kernel.org
 help / color / mirror / Atom feed
From: Jani Nikula <jani.nikula@linux.intel.com>
To: Michal Wajdeczko <michal.wajdeczko@intel.com>,
	intel-xe@lists.freedesktop.org
Cc: "Michal Wajdeczko" <michal.wajdeczko@intel.com>,
	"Mallesh Koujalagi" <mallesh.koujalagi@intel.com>,
	"Rodrigo Vivi" <rodrigo.vivi@intel.com>,
	"Thomas Hellström" <thomas.hellstrom@linux.intel.com>,
	"Matthew Brost" <matthew.brost@intel.com>,
	"Aravind Iddamsetty" <aravind.iddamsetty@intel.com>,
	"Riana Tauro" <riana.tauro@intel.com>,
	"Raag Jadav" <raag.jadav@intel.com>,
	"Badal Nilawar" <badal.nilawar@intel.com>
Subject: Re: [PATCH v2 00/22] drm/xe: Add structured SIGID error logging infrastructure
Date: Wed, 29 Jul 2026 11:31:46 +0300	[thread overview]
Message-ID: <dfe82413d317aee8bd39dbbc45ece77f9b6d7610@intel.com> (raw)
In-Reply-To: <20260728161039.579-1-michal.wajdeczko@intel.com>

On Tue, 28 Jul 2026, Michal Wajdeczko <michal.wajdeczko@intel.com> wrote:
> Today the driver reports faults with ad-hoc drm_err()/xe_gt_err()
> strings that have no stable shape. That is readable for a human, but it
> gives fleet tooling nothing durable to match on: the wording changes
> between releases, lines can be rate-limited or dropped under an error
> storm, and there is no consistent way to ask "which recognised fault
> just happened?".
>
> Introduce a signature identifier (SIGID): a small, stable integer that
> names one recognised Xe fault situation and serves as the primary handle
> for triage. A SIGID maps, through published end-user documentation, to a
> description and a recommended action; the driver only has to emit the
> right SIGID next to the usual human-readable text.
>
> Design decisions:
>
>  - Software-emitted signatures only. This header enumerates just the
>    situations the driver detects and reports itself. Signatures that
>    originate in firmware or hardware are identified by those layers (via
>    their own records/counters) and are logged as received -- minting a
>    driver-side id for them would duplicate an id the reporting layer
>    already owns.
>
>  - Flat catalogue, chosen per report site. Each site emits the single
>    most specific situation for that site, so a multi-layer failure
>    produces a chain of reports rather than one ambiguous classification
>    (e.g. a failed GT reset reports GT_TDR and then WEDGED). A site that
>    matches no defined situation keeps using ordinary xe_err() /
>    xe_gt_err() rather than forcing a wrong id.
>
>  - Stable numbering. A single flat list numbered sequentially from 1, in
>    introduction order. Values are only ever appended, never renumbered
>    or reused.
>
>  - First-order action. Each SIGID carries a coarse, in-tree resolution
>    bucket (COLLECT / RETRY / UPDATE / RECOVER) so it is actionable
>    without an external reference, and so every new id must declare what
>    to do about it.
>
>  - Severity is decoupled from the SIGID and chosen at the call site via
>    xe_ras_log_fatal() / _recoverable() / _info(); the same situation can
>    be reported at different severities depending on the instance.
>
>  - dmesg stays close to a normal xe error line by reusing xe_err() /
>    xe_gt_err() (and their Tile/GT decoration); the only stable,
>    machine-matchable token added is SIGID=<n>. dmesg is not an ABI --
>    the durable machine record is the CPER carrying the same SIGID (a
>    planned follow-up, left as a TODO).
>
> Wire up a representative site for each software signature so the set is
> exercised rather than merely declared:
>
>  - PROBE: xe workqueue allocation failure during early init
>  - WEDGED: xe_device_declare_wedged() (drop redundant "CRITICAL" + BDF)
>  - SURVIVABILITY: entering boot survivability mode
>  - RUNTIME_FW: GuC mmio request failure
>  - DEVICE_FW: PCODE mailbox failure
>  - GT_TDR: GT reset failure (which then chains into a WEDGED report)
>  - MEM_FAULT: page-fault queue overflow
>  - IO_BUS: PCI re-enable failure after a bus reset

The idea is not new, see for example [1] and [2]. I'm not sure if that
was ever merged, though. Would be good to know what came of it, and why.

There's also the printk index support already in the kernel [3]. Did you
look into that? If not, please do. If yes, please iterate why that can't
be used, or extended, and a local mechanism is required.

FWIW, I think merging driver local mechanisms for what should really be
generic kernel stuff usually ends up being a maintenance PITA. It's easy
in the beginning, a burden in the end, and even harder to migrate to
generic mechanisms. We've seen this over and over in i915, even for
superficially small things. Part of the reason for xe existing in the
first place is to use common stuff instead of reinventing the weels.

BR,
Jani.


[1] https://lwn.net/Articles/464276/
[2] https://lwn.net/Articles/492125/
[3] https://lkml.org/lkml/2021/6/15/1053

>
> Signed-off-by: Mallesh Koujalagi <mallesh.koujalagi@intel.com>
> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
> Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
> Cc: Thomas Hellström <thomas.hellstrom@linux.intel.com>
> Cc: Matthew Brost <matthew.brost@intel.com>
> Cc: Aravind Iddamsetty <aravind.iddamsetty@intel.com>
> Cc: Riana Tauro <riana.tauro@intel.com>
> Cc: Raag Jadav <raag.jadav@intel.com>
> Cc: Badal Nilawar <badal.nilawar@intel.com>
>
> v1: https://patchwork.freedesktop.org/series/171022/#rev1
> v2: component/location dmesg decorations moved away from macros
>     HW component identifiers related to RAS/FW enum values
>     CORRECTED severity rendered as an error
>     more support for const pointers
>     log 'probe blocked' as INFO
>     fix kunit device setup
>     more tests
>
> Mallesh Koujalagi (1):
>   drm/xe/log: Add structured SIGID error logging infrastructure
>
> Michal Wajdeczko (21):
>   drm/xe: Introduce xe_any helpers
>   drm/xe/log: Introduce structured component/location identifiers
>   drm/xe/log: Add component/location decorations to dmesg
>   drm/xe/log: Add SIGID log helpers for severity
>   drm/xe/log: Add SIGID log helpers for location
>   drm/xe/log: Add SIGID log helpers for location & severity
>   drm/xe/log: Add SIGID log helpers for components
>   drm/xe/log: Add SIGID log helpers for errno-only
>   drm/xe/log: Add hardware error signatures
>   drm/xe/log: Extend components list with hardware items
>   drm/xe/ras: Check RAS and LOG component definitions
>   drm/xe/kunit: Setup driver data in the test device
>   drm/xe/tests: Add Kunit tests for xe_log
>   drm/xe: Report 'probe blocked' error using SIGID
>   drm/xe: Report 'device wedged' errors using SIGID
>   drm/xe: Report 'Survivability Mode' errors using SIGID
>   drm/xe/guc: Report 'GuC mmio' errors using SIGID
>   drm/xe/pcode: Report 'Mailbox failed' error using SIGID
>   drm/xe/gt: Report 'reset failed' errors using SIGID
>   drm/xe/gt: Report 'pagefault' errors using SIGID
>   drm/xe/pci: Report 'cannot re-enable' error using SIGID
>
>  Documentation/gpu/xe/index.rst              |   1 +
>  Documentation/gpu/xe/xe_sigid.rst           |  14 +
>  drivers/gpu/drm/xe/Makefile                 |   1 +
>  drivers/gpu/drm/xe/abi/xe_log_abi.h         | 197 ++++++++
>  drivers/gpu/drm/xe/abi/xe_sigid_abi.h       | 195 ++++++++
>  drivers/gpu/drm/xe/tests/xe_kunit_helpers.c |   4 +
>  drivers/gpu/drm/xe/tests/xe_log_kunit.c     | 480 ++++++++++++++++++++
>  drivers/gpu/drm/xe/xe_any.h                 | 112 +++++
>  drivers/gpu/drm/xe/xe_device.c              |  16 +-
>  drivers/gpu/drm/xe/xe_gt.c                  |   7 +-
>  drivers/gpu/drm/xe/xe_guc.c                 |  18 +-
>  drivers/gpu/drm/xe/xe_log.c                 | 212 +++++++++
>  drivers/gpu/drm/xe/xe_log.h                 | 158 +++++++
>  drivers/gpu/drm/xe/xe_pagefault.c           |  10 +-
>  drivers/gpu/drm/xe/xe_pci.c                 |   5 +-
>  drivers/gpu/drm/xe/xe_pci_error.c           |   7 +-
>  drivers/gpu/drm/xe/xe_pcode.c               |   5 +-
>  drivers/gpu/drm/xe/xe_ras.c                 |  11 +
>  drivers/gpu/drm/xe/xe_survivability_mode.c  |  24 +-
>  19 files changed, 1438 insertions(+), 39 deletions(-)
>  create mode 100644 Documentation/gpu/xe/xe_sigid.rst
>  create mode 100644 drivers/gpu/drm/xe/abi/xe_log_abi.h
>  create mode 100644 drivers/gpu/drm/xe/abi/xe_sigid_abi.h
>  create mode 100644 drivers/gpu/drm/xe/tests/xe_log_kunit.c
>  create mode 100644 drivers/gpu/drm/xe/xe_any.h
>  create mode 100644 drivers/gpu/drm/xe/xe_log.c
>  create mode 100644 drivers/gpu/drm/xe/xe_log.h

-- 
Jani Nikula, Intel

      parent reply	other threads:[~2026-07-29  8:31 UTC|newest]

Thread overview: 34+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-07-28 16:10 [PATCH v2 00/22] drm/xe: Add structured SIGID error logging infrastructure Michal Wajdeczko
2026-07-28 16:10 ` [PATCH v2 01/22] drm/xe: Introduce xe_any helpers Michal Wajdeczko
2026-07-28 19:11   ` Rodrigo Vivi
2026-07-29  7:23   ` Mallesh, Koujalagi
2026-07-29  7:58   ` Jani Nikula
2026-07-28 16:10 ` [PATCH v2 02/22] drm/xe/log: Add structured SIGID error logging infrastructure Michal Wajdeczko
2026-07-28 16:10 ` [PATCH v2 03/22] drm/xe/log: Introduce structured component/location identifiers Michal Wajdeczko
2026-07-28 19:13   ` Rodrigo Vivi
2026-07-28 16:10 ` [PATCH v2 04/22] drm/xe/log: Add component/location decorations to dmesg Michal Wajdeczko
2026-07-28 19:16   ` Rodrigo Vivi
2026-07-28 16:10 ` [PATCH v2 05/22] drm/xe/log: Add SIGID log helpers for severity Michal Wajdeczko
2026-07-28 16:10 ` [PATCH v2 06/22] drm/xe/log: Add SIGID log helpers for location Michal Wajdeczko
2026-07-28 16:10 ` [PATCH v2 07/22] drm/xe/log: Add SIGID log helpers for location & severity Michal Wajdeczko
2026-07-28 16:10 ` [PATCH v2 08/22] drm/xe/log: Add SIGID log helpers for components Michal Wajdeczko
2026-07-29  8:06   ` Jani Nikula
2026-07-28 16:10 ` [PATCH v2 09/22] drm/xe/log: Add SIGID log helpers for errno-only Michal Wajdeczko
2026-07-28 16:10 ` [PATCH v2 10/22] drm/xe/log: Add hardware error signatures Michal Wajdeczko
2026-07-28 16:10 ` [PATCH v2 11/22] drm/xe/log: Extend components list with hardware items Michal Wajdeczko
2026-07-28 16:10 ` [PATCH v2 12/22] drm/xe/ras: Check RAS and LOG component definitions Michal Wajdeczko
2026-07-28 16:10 ` [PATCH v2 13/22] drm/xe/kunit: Setup driver data in the test device Michal Wajdeczko
2026-07-28 16:10 ` [PATCH v2 14/22] drm/xe/tests: Add Kunit tests for xe_log Michal Wajdeczko
2026-07-28 16:10 ` [PATCH v2 15/22] drm/xe: Report 'probe blocked' error using SIGID Michal Wajdeczko
2026-07-28 16:10 ` [PATCH v2 16/22] drm/xe: Report 'device wedged' errors " Michal Wajdeczko
2026-07-28 16:10 ` [PATCH v2 17/22] drm/xe: Report 'Survivability Mode' " Michal Wajdeczko
2026-07-28 16:10 ` [PATCH v2 18/22] drm/xe/guc: Report 'GuC mmio' " Michal Wajdeczko
2026-07-28 16:10 ` [PATCH v2 19/22] drm/xe/pcode: Report 'Mailbox failed' error " Michal Wajdeczko
2026-07-28 16:10 ` [PATCH v2 20/22] drm/xe/gt: Report 'reset failed' errors " Michal Wajdeczko
2026-07-28 16:10 ` [PATCH v2 21/22] drm/xe/gt: Report 'pagefault' " Michal Wajdeczko
2026-07-28 16:10 ` [PATCH v2 22/22] drm/xe/pci: Report 'cannot re-enable' error " Michal Wajdeczko
2026-07-28 16:17 ` ✗ CI.checkpatch: warning for drm/xe: Add structured SIGID error logging infrastructure (rev2) Patchwork
2026-07-28 16:18 ` ✓ CI.KUnit: success " Patchwork
2026-07-28 17:01 ` ✗ Xe.CI.BAT: failure " Patchwork
2026-07-28 20:57 ` ✗ Xe.CI.FULL: " Patchwork
2026-07-29  8:31 ` Jani Nikula [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=dfe82413d317aee8bd39dbbc45ece77f9b6d7610@intel.com \
    --to=jani.nikula@linux.intel.com \
    --cc=aravind.iddamsetty@intel.com \
    --cc=badal.nilawar@intel.com \
    --cc=intel-xe@lists.freedesktop.org \
    --cc=mallesh.koujalagi@intel.com \
    --cc=matthew.brost@intel.com \
    --cc=michal.wajdeczko@intel.com \
    --cc=raag.jadav@intel.com \
    --cc=riana.tauro@intel.com \
    --cc=rodrigo.vivi@intel.com \
    --cc=thomas.hellstrom@linux.intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox