Intel-XE Archive on lore.kernel.org
 help / color / mirror / Atom feed
From: Matthew Brost <matthew.brost@intel.com>
To: "Tales A. Mendonça" <talesam@gmail.com>
Cc: <intel-xe@lists.freedesktop.org>,
	<thomas.hellstrom@linux.intel.com>, <rodrigo.vivi@intel.com>,
	<dri-devel@lists.freedesktop.org>
Subject: Re: [RFC PATCH 0/3] drm/xe: diagnostics and workaround for GuC TLB invalidation ack stalls on ARL
Date: Wed, 5 Aug 2026 13:24:58 -0700	[thread overview]
Message-ID: <anOcGqIyrA8LUHo/@gsse-cloud1.jf.intel.com> (raw)
In-Reply-To: <anJZt4PzCE5t49Qg@gsse-cloud1.jf.intel.com>

On Tue, Aug 04, 2026 at 02:29:27PM -0700, Matthew Brost wrote:
> On Tue, Aug 04, 2026 at 01:34:19PM -0300, Tales A. Mendonça wrote:
> > Patchwork reports my address is not on the CI allowlist, so CI was not
> > triggered for this series:
> > 
> >     Series author address 'talesam@gmail.com' is not on the allowlist,
> > which prevents CI from being automatically triggered.
> > 
> >   Could one of the project owners click 'retest' on the series (and/or
> > add me to the allowlist)? Series URL:
> > 
> >     https://patchwork.freedesktop.org/series/171539/
> > 
> 
> We'd have to resend this ourselves. I can do this, but I've requested
> for you to be on our allow list as well. I'll ping here once that goes
> through.
> 

You are approved on our CI for future patches.

Also I came across this issue in the i915 for MTL [1] which seems to
indicate the same issue (ARL and MTL are very close and share same GuC
firmware), the suggestion there is turn off rc6. Unfortunately Xe
doesn't have a knob to do this but according to [1] you can turn off rc6
in the BIOS. Might be worth a try.

Matt

[1] https://gitlab.freedesktop.org/drm/i915/kernel/-/work_items/14469

> Matt 
> 
> >   Thanks!
> >   Tales
> > 
> > 
> > Em seg., 3 de ago. de 2026 às 23:14, Tales A. Mendonça
> > <talesam@gmail.com> escreveu:
> > >
> > > Hi,
> > >
> > > This series is a follow-up to the TLB invalidation ack stall I have
> > > been debugging on ARL, tracked in:
> > >
> > >   https://gitlab.freedesktop.org/drm/xe/kernel/-/work_items/8678
> > >
> > > Summary of the issue: with GuC 70.53.0 on ARL (reproduced on 7d51 and
> > > 7dd1 machines here, plus an independent Arc Pro 130T report on the
> > > issue above), TLB invalidation acks intermittently stall for ~2.3s.
> > > The H2G request is consumed from the CTB immediately and the G2H CTB
> > > is empty the whole time - the firmware simply does not send the ack
> > > until much later. The fence timeout fires at 2.25s and the ack lands
> > > tens of ms after it. Userspace blocked on the invalidation (compositor
> > > buffer unmaps etc.) hitches for the full window.
> > >
> > > Patch 1 adds xe_devcoredump_gt() so this kind of hang - which has no
> > > exec queue or job to blame - leaves a devcoredump with the GuC log and
> > > CT state behind (Matt suggested capturing devcoredumps when we
> > > discussed the issue; devcoredumps from both machines are attached to
> > > the issue above).
> > >
> > > Patch 2 logs when the ack for a timed out invalidation finally
> > > arrives. This is what established that the acks are late rather than
> > > lost.
> > >
> > > Patch 3 is the RFC part: a delayed work that pokes the GuC (status
> > > register read, CT flush, doorbell ring) every 250ms while an ack is
> > > overdue. On my machines this converts the guaranteed 2.3s stall into a
> > > sub-500ms hiccup for the majority of occurrences; a minority of severe
> > > episodes ignore 8-9 consecutive doorbells, which points at the GuC
> > > firmware being internally blocked for the whole window. Full data on
> > > the issue. I am happy to rework the approach (different delay,
> > > tying it to the G2H handler, dropping the status read, etc.) - mainly
> > > I would like the firmware side investigated, since no host-side poke
> > > can fix the severe cases.
> > >
> > > Based on drm-tip. Tested for several days on both ARL machines under
> > > desktop and VM-heavy workloads.
> > >
> > > Thanks,
> > > Tales
> > >
> > > Tales A. Mendonça (3):
> > >   drm/xe: Capture devcoredump on TLB invalidation timeout
> > >   drm/xe: Log when a timed out TLB invalidation ack finally arrives
> > >   drm/xe: Kick GuC while TLB invalidation acks are overdue
> > >
> > >  drivers/gpu/drm/xe/xe_devcoredump.c     |  68 ++++++++++++
> > >  drivers/gpu/drm/xe/xe_devcoredump.h     |   6 ++
> > >  drivers/gpu/drm/xe/xe_tlb_inval.c       | 131 +++++++++++++++++++++++-
> > >  drivers/gpu/drm/xe/xe_tlb_inval_types.h |  42 ++++++++
> > >  4 files changed, 243 insertions(+), 4 deletions(-)
> > >
> > > --
> > > 2.55.0
> > >
> > 
> > 
> > -- 
> > Com os cumprimentos,
> > 
> > Tales A. Mendonça
> > talesam.org
> > communitybig.org

  reply	other threads:[~2026-08-05 20:25 UTC|newest]

Thread overview: 25+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-08-04  2:14 [RFC PATCH 0/3] drm/xe: diagnostics and workaround for GuC TLB invalidation ack stalls on ARL Tales A. Mendonça
2026-08-04  2:14 ` [RFC PATCH 1/3] drm/xe: Capture devcoredump on TLB invalidation timeout Tales A. Mendonça
2026-08-04 22:05   ` Matthew Brost
2026-08-04  2:14 ` [RFC PATCH 2/3] drm/xe: Log when a timed out TLB invalidation ack finally arrives Tales A. Mendonça
2026-08-04 22:17   ` Matthew Brost
2026-08-04  2:14 ` [RFC PATCH 3/3] drm/xe: Kick GuC while TLB invalidation acks are overdue Tales A. Mendonça
2026-08-04  2:15 ` ✗ LGCI.VerificationFailed: failure for drm/xe: diagnostics and workaround for GuC TLB invalidation ack stalls on ARL Patchwork
2026-08-04 16:34 ` [RFC PATCH 0/3] " Tales A. Mendonça
2026-08-04 21:29   ` Matthew Brost
2026-08-05 20:24     ` Matthew Brost [this message]
2026-08-06 17:37       ` Tales A. Mendonça
2026-08-06 17:56         ` Daniele Ceraolo Spurio
2026-08-04 21:02 ` Summers, Stuart
2026-08-04 21:27   ` Matthew Brost
2026-08-04 21:33     ` Summers, Stuart
2026-08-04 22:08       ` Daniele Ceraolo Spurio
2026-08-04 23:00         ` Tales A. Mendonça
2026-08-04 23:50           ` Daniele Ceraolo Spurio
2026-08-06 17:36             ` Tales A. Mendonça
2026-08-06 21:13               ` Daniele Ceraolo Spurio
2026-08-08  0:20                 ` Tales A. Mendonça
2026-08-05 12:32 ` ✗ CI.checkpatch: warning for drm/xe: diagnostics and workaround for GuC TLB invalidation ack stalls on ARL (rev2) Patchwork
2026-08-05 12:34 ` ✓ CI.KUnit: success " Patchwork
2026-08-05 13:11 ` ✗ Xe.CI.BAT: failure " Patchwork
2026-08-05 23:39 ` ✗ Xe.CI.FULL: " Patchwork

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=anOcGqIyrA8LUHo/@gsse-cloud1.jf.intel.com \
    --to=matthew.brost@intel.com \
    --cc=dri-devel@lists.freedesktop.org \
    --cc=intel-xe@lists.freedesktop.org \
    --cc=rodrigo.vivi@intel.com \
    --cc=talesam@gmail.com \
    --cc=thomas.hellstrom@linux.intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox