Re: [PATCH 0/7] Fix a couple of wedge corner-case memory leaks

["Summers, Stuart" <stuart.summers@intel.com>]

On Mon, 2025-10-13 at 17:13 +0000, Summers, Stuart wrote:
> On Mon, 2025-10-13 at 10:04 -0700, Matthew Brost wrote:
> > On Mon, Oct 13, 2025 at 04:24:57PM +0000, Stuart Summers wrote:
> > > Most of the patches in this series are just adding
> > > some debug hints to help track these down. I split
> > > these up in case we want to pick and choose which ones
> > > to include in the tree. I found them useful.
> > > 
> > > The main two interesting patches are the last two in the
> > > series which are fixing some corner cases when the
> > > driver becomes wedged in the middle of either communication
> > > with the DRM scheduler or in the event the GuC becomes
> > > unresponsive. In both of these cases there is a chance
> > > we could leak memory around the exec queue members
> > > like the LRC and the LRC BO. These patches fix those
> > > scenarios.
> > > 
> > 
> > Ok, I think I see the problem. I believe the correct approach is:
> > 
> > - Apply [1]
> 
> I get the need for [1], but I don't think this is going to solve the
> issue I'm facing specifically since, in my case, GuC is alive at the
> time of schedule disable and becomes unresponsive only after that H2G
> is sent.
> 
> Basically I think there are a few things going on here.
> 
> I'm pulling [1] now though and I'll test and get back.

Yeah unfortunately this doesn't cover the case where we can submit to
GuC (GuC is alive on deregistration/schedule disable) but then GuC
stops responding before we get any of those responses back. We just
aren't tracking that. I still think the second patch in this series is
needed for that reason.

> 
> > 
> > - Ensure all schedulers are not stopped in guc_submit_wedged_fini
> > 
> > - Clean up any lost H2G in guc_submit_wedged_fini similar to a GT
> > reset
> > 
> > - Wait on all async scheduler work queue opertaions complete
> > somewhere
> >   (this is part of VLK-80263, I was going to try to look at this
> > part
> >   sometime this week). 
> 
> Oh thanks for the link here too! I'll see if this fits in to my
> scenario as well.

I did try adding a guc_sanitize() into the wedge_fini() function and it
isn't helping. I'm still debugging here though. I agree it seems like
we could use something there.

w.r.t. vlk-80263, we do already have a 5s wait on all exec queues to
clear there. I think this isn't the only issue though, although I
haven't looked through your vf case specifically there.

Thanks,
Stuart

> 
> Thanks,
> Stuart
> 
> > 
> > Matt
> > 
> > [1] https://patchwork.freedesktop.org/series/155417/
> > 
> > > v2: Address feedback from Matt:
> > >     - Let the DRM scheduler handle pausing/unpausing
> > >     - Still do the wait after scheduling disable/deregister
> > >       as with the previous patch, but skip the intermediate
> > >       software-based schedule disable using the "banned"
> > >       flag and instead just jump straight to the deregister
> > >       handling which will fully reset the queue state.
> > >       Note that for this case I am seeing a hardware failure
> > >       after submitting to GuC but before receiving the
> > >       response from GuC. So even if we wedge in this case
> > >       (monitoring the hardware state change), the queue
> > >       itself is not wedged because of the active GuC
> > >       submission (CT is not stalled at that point).
> > > 
> > > Stuart Summers (7):
> > >   drm/xe: Add additional trace points for LRCs
> > >   drm/xe: Add a trace point for VM close
> > >   drm/xe: Add the BO pointer info to the BO trace
> > >   drm/xe: Add new exec queue trace points
> > >   drm/xe: Correct migration VM teardown order
> > >   drm/xe: Don't block messages to the GPU scheduler
> > >   drm/xe: Check for GuC responses on disabling scheduling
> > > 
> > >  drivers/gpu/drm/xe/xe_exec_queue.c    |  4 +++
> > >  drivers/gpu/drm/xe/xe_gpu_scheduler.c |  6 +---
> > >  drivers/gpu/drm/xe/xe_guc_submit.c    | 24 ++++++++++++---
> > >  drivers/gpu/drm/xe/xe_lrc.c           |  4 +++
> > >  drivers/gpu/drm/xe/xe_lrc.h           |  3 ++
> > >  drivers/gpu/drm/xe/xe_migrate.c       |  2 +-
> > >  drivers/gpu/drm/xe/xe_trace.h         | 22 ++++++++++++--
> > >  drivers/gpu/drm/xe/xe_trace_bo.h      | 12 ++++++--
> > >  drivers/gpu/drm/xe/xe_trace_lrc.h     | 42
> > > ++++++++++++++++++++++++++-
> > >  drivers/gpu/drm/xe/xe_vm.c            |  2 ++
> > >  10 files changed, 106 insertions(+), 15 deletions(-)
> > > 
> > > -- 
> > > 2.34.1
> > > 
>