Intel-XE Archive on lore.kernel.org
 help / color / mirror / Atom feed
From: Simona Vetter <simona.vetter@ffwll.ch>
To: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: "Miklos Szeredi" <miklos@szeredi.hu>,
	"Bernd Schubert" <bernd@bsbernd.com>,
	"Joanne Koong" <joannelkoong@gmail.com>,
	"Amir Goldstein" <amir73il@gmail.com>,
	intel-xe@lists.freedesktop.org, fuse-devel@lists.linux.dev,
	sashiko-bot@kernel.org, sashiko-reviews@lists.linux.dev,
	"David Airlie" <airlied@gmail.com>,
	"Simona Vetter" <simona@ffwll.ch>,
	"Srinivasan Shanmugam" <srinivasan.shanmugam@amd.com>,
	"Christian König" <christian.koenig@amd.com>,
	"Alex Deucher" <alexander.deucher@amd.com>,
	"Matthew Brost" <matthew.brost@intel.com>,
	"Thomas Hellström" <thomas.hellstrom@linux.intel.com>,
	dri-devel@lists.freedesktop.org,
	"Mika Kuoppala" <mika.kuoppala@linux.intel.com>
Subject: Re: FUSE deadlocks vs. copy_from_user() and locks (Was: Re: [PATCH v10 04/27] drm/xe/eudebug: Introduce discovery for resources)
Date: Wed, 9 Sep 2026 12:21:17 +0200	[thread overview]
Message-ID: <aqEzHQMrI8p9XVcu@phenom.ffwll.local> (raw)
In-Reply-To: <178894801687.37859.4186888279082797860@jlahtine-mobl>

On Wed, Sep 09, 2026 at 01:00:16PM +0300, Joonas Lahtinen wrote:
> (Adding bunch of FUSE folks and DRM folks in the hopes of coming to a
> resolution here.)
> 
> Quoting Joonas Lahtinen (2026-09-07 16:24:40)
> > Quoting sashiko-bot@kernel.org (2026-09-03 18:22:23)
> > > Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
> > > - [High] Global ordered workqueue stall via copy_from_user block while holding ioctl_lock.
> > > - [Medium] Event sequence numbers (seqno) can be queued out-of-order in the event stream due to a race between seqno generation and event queuing.
> > > --
> > > 
> > > commit e2061e72d8e954a17bde35e9c970ee41bc2db0ad
> > > Author: Mika Kuoppala <mika.kuoppala@linux.intel.com>
> > > 
> > >     drm/xe/eudebug: Introduce discovery for resources
> > > 
> > > This commit introduces a resource discovery mechanism for debugger connections.
> > > When a debugger connects, all existing resources for the target client are
> > > relayed as events via an ordered workqueue, and DRM ioctls are blocked using
> > > an rw_semaphore until discovery is complete.
> > > 
> > > > diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c
> > > > index 032a7ec5fbc8d..f10959b898297 100644
> > > > --- a/drivers/gpu/drm/xe/xe_device.c
> > > > +++ b/drivers/gpu/drm/xe/xe_device.c
> > > [ ... ]
> > > > @@ -238,8 +239,12 @@ static long xe_drm_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
> > > >  
> > > >       ACQUIRE(xe_pm_runtime_ioctl, pm)(xe);
> > > >       ret = ACQUIRE_ERR(xe_pm_runtime_ioctl, &pm);
> > > > -     if (ret >= 0)
> > > > +     if (ret >= 0) {
> > > > +             bool lock = xe_eudebug_discovery_lock(file, cmd);
> > > >               ret = drm_ioctl(file, cmd, arg);
> > > 
> > > [Severity: High]
> > > Does holding the discovery lock across drm_ioctl() create a vector for a
> > > workqueue stall? 
> > > 
> > > If an unprivileged user maps the ioctl argument pointer to a blocking FUSE
> > > or userfaultfd region, drm_ioctl() can be stalled indefinitely during
> > > copy_from_user() while holding xef->eudebug.ioctl_lock for read.
> > 
> > Don't think this is a very realistic vector to address, as it would
> > also extend to every other copy_from_user() and also to userptr across
> > all drivers.
> > 
> > Having a malfunctioning FUSE driver and getting a malfunctioning system
> > as a result is probably somewhat expected.
> 
> Based on further chatting on this with Sima, I was volunteered to pull
> together the discussion here.
> 
> We seem to have Sashiko picking up on patterns about accessing userspace
> memory with locks held and potential for copy_from_user() (or userptr) to
> then take indefinitely long to resolve. And that spreads to deadlocks
> everywhere situation very fast.
> 
> Based on reading of [1] and [2], it seems pretty much expected FUSE
> drivers can trivially deadlock and ultimately in worst case the situation
> can only be solved by manually aborting those connections by sysadmin.
> 
> It also seems (from the Sashiko comments) that by design, there's no
> upper bound for how long an operation can take, so a bad FUSE driver
> may stall for however long it sees fit to serve page-fault or in the
> case of [3] it may decide to not actually populate the PTEs (or maybe
> invalidate them immediately).
> 
> Should we really be refactoring the whole kernel for the sake of
> knowingly allowing potentially malicious userspace driver to idefinitely
> stall or incorrectly resolve page faults? That'll be quite a lot of
> complexity added to all the other drivers.
> 
> Or should there be more protections on FUSE / uffd to ensure such
> idefinitive stall can't happen? Or maybe this is just an academic
> problem and we amend review-prompts not to bring it up?
> 
> Or maybe I missed some part of the FUSE docs and this isn't a real
> problem?

Thanks for typing this up, matches what I think is going on here.

> Regards, Joonas
> 
> PS. There is a related patch in [3] which tries to address the problem,
> but we'll quickly run into live-locks and other issues even if we
> refactored things into: pre-fault, take locks, do _nofault() access, and
> retry if that fails.

Yeah just quickly wanting to add here that in my opinion, trying to sort
this out in all the various subsystem is not how we should even start to
think about this issue. This would be a fundamental change in how
subsystems are allowed to nest locking with stuff that can trigger
userspace faults.

I did ponder a bit how this could be solved on the fuse side of things,
maybe with some seccomp style filters. Like maybe lockdep could be
enlisted to help catch deadlocks, with a special "this is a fuse process,
it all defacto runs in fault handler context. But that only catches bugs
in normal use, not malicious exploits. And given that userspace can choose
the timing and unblock at will (I think so at least), this is pretty
powerful tool for being nasty to the kernel.

But mostly I want to really, really stand back in awe about this issue and
not think too hard about it.

Cheers, Sima

> [1] https://www.kernel.org/doc/html/next/filesystems/fuse.html#kernel-userspace-interface
> [2] https://www.kernel.org/doc/html/next/filesystems/fuse.html#aborting-a-filesystem-connection
> [3] https://sashiko.dev/#/patchset/20260827062142.4038272-1-srinivasan.shanmugam%40amd.com

-- 
Simona Vetter
Software Engineer
http://blog.ffwll.ch

  reply	other threads:[~2026-09-09 10:21 UTC|newest]

Thread overview: 61+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-09-03 14:59 [PATCH v10 00/27] Intel Xe GPU Debug Support (eudebug) v10 Mika Kuoppala
2026-09-03 14:59 ` [PATCH v10 01/27] drm/xe/eudebug: Introduce eudebug interface Mika Kuoppala
2026-09-03 15:16   ` sashiko-bot
2026-09-03 14:59 ` [PATCH v10 02/27] drm/xe/eudebug: Add documentation Mika Kuoppala
2026-09-03 14:59 ` [PATCH v10 03/27] drm/xe/eudebug: Add connection establishment documentation Mika Kuoppala
2026-09-03 14:59 ` [PATCH v10 04/27] drm/xe/eudebug: Introduce discovery for resources Mika Kuoppala
2026-09-03 15:22   ` sashiko-bot
2026-09-07 13:24     ` Joonas Lahtinen
2026-09-09 10:00       ` FUSE deadlocks vs. copy_from_user() and locks (Was: Re: [PATCH v10 04/27] drm/xe/eudebug: Introduce discovery for resources) Joonas Lahtinen
2026-09-09 10:21         ` Simona Vetter [this message]
2026-09-09 11:02           ` Christian König
2026-09-09 11:24             ` Joonas Lahtinen
2026-09-09 11:44               ` Miklos Szeredi
2026-09-09 13:03               ` Christian König
2026-09-09 14:51                 ` Joonas Lahtinen
2026-09-09 10:30         ` Miklos Szeredi
2026-09-03 14:59 ` [PATCH v10 05/27] drm/xe: Add EUDEBUG_ENABLE exec queue property Mika Kuoppala
2026-09-03 15:14   ` sashiko-bot
2026-09-03 14:59 ` [PATCH v10 06/27] drm/xe/eudebug: Introduce exec_queue events Mika Kuoppala
2026-09-03 14:59 ` [PATCH v10 07/27] drm/xe/eudebug: Mark guc contexts as debuggable Mika Kuoppala
2026-09-03 14:59 ` [PATCH v10 08/27] drm/xe: Remove ifdef in DRM_GPUVA_OP_DRIVER svm subop checking Mika Kuoppala
2026-09-03 14:59 ` [PATCH v10 09/27] drm/xe: Introduce ADD_DEBUG_DATA and REMOVE_DEBUG_DATA vm bind ops Mika Kuoppala
2026-09-03 15:22   ` sashiko-bot
2026-09-03 14:59 ` [PATCH v10 10/27] drm/xe/eudebug: Introduce vm bind and vm bind debug data events Mika Kuoppala
2026-09-03 15:26   ` sashiko-bot
2026-09-03 14:59 ` [PATCH v10 11/27] drm/xe/eudebug: Add ufence events with acks Mika Kuoppala
2026-09-03 15:20   ` sashiko-bot
2026-09-03 14:59 ` [PATCH v10 12/27] drm/xe/eudebug: Add vm open/pread/pwrite Mika Kuoppala
2026-09-03 15:27   ` sashiko-bot
2026-09-03 14:59 ` [PATCH v10 13/27] drm/xe/eudebug: Add userptr vm pread/pwrite Mika Kuoppala
2026-09-03 15:24   ` sashiko-bot
2026-09-03 14:59 ` [PATCH v10 14/27] drm/xe/eudebug: Add hw enablement Mika Kuoppala
2026-09-03 15:15   ` sashiko-bot
2026-09-03 14:59 ` [PATCH v10 15/27] drm/xe/eudebug: Introduce EU control interface Mika Kuoppala
2026-09-03 15:34   ` sashiko-bot
2026-09-03 14:59 ` [PATCH v10 16/27] drm/xe/eudebug: Introduce per device attention scan worker Mika Kuoppala
2026-09-03 14:59 ` [PATCH v10 17/27] drm/xe/eudebug_test: Introduce eudebug live tests Mika Kuoppala
2026-09-03 14:59 ` [PATCH v10 18/27] drm/xe: Implement SR-IOV and eudebug exclusivity Mika Kuoppala
2026-09-03 15:32   ` sashiko-bot
2026-09-03 14:59 ` [PATCH v10 19/27] drm/xe: Add xe_client_debugfs and introduce debug_data file Mika Kuoppala
2026-09-03 14:59 ` [PATCH v10 20/27] drm/xe/pagefault: export pagefault queue properties Mika Kuoppala
2026-09-03 14:59 ` [PATCH v10 21/27] drm/xe/eudebug: Add read/count/compare helper for eu attention Mika Kuoppala
2026-09-03 15:31   ` sashiko-bot
2026-09-03 14:59 ` [PATCH v10 22/27] drm/xe/vm: Support for adding null page VMA to VM on request Mika Kuoppala
2026-09-03 14:59 ` [PATCH v10 23/27] drm/xe/vm: Add xe_vm_svm_vma_subtract() to carve out a sub-range from an SVM VMA Mika Kuoppala
2026-09-03 14:59 ` [PATCH v10 24/27] drm/xe: Support for xe_vma_unbind() Mika Kuoppala
2026-09-03 14:59 ` [PATCH v10 25/27] drm/xe: export prep_vma_destroy as xe_vm_prep_vma_destroy Mika Kuoppala
2026-09-03 14:59 ` [PATCH v10 26/27] drm/xe/eudebug: Introduce EU pagefault handling interface Mika Kuoppala
2026-09-03 15:43   ` sashiko-bot
2026-09-08 15:12     ` Maciej Patelczyk
2026-09-03 14:59 ` [PATCH v10 27/27] drm/xe/eudebug: Enable EU pagefault handling Mika Kuoppala
2026-09-03 15:46   ` sashiko-bot
2026-09-08  9:35     ` Joonas Lahtinen
2026-09-08 15:28     ` Maciej Patelczyk
2026-09-03 15:35 ` ✗ CI.checkpatch: warning for Intel Xe GPU Debug Support (eudebug) v10 Patchwork
2026-09-03 15:37 ` ✓ CI.KUnit: success " Patchwork
2026-09-03 15:53 ` ✗ CI.checksparse: warning " Patchwork
2026-09-03 16:17 ` ✓ Xe.CI.BAT: success " Patchwork
2026-09-03 16:30 ` [PATCH v10 00/27] " Rodrigo Vivi
2026-09-07  5:32   ` Joonas Lahtinen
2026-09-04  3:21 ` ✗ Xe.CI.FULL: failure for " Patchwork

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=aqEzHQMrI8p9XVcu@phenom.ffwll.local \
    --to=simona.vetter@ffwll.ch \
    --cc=airlied@gmail.com \
    --cc=alexander.deucher@amd.com \
    --cc=amir73il@gmail.com \
    --cc=bernd@bsbernd.com \
    --cc=christian.koenig@amd.com \
    --cc=dri-devel@lists.freedesktop.org \
    --cc=fuse-devel@lists.linux.dev \
    --cc=intel-xe@lists.freedesktop.org \
    --cc=joannelkoong@gmail.com \
    --cc=joonas.lahtinen@linux.intel.com \
    --cc=matthew.brost@intel.com \
    --cc=mika.kuoppala@linux.intel.com \
    --cc=miklos@szeredi.hu \
    --cc=sashiko-bot@kernel.org \
    --cc=sashiko-reviews@lists.linux.dev \
    --cc=simona@ffwll.ch \
    --cc=srinivasan.shanmugam@amd.com \
    --cc=thomas.hellstrom@linux.intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox