linux-nfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Mike Snitzer <snitzer@kernel.org>
To: Yongcheng Yang <yoyang@redhat.com>
Cc: Trond Myklebust <trondmy@kernel.org>, linux-nfs@vger.kernel.org
Subject: Re: [Bug report] xfstests generic/323 over NFS hit BUG: KASAN: slab-use-after-free in nfs_local_call_read on 6.18.0-rc1
Date: Mon, 20 Oct 2025 14:24:52 -0400	[thread overview]
Message-ID: <aPZ-dIObXH8Z06la@kernel.org> (raw)
In-Reply-To: <aPURMSaH1rXQJkdj@kernel.org>

On Sun, Oct 19, 2025 at 12:26:25PM -0400, Mike Snitzer wrote:
> On Sun, Oct 19, 2025 at 11:18:57AM -0400, Trond Myklebust wrote:
> > On Sun, 2025-10-19 at 17:29 +0800, Yongcheng Yang wrote:
> > > Hi All,
> > > 
> > > There is a new nfs slab-use-after-free issue since 6.18.0-rc1.
> > > It appears to be reliably reproducible on my side when running
> > > xfstests
> > > generic/323 over NFSv4.2 in *debug* kernel mode:
> > 
> > Thanks for the report! I think I see the problem.
> > 
> > Mike,
> > 
> > When you iterate over the iocb in nfs_local_call_read(), you're calling
> > nfs_local_pgio_done(), nfs_local_read_done() and
> > nfs_local_pgio_release() multiple times.
> 
> I purposely made nfs_local_pgio_done() safe to call multiple times.
> 
> And nfs_local_{read,write}_done() and nfs_local_pgio_release()
> _should_ only be called once.
> 
> >  * You're calling nfs_local_read_aio_complete() and
> >    nfs_local_read_aio_complete_work() once for each and every
> >    asynchronous call.
> 
> There is only the possibility of a single async call for the single
> aligned DIO.
> 
> For any given pgio entering LOCALIO, it may be split into 3 pieces:
> The misaligned head and tail are first handled sync and only then the
> aligned middle async (or possibly sync if underlying device imposes
> sync, e.g. ramdisk).
> 
> >  * You're calling nfs_local_pgio_done() for each synchronous call.
> 
> Yes, which is safe.  It just updates status, deals with partial
> completion.
> 
> >  * In addition, if there is a synchronous call at the very end of the
> >    iteration, so that status != -EIOCBQUEUED, then you're also calling
> >    nfs_local_read_done() one extra time, and then calling
> >    nfs_local_pgio_release().
> 
> It isn't in addition, its only for the last piece of IO (be it sync or
> async).
> 
> > The same thing appears to be happening in nfs_local_call_write().
> 
> I fully acknolwdge this isn't an easy audit.  And there could be
> something wrong.  But I'm not seeing it.  Obviously this BUG report
> puts onus on me to figure it out...
> 
> BUT, I have used this code extensively on non-debug and had no issues.
> Is it at all possible KASAN is triggering a false-positive!?

I haven't been able to reproduce this (NFS LOCALIO and KASAN is
enabled):

[root@snitzer xfstests-dev]# cat local.config
export TEST_DIR="/mnt/share1"
export TEST_DEV="10.200.111.104:/share1"
export SCRATCH_MNT="/mnt/scratch"
export SCRATCH_DEV="10.200.111.104:/"
export TEST_FS_MOUNT_OPTS="-overs=4.2,sec=sys,acl,nconnect=5"

[root@snitzer xfstests-dev]# ./check -nfs generic/323
FSTYP         -- nfs
PLATFORM      -- Linux/x86_64 snitzer 6.12.53.1.hs.snitm+ #75 SMP PREEMPT_DYNAMIC Fri Oct 17 03:55:21 UTC 2025
MKFS_OPTIONS  -- 10.200.111.104:/
MOUNT_OPTIONS -- 10.200.111.104:/ /mnt/scratch

generic/323        121s
Ran: generic/323
Passed all 1 tests

My kernel is 6.12-stable based, but includes all NFS and NFSD changes
through 6.18-rc1 (and also most of chuck's nfsd-testing), see:
https://git.kernel.org/pub/scm/linux/kernel/git/snitzer/linux.git/log/?h=kernel-6.12.53/main

Please provide your .config (off-list is fine!) and I'll see if I'm
somehow missing something.

(I suppose it could be that by test system is too slow...)

Thanks,
Mike

  reply	other threads:[~2025-10-20 18:24 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-10-19  9:29 [Bug report] xfstests generic/323 over NFS hit BUG: KASAN: slab-use-after-free in nfs_local_call_read on 6.18.0-rc1 Yongcheng Yang
2025-10-19 15:18 ` Trond Myklebust
2025-10-19 16:26   ` Mike Snitzer
2025-10-20 18:24     ` Mike Snitzer [this message]
2025-10-27 13:08       ` [v6.18-rcX PATCH 0/3] nfs/localio: fixes for recent misaligned DIO changes Mike Snitzer
2025-10-27 13:08       ` [v6.18-rcX PATCH 1/3] nfs/localio: remove unecessary ENOTBLK handling in DIO WRITE support Mike Snitzer
2025-10-27 13:08       ` [v6.18-rcX PATCH 2/3] nfs/localio: add refcounting for each iocb IO associated with NFS pgio header Mike Snitzer
2025-10-27 13:19         ` Christoph Hellwig
2025-10-27 13:55           ` Mike Snitzer
2025-10-27 14:45             ` Christoph Hellwig
2025-10-27 13:08       ` [v6.18-rcX PATCH 3/3] nfs/localio: backfill missing partial read support for misaligned DIO Mike Snitzer
2025-10-27 17:52       ` [v6.18-rcX PATCH 4/3] nfs/localio: Ensure DIO WRITE's IO on stable storage upon completion Mike Snitzer
2025-10-29 23:19         ` [v6.18-rcX PATCH 5/3] nfs/localio: do not issue misaligned DIO out-of-order Mike Snitzer
2025-10-31  1:50           ` Mike Snitzer
2025-10-31 13:33             ` Anna Schumaker
2025-11-04 18:02               ` [v6.18-rcX PATCH v2] " Mike Snitzer
2025-11-06  2:50                 ` Mike Snitzer
2025-11-06  3:03                   ` [v6.18-rcX PATCH v3 5/3] " Mike Snitzer

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=aPZ-dIObXH8Z06la@kernel.org \
    --to=snitzer@kernel.org \
    --cc=linux-nfs@vger.kernel.org \
    --cc=trondmy@kernel.org \
    --cc=yoyang@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).