From: Mike Snitzer <snitzer@kernel.org>
To: Yongcheng Yang <yoyang@redhat.com>
Cc: Trond Myklebust <trondmy@kernel.org>, linux-nfs@vger.kernel.org
Subject: Re: [Bug report] xfstests generic/323 over NFS hit BUG: KASAN: slab-use-after-free in nfs_local_call_read on 6.18.0-rc1
Date: Mon, 20 Oct 2025 14:24:52 -0400 [thread overview]
Message-ID: <aPZ-dIObXH8Z06la@kernel.org> (raw)
In-Reply-To: <aPURMSaH1rXQJkdj@kernel.org>
On Sun, Oct 19, 2025 at 12:26:25PM -0400, Mike Snitzer wrote:
> On Sun, Oct 19, 2025 at 11:18:57AM -0400, Trond Myklebust wrote:
> > On Sun, 2025-10-19 at 17:29 +0800, Yongcheng Yang wrote:
> > > Hi All,
> > >
> > > There is a new nfs slab-use-after-free issue since 6.18.0-rc1.
> > > It appears to be reliably reproducible on my side when running
> > > xfstests
> > > generic/323 over NFSv4.2 in *debug* kernel mode:
> >
> > Thanks for the report! I think I see the problem.
> >
> > Mike,
> >
> > When you iterate over the iocb in nfs_local_call_read(), you're calling
> > nfs_local_pgio_done(), nfs_local_read_done() and
> > nfs_local_pgio_release() multiple times.
>
> I purposely made nfs_local_pgio_done() safe to call multiple times.
>
> And nfs_local_{read,write}_done() and nfs_local_pgio_release()
> _should_ only be called once.
>
> > * You're calling nfs_local_read_aio_complete() and
> > nfs_local_read_aio_complete_work() once for each and every
> > asynchronous call.
>
> There is only the possibility of a single async call for the single
> aligned DIO.
>
> For any given pgio entering LOCALIO, it may be split into 3 pieces:
> The misaligned head and tail are first handled sync and only then the
> aligned middle async (or possibly sync if underlying device imposes
> sync, e.g. ramdisk).
>
> > * You're calling nfs_local_pgio_done() for each synchronous call.
>
> Yes, which is safe. It just updates status, deals with partial
> completion.
>
> > * In addition, if there is a synchronous call at the very end of the
> > iteration, so that status != -EIOCBQUEUED, then you're also calling
> > nfs_local_read_done() one extra time, and then calling
> > nfs_local_pgio_release().
>
> It isn't in addition, its only for the last piece of IO (be it sync or
> async).
>
> > The same thing appears to be happening in nfs_local_call_write().
>
> I fully acknolwdge this isn't an easy audit. And there could be
> something wrong. But I'm not seeing it. Obviously this BUG report
> puts onus on me to figure it out...
>
> BUT, I have used this code extensively on non-debug and had no issues.
> Is it at all possible KASAN is triggering a false-positive!?
I haven't been able to reproduce this (NFS LOCALIO and KASAN is
enabled):
[root@snitzer xfstests-dev]# cat local.config
export TEST_DIR="/mnt/share1"
export TEST_DEV="10.200.111.104:/share1"
export SCRATCH_MNT="/mnt/scratch"
export SCRATCH_DEV="10.200.111.104:/"
export TEST_FS_MOUNT_OPTS="-overs=4.2,sec=sys,acl,nconnect=5"
[root@snitzer xfstests-dev]# ./check -nfs generic/323
FSTYP -- nfs
PLATFORM -- Linux/x86_64 snitzer 6.12.53.1.hs.snitm+ #75 SMP PREEMPT_DYNAMIC Fri Oct 17 03:55:21 UTC 2025
MKFS_OPTIONS -- 10.200.111.104:/
MOUNT_OPTIONS -- 10.200.111.104:/ /mnt/scratch
generic/323 121s
Ran: generic/323
Passed all 1 tests
My kernel is 6.12-stable based, but includes all NFS and NFSD changes
through 6.18-rc1 (and also most of chuck's nfsd-testing), see:
https://git.kernel.org/pub/scm/linux/kernel/git/snitzer/linux.git/log/?h=kernel-6.12.53/main
Please provide your .config (off-list is fine!) and I'll see if I'm
somehow missing something.
(I suppose it could be that by test system is too slow...)
Thanks,
Mike
next prev parent reply other threads:[~2025-10-20 18:24 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-10-19 9:29 [Bug report] xfstests generic/323 over NFS hit BUG: KASAN: slab-use-after-free in nfs_local_call_read on 6.18.0-rc1 Yongcheng Yang
2025-10-19 15:18 ` Trond Myklebust
2025-10-19 16:26 ` Mike Snitzer
2025-10-20 18:24 ` Mike Snitzer [this message]
2025-10-27 13:08 ` [v6.18-rcX PATCH 0/3] nfs/localio: fixes for recent misaligned DIO changes Mike Snitzer
2025-10-27 13:08 ` [v6.18-rcX PATCH 1/3] nfs/localio: remove unecessary ENOTBLK handling in DIO WRITE support Mike Snitzer
2025-10-27 13:08 ` [v6.18-rcX PATCH 2/3] nfs/localio: add refcounting for each iocb IO associated with NFS pgio header Mike Snitzer
2025-10-27 13:19 ` Christoph Hellwig
2025-10-27 13:55 ` Mike Snitzer
2025-10-27 14:45 ` Christoph Hellwig
2025-10-27 13:08 ` [v6.18-rcX PATCH 3/3] nfs/localio: backfill missing partial read support for misaligned DIO Mike Snitzer
2025-10-27 17:52 ` [v6.18-rcX PATCH 4/3] nfs/localio: Ensure DIO WRITE's IO on stable storage upon completion Mike Snitzer
2025-10-29 23:19 ` [v6.18-rcX PATCH 5/3] nfs/localio: do not issue misaligned DIO out-of-order Mike Snitzer
2025-10-31 1:50 ` Mike Snitzer
2025-10-31 13:33 ` Anna Schumaker
2025-11-04 18:02 ` [v6.18-rcX PATCH v2] " Mike Snitzer
2025-11-06 2:50 ` Mike Snitzer
2025-11-06 3:03 ` [v6.18-rcX PATCH v3 5/3] " Mike Snitzer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=aPZ-dIObXH8Z06la@kernel.org \
--to=snitzer@kernel.org \
--cc=linux-nfs@vger.kernel.org \
--cc=trondmy@kernel.org \
--cc=yoyang@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).