From: "Darrick J. Wong" <djwong@kernel.org>
To: John Garry <john.g.garry@oracle.com>
Cc: Carlos Maiolino <cem@kernel.org>,
Ojaswin Mujoo <ojaswin@linux.ibm.com>,
Zorro Lang <zlang@redhat.com>,
fstests@vger.kernel.org, Ritesh Harjani <ritesh.list@gmail.com>,
linux-xfs@vger.kernel.org
Subject: Re: [PATCH] xfs: fix write failures in software-provided atomic writes
Date: Thu, 30 Oct 2025 08:01:38 -0700 [thread overview]
Message-ID: <20251030150138.GW4015566@frogsfrogsfrogs> (raw)
In-Reply-To: <02af7e21-1a0f-4035-b2d1-b96c9db2f5c7@oracle.com>
On Thu, Oct 30, 2025 at 01:52:46PM +0000, John Garry wrote:
> On 29/10/2025 18:11, Darrick J. Wong wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> >
> > With the 5 Oct 2025 release of fstests, generic/521 fails for me on
> > regular (aka non-block-atomic-writes) storage:
> >
> > QA output created by 521
> > dowrite: write: Input/output error
> > LOG DUMP (8553 total operations):
> > 1( 1 mod 256): SKIPPED (no operation)
> > 2( 2 mod 256): WRITE 0x7e000 thru 0x8dfff (0x10000 bytes) HOLE
> > 3( 3 mod 256): READ 0x69000 thru 0x79fff (0x11000 bytes)
> > 4( 4 mod 256): FALLOC 0x53c38 thru 0x5e853 (0xac1b bytes) INTERIOR
> > 5( 5 mod 256): COPY 0x55000 thru 0x59fff (0x5000 bytes) to 0x25000 thru 0x29fff
> > 6( 6 mod 256): WRITE 0x74000 thru 0x88fff (0x15000 bytes)
> > 7( 7 mod 256): ZERO 0xedb1 thru 0x11693 (0x28e3 bytes)
> > <snip>
> >
> > with a warning in dmesg from iomap about XFS trying to give it a
> > delalloc mapping for a directio write. Fix the software atomic write
> > iomap_begin code to convert the reservation into a written mapping.
> > This doesn't fix the data corruption problems reported by generic/760,
> > but it's a start.
>
> I was seeing the corruption and, as expected, unfortunately this does not
> fix the issue. Indeed, I don't even touch the new codepath when testing (for
> that corruption).
Yeah, I know. This fix enables me to move on to what I think is the
corruption that you and Ojaswin are seeing.
> As for that corruption, I am seeing the same behaviour as Ojaswin described.
> The failure is in a read operation.
>
> It seems to be a special combo of atomic write, write, and then read which
> reliably shows the issue. The regular write seems to write to the cow fork,
> so I am guessing that the atomic write does not leave it in proper state.
>
> I do notice for the atomic write that we are writing (calling
> xfs_atomic_write_cow_iomap_begin() -> xfs_bmapi_write()) for more blocks
> that are required for the atomic write. The regular write overwrites these
> blocks, and the read is corrupted in the blocks just after the atomic write.
> It's as if the blocks just after atomic write are not left in the proper
> state.
That's a good breadcrumb for me to follow; I will turn on the rmap
tracepoints to see if they give me a better idea of what's going on.
I mentioned earlier that I think the problem could be that iomap treats
srcmap::type == IOMAP_HOLE as if the srcmap isn't there, and so it'll
read from the cow fork blocks even though that's not right.
--D
> >
> > Cc: <stable@vger.kernel.org> # v6.16
> > Fixes: bd1d2c21d5d249 ("xfs: add xfs_atomic_write_cow_iomap_begin()")
> > Signed-off-by: "Darrick J. Wong" <djwong@kernel.org>
> > ---
> > fs/xfs/xfs_iomap.c | 21 +++++++++++++++++++--
> > 1 file changed, 19 insertions(+), 2 deletions(-)
> >
> > diff --git a/fs/xfs/xfs_iomap.c b/fs/xfs/xfs_iomap.c
> > index d3f6e3e42a1191..e1da06b157cf94 100644
> > --- a/fs/xfs/xfs_iomap.c
> > +++ b/fs/xfs/xfs_iomap.c
> > @@ -1130,7 +1130,7 @@ xfs_atomic_write_cow_iomap_begin(
> > return -EAGAIN;
> > trace_xfs_iomap_atomic_write_cow(ip, offset, length);
> > -
> > +retry:
> > xfs_ilock(ip, XFS_ILOCK_EXCL);
> > if (!ip->i_cowfp) {
> > @@ -1141,6 +1141,8 @@ xfs_atomic_write_cow_iomap_begin(
> > if (!xfs_iext_lookup_extent(ip, ip->i_cowfp, offset_fsb, &icur, &cmap))
> > cmap.br_startoff = end_fsb;
> > if (cmap.br_startoff <= offset_fsb) {
> > + if (isnullstartblock(cmap.br_startblock))
> > + goto convert;
> > xfs_trim_extent(&cmap, offset_fsb, count_fsb);
> > goto found;
> > }
> > @@ -1169,8 +1171,10 @@ xfs_atomic_write_cow_iomap_begin(
> > if (!xfs_iext_lookup_extent(ip, ip->i_cowfp, offset_fsb, &icur, &cmap))
> > cmap.br_startoff = end_fsb;
> > if (cmap.br_startoff <= offset_fsb) {
> > - xfs_trim_extent(&cmap, offset_fsb, count_fsb);
> > xfs_trans_cancel(tp);
> > + if (isnullstartblock(cmap.br_startblock))
> > + goto convert;
> > + xfs_trim_extent(&cmap, offset_fsb, count_fsb);
> > goto found;
> > }
> > @@ -1210,6 +1214,19 @@ xfs_atomic_write_cow_iomap_begin(
> > xfs_iunlock(ip, XFS_ILOCK_EXCL);
> > return xfs_bmbt_to_iomap(ip, iomap, &cmap, flags, IOMAP_F_SHARED, seq);
> > +convert:
> > + xfs_iunlock(ip, XFS_ILOCK_EXCL);
> > + error = xfs_bmapi_convert_delalloc(ip, XFS_COW_FORK, offset, iomap,
> > + NULL);
> > + if (error)
> > + return error;
> > +
> > + /*
> > + * Try the lookup again, because the delalloc conversion might have
> > + * turned the COW mapping into unwritten, but we need it to be in
> > + * written state.
> > + */
> > + goto retry;
> > out_unlock:
> > xfs_iunlock(ip, XFS_ILOCK_EXCL);
> > return error;
>
next prev parent reply other threads:[~2025-10-30 15:01 UTC|newest]
Thread overview: 49+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-09-19 6:47 [PATCH v7 00/11] Add more tests for multi fs block atomic writes Ojaswin Mujoo
2025-09-19 6:47 ` [PATCH v7 01/12] common/rc: Add _min() and _max() helpers Ojaswin Mujoo
2025-09-19 6:47 ` [PATCH v7 02/12] common/rc: Add fio atomic write helpers Ojaswin Mujoo
2025-09-19 16:27 ` Darrick J. Wong
2025-09-19 6:47 ` [PATCH v7 03/12] common/rc: Add a helper to run fsx on a given file Ojaswin Mujoo
2025-09-19 6:47 ` [PATCH v7 04/12] ltp/fsx.c: Add atomic writes support to fsx Ojaswin Mujoo
2025-09-28 8:55 ` Zorro Lang
2025-09-28 13:19 ` Zorro Lang
2025-10-02 17:56 ` Ojaswin Mujoo
2025-10-03 17:19 ` Zorro Lang
2025-10-05 12:57 ` Ojaswin Mujoo
2025-10-05 15:39 ` Zorro Lang
2025-10-06 13:20 ` Ojaswin Mujoo
2025-10-07 9:58 ` Ojaswin Mujoo
2025-10-17 16:01 ` Zorro Lang
2025-10-17 16:27 ` Darrick J. Wong
2025-10-17 18:47 ` Zorro Lang
2025-10-17 22:52 ` Darrick J. Wong
2025-10-20 10:33 ` John Garry
2025-10-21 10:28 ` Ojaswin Mujoo
2025-10-21 11:30 ` Brian Foster
2025-10-21 11:58 ` Ojaswin Mujoo
2025-10-21 17:44 ` Darrick J. Wong
2025-10-22 7:40 ` Ojaswin Mujoo
2025-10-23 15:44 ` John Garry
2025-10-23 17:55 ` Darrick J. Wong
2025-10-29 18:11 ` [PATCH] xfs: fix write failures in software-provided atomic writes Darrick J. Wong
2025-10-29 18:13 ` Darrick J. Wong
2025-10-30 13:52 ` John Garry
2025-10-30 15:01 ` Darrick J. Wong [this message]
2025-10-30 16:35 ` John Garry
2025-10-30 19:38 ` John Garry
2025-10-31 4:30 ` Darrick J. Wong
2025-10-31 10:17 ` John Garry
2025-10-31 17:13 ` Darrick J. Wong
2025-11-03 12:16 ` John Garry
2025-11-03 18:01 ` Darrick J. Wong
2025-10-31 8:08 ` Ojaswin Mujoo
2025-10-31 10:04 ` John Garry
2025-09-19 6:47 ` [PATCH v7 05/12] generic: Add atomic write test using fio crc check verifier Ojaswin Mujoo
2025-10-28 9:42 ` Ojaswin Mujoo
2025-11-01 9:00 ` Zorro Lang
2025-09-19 6:47 ` [PATCH v7 06/12] generic: Add atomic write test using fio verify on file mixed mappings Ojaswin Mujoo
2025-09-19 6:48 ` [PATCH v7 07/12] generic: Add atomic write multi-fsblock O_[D]SYNC tests Ojaswin Mujoo
2025-09-19 6:48 ` [PATCH v7 08/12] generic: Stress fsx with atomic writes enabled Ojaswin Mujoo
2025-09-19 6:48 ` [PATCH v7 09/12] generic: Add sudden shutdown tests for multi block atomic writes Ojaswin Mujoo
2025-09-19 6:48 ` [PATCH v7 10/12] ext4: Test atomic write and ioend codepaths with bigalloc Ojaswin Mujoo
2025-09-19 6:48 ` [PATCH v7 11/12] ext4: Test atomic writes allocation and write " Ojaswin Mujoo
2025-09-19 6:48 ` [PATCH v7 12/12] ext4: Atomic write test for extent split across leaf nodes Ojaswin Mujoo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20251030150138.GW4015566@frogsfrogsfrogs \
--to=djwong@kernel.org \
--cc=cem@kernel.org \
--cc=fstests@vger.kernel.org \
--cc=john.g.garry@oracle.com \
--cc=linux-xfs@vger.kernel.org \
--cc=ojaswin@linux.ibm.com \
--cc=ritesh.list@gmail.com \
--cc=zlang@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox