linux-xfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Ojaswin Mujoo <ojaswin@linux.ibm.com>
To: Zorro Lang <zlang@redhat.com>, fstests@vger.kernel.org
Cc: Ritesh Harjani <ritesh.list@gmail.com>,
	djwong@kernel.org, john.g.garry@oracle.com, tytso@mit.edu,
	linux-xfs@vger.kernel.org, linux-kernel@vger.kernel.org,
	linux-ext4@vger.kernel.org
Subject: [PATCH v4 05/11] generic: Add atomic write test using fio verify on file mixed mappings
Date: Sun, 10 Aug 2025 19:11:56 +0530	[thread overview]
Message-ID: <508d55ad8e3b8efde87ffbe3354e9e1d9ee8c908.1754833177.git.ojaswin@linux.ibm.com> (raw)
In-Reply-To: <cover.1754833177.git.ojaswin@linux.ibm.com>

This tests uses fio to first create a file with mixed mappings. Then it
does atomic writes using aio dio with parallel jobs to the same file with
mixed mappings. This forces the filesystem allocator to allocate extents
over mixed mapping regions to stress FS block allocators.

Avoid doing overlapping parallel atomic writes because it might give
unexpected results. Use offset_increment=, size= fio options to achieve
this behavior.

Co-developed-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
Signed-off-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
Signed-off-by: Ojaswin Mujoo <ojaswin@linux.ibm.com>
---
 tests/generic/1227     | 131 +++++++++++++++++++++++++++++++++++++++++
 tests/generic/1227.out |   2 +
 2 files changed, 133 insertions(+)
 create mode 100755 tests/generic/1227
 create mode 100644 tests/generic/1227.out

diff --git a/tests/generic/1227 b/tests/generic/1227
new file mode 100755
index 00000000..7423e67c
--- /dev/null
+++ b/tests/generic/1227
@@ -0,0 +1,131 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2025 IBM Corporation. All Rights Reserved.
+#
+# FS QA Test 1227
+#
+# Validate FS atomic write using fio crc check verifier on mixed mappings
+# of a file.
+#
+. ./common/preamble
+. ./common/atomicwrites
+
+_begin_fstest auto aio rw atomicwrites
+
+_require_scratch_write_atomic_multi_fsblock
+_require_odirect
+_require_aio
+_require_xfs_io_command "truncate"
+
+_scratch_mkfs >> $seqres.full 2>&1
+_scratch_mount
+
+touch "$SCRATCH_MNT/f1"
+awu_min_write=$(_get_atomic_write_unit_min "$SCRATCH_MNT/f1")
+awu_max_write=$(_get_atomic_write_unit_max "$SCRATCH_MNT/f1")
+
+aw_bsize=$(_max "$awu_min_write" "$((awu_max_write/4))")
+fsbsize=$(_get_block_size $SCRATCH_MNT)
+
+threads=$(_min "$(($(nproc) * 2 * LOAD_FACTOR))" "100")
+filesize=$((aw_bsize * threads * 100))
+depth=$threads
+aw_io_size=$((filesize / threads))
+aw_io_inc=$aw_io_size
+testfile=$SCRATCH_MNT/test-file
+
+fio_prep_config=$tmp.prep.fio
+fio_aw_config=$tmp.aw.fio
+fio_verify_config=$tmp.verify.fio
+fio_out=$tmp.fio.out
+
+cat >$fio_prep_config <<EOF
+# prep file to have mixed mappings
+[global]
+ioengine=libaio
+filename=$testfile
+size=$filesize
+bs=$fsbsize
+direct=1
+iodepth=$depth
+group_reporting=1
+
+# Create written extents
+[prep_written_blocks]
+ioengine=libaio
+rw=randwrite
+io_size=$((filesize/3))
+random_generator=lfsr
+
+# Create unwritten extents
+[prep_unwritten_blocks]
+ioengine=falloc
+rw=randwrite
+io_size=$((filesize/3))
+random_generator=lfsr
+EOF
+
+cat >$fio_aw_config <<EOF
+# atomic write to mixed mappings of written/unwritten/holes
+[atomic_write_job]
+ioengine=libaio
+rw=randwrite
+direct=1
+atomic=1
+random_generator=lfsr
+group_reporting=1
+
+filename=$testfile
+bs=$aw_bsize
+size=$aw_io_size
+offset_increment=$aw_io_inc
+iodepth=$depth
+numjobs=$threads
+
+verify_state_save=0
+verify=crc32c
+do_verify=0
+EOF
+
+cat >$fio_verify_config <<EOF
+# verify atomic writes done by previous job
+[verify_job]
+ioengine=libaio
+rw=read
+random_generator=lfsr
+group_reporting=1
+
+filename=$testfile
+size=$filesize
+bs=$aw_bsize
+iodepth=$depth
+
+verify_state_save=0
+verify_only=1
+verify=crc32c
+verify_fatal=1
+verify_write_sequence=0
+EOF
+
+_require_fio $fio_aw_config
+_require_fio $fio_verify_config
+
+cat $fio_prep_config >> $seqres.full
+cat $fio_aw_config >> $seqres.full
+cat $fio_verify_config >> $seqres.full
+
+$XFS_IO_PROG -fc "truncate $filesize" $testfile >> $seqres.full
+
+#prepare file with mixed mappings
+$FIO_PROG $fio_prep_config >> $seqres.full
+
+# do atomic writes without verifying
+$FIO_PROG $fio_aw_config >> $seqres.full
+
+# verify data is not torn
+$FIO_PROG $fio_verify_config >> $seqres.full
+
+# success, all done
+echo Silence is golden
+status=0
+exit
diff --git a/tests/generic/1227.out b/tests/generic/1227.out
new file mode 100644
index 00000000..2605d062
--- /dev/null
+++ b/tests/generic/1227.out
@@ -0,0 +1,2 @@
+QA output created by 1227
+Silence is golden
-- 
2.49.0


  parent reply	other threads:[~2025-08-10 13:42 UTC|newest]

Thread overview: 38+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-08-10 13:41 [PATCH v4 00/11] Add more tests for multi fs block atomic writes Ojaswin Mujoo
2025-08-10 13:41 ` [PATCH v4 01/11] common/rc: Add _min() and _max() helpers Ojaswin Mujoo
2025-08-13 12:20   ` David Laight
2025-08-21 10:35     ` Ojaswin Mujoo
2025-08-10 13:41 ` [PATCH v4 02/11] common/rc: Add a helper to run fsx on a given file Ojaswin Mujoo
2025-08-10 13:41 ` [PATCH v4 03/11] ltp/fsx.c: Add atomic writes support to fsx Ojaswin Mujoo
2025-08-13 13:42   ` John Garry
2025-08-21  9:45     ` Ojaswin Mujoo
2025-08-10 13:41 ` [PATCH v4 04/11] generic: Add atomic write test using fio crc check verifier Ojaswin Mujoo
2025-08-12 17:16   ` Darrick J. Wong
2025-08-13 13:39   ` John Garry
2025-08-21  8:42     ` Ojaswin Mujoo
2025-08-21  9:24       ` John Garry
2025-08-21 12:18         ` Ojaswin Mujoo
2025-08-10 13:41 ` Ojaswin Mujoo [this message]
2025-08-12 17:16   ` [PATCH v4 05/11] generic: Add atomic write test using fio verify on file mixed mappings Darrick J. Wong
2025-08-10 13:41 ` [PATCH v4 06/11] generic: Add atomic write multi-fsblock O_[D]SYNC tests Ojaswin Mujoo
2025-08-11 15:29   ` Darrick J. Wong
2025-08-10 13:41 ` [PATCH v4 07/11] generic: Stress fsx with atomic writes enabled Ojaswin Mujoo
2025-08-12 17:18   ` Darrick J. Wong
2025-08-13  5:45     ` Ojaswin Mujoo
2025-08-10 13:41 ` [PATCH v4 08/11] generic: Add sudden shutdown tests for multi block atomic writes Ojaswin Mujoo
2025-08-10 13:42 ` [PATCH v4 09/11] ext4: Atomic writes stress test for bigalloc using fio crc verifier Ojaswin Mujoo
2025-08-12  8:08   ` John Garry
2025-08-13  7:08     ` Ojaswin Mujoo
2025-08-13  7:33       ` John Garry
2025-08-21  8:29         ` Ojaswin Mujoo
2025-08-10 13:42 ` [PATCH v4 10/11] ext4: Atomic writes test for bigalloc using fio crc verifier on multiple files Ojaswin Mujoo
2025-08-13 13:45   ` John Garry
2025-08-21  8:28     ` Ojaswin Mujoo
2025-08-21  9:28       ` John Garry
2025-08-21 12:19         ` Ojaswin Mujoo
2025-08-10 13:42 ` [PATCH v4 11/11] ext4: Atomic write test for extent split across leaf nodes Ojaswin Mujoo
2025-08-12 17:19   ` Darrick J. Wong
2025-08-13  5:45     ` Ojaswin Mujoo
2025-08-13 13:54   ` John Garry
2025-08-21  8:25     ` Ojaswin Mujoo
2025-08-21  9:23       ` John Garry

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=508d55ad8e3b8efde87ffbe3354e9e1d9ee8c908.1754833177.git.ojaswin@linux.ibm.com \
    --to=ojaswin@linux.ibm.com \
    --cc=djwong@kernel.org \
    --cc=fstests@vger.kernel.org \
    --cc=john.g.garry@oracle.com \
    --cc=linux-ext4@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-xfs@vger.kernel.org \
    --cc=ritesh.list@gmail.com \
    --cc=tytso@mit.edu \
    --cc=zlang@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).