Re: [PATCH -v2] generic: test which tries to exercise AIO/DIO into unwritten space

["Darrick J. Wong" <djwong@kernel.org>]

On Sun, Mar 07, 2021 at 08:22:47PM -0500, Theodore Ts'o wrote:
> This test verifies that the an unwritten extent is properly marked as
> written after writing into it.
> 
> There was a hard-to-hit bug which would occasionally trigger with ext4
> for which this test was a reproducer.  This has been fixed after
> moving ext4 to use iomap for Direct I/O's, although as of this
> writing, there are still some occasional failures on ext4 when block
> size < page size.
> 
> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
> ---
> 
> Differences between v1 and v2:
> * Addressed review comments from Eryu
> * Renumbered from generic/623 to generic/630 (since 623 has been assigned)
> 
>  tests/generic/630     | 105 ++++++++++++++++++++++++++++++++++++++++++
>  tests/generic/630.out |   4 ++
>  tests/generic/group   |   1 +
>  3 files changed, 110 insertions(+)
>  create mode 100755 tests/generic/630
>  create mode 100644 tests/generic/630.out
> 
> diff --git a/tests/generic/630 b/tests/generic/630
> new file mode 100755
> index 00000000..7a04b899
> --- /dev/null
> +++ b/tests/generic/630
> @@ -0,0 +1,105 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +
> +#
> +# FSQA Test No. 630
> +#
> +# AIO/DIO stress test
> +# Run random AIO/DIO activity on an file system with unwritten regions
> +#
> +# This test verifies that the an unwritten extent is properly marked
> +# as written after writing into it.
> +#
> +# There was a hard-to-hit bug which would occasionally trigger with
> +# ext4 for which this test was a reproducer.  This has been fixed
> +# after moving ext4 to use iomap for Direct I/O's, although as of this
> +# writing, there are still some occasional failures on ext4 when block
> +# size < page size.
> +#
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +fio_config=$tmp.fio
> +fio_out=$tmp.fio.out
> +status=1	# failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +    rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# real QA test starts here
> +_supported_fs generic
> +_require_test
> +_require_scratch
> +_require_odirect
> +_require_aio
> +_require_block_device $SCRATCH_DEV
> +
> +NUM_JOBS=$((4*LOAD_FACTOR))
> +BLK_DEV_SIZE=`blockdev --getsz $SCRATCH_DEV`
> +FILE_SIZE=$(((BLK_DEV_SIZE * 512) * 3 / 4))
> +
> +max_file_size=$((5 * 1024 * 1024 * 1024))
> +if [ $max_file_size -lt $FILE_SIZE ]; then
> +	FILE_SIZE=$max_file_size
> +fi
> +SIZE=$((FILE_SIZE / 2))
> +
> +cat >$fio_config <<EOF
> +###########
> +# $seq test fio activity
> +# Filenames derived from jobsname and jobid like follows:
> +# ${JOB_NAME}.${JOB_ID}.${ITERATION_ID}
> +[global]
> +ioengine=libaio
> +bs=128k
> +directory=${SCRATCH_MNT}
> +filesize=${FILE_SIZE}
> +size=${FILE_SIZE}
> +io_size=${SIZE}
> +iodepth=$((128*$LOAD_FACTOR))
> +fallocate=native
> +
> +# Perform direct aio and verify data
> +# This test case should check use-after-free issues
> +[aio-dio-verifier]
> +numjobs=1
> +verify=crc32c-intel
> +verify_fatal=1
> +verify_dump=1
> +verify_backlog=1024
> +verify_async=4
> +direct=1
> +random_generator=lfsr
> +blocksize_range=4k-8200k

Something I hadn't noticed before this got committed -- setting the
maximum blocksize to 8200k and the iodepth to a multiple of 128 means
that fio will allocate an 8200k buffer per iodepth.  Given that the
iodepth is (128 * LOAD_FACTOR), this test therefore requires (1GB *
LOAD_FACTOR) memory.  On my testing VMs which have a generous 1200M of
memory, this causes the oom killer to zap fio, which in turn causes the
test to fail due to the SIGKILL being reported in the golden output.

Is it necessary to allocate 8MB of memory per io thread?  Given that
this is a regression test to probe a bug in ext4's unwritten extent
conversion I'm loath to touch it too much, but it seems to me that a
smaller blocksize would stress the extent mapping structures harder and
reduce the memory requirements at the same time.

--D

> +rw=randwrite
> +filename=test-file
> +
> +EOF
> +
> +rm -f $seqres.full
> +
> +_require_fio $fio_config
> +_require_xfs_io_command "falloc"
> +
> +_scratch_mkfs >> $seqres.full 2>&1
> +_scratch_mount
> +
> +echo ""
> +echo "Run fio with random aio-dio pattern"
> +echo ""
> +cat $fio_config >> $seqres.full
> +$FIO_PROG $fio_config --output=$fio_out
> +cat $fio_out >> $seqres.full
> +
> +status=0
> +exit
> diff --git a/tests/generic/630.out b/tests/generic/630.out
> new file mode 100644
> index 00000000..7f3adb50
> --- /dev/null
> +++ b/tests/generic/630.out
> @@ -0,0 +1,4 @@
> +QA output created by 630
> +
> +Run fio with random aio-dio pattern
> +
> diff --git a/tests/generic/group b/tests/generic/group
> index 84db3789..1ff896a1 100644
> --- a/tests/generic/group
> +++ b/tests/generic/group
> @@ -628,3 +628,4 @@
>  623 auto quick shutdown
>  624 auto quick verity
>  625 auto quick verity
> +630 aio rw stress
> -- 
> 2.30.0
>