Re: [RFC 2/2] generic: Add integrity tests for O_DSYNC and RWF_DSYNC writes

[Ritesh Harjani (IBM) <ritesh.list@gmail.com>]

Zorro Lang <zlang@redhat.com> writes:

> On Thu, Jul 31, 2025 at 06:05:55PM +0530, Ritesh Harjani (IBM) wrote:
>> This test verifies the data & required metadata (e.g. inode i_size for extending
>> writes) integrity when using O_DSYNC and RWF_DSYNC during writes operations,
>> across buffered-io, aio-dio and dio, in the event of a sudden filesystem
>> shutdown after write completion.
>> 
>> Man page of open says that -
>> O_DSYNC provides synchronized I/O data integrity completion, meaning
>> write operations will flush data to the underlying hardware, but will
>> only flush metadata updates that are required to allow a subsequent read
>> operation to complete successfully.
>> 
>> Signed-off-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
>> ---
>>  tests/generic/737     | 30 +++++++++++++++++++++++++++++-
>>  tests/generic/737.out | 21 +++++++++++++++++++++
>>  2 files changed, 50 insertions(+), 1 deletion(-)
>> 
>> diff --git a/tests/generic/737 b/tests/generic/737
>> index 99ca1f39..0f27c82b 100755
>> --- a/tests/generic/737
>> +++ b/tests/generic/737
>> @@ -4,7 +4,8 @@
>>  #
>>  # FS QA Test No. 737
>>  #
>> -# Integrity test for O_SYNC with buff-io, dio, aio-dio with sudden shutdown.
>> +# Integrity test for O_[D]SYNC and/or RWF_DSYNC with buff-io, dio, aio-dio with
>> +# sudden shutdown.
>>  # Based on a testcase reported by Gao Xiang <hsiangkao@linux.alibaba.com>
>>  #
>> 
>> @@ -21,6 +22,15 @@ _require_aiodio aio-dio-write-verify
>>  _scratch_mkfs > $seqres.full 2>&1
>>  _scratch_mount
>> 
>> +echo "T-0: Create a 1M file using buff-io & RWF_DSYNC"
>> +$XFS_IO_PROG -f -c "pwrite -V 1 -D -S 0x5a 0 1M" $SCRATCH_MNT/testfile.t1 > /dev/null 2>&1
>> +echo "T-0: Shutdown the fs suddenly"
>> +_scratch_shutdown
>> +echo "T-0: Cycle mount"
>> +_scratch_cycle_mount
>> +echo "T-0: File contents after cycle mount"
>> +_hexdump $SCRATCH_MNT/testfile.t1
>> +
>>  echo "T-1: Create a 1M file using buff-io & O_SYNC"
>>  $XFS_IO_PROG -fs -c "pwrite -S 0x5a 0 1M" $SCRATCH_MNT/testfile.t1 > /dev/null 2>&1
>>  echo "T-1: Shutdown the fs suddenly"
>> @@ -48,5 +58,23 @@ _scratch_cycle_mount
>>  echo "T-3: File contents after cycle mount"
>>  _hexdump $SCRATCH_MNT/testfile.t3
>> 
>> +echo "T-4: Create a 1M file using DIO & RWF_DSYNC"
>> +$XFS_IO_PROG -fdc "pwrite -V 1 -S 0x5a -D 0 1M" $SCRATCH_MNT/testfile.t4 > /dev/null 2>&1
>> +echo "T-4: Shutdown the fs suddenly"
>> +_scratch_shutdown
>> +echo "T-4: Cycle mount"
>> +_scratch_cycle_mount
>> +echo "T-4: File contents after cycle mount"
>> +_hexdump $SCRATCH_MNT/testfile.t4
>> +
>> +echo "T-5: Create a 1M file using AIO-DIO & O_DSYNC"
>> +$AIO_TEST -a size=1048576 -D -N $SCRATCH_MNT/testfile.t5 > /dev/null 2>&1
>> +echo "T-5: Shutdown the fs suddenly"
>> +_scratch_shutdown
>> +echo "T-5: Cycle mount"
>> +_scratch_cycle_mount
>> +echo "T-5: File contents after cycle mount"
>> +_hexdump $SCRATCH_MNT/testfile.t5
>
> I always hit "No such file or directory" [1], is this an expected test failure
> which you hope to uncover?

Yes, we will need this fix [1] from Jan. Sorry I missed to add that in the commit
message. Could you please give it a try with the fix maybe?

[1]:
https://lore.kernel.org/linux-fsdevel/20250730102840.20470-2-jack@suse.cz/

-ritesh


>
> [1]
> # diff -u /root/git/xfstests/tests/generic/737.out /root/git/xfstests/results//generic/737.out.bad
> --- /root/git/xfstests/tests/generic/737.out    2025-08-02 04:04:57.334489725 +0800
> +++ /root/git/xfstests/results//generic/737.out.bad     2025-08-02 04:12:08.167934723 +0800
> @@ -28,16 +28,14 @@
>  *
>  100000
>  T-4: Create a 1M file using DIO & RWF_DSYNC
>  T-4: Shutdown the fs suddenly
>  T-4: Cycle mount
>  T-4: File contents after cycle mount
> -000000 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a  >ZZZZZZZZZZZZZZZZ<
> -*
> -100000
> +od: /mnt/scratch/testfile.t4: No such file or directory
>  T-5: Create a 1M file using AIO-DIO & O_DSYNC
>  T-5: Shutdown the fs suddenly
>  T-5: Cycle mount
>  T-5: File contents after cycle mount
> -000000 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a  >ZZZZZZZZZZZZZZZZ<
> -*
> -100000
> +od: /mnt/scratch/testfile.t5: No such file or directory
>
>> +
>>  status=0
>>  exit
>> diff --git a/tests/generic/737.out b/tests/generic/737.out
>> index efe6ff1f..2bafeefa 100644
>> --- a/tests/generic/737.out
>> +++ b/tests/generic/737.out
>> @@ -1,4 +1,11 @@
>>  QA output created by 737
>> +T-0: Create a 1M file using buff-io & RWF_DSYNC
>> +T-0: Shutdown the fs suddenly
>> +T-0: Cycle mount
>> +T-0: File contents after cycle mount
>> +000000 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a  >ZZZZZZZZZZZZZZZZ<
>> +*
>> +100000
>>  T-1: Create a 1M file using buff-io & O_SYNC
>>  T-1: Shutdown the fs suddenly
>>  T-1: Cycle mount
>> @@ -20,3 +27,17 @@ T-3: File contents after cycle mount
>>  000000 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a  >ZZZZZZZZZZZZZZZZ<
>>  *
>>  100000
>> +T-4: Create a 1M file using DIO & RWF_DSYNC
>> +T-4: Shutdown the fs suddenly
>> +T-4: Cycle mount
>> +T-4: File contents after cycle mount
>> +000000 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a  >ZZZZZZZZZZZZZZZZ<
>> +*
>> +100000
>> +T-5: Create a 1M file using AIO-DIO & O_DSYNC
>> +T-5: Shutdown the fs suddenly
>> +T-5: Cycle mount
>> +T-5: File contents after cycle mount
>> +000000 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a  >ZZZZZZZZZZZZZZZZ<
>> +*
>> +100000
>> --
>> 2.49.0
>> 
>>