* [RFC 1/2] aio-dio-write-verify: Add O_DSYNC option
@ 2025-07-31 12:35 Ritesh Harjani (IBM)
2025-07-31 12:35 ` [RFC 2/2] generic: Add integrity tests for O_DSYNC and RWF_DSYNC writes Ritesh Harjani (IBM)
` (2 more replies)
0 siblings, 3 replies; 8+ messages in thread
From: Ritesh Harjani (IBM) @ 2025-07-31 12:35 UTC (permalink / raw)
To: fstests
Cc: linux-fsdevel, Jan Kara, Darrick J . Wong, John Garry,
Ritesh Harjani (IBM)
This patch adds -D for O_DSYNC open flag to aio-dio-write-verify test.
We will use this in later patch for integrity verification test with
aio-dio.
Signed-off-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
---
src/aio-dio-regress/aio-dio-write-verify.c | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/src/aio-dio-regress/aio-dio-write-verify.c b/src/aio-dio-regress/aio-dio-write-verify.c
index 513a338b..0cf14a2a 100644
--- a/src/aio-dio-regress/aio-dio-write-verify.c
+++ b/src/aio-dio-regress/aio-dio-write-verify.c
@@ -40,6 +40,7 @@ void usage(char *progname)
"\t\tsize=N: AIO write size\n"
"\t\toff=M: AIO write startoff\n"
"\t-S: uses O_SYNC flag for open. By default O_SYNC is not used\n"
+ "\t-D: uses O_DSYNC flag for open. By default O_DSYNC is not used\n"
"\t-N: no_verify: means no write verification. By default noverify is false\n"
"e.g: %s -t 4608 -a size=4096,off=512 -a size=4096,off=4608 filename\n"
"e.g: %s -t 1048576 -a size=1048576 -S -N filename\n",
@@ -298,7 +299,7 @@ int main(int argc, char *argv[])
int o_sync = 0;
int no_verify = 0;
- while ((c = getopt(argc, argv, "a:t:SN")) != -1) {
+ while ((c = getopt(argc, argv, "a:t:SND")) != -1) {
char *endp;
switch (c) {
@@ -316,6 +317,9 @@ int main(int argc, char *argv[])
case 'S':
o_sync = O_SYNC;
break;
+ case 'D':
+ o_sync = O_DSYNC;
+ break;
case 'N':
no_verify = 1;
break;
--
2.49.0
^ permalink raw reply related [flat|nested] 8+ messages in thread* [RFC 2/2] generic: Add integrity tests for O_DSYNC and RWF_DSYNC writes 2025-07-31 12:35 [RFC 1/2] aio-dio-write-verify: Add O_DSYNC option Ritesh Harjani (IBM) @ 2025-07-31 12:35 ` Ritesh Harjani (IBM) 2025-08-01 20:18 ` Zorro Lang 2025-07-31 15:00 ` [RFC 1/2] aio-dio-write-verify: Add O_DSYNC option Jan Kara 2025-08-02 8:18 ` Zorro Lang 2 siblings, 1 reply; 8+ messages in thread From: Ritesh Harjani (IBM) @ 2025-07-31 12:35 UTC (permalink / raw) To: fstests Cc: linux-fsdevel, Jan Kara, Darrick J . Wong, John Garry, Ritesh Harjani (IBM) This test verifies the data & required metadata (e.g. inode i_size for extending writes) integrity when using O_DSYNC and RWF_DSYNC during writes operations, across buffered-io, aio-dio and dio, in the event of a sudden filesystem shutdown after write completion. Man page of open says that - O_DSYNC provides synchronized I/O data integrity completion, meaning write operations will flush data to the underlying hardware, but will only flush metadata updates that are required to allow a subsequent read operation to complete successfully. Signed-off-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com> --- tests/generic/737 | 30 +++++++++++++++++++++++++++++- tests/generic/737.out | 21 +++++++++++++++++++++ 2 files changed, 50 insertions(+), 1 deletion(-) diff --git a/tests/generic/737 b/tests/generic/737 index 99ca1f39..0f27c82b 100755 --- a/tests/generic/737 +++ b/tests/generic/737 @@ -4,7 +4,8 @@ # # FS QA Test No. 737 # -# Integrity test for O_SYNC with buff-io, dio, aio-dio with sudden shutdown. +# Integrity test for O_[D]SYNC and/or RWF_DSYNC with buff-io, dio, aio-dio with +# sudden shutdown. # Based on a testcase reported by Gao Xiang <hsiangkao@linux.alibaba.com> # @@ -21,6 +22,15 @@ _require_aiodio aio-dio-write-verify _scratch_mkfs > $seqres.full 2>&1 _scratch_mount +echo "T-0: Create a 1M file using buff-io & RWF_DSYNC" +$XFS_IO_PROG -f -c "pwrite -V 1 -D -S 0x5a 0 1M" $SCRATCH_MNT/testfile.t1 > /dev/null 2>&1 +echo "T-0: Shutdown the fs suddenly" +_scratch_shutdown +echo "T-0: Cycle mount" +_scratch_cycle_mount +echo "T-0: File contents after cycle mount" +_hexdump $SCRATCH_MNT/testfile.t1 + echo "T-1: Create a 1M file using buff-io & O_SYNC" $XFS_IO_PROG -fs -c "pwrite -S 0x5a 0 1M" $SCRATCH_MNT/testfile.t1 > /dev/null 2>&1 echo "T-1: Shutdown the fs suddenly" @@ -48,5 +58,23 @@ _scratch_cycle_mount echo "T-3: File contents after cycle mount" _hexdump $SCRATCH_MNT/testfile.t3 +echo "T-4: Create a 1M file using DIO & RWF_DSYNC" +$XFS_IO_PROG -fdc "pwrite -V 1 -S 0x5a -D 0 1M" $SCRATCH_MNT/testfile.t4 > /dev/null 2>&1 +echo "T-4: Shutdown the fs suddenly" +_scratch_shutdown +echo "T-4: Cycle mount" +_scratch_cycle_mount +echo "T-4: File contents after cycle mount" +_hexdump $SCRATCH_MNT/testfile.t4 + +echo "T-5: Create a 1M file using AIO-DIO & O_DSYNC" +$AIO_TEST -a size=1048576 -D -N $SCRATCH_MNT/testfile.t5 > /dev/null 2>&1 +echo "T-5: Shutdown the fs suddenly" +_scratch_shutdown +echo "T-5: Cycle mount" +_scratch_cycle_mount +echo "T-5: File contents after cycle mount" +_hexdump $SCRATCH_MNT/testfile.t5 + status=0 exit diff --git a/tests/generic/737.out b/tests/generic/737.out index efe6ff1f..2bafeefa 100644 --- a/tests/generic/737.out +++ b/tests/generic/737.out @@ -1,4 +1,11 @@ QA output created by 737 +T-0: Create a 1M file using buff-io & RWF_DSYNC +T-0: Shutdown the fs suddenly +T-0: Cycle mount +T-0: File contents after cycle mount +000000 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a >ZZZZZZZZZZZZZZZZ< +* +100000 T-1: Create a 1M file using buff-io & O_SYNC T-1: Shutdown the fs suddenly T-1: Cycle mount @@ -20,3 +27,17 @@ T-3: File contents after cycle mount 000000 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a >ZZZZZZZZZZZZZZZZ< * 100000 +T-4: Create a 1M file using DIO & RWF_DSYNC +T-4: Shutdown the fs suddenly +T-4: Cycle mount +T-4: File contents after cycle mount +000000 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a >ZZZZZZZZZZZZZZZZ< +* +100000 +T-5: Create a 1M file using AIO-DIO & O_DSYNC +T-5: Shutdown the fs suddenly +T-5: Cycle mount +T-5: File contents after cycle mount +000000 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a >ZZZZZZZZZZZZZZZZ< +* +100000 -- 2.49.0 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [RFC 2/2] generic: Add integrity tests for O_DSYNC and RWF_DSYNC writes 2025-07-31 12:35 ` [RFC 2/2] generic: Add integrity tests for O_DSYNC and RWF_DSYNC writes Ritesh Harjani (IBM) @ 2025-08-01 20:18 ` Zorro Lang 2025-08-01 20:29 ` Ritesh Harjani 0 siblings, 1 reply; 8+ messages in thread From: Zorro Lang @ 2025-08-01 20:18 UTC (permalink / raw) To: Ritesh Harjani (IBM) Cc: fstests, linux-fsdevel, Jan Kara, Darrick J . Wong, John Garry On Thu, Jul 31, 2025 at 06:05:55PM +0530, Ritesh Harjani (IBM) wrote: > This test verifies the data & required metadata (e.g. inode i_size for extending > writes) integrity when using O_DSYNC and RWF_DSYNC during writes operations, > across buffered-io, aio-dio and dio, in the event of a sudden filesystem > shutdown after write completion. > > Man page of open says that - > O_DSYNC provides synchronized I/O data integrity completion, meaning > write operations will flush data to the underlying hardware, but will > only flush metadata updates that are required to allow a subsequent read > operation to complete successfully. > > Signed-off-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com> > --- > tests/generic/737 | 30 +++++++++++++++++++++++++++++- > tests/generic/737.out | 21 +++++++++++++++++++++ > 2 files changed, 50 insertions(+), 1 deletion(-) > > diff --git a/tests/generic/737 b/tests/generic/737 > index 99ca1f39..0f27c82b 100755 > --- a/tests/generic/737 > +++ b/tests/generic/737 > @@ -4,7 +4,8 @@ > # > # FS QA Test No. 737 > # > -# Integrity test for O_SYNC with buff-io, dio, aio-dio with sudden shutdown. > +# Integrity test for O_[D]SYNC and/or RWF_DSYNC with buff-io, dio, aio-dio with > +# sudden shutdown. > # Based on a testcase reported by Gao Xiang <hsiangkao@linux.alibaba.com> > # > > @@ -21,6 +22,15 @@ _require_aiodio aio-dio-write-verify > _scratch_mkfs > $seqres.full 2>&1 > _scratch_mount > > +echo "T-0: Create a 1M file using buff-io & RWF_DSYNC" > +$XFS_IO_PROG -f -c "pwrite -V 1 -D -S 0x5a 0 1M" $SCRATCH_MNT/testfile.t1 > /dev/null 2>&1 > +echo "T-0: Shutdown the fs suddenly" > +_scratch_shutdown > +echo "T-0: Cycle mount" > +_scratch_cycle_mount > +echo "T-0: File contents after cycle mount" > +_hexdump $SCRATCH_MNT/testfile.t1 > + > echo "T-1: Create a 1M file using buff-io & O_SYNC" > $XFS_IO_PROG -fs -c "pwrite -S 0x5a 0 1M" $SCRATCH_MNT/testfile.t1 > /dev/null 2>&1 > echo "T-1: Shutdown the fs suddenly" > @@ -48,5 +58,23 @@ _scratch_cycle_mount > echo "T-3: File contents after cycle mount" > _hexdump $SCRATCH_MNT/testfile.t3 > > +echo "T-4: Create a 1M file using DIO & RWF_DSYNC" > +$XFS_IO_PROG -fdc "pwrite -V 1 -S 0x5a -D 0 1M" $SCRATCH_MNT/testfile.t4 > /dev/null 2>&1 > +echo "T-4: Shutdown the fs suddenly" > +_scratch_shutdown > +echo "T-4: Cycle mount" > +_scratch_cycle_mount > +echo "T-4: File contents after cycle mount" > +_hexdump $SCRATCH_MNT/testfile.t4 > + > +echo "T-5: Create a 1M file using AIO-DIO & O_DSYNC" > +$AIO_TEST -a size=1048576 -D -N $SCRATCH_MNT/testfile.t5 > /dev/null 2>&1 > +echo "T-5: Shutdown the fs suddenly" > +_scratch_shutdown > +echo "T-5: Cycle mount" > +_scratch_cycle_mount > +echo "T-5: File contents after cycle mount" > +_hexdump $SCRATCH_MNT/testfile.t5 I always hit "No such file or directory" [1], is this an expected test failure which you hope to uncover? [1] # diff -u /root/git/xfstests/tests/generic/737.out /root/git/xfstests/results//generic/737.out.bad --- /root/git/xfstests/tests/generic/737.out 2025-08-02 04:04:57.334489725 +0800 +++ /root/git/xfstests/results//generic/737.out.bad 2025-08-02 04:12:08.167934723 +0800 @@ -28,16 +28,14 @@ * 100000 T-4: Create a 1M file using DIO & RWF_DSYNC T-4: Shutdown the fs suddenly T-4: Cycle mount T-4: File contents after cycle mount -000000 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a >ZZZZZZZZZZZZZZZZ< -* -100000 +od: /mnt/scratch/testfile.t4: No such file or directory T-5: Create a 1M file using AIO-DIO & O_DSYNC T-5: Shutdown the fs suddenly T-5: Cycle mount T-5: File contents after cycle mount -000000 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a >ZZZZZZZZZZZZZZZZ< -* -100000 +od: /mnt/scratch/testfile.t5: No such file or directory > + > status=0 > exit > diff --git a/tests/generic/737.out b/tests/generic/737.out > index efe6ff1f..2bafeefa 100644 > --- a/tests/generic/737.out > +++ b/tests/generic/737.out > @@ -1,4 +1,11 @@ > QA output created by 737 > +T-0: Create a 1M file using buff-io & RWF_DSYNC > +T-0: Shutdown the fs suddenly > +T-0: Cycle mount > +T-0: File contents after cycle mount > +000000 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a >ZZZZZZZZZZZZZZZZ< > +* > +100000 > T-1: Create a 1M file using buff-io & O_SYNC > T-1: Shutdown the fs suddenly > T-1: Cycle mount > @@ -20,3 +27,17 @@ T-3: File contents after cycle mount > 000000 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a >ZZZZZZZZZZZZZZZZ< > * > 100000 > +T-4: Create a 1M file using DIO & RWF_DSYNC > +T-4: Shutdown the fs suddenly > +T-4: Cycle mount > +T-4: File contents after cycle mount > +000000 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a >ZZZZZZZZZZZZZZZZ< > +* > +100000 > +T-5: Create a 1M file using AIO-DIO & O_DSYNC > +T-5: Shutdown the fs suddenly > +T-5: Cycle mount > +T-5: File contents after cycle mount > +000000 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a >ZZZZZZZZZZZZZZZZ< > +* > +100000 > -- > 2.49.0 > > ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [RFC 2/2] generic: Add integrity tests for O_DSYNC and RWF_DSYNC writes 2025-08-01 20:18 ` Zorro Lang @ 2025-08-01 20:29 ` Ritesh Harjani 2025-08-02 8:17 ` Zorro Lang 0 siblings, 1 reply; 8+ messages in thread From: Ritesh Harjani @ 2025-08-01 20:29 UTC (permalink / raw) To: Zorro Lang; +Cc: fstests, linux-fsdevel, Jan Kara, Darrick J . Wong, John Garry Zorro Lang <zlang@redhat.com> writes: > On Thu, Jul 31, 2025 at 06:05:55PM +0530, Ritesh Harjani (IBM) wrote: >> This test verifies the data & required metadata (e.g. inode i_size for extending >> writes) integrity when using O_DSYNC and RWF_DSYNC during writes operations, >> across buffered-io, aio-dio and dio, in the event of a sudden filesystem >> shutdown after write completion. >> >> Man page of open says that - >> O_DSYNC provides synchronized I/O data integrity completion, meaning >> write operations will flush data to the underlying hardware, but will >> only flush metadata updates that are required to allow a subsequent read >> operation to complete successfully. >> >> Signed-off-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com> >> --- >> tests/generic/737 | 30 +++++++++++++++++++++++++++++- >> tests/generic/737.out | 21 +++++++++++++++++++++ >> 2 files changed, 50 insertions(+), 1 deletion(-) >> >> diff --git a/tests/generic/737 b/tests/generic/737 >> index 99ca1f39..0f27c82b 100755 >> --- a/tests/generic/737 >> +++ b/tests/generic/737 >> @@ -4,7 +4,8 @@ >> # >> # FS QA Test No. 737 >> # >> -# Integrity test for O_SYNC with buff-io, dio, aio-dio with sudden shutdown. >> +# Integrity test for O_[D]SYNC and/or RWF_DSYNC with buff-io, dio, aio-dio with >> +# sudden shutdown. >> # Based on a testcase reported by Gao Xiang <hsiangkao@linux.alibaba.com> >> # >> >> @@ -21,6 +22,15 @@ _require_aiodio aio-dio-write-verify >> _scratch_mkfs > $seqres.full 2>&1 >> _scratch_mount >> >> +echo "T-0: Create a 1M file using buff-io & RWF_DSYNC" >> +$XFS_IO_PROG -f -c "pwrite -V 1 -D -S 0x5a 0 1M" $SCRATCH_MNT/testfile.t1 > /dev/null 2>&1 >> +echo "T-0: Shutdown the fs suddenly" >> +_scratch_shutdown >> +echo "T-0: Cycle mount" >> +_scratch_cycle_mount >> +echo "T-0: File contents after cycle mount" >> +_hexdump $SCRATCH_MNT/testfile.t1 >> + >> echo "T-1: Create a 1M file using buff-io & O_SYNC" >> $XFS_IO_PROG -fs -c "pwrite -S 0x5a 0 1M" $SCRATCH_MNT/testfile.t1 > /dev/null 2>&1 >> echo "T-1: Shutdown the fs suddenly" >> @@ -48,5 +58,23 @@ _scratch_cycle_mount >> echo "T-3: File contents after cycle mount" >> _hexdump $SCRATCH_MNT/testfile.t3 >> >> +echo "T-4: Create a 1M file using DIO & RWF_DSYNC" >> +$XFS_IO_PROG -fdc "pwrite -V 1 -S 0x5a -D 0 1M" $SCRATCH_MNT/testfile.t4 > /dev/null 2>&1 >> +echo "T-4: Shutdown the fs suddenly" >> +_scratch_shutdown >> +echo "T-4: Cycle mount" >> +_scratch_cycle_mount >> +echo "T-4: File contents after cycle mount" >> +_hexdump $SCRATCH_MNT/testfile.t4 >> + >> +echo "T-5: Create a 1M file using AIO-DIO & O_DSYNC" >> +$AIO_TEST -a size=1048576 -D -N $SCRATCH_MNT/testfile.t5 > /dev/null 2>&1 >> +echo "T-5: Shutdown the fs suddenly" >> +_scratch_shutdown >> +echo "T-5: Cycle mount" >> +_scratch_cycle_mount >> +echo "T-5: File contents after cycle mount" >> +_hexdump $SCRATCH_MNT/testfile.t5 > > I always hit "No such file or directory" [1], is this an expected test failure > which you hope to uncover? Yes, we will need this fix [1] from Jan. Sorry I missed to add that in the commit message. Could you please give it a try with the fix maybe? [1]: https://lore.kernel.org/linux-fsdevel/20250730102840.20470-2-jack@suse.cz/ -ritesh > > [1] > # diff -u /root/git/xfstests/tests/generic/737.out /root/git/xfstests/results//generic/737.out.bad > --- /root/git/xfstests/tests/generic/737.out 2025-08-02 04:04:57.334489725 +0800 > +++ /root/git/xfstests/results//generic/737.out.bad 2025-08-02 04:12:08.167934723 +0800 > @@ -28,16 +28,14 @@ > * > 100000 > T-4: Create a 1M file using DIO & RWF_DSYNC > T-4: Shutdown the fs suddenly > T-4: Cycle mount > T-4: File contents after cycle mount > -000000 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a >ZZZZZZZZZZZZZZZZ< > -* > -100000 > +od: /mnt/scratch/testfile.t4: No such file or directory > T-5: Create a 1M file using AIO-DIO & O_DSYNC > T-5: Shutdown the fs suddenly > T-5: Cycle mount > T-5: File contents after cycle mount > -000000 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a >ZZZZZZZZZZZZZZZZ< > -* > -100000 > +od: /mnt/scratch/testfile.t5: No such file or directory > >> + >> status=0 >> exit >> diff --git a/tests/generic/737.out b/tests/generic/737.out >> index efe6ff1f..2bafeefa 100644 >> --- a/tests/generic/737.out >> +++ b/tests/generic/737.out >> @@ -1,4 +1,11 @@ >> QA output created by 737 >> +T-0: Create a 1M file using buff-io & RWF_DSYNC >> +T-0: Shutdown the fs suddenly >> +T-0: Cycle mount >> +T-0: File contents after cycle mount >> +000000 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a >ZZZZZZZZZZZZZZZZ< >> +* >> +100000 >> T-1: Create a 1M file using buff-io & O_SYNC >> T-1: Shutdown the fs suddenly >> T-1: Cycle mount >> @@ -20,3 +27,17 @@ T-3: File contents after cycle mount >> 000000 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a >ZZZZZZZZZZZZZZZZ< >> * >> 100000 >> +T-4: Create a 1M file using DIO & RWF_DSYNC >> +T-4: Shutdown the fs suddenly >> +T-4: Cycle mount >> +T-4: File contents after cycle mount >> +000000 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a >ZZZZZZZZZZZZZZZZ< >> +* >> +100000 >> +T-5: Create a 1M file using AIO-DIO & O_DSYNC >> +T-5: Shutdown the fs suddenly >> +T-5: Cycle mount >> +T-5: File contents after cycle mount >> +000000 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a >ZZZZZZZZZZZZZZZZ< >> +* >> +100000 >> -- >> 2.49.0 >> >> ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [RFC 2/2] generic: Add integrity tests for O_DSYNC and RWF_DSYNC writes 2025-08-01 20:29 ` Ritesh Harjani @ 2025-08-02 8:17 ` Zorro Lang 2025-08-03 5:23 ` Ritesh Harjani 0 siblings, 1 reply; 8+ messages in thread From: Zorro Lang @ 2025-08-02 8:17 UTC (permalink / raw) To: Ritesh Harjani Cc: fstests, linux-fsdevel, Jan Kara, Darrick J . Wong, John Garry On Sat, Aug 02, 2025 at 01:59:18AM +0530, Ritesh Harjani wrote: > Zorro Lang <zlang@redhat.com> writes: > > > On Thu, Jul 31, 2025 at 06:05:55PM +0530, Ritesh Harjani (IBM) wrote: > >> This test verifies the data & required metadata (e.g. inode i_size for extending > >> writes) integrity when using O_DSYNC and RWF_DSYNC during writes operations, > >> across buffered-io, aio-dio and dio, in the event of a sudden filesystem > >> shutdown after write completion. > >> > >> Man page of open says that - > >> O_DSYNC provides synchronized I/O data integrity completion, meaning > >> write operations will flush data to the underlying hardware, but will > >> only flush metadata updates that are required to allow a subsequent read > >> operation to complete successfully. > >> > >> Signed-off-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com> > >> --- > >> tests/generic/737 | 30 +++++++++++++++++++++++++++++- > >> tests/generic/737.out | 21 +++++++++++++++++++++ BTW, as you change a testcase with known case number, your subject can be "generic/737: ..." (instead of "generic: ..."). > >> 2 files changed, 50 insertions(+), 1 deletion(-) > >> > >> diff --git a/tests/generic/737 b/tests/generic/737 > >> index 99ca1f39..0f27c82b 100755 > >> --- a/tests/generic/737 > >> +++ b/tests/generic/737 > >> @@ -4,7 +4,8 @@ > >> # > >> # FS QA Test No. 737 > >> # > >> -# Integrity test for O_SYNC with buff-io, dio, aio-dio with sudden shutdown. > >> +# Integrity test for O_[D]SYNC and/or RWF_DSYNC with buff-io, dio, aio-dio with > >> +# sudden shutdown. > >> # Based on a testcase reported by Gao Xiang <hsiangkao@linux.alibaba.com> > >> # > >> > >> @@ -21,6 +22,15 @@ _require_aiodio aio-dio-write-verify > >> _scratch_mkfs > $seqres.full 2>&1 > >> _scratch_mount > >> > >> +echo "T-0: Create a 1M file using buff-io & RWF_DSYNC" > >> +$XFS_IO_PROG -f -c "pwrite -V 1 -D -S 0x5a 0 1M" $SCRATCH_MNT/testfile.t1 > /dev/null 2>&1 > >> +echo "T-0: Shutdown the fs suddenly" > >> +_scratch_shutdown > >> +echo "T-0: Cycle mount" > >> +_scratch_cycle_mount > >> +echo "T-0: File contents after cycle mount" > >> +_hexdump $SCRATCH_MNT/testfile.t1 > >> + > >> echo "T-1: Create a 1M file using buff-io & O_SYNC" > >> $XFS_IO_PROG -fs -c "pwrite -S 0x5a 0 1M" $SCRATCH_MNT/testfile.t1 > /dev/null 2>&1 > >> echo "T-1: Shutdown the fs suddenly" > >> @@ -48,5 +58,23 @@ _scratch_cycle_mount > >> echo "T-3: File contents after cycle mount" > >> _hexdump $SCRATCH_MNT/testfile.t3 > >> > >> +echo "T-4: Create a 1M file using DIO & RWF_DSYNC" > >> +$XFS_IO_PROG -fdc "pwrite -V 1 -S 0x5a -D 0 1M" $SCRATCH_MNT/testfile.t4 > /dev/null 2>&1 > >> +echo "T-4: Shutdown the fs suddenly" > >> +_scratch_shutdown > >> +echo "T-4: Cycle mount" > >> +_scratch_cycle_mount > >> +echo "T-4: File contents after cycle mount" > >> +_hexdump $SCRATCH_MNT/testfile.t4 > >> + > >> +echo "T-5: Create a 1M file using AIO-DIO & O_DSYNC" > >> +$AIO_TEST -a size=1048576 -D -N $SCRATCH_MNT/testfile.t5 > /dev/null 2>&1 > >> +echo "T-5: Shutdown the fs suddenly" > >> +_scratch_shutdown > >> +echo "T-5: Cycle mount" > >> +_scratch_cycle_mount > >> +echo "T-5: File contents after cycle mount" > >> +_hexdump $SCRATCH_MNT/testfile.t5 > > > > I always hit "No such file or directory" [1], is this an expected test failure > > which you hope to uncover? > > Yes, we will need this fix [1] from Jan. Sorry I missed to add that in the commit > message. Could you please give it a try with the fix maybe? Sure, with this patch, this test passed on my side: FSTYP -- xfs (debug) PLATFORM -- Linux/x86_64 dell-per750-41 6.16.0-mainline+ #9 SMP PREEMPT_DYNAMIC Sat Aug 2 16:01:22 CST 2025 MKFS_OPTIONS -- -f /dev/mapper/testvg-scratch--devA MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /dev/mapper/testvg-scratch--devA /mnt/scratch generic/737 4s ... 9s Ran: generic/737 Passed all 1 tests So this "No such file or directory" failure is a known bug, and this patch trys to uncover it. You didn't metion that, so I thought you just try to write a new integrity test (rather than a regression test) :-D If this change uncover a known fix, please feel free to add _fixed_by_kernel_commit. BTW, I saw you mark this patchset with "RFC", do you need to change it more? Generally I won't merge RFC patches directly, although this patch looks good to me. So when you think you've gotten enough review points, please feel free to remove the "RFC" label and resend it :) Then ... Reviewed-by: Zorro Lang <zlang@redhat.com> Thanks, Zorro > > [1]: > https://lore.kernel.org/linux-fsdevel/20250730102840.20470-2-jack@suse.cz/ > > -ritesh > > > > > > [1] > > # diff -u /root/git/xfstests/tests/generic/737.out /root/git/xfstests/results//generic/737.out.bad > > --- /root/git/xfstests/tests/generic/737.out 2025-08-02 04:04:57.334489725 +0800 > > +++ /root/git/xfstests/results//generic/737.out.bad 2025-08-02 04:12:08.167934723 +0800 > > @@ -28,16 +28,14 @@ > > * > > 100000 > > T-4: Create a 1M file using DIO & RWF_DSYNC > > T-4: Shutdown the fs suddenly > > T-4: Cycle mount > > T-4: File contents after cycle mount > > -000000 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a >ZZZZZZZZZZZZZZZZ< > > -* > > -100000 > > +od: /mnt/scratch/testfile.t4: No such file or directory > > T-5: Create a 1M file using AIO-DIO & O_DSYNC > > T-5: Shutdown the fs suddenly > > T-5: Cycle mount > > T-5: File contents after cycle mount > > -000000 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a >ZZZZZZZZZZZZZZZZ< > > -* > > -100000 > > +od: /mnt/scratch/testfile.t5: No such file or directory > > > >> + > >> status=0 > >> exit > >> diff --git a/tests/generic/737.out b/tests/generic/737.out > >> index efe6ff1f..2bafeefa 100644 > >> --- a/tests/generic/737.out > >> +++ b/tests/generic/737.out > >> @@ -1,4 +1,11 @@ > >> QA output created by 737 > >> +T-0: Create a 1M file using buff-io & RWF_DSYNC > >> +T-0: Shutdown the fs suddenly > >> +T-0: Cycle mount > >> +T-0: File contents after cycle mount > >> +000000 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a >ZZZZZZZZZZZZZZZZ< > >> +* > >> +100000 > >> T-1: Create a 1M file using buff-io & O_SYNC > >> T-1: Shutdown the fs suddenly > >> T-1: Cycle mount > >> @@ -20,3 +27,17 @@ T-3: File contents after cycle mount > >> 000000 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a >ZZZZZZZZZZZZZZZZ< > >> * > >> 100000 > >> +T-4: Create a 1M file using DIO & RWF_DSYNC > >> +T-4: Shutdown the fs suddenly > >> +T-4: Cycle mount > >> +T-4: File contents after cycle mount > >> +000000 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a >ZZZZZZZZZZZZZZZZ< > >> +* > >> +100000 > >> +T-5: Create a 1M file using AIO-DIO & O_DSYNC > >> +T-5: Shutdown the fs suddenly > >> +T-5: Cycle mount > >> +T-5: File contents after cycle mount > >> +000000 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a >ZZZZZZZZZZZZZZZZ< > >> +* > >> +100000 > >> -- > >> 2.49.0 > >> > >> > ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [RFC 2/2] generic: Add integrity tests for O_DSYNC and RWF_DSYNC writes 2025-08-02 8:17 ` Zorro Lang @ 2025-08-03 5:23 ` Ritesh Harjani 0 siblings, 0 replies; 8+ messages in thread From: Ritesh Harjani @ 2025-08-03 5:23 UTC (permalink / raw) To: Zorro Lang; +Cc: fstests, linux-fsdevel, Jan Kara, Darrick J . Wong, John Garry Zorro Lang <zlang@redhat.com> writes: > On Sat, Aug 02, 2025 at 01:59:18AM +0530, Ritesh Harjani wrote: >> Zorro Lang <zlang@redhat.com> writes: >> >> > On Thu, Jul 31, 2025 at 06:05:55PM +0530, Ritesh Harjani (IBM) wrote: >> >> This test verifies the data & required metadata (e.g. inode i_size for extending >> >> writes) integrity when using O_DSYNC and RWF_DSYNC during writes operations, >> >> across buffered-io, aio-dio and dio, in the event of a sudden filesystem >> >> shutdown after write completion. >> >> >> >> Man page of open says that - >> >> O_DSYNC provides synchronized I/O data integrity completion, meaning >> >> write operations will flush data to the underlying hardware, but will >> >> only flush metadata updates that are required to allow a subsequent read >> >> operation to complete successfully. >> >> >> >> Signed-off-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com> >> >> --- >> >> tests/generic/737 | 30 +++++++++++++++++++++++++++++- >> >> tests/generic/737.out | 21 +++++++++++++++++++++ > > BTW, as you change a testcase with known case number, your subject can be > "generic/737: ..." (instead of "generic: ..."). > Make sense. I will fix in v2. >> >> 2 files changed, 50 insertions(+), 1 deletion(-) >> >> >> >> diff --git a/tests/generic/737 b/tests/generic/737 >> >> index 99ca1f39..0f27c82b 100755 >> >> --- a/tests/generic/737 >> >> +++ b/tests/generic/737 >> >> @@ -4,7 +4,8 @@ >> >> # >> >> # FS QA Test No. 737 >> >> # >> >> -# Integrity test for O_SYNC with buff-io, dio, aio-dio with sudden shutdown. >> >> +# Integrity test for O_[D]SYNC and/or RWF_DSYNC with buff-io, dio, aio-dio with >> >> +# sudden shutdown. >> >> # Based on a testcase reported by Gao Xiang <hsiangkao@linux.alibaba.com> >> >> # >> >> >> >> @@ -21,6 +22,15 @@ _require_aiodio aio-dio-write-verify >> >> _scratch_mkfs > $seqres.full 2>&1 >> >> _scratch_mount >> >> >> >> +echo "T-0: Create a 1M file using buff-io & RWF_DSYNC" >> >> +$XFS_IO_PROG -f -c "pwrite -V 1 -D -S 0x5a 0 1M" $SCRATCH_MNT/testfile.t1 > /dev/null 2>&1 >> >> +echo "T-0: Shutdown the fs suddenly" >> >> +_scratch_shutdown >> >> +echo "T-0: Cycle mount" >> >> +_scratch_cycle_mount >> >> +echo "T-0: File contents after cycle mount" >> >> +_hexdump $SCRATCH_MNT/testfile.t1 >> >> + >> >> echo "T-1: Create a 1M file using buff-io & O_SYNC" >> >> $XFS_IO_PROG -fs -c "pwrite -S 0x5a 0 1M" $SCRATCH_MNT/testfile.t1 > /dev/null 2>&1 >> >> echo "T-1: Shutdown the fs suddenly" >> >> @@ -48,5 +58,23 @@ _scratch_cycle_mount >> >> echo "T-3: File contents after cycle mount" >> >> _hexdump $SCRATCH_MNT/testfile.t3 >> >> >> >> +echo "T-4: Create a 1M file using DIO & RWF_DSYNC" >> >> +$XFS_IO_PROG -fdc "pwrite -V 1 -S 0x5a -D 0 1M" $SCRATCH_MNT/testfile.t4 > /dev/null 2>&1 >> >> +echo "T-4: Shutdown the fs suddenly" >> >> +_scratch_shutdown >> >> +echo "T-4: Cycle mount" >> >> +_scratch_cycle_mount >> >> +echo "T-4: File contents after cycle mount" >> >> +_hexdump $SCRATCH_MNT/testfile.t4 >> >> + >> >> +echo "T-5: Create a 1M file using AIO-DIO & O_DSYNC" >> >> +$AIO_TEST -a size=1048576 -D -N $SCRATCH_MNT/testfile.t5 > /dev/null 2>&1 >> >> +echo "T-5: Shutdown the fs suddenly" >> >> +_scratch_shutdown >> >> +echo "T-5: Cycle mount" >> >> +_scratch_cycle_mount >> >> +echo "T-5: File contents after cycle mount" >> >> +_hexdump $SCRATCH_MNT/testfile.t5 >> > >> > I always hit "No such file or directory" [1], is this an expected test failure >> > which you hope to uncover? >> >> Yes, we will need this fix [1] from Jan. Sorry I missed to add that in the commit >> message. Could you please give it a try with the fix maybe? > > Sure, with this patch, this test passed on my side: > FSTYP -- xfs (debug) > PLATFORM -- Linux/x86_64 dell-per750-41 6.16.0-mainline+ #9 SMP PREEMPT_DYNAMIC Sat Aug 2 16:01:22 CST 2025 > MKFS_OPTIONS -- -f /dev/mapper/testvg-scratch--devA > MOUNT_OPTIONS -- -o context=system_u:object_r:root_t:s0 /dev/mapper/testvg-scratch--devA /mnt/scratch > > generic/737 4s ... 9s > Ran: generic/737 > Passed all 1 tests > > So this "No such file or directory" failure is a known bug, and this patch trys to > uncover it. You didn't metion that, so I thought you just try to write a > new integrity test (rather than a regression test) :-D Your understanding is correct. Sorry, I missed to add the fix details. I posted the test before the fix got merged. But yes, I will add the necessary details in v2. > If this change uncover a known fix, please feel free to add _fixed_by_kernel_commit. The commit isn't yet showing up in the VFS tree. Once it gets in I will share the v2 with the following changes. diff --git a/tests/generic/737 b/tests/generic/737 index 0f27c82b..a51e9623 100755 --- a/tests/generic/737 +++ b/tests/generic/737 @@ -18,6 +18,9 @@ _require_aiodio aio-dio-write-verify [[ "$FSTYP" =~ ext[0-9]+ ]] && _fixed_by_kernel_commit 91562895f803 \ "ext4: properly sync file size update after O_SYNC direct IO" +# which is further fixed by +_fixed_by_kernel_commit 16f206eebbf8 \ + "iomap: Fix broken data integrity guarantees for O_SYNC writes" _scratch_mkfs > $seqres.full 2>&1 _scratch_mount > BTW, I saw you mark this patchset with "RFC", do you need to change it more? > Generally I won't merge RFC patches directly, although this patch looks good to > me. So when you think you've gotten enough review points, please feel free to > remove the "RFC" label and resend it :) Then ... > Right. I will drop RFC and will resend it with above mentioned points (once the fix appears in VFS tree). > Reviewed-by: Zorro Lang <zlang@redhat.com> > Thanks! -ritesh > Thanks, > Zorro > ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [RFC 1/2] aio-dio-write-verify: Add O_DSYNC option 2025-07-31 12:35 [RFC 1/2] aio-dio-write-verify: Add O_DSYNC option Ritesh Harjani (IBM) 2025-07-31 12:35 ` [RFC 2/2] generic: Add integrity tests for O_DSYNC and RWF_DSYNC writes Ritesh Harjani (IBM) @ 2025-07-31 15:00 ` Jan Kara 2025-08-02 8:18 ` Zorro Lang 2 siblings, 0 replies; 8+ messages in thread From: Jan Kara @ 2025-07-31 15:00 UTC (permalink / raw) To: Ritesh Harjani (IBM) Cc: fstests, linux-fsdevel, Jan Kara, Darrick J . Wong, John Garry On Thu 31-07-25 18:05:54, Ritesh Harjani (IBM) wrote: > This patch adds -D for O_DSYNC open flag to aio-dio-write-verify test. > We will use this in later patch for integrity verification test with > aio-dio. > > Signed-off-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com> Cool. Both patches look good to me and they fail without the iomap fix I've submitted so feel free to add: Reviewed-by: Jan Kara <jack@suse.cz> and Tested-by: Jan Kara <jack@suse.cz> Honza > --- > src/aio-dio-regress/aio-dio-write-verify.c | 6 +++++- > 1 file changed, 5 insertions(+), 1 deletion(-) > > diff --git a/src/aio-dio-regress/aio-dio-write-verify.c b/src/aio-dio-regress/aio-dio-write-verify.c > index 513a338b..0cf14a2a 100644 > --- a/src/aio-dio-regress/aio-dio-write-verify.c > +++ b/src/aio-dio-regress/aio-dio-write-verify.c > @@ -40,6 +40,7 @@ void usage(char *progname) > "\t\tsize=N: AIO write size\n" > "\t\toff=M: AIO write startoff\n" > "\t-S: uses O_SYNC flag for open. By default O_SYNC is not used\n" > + "\t-D: uses O_DSYNC flag for open. By default O_DSYNC is not used\n" > "\t-N: no_verify: means no write verification. By default noverify is false\n" > "e.g: %s -t 4608 -a size=4096,off=512 -a size=4096,off=4608 filename\n" > "e.g: %s -t 1048576 -a size=1048576 -S -N filename\n", > @@ -298,7 +299,7 @@ int main(int argc, char *argv[]) > int o_sync = 0; > int no_verify = 0; > > - while ((c = getopt(argc, argv, "a:t:SN")) != -1) { > + while ((c = getopt(argc, argv, "a:t:SND")) != -1) { > char *endp; > > switch (c) { > @@ -316,6 +317,9 @@ int main(int argc, char *argv[]) > case 'S': > o_sync = O_SYNC; > break; > + case 'D': > + o_sync = O_DSYNC; > + break; > case 'N': > no_verify = 1; > break; > -- > 2.49.0 > -- Jan Kara <jack@suse.com> SUSE Labs, CR ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [RFC 1/2] aio-dio-write-verify: Add O_DSYNC option 2025-07-31 12:35 [RFC 1/2] aio-dio-write-verify: Add O_DSYNC option Ritesh Harjani (IBM) 2025-07-31 12:35 ` [RFC 2/2] generic: Add integrity tests for O_DSYNC and RWF_DSYNC writes Ritesh Harjani (IBM) 2025-07-31 15:00 ` [RFC 1/2] aio-dio-write-verify: Add O_DSYNC option Jan Kara @ 2025-08-02 8:18 ` Zorro Lang 2 siblings, 0 replies; 8+ messages in thread From: Zorro Lang @ 2025-08-02 8:18 UTC (permalink / raw) To: Ritesh Harjani (IBM) Cc: fstests, linux-fsdevel, Jan Kara, Darrick J . Wong, John Garry On Thu, Jul 31, 2025 at 06:05:54PM +0530, Ritesh Harjani (IBM) wrote: > This patch adds -D for O_DSYNC open flag to aio-dio-write-verify test. > We will use this in later patch for integrity verification test with > aio-dio. > > Signed-off-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com> > --- Reviewed-by: Zorro Lang <zlang@redhat.com> > src/aio-dio-regress/aio-dio-write-verify.c | 6 +++++- > 1 file changed, 5 insertions(+), 1 deletion(-) > > diff --git a/src/aio-dio-regress/aio-dio-write-verify.c b/src/aio-dio-regress/aio-dio-write-verify.c > index 513a338b..0cf14a2a 100644 > --- a/src/aio-dio-regress/aio-dio-write-verify.c > +++ b/src/aio-dio-regress/aio-dio-write-verify.c > @@ -40,6 +40,7 @@ void usage(char *progname) > "\t\tsize=N: AIO write size\n" > "\t\toff=M: AIO write startoff\n" > "\t-S: uses O_SYNC flag for open. By default O_SYNC is not used\n" > + "\t-D: uses O_DSYNC flag for open. By default O_DSYNC is not used\n" > "\t-N: no_verify: means no write verification. By default noverify is false\n" > "e.g: %s -t 4608 -a size=4096,off=512 -a size=4096,off=4608 filename\n" > "e.g: %s -t 1048576 -a size=1048576 -S -N filename\n", > @@ -298,7 +299,7 @@ int main(int argc, char *argv[]) > int o_sync = 0; > int no_verify = 0; > > - while ((c = getopt(argc, argv, "a:t:SN")) != -1) { > + while ((c = getopt(argc, argv, "a:t:SND")) != -1) { > char *endp; > > switch (c) { > @@ -316,6 +317,9 @@ int main(int argc, char *argv[]) > case 'S': > o_sync = O_SYNC; > break; > + case 'D': > + o_sync = O_DSYNC; > + break; > case 'N': > no_verify = 1; > break; > -- > 2.49.0 > > ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2025-08-03 5:38 UTC | newest] Thread overview: 8+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2025-07-31 12:35 [RFC 1/2] aio-dio-write-verify: Add O_DSYNC option Ritesh Harjani (IBM) 2025-07-31 12:35 ` [RFC 2/2] generic: Add integrity tests for O_DSYNC and RWF_DSYNC writes Ritesh Harjani (IBM) 2025-08-01 20:18 ` Zorro Lang 2025-08-01 20:29 ` Ritesh Harjani 2025-08-02 8:17 ` Zorro Lang 2025-08-03 5:23 ` Ritesh Harjani 2025-07-31 15:00 ` [RFC 1/2] aio-dio-write-verify: Add O_DSYNC option Jan Kara 2025-08-02 8:18 ` Zorro Lang
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).