[PATCH] fstests: generic/563: use fs blocksize to do the writes

linux-btrfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed

* [PATCH] fstests: generic/563: use fs blocksize to do the writes
@ 2024-09-29 23:50 Qu Wenruo
  2024-10-03 19:15 ` Boris Burkov
                   ` (2 more replies)
  0 siblings, 3 replies; 7+ messages in thread
From: Qu Wenruo @ 2024-09-29 23:50 UTC (permalink / raw)
  To: linux-btrfs, fstests

[FALSE ALERTS]
If the system has a page size larger than 4K, and the fs block size
matches the page size, test case generic/563 will fail:

    --- tests/generic/563.out	2024-04-25 18:13:45.178550333 +0930
    +++ /home/adam/xfstests-dev/results//generic/563.out.bad	2024-09-30 09:09:16.155312379 +0930
    @@ -3,7 +3,8 @@
     read is in range
     write is in range
     write -> read/write
    -read is in range
    +read has value of 8388608
    +read is NOT in range -33792 .. 33792
     write is in range
    ...

Both Ext4 and btrfs fail with 64K block size and 64K page size

[CAUSE]
The test case writes the 8MiB file using the default block size xfs_io
pwrite, which is 4KiB.

Since the fs block size is 64K, such 4KiB write is unaligned inside a
block, causing the fs to read out the full page.

Thus the pwrite will cause the fs to read out every page, resulting the
above 8MiB+ read value.

[FIX]
Fix the test case by using the fs block size to avoid such unaligned
buffered write.

Signed-off-by: Qu Wenruo <wqu@suse.com>
---
 tests/generic/563 | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/tests/generic/563 b/tests/generic/563
index 0a8129a6..e8db8acf 100755
--- a/tests/generic/563
+++ b/tests/generic/563
@@ -94,6 +94,8 @@ sminor=$((0x`stat -L -c %T $LOOP_DEV`))
 _mkfs_dev $LOOP_DEV >> $seqres.full 2>&1
 _mount $LOOP_DEV $SCRATCH_MNT || _fail "mount failed"
 
+blksize=$(_get_block_size "$SCRATCH_MNT")
+
 drop_io_cgroup=
 grep -q -w io $cgdir/cgroup.subtree_control || drop_io_cgroup=1
 
@@ -103,7 +105,7 @@ echo "+io" > $cgdir/cgroup.subtree_control || _fail "subtree control"
 echo "read/write"
 reset
 switch_cg $cgdir/$seq-cg
-$XFS_IO_PROG -c "pread 0 $iosize" -c "pwrite 0 $iosize" -c fsync \
+$XFS_IO_PROG -c "pread 0 $iosize" -c "pwrite -b $blksize 0 $iosize" -c fsync \
 	$SCRATCH_MNT/file >> $seqres.full 2>&1
 switch_cg $cgdir
 $XFS_IO_PROG -c fsync $SCRATCH_MNT/file
@@ -114,9 +116,9 @@ check_cg $cgdir/$seq-cg $iosize $iosize 5% 5%
 echo "write -> read/write"
 reset
 switch_cg $cgdir/$seq-cg
-$XFS_IO_PROG -c "pwrite 0 $iosize" $SCRATCH_MNT/file >> $seqres.full 2>&1
+$XFS_IO_PROG -c "pwrite -b $blksize 0 $iosize" $SCRATCH_MNT/file >> $seqres.full 2>&1
 switch_cg $cgdir/$seq-cg-2
-$XFS_IO_PROG -c "pread 0 $iosize" -c "pwrite 0 $iosize" $SCRATCH_MNT/file \
+$XFS_IO_PROG -c "pread 0 $iosize" -c "pwrite -b $blksize 0 $iosize" $SCRATCH_MNT/file \
 	>> $seqres.full 2>&1
 switch_cg $cgdir
 $XFS_IO_PROG -c fsync $SCRATCH_MNT/file
@@ -134,7 +136,7 @@ reset
 switch_cg $cgdir/$seq-cg
 $XFS_IO_PROG -c "pread 0 $iosize" $SCRATCH_MNT/file >> $seqres.full 2>&1
 switch_cg $cgdir/$seq-cg-2
-$XFS_IO_PROG -c "pread 0 $iosize" -c "pwrite 0 $iosize" $SCRATCH_MNT/file \
+$XFS_IO_PROG -c "pread 0 $iosize" -c "pwrite -b $blksize 0 $iosize" $SCRATCH_MNT/file \
 	>> $seqres.full 2>&1
 switch_cg $cgdir
 $XFS_IO_PROG -c fsync $SCRATCH_MNT/file
-- 
2.46.0


^ permalink raw reply related	[flat|nested] 7+ messages in thread

* Re: [PATCH] fstests: generic/563: use fs blocksize to do the writes
  2024-09-29 23:50 [PATCH] fstests: generic/563: use fs blocksize to do the writes Qu Wenruo
@ 2024-10-03 19:15 ` Boris Burkov
  2024-10-04  9:45 ` Mark Harmstone
  2024-10-09  9:28 ` Qu Wenruo
  2 siblings, 0 replies; 7+ messages in thread
From: Boris Burkov @ 2024-10-03 19:15 UTC (permalink / raw)
  To: Qu Wenruo; +Cc: linux-btrfs, fstests

On Mon, Sep 30, 2024 at 09:20:38AM +0930, Qu Wenruo wrote:
> [FALSE ALERTS]
> If the system has a page size larger than 4K, and the fs block size
> matches the page size, test case generic/563 will fail:
> 
>     --- tests/generic/563.out	2024-04-25 18:13:45.178550333 +0930
>     +++ /home/adam/xfstests-dev/results//generic/563.out.bad	2024-09-30 09:09:16.155312379 +0930
>     @@ -3,7 +3,8 @@
>      read is in range
>      write is in range
>      write -> read/write
>     -read is in range
>     +read has value of 8388608
>     +read is NOT in range -33792 .. 33792
>      write is in range
>     ...
> 
> Both Ext4 and btrfs fail with 64K block size and 64K page size
> 
> [CAUSE]
> The test case writes the 8MiB file using the default block size xfs_io
> pwrite, which is 4KiB.
> 
> Since the fs block size is 64K, such 4KiB write is unaligned inside a
> block, causing the fs to read out the full page.
> 
> Thus the pwrite will cause the fs to read out every page, resulting the
> above 8MiB+ read value.
> 
> [FIX]
> Fix the test case by using the fs block size to avoid such unaligned
> buffered write.
> 
> Signed-off-by: Qu Wenruo <wqu@suse.com>

The rationale and change look good to me. I also tested it on my system
with blocksize=pagesize=4k, though I don't have access to one with a 64k
page size for the more interesting confirmation.

Reviewed-by: Boris Burkov <boris@bur.io>

> ---
>  tests/generic/563 | 10 ++++++----
>  1 file changed, 6 insertions(+), 4 deletions(-)
> 
> diff --git a/tests/generic/563 b/tests/generic/563
> index 0a8129a6..e8db8acf 100755
> --- a/tests/generic/563
> +++ b/tests/generic/563
> @@ -94,6 +94,8 @@ sminor=$((0x`stat -L -c %T $LOOP_DEV`))
>  _mkfs_dev $LOOP_DEV >> $seqres.full 2>&1
>  _mount $LOOP_DEV $SCRATCH_MNT || _fail "mount failed"
>  
> +blksize=$(_get_block_size "$SCRATCH_MNT")
> +
>  drop_io_cgroup=
>  grep -q -w io $cgdir/cgroup.subtree_control || drop_io_cgroup=1
>  
> @@ -103,7 +105,7 @@ echo "+io" > $cgdir/cgroup.subtree_control || _fail "subtree control"
>  echo "read/write"
>  reset
>  switch_cg $cgdir/$seq-cg
> -$XFS_IO_PROG -c "pread 0 $iosize" -c "pwrite 0 $iosize" -c fsync \
> +$XFS_IO_PROG -c "pread 0 $iosize" -c "pwrite -b $blksize 0 $iosize" -c fsync \
>  	$SCRATCH_MNT/file >> $seqres.full 2>&1
>  switch_cg $cgdir
>  $XFS_IO_PROG -c fsync $SCRATCH_MNT/file
> @@ -114,9 +116,9 @@ check_cg $cgdir/$seq-cg $iosize $iosize 5% 5%
>  echo "write -> read/write"
>  reset
>  switch_cg $cgdir/$seq-cg
> -$XFS_IO_PROG -c "pwrite 0 $iosize" $SCRATCH_MNT/file >> $seqres.full 2>&1
> +$XFS_IO_PROG -c "pwrite -b $blksize 0 $iosize" $SCRATCH_MNT/file >> $seqres.full 2>&1
>  switch_cg $cgdir/$seq-cg-2
> -$XFS_IO_PROG -c "pread 0 $iosize" -c "pwrite 0 $iosize" $SCRATCH_MNT/file \
> +$XFS_IO_PROG -c "pread 0 $iosize" -c "pwrite -b $blksize 0 $iosize" $SCRATCH_MNT/file \
>  	>> $seqres.full 2>&1
>  switch_cg $cgdir
>  $XFS_IO_PROG -c fsync $SCRATCH_MNT/file
> @@ -134,7 +136,7 @@ reset
>  switch_cg $cgdir/$seq-cg
>  $XFS_IO_PROG -c "pread 0 $iosize" $SCRATCH_MNT/file >> $seqres.full 2>&1
>  switch_cg $cgdir/$seq-cg-2
> -$XFS_IO_PROG -c "pread 0 $iosize" -c "pwrite 0 $iosize" $SCRATCH_MNT/file \
> +$XFS_IO_PROG -c "pread 0 $iosize" -c "pwrite -b $blksize 0 $iosize" $SCRATCH_MNT/file \
>  	>> $seqres.full 2>&1
>  switch_cg $cgdir
>  $XFS_IO_PROG -c fsync $SCRATCH_MNT/file
> -- 
> 2.46.0
> 

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH] fstests: generic/563: use fs blocksize to do the writes
  2024-09-29 23:50 [PATCH] fstests: generic/563: use fs blocksize to do the writes Qu Wenruo
  2024-10-03 19:15 ` Boris Burkov
@ 2024-10-04  9:45 ` Mark Harmstone
  2024-10-04  9:52   ` Qu Wenruo
  2024-10-09  9:28 ` Qu Wenruo
  2 siblings, 1 reply; 7+ messages in thread
From: Mark Harmstone @ 2024-10-04  9:45 UTC (permalink / raw)
  To: Qu Wenruo, linux-btrfs@vger.kernel.org, fstests@vger.kernel.org,
	Boris Burkov

On 30/9/24 00:50, Qu Wenruo wrote:
> > 
> [FALSE ALERTS]
> If the system has a page size larger than 4K, and the fs block size
> matches the page size, test case generic/563 will fail:
> 
>      --- tests/generic/563.out	2024-04-25 18:13:45.178550333 +0930
>      +++ /home/adam/xfstests-dev/results//generic/563.out.bad	2024-09-30 09:09:16.155312379 +0930
>      @@ -3,7 +3,8 @@
>       read is in range
>       write is in range
>       write -> read/write
>      -read is in range
>      +read has value of 8388608
>      +read is NOT in range -33792 .. 33792
>       write is in range
>      ...
> 
> Both Ext4 and btrfs fail with 64K block size and 64K page size
> 
> [CAUSE]
> The test case writes the 8MiB file using the default block size xfs_io
> pwrite, which is 4KiB.
> 
> Since the fs block size is 64K, such 4KiB write is unaligned inside a
> block, causing the fs to read out the full page.
> 
> Thus the pwrite will cause the fs to read out every page, resulting the
> above 8MiB+ read value.
> 
> [FIX]
> Fix the test case by using the fs block size to avoid such unaligned
> buffered write.
> 

I ran generic/563 on a Raspberry Pi running 6.4 and with a 64K page 
size, and got a similar error:

FSTYP         -- btrfs
PLATFORM      -- Linux/aarch64 fstests-aarch64 6.4.3-arm64-g0ef0e2e48724 
#61 SMP Tue Aug  6 16:51:45 BST 2024
MKFS_OPTIONS  -- /dev/vdc
MOUNT_OPTIONS -- /dev/vdc /mnt/scratch-dir

generic/563       - output mismatch (see 
/root/xfstests/results//generic/563.out.bad)
     --- tests/generic/563.out   2024-08-05 10:33:23.000000000 -0000
     +++ /root/xfstests/results//generic/563.out.bad     2024-10-04 
09:35:51.433413098 -0000
     @@ -3,7 +3,8 @@
      read is in range
      write is in range
      write -> read/write
     -read is in range
     +read has value of 8421376
     +read is NOT in range -33792 .. 33792
      write is in range
     ...
     (Run 'diff -u /root/xfstests/tests/generic/563.out 
/root/xfstests/results//generic/563.out.bad'  to see the entire diff)
Ran: generic/563
Failures: generic/563
Failed 1 of 1 tests

The same happens whether the btrfs volume has a sector size of 4K or 
64K, and the patch doesn't seem to fix it.

Mark

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH] fstests: generic/563: use fs blocksize to do the writes
  2024-10-04  9:45 ` Mark Harmstone
@ 2024-10-04  9:52   ` Qu Wenruo
  2024-10-04 11:18     ` Mark Harmstone
  0 siblings, 1 reply; 7+ messages in thread
From: Qu Wenruo @ 2024-10-04  9:52 UTC (permalink / raw)
  To: Mark Harmstone, Qu Wenruo, linux-btrfs@vger.kernel.org,
	fstests@vger.kernel.org, Boris Burkov



在 2024/10/4 19:15, Mark Harmstone 写道:
> On 30/9/24 00:50, Qu Wenruo wrote:
>>>
>> [FALSE ALERTS]
>> If the system has a page size larger than 4K, and the fs block size
>> matches the page size, test case generic/563 will fail:
>>
>>       --- tests/generic/563.out	2024-04-25 18:13:45.178550333 +0930
>>       +++ /home/adam/xfstests-dev/results//generic/563.out.bad	2024-09-30 09:09:16.155312379 +0930
>>       @@ -3,7 +3,8 @@
>>        read is in range
>>        write is in range
>>        write -> read/write
>>       -read is in range
>>       +read has value of 8388608
>>       +read is NOT in range -33792 .. 33792
>>        write is in range
>>       ...
>>
>> Both Ext4 and btrfs fail with 64K block size and 64K page size
>>
>> [CAUSE]
>> The test case writes the 8MiB file using the default block size xfs_io
>> pwrite, which is 4KiB.
>>
>> Since the fs block size is 64K, such 4KiB write is unaligned inside a
>> block, causing the fs to read out the full page.
>>
>> Thus the pwrite will cause the fs to read out every page, resulting the
>> above 8MiB+ read value.
>>
>> [FIX]
>> Fix the test case by using the fs block size to avoid such unaligned
>> buffered write.
>>
>
> I ran generic/563 on a Raspberry Pi running 6.4 and with a 64K page
> size, and got a similar error:
>
> FSTYP         -- btrfs
> PLATFORM      -- Linux/aarch64 fstests-aarch64 6.4.3-arm64-g0ef0e2e48724
> #61 SMP Tue Aug  6 16:51:45 BST 2024
> MKFS_OPTIONS  -- /dev/vdc
> MOUNT_OPTIONS -- /dev/vdc /mnt/scratch-dir
>
> generic/563       - output mismatch (see
> /root/xfstests/results//generic/563.out.bad)
>       --- tests/generic/563.out   2024-08-05 10:33:23.000000000 -0000
>       +++ /root/xfstests/results//generic/563.out.bad     2024-10-04
> 09:35:51.433413098 -0000
>       @@ -3,7 +3,8 @@
>        read is in range
>        write is in range
>        write -> read/write
>       -read is in range
>       +read has value of 8421376
>       +read is NOT in range -33792 .. 33792
>        write is in range
>       ...
>       (Run 'diff -u /root/xfstests/tests/generic/563.out
> /root/xfstests/results//generic/563.out.bad'  to see the entire diff)
> Ran: generic/563
> Failures: generic/563
> Failed 1 of 1 tests
>
> The same happens whether the btrfs volume has a sector size of 4K or
> 64K, and the patch doesn't seem to fix it.

For 4K sector size 64K page size btrfs, it needs several kernel patches
to proper fix it.
(https://github.com/adam900710/linux/tree/subpage_read)


The above case only shows the 4K sector size case (the new default of
mkfs.btrfs, no matter page size now).

For 64K sectorsize with 64K page size case, you need to specify the "-s
64K" mkfs option, apply the patch, only after that the test can pass:

Unpatched:

FSTYP         -- btrfs
PLATFORM      -- Linux/aarch64 btrfs-aarch64 6.11.0-rc7-custom+ #70 SMP
PREEMPT_DYNAMIC Thu Oct  3 07:25:40 ACST 2024
MKFS_OPTIONS  -- -s 64k /dev/mapper/test-scratch1
MOUNT_OPTIONS -- /dev/mapper/test-scratch1 /mnt/scratch

generic/563 4s ... - output mismatch (see
/home/adam/xfstests-dev/results//generic/563.out.bad)
     --- tests/generic/563.out	2024-04-25 18:13:45.178550333 +0930
     +++ /home/adam/xfstests-dev/results//generic/563.out.bad	2024-10-04
19:19:19.699153483 +0930
     @@ -3,7 +3,8 @@
      read is in range
      write is in range
      write -> read/write
     -read is in range
     +read has value of 8388608
     +read is NOT in range -33792 .. 33792
      write is in range
     ...
     (Run 'diff -u /home/adam/xfstests-dev/tests/generic/563.out
/home/adam/xfstests-dev/results//generic/563.out.bad'  to see the entire
diff)
Ran: generic/563
Failures: generic/563
Failed 1 of 1 tests

Patched:

FSTYP         -- btrfs
PLATFORM      -- Linux/aarch64 btrfs-aarch64 6.11.0-rc7-custom+ #70 SMP
PREEMPT_DYNAMIC Thu Oct  3 07:25:40 ACST 2024
MKFS_OPTIONS  -- -s 64k /dev/mapper/test-scratch1
MOUNT_OPTIONS -- /dev/mapper/test-scratch1 /mnt/scratch

generic/563 4s ...  1s
Ran: generic/563
Passed all 1 tests


You can also do the same using ext4:

Unpatched:

FSTYP         -- ext4
PLATFORM      -- Linux/aarch64 btrfs-aarch64 6.11.0-rc7-custom+ #70 SMP
PREEMPT_DYNAMIC Thu Oct  3 07:25:40 ACST 2024
MKFS_OPTIONS  -- -F -b 64k /dev/mapper/test-scratch1
MOUNT_OPTIONS -- -o acl,user_xattr /dev/mapper/test-scratch1 /mnt/scratch

generic/563 1s ... - output mismatch (see
/home/adam/xfstests-dev/results//generic/563.out.bad)
     --- tests/generic/563.out	2024-04-25 18:13:45.178550333 +0930
     +++ /home/adam/xfstests-dev/results//generic/563.out.bad	2024-10-04
19:21:23.377651352 +0930
     @@ -3,7 +3,8 @@
      read is in range
      write is in range
      write -> read/write
     -read is in range
     +read has value of 8388608
     +read is NOT in range -33792 .. 33792
      write is in range
     ...
     (Run 'diff -u /home/adam/xfstests-dev/tests/generic/563.out
/home/adam/xfstests-dev/results//generic/563.out.bad'  to see the entire
diff)
Ran: generic/563
Failures: generic/563
Failed 1 of 1 tests

Patched:

FSTYP         -- ext4
PLATFORM      -- Linux/aarch64 btrfs-aarch64 6.11.0-rc7-custom+ #70 SMP
PREEMPT_DYNAMIC Thu Oct  3 07:25:40 ACST 2024
MKFS_OPTIONS  -- -F -b 64k /dev/mapper/test-scratch1
MOUNT_OPTIONS -- -o acl,user_xattr /dev/mapper/test-scratch1 /mnt/scratch

generic/563 1s ...  1s
Ran: generic/563
Passed all 1 tests


Mind to re-verify with the proper mkfs options?

Thanks,
Qu
>
> Mark
>


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH] fstests: generic/563: use fs blocksize to do the writes
  2024-10-04  9:52   ` Qu Wenruo
@ 2024-10-04 11:18     ` Mark Harmstone
  0 siblings, 0 replies; 7+ messages in thread
From: Mark Harmstone @ 2024-10-04 11:18 UTC (permalink / raw)
  To: Qu Wenruo, Qu Wenruo, linux-btrfs@vger.kernel.org,
	fstests@vger.kernel.org, Boris Burkov

On 4/10/24 10:52, Qu Wenruo wrote:
> For 4K sector size 64K page size btrfs, it needs several kernel patches
> to proper fix it.
> (https://github.com/adam900710/linux/tree/subpage_read)
> 
> 
> The above case only shows the 4K sector size case (the new default of
> mkfs.btrfs, no matter page size now).
> 
> For 64K sectorsize with 64K page size case, you need to specify the "-s
> 64K" mkfs option, apply the patch, only after that the test can pass:

Ah, you have to specify an environment variable, because fstests wipes 
the filesystem every time. I knew I must have been doing something wrong.

Yes, with this change it works for me.

Reviewed-by: Mark Harmstone <maharmstone@fb.com>


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH] fstests: generic/563: use fs blocksize to do the writes
  2024-09-29 23:50 [PATCH] fstests: generic/563: use fs blocksize to do the writes Qu Wenruo
  2024-10-03 19:15 ` Boris Burkov
  2024-10-04  9:45 ` Mark Harmstone
@ 2024-10-09  9:28 ` Qu Wenruo
  2024-10-10  6:43   ` Zorro Lang
  2 siblings, 1 reply; 7+ messages in thread
From: Qu Wenruo @ 2024-10-09  9:28 UTC (permalink / raw)
  To: linux-btrfs, fstests, Zorro Lang

Hi Zorro,

Mind to merge this fix?

It got 2 reviews and at least one verification on 64K page systems.

Thanks,
Qu

在 2024/9/30 09:20, Qu Wenruo 写道:
> [FALSE ALERTS]
> If the system has a page size larger than 4K, and the fs block size
> matches the page size, test case generic/563 will fail:
> 
>      --- tests/generic/563.out	2024-04-25 18:13:45.178550333 +0930
>      +++ /home/adam/xfstests-dev/results//generic/563.out.bad	2024-09-30 09:09:16.155312379 +0930
>      @@ -3,7 +3,8 @@
>       read is in range
>       write is in range
>       write -> read/write
>      -read is in range
>      +read has value of 8388608
>      +read is NOT in range -33792 .. 33792
>       write is in range
>      ...
> 
> Both Ext4 and btrfs fail with 64K block size and 64K page size
> 
> [CAUSE]
> The test case writes the 8MiB file using the default block size xfs_io
> pwrite, which is 4KiB.
> 
> Since the fs block size is 64K, such 4KiB write is unaligned inside a
> block, causing the fs to read out the full page.
> 
> Thus the pwrite will cause the fs to read out every page, resulting the
> above 8MiB+ read value.
> 
> [FIX]
> Fix the test case by using the fs block size to avoid such unaligned
> buffered write.
> 
> Signed-off-by: Qu Wenruo <wqu@suse.com>
> ---
>   tests/generic/563 | 10 ++++++----
>   1 file changed, 6 insertions(+), 4 deletions(-)
> 
> diff --git a/tests/generic/563 b/tests/generic/563
> index 0a8129a6..e8db8acf 100755
> --- a/tests/generic/563
> +++ b/tests/generic/563
> @@ -94,6 +94,8 @@ sminor=$((0x`stat -L -c %T $LOOP_DEV`))
>   _mkfs_dev $LOOP_DEV >> $seqres.full 2>&1
>   _mount $LOOP_DEV $SCRATCH_MNT || _fail "mount failed"
>   
> +blksize=$(_get_block_size "$SCRATCH_MNT")
> +
>   drop_io_cgroup=
>   grep -q -w io $cgdir/cgroup.subtree_control || drop_io_cgroup=1
>   
> @@ -103,7 +105,7 @@ echo "+io" > $cgdir/cgroup.subtree_control || _fail "subtree control"
>   echo "read/write"
>   reset
>   switch_cg $cgdir/$seq-cg
> -$XFS_IO_PROG -c "pread 0 $iosize" -c "pwrite 0 $iosize" -c fsync \
> +$XFS_IO_PROG -c "pread 0 $iosize" -c "pwrite -b $blksize 0 $iosize" -c fsync \
>   	$SCRATCH_MNT/file >> $seqres.full 2>&1
>   switch_cg $cgdir
>   $XFS_IO_PROG -c fsync $SCRATCH_MNT/file
> @@ -114,9 +116,9 @@ check_cg $cgdir/$seq-cg $iosize $iosize 5% 5%
>   echo "write -> read/write"
>   reset
>   switch_cg $cgdir/$seq-cg
> -$XFS_IO_PROG -c "pwrite 0 $iosize" $SCRATCH_MNT/file >> $seqres.full 2>&1
> +$XFS_IO_PROG -c "pwrite -b $blksize 0 $iosize" $SCRATCH_MNT/file >> $seqres.full 2>&1
>   switch_cg $cgdir/$seq-cg-2
> -$XFS_IO_PROG -c "pread 0 $iosize" -c "pwrite 0 $iosize" $SCRATCH_MNT/file \
> +$XFS_IO_PROG -c "pread 0 $iosize" -c "pwrite -b $blksize 0 $iosize" $SCRATCH_MNT/file \
>   	>> $seqres.full 2>&1
>   switch_cg $cgdir
>   $XFS_IO_PROG -c fsync $SCRATCH_MNT/file
> @@ -134,7 +136,7 @@ reset
>   switch_cg $cgdir/$seq-cg
>   $XFS_IO_PROG -c "pread 0 $iosize" $SCRATCH_MNT/file >> $seqres.full 2>&1
>   switch_cg $cgdir/$seq-cg-2
> -$XFS_IO_PROG -c "pread 0 $iosize" -c "pwrite 0 $iosize" $SCRATCH_MNT/file \
> +$XFS_IO_PROG -c "pread 0 $iosize" -c "pwrite -b $blksize 0 $iosize" $SCRATCH_MNT/file \
>   	>> $seqres.full 2>&1
>   switch_cg $cgdir
>   $XFS_IO_PROG -c fsync $SCRATCH_MNT/file


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH] fstests: generic/563: use fs blocksize to do the writes
  2024-10-09  9:28 ` Qu Wenruo
@ 2024-10-10  6:43   ` Zorro Lang
  0 siblings, 0 replies; 7+ messages in thread
From: Zorro Lang @ 2024-10-10  6:43 UTC (permalink / raw)
  To: Qu Wenruo; +Cc: linux-btrfs, fstests

On Wed, Oct 09, 2024 at 07:58:19PM +1030, Qu Wenruo wrote:
> Hi Zorro,
> 
> Mind to merge this fix?
> 
> It got 2 reviews and at least one verification on 64K page systems.

Sure, it'll be merged in next release. Thanks for fix this large block size
test error. More filesystems support bigger blocksize, e.g. xfs supports
blocksize > pagesize. It's great to fix more errors like that :)

Thanks,
Zorro

> 
> Thanks,
> Qu
> 
> 在 2024/9/30 09:20, Qu Wenruo 写道:
> > [FALSE ALERTS]
> > If the system has a page size larger than 4K, and the fs block size
> > matches the page size, test case generic/563 will fail:
> > 
> >      --- tests/generic/563.out	2024-04-25 18:13:45.178550333 +0930
> >      +++ /home/adam/xfstests-dev/results//generic/563.out.bad	2024-09-30 09:09:16.155312379 +0930
> >      @@ -3,7 +3,8 @@
> >       read is in range
> >       write is in range
> >       write -> read/write
> >      -read is in range
> >      +read has value of 8388608
> >      +read is NOT in range -33792 .. 33792
> >       write is in range
> >      ...
> > 
> > Both Ext4 and btrfs fail with 64K block size and 64K page size
> > 
> > [CAUSE]
> > The test case writes the 8MiB file using the default block size xfs_io
> > pwrite, which is 4KiB.
> > 
> > Since the fs block size is 64K, such 4KiB write is unaligned inside a
> > block, causing the fs to read out the full page.
> > 
> > Thus the pwrite will cause the fs to read out every page, resulting the
> > above 8MiB+ read value.
> > 
> > [FIX]
> > Fix the test case by using the fs block size to avoid such unaligned
> > buffered write.
> > 
> > Signed-off-by: Qu Wenruo <wqu@suse.com>
> > ---
> >   tests/generic/563 | 10 ++++++----
> >   1 file changed, 6 insertions(+), 4 deletions(-)
> > 
> > diff --git a/tests/generic/563 b/tests/generic/563
> > index 0a8129a6..e8db8acf 100755
> > --- a/tests/generic/563
> > +++ b/tests/generic/563
> > @@ -94,6 +94,8 @@ sminor=$((0x`stat -L -c %T $LOOP_DEV`))
> >   _mkfs_dev $LOOP_DEV >> $seqres.full 2>&1
> >   _mount $LOOP_DEV $SCRATCH_MNT || _fail "mount failed"
> > +blksize=$(_get_block_size "$SCRATCH_MNT")
> > +
> >   drop_io_cgroup=
> >   grep -q -w io $cgdir/cgroup.subtree_control || drop_io_cgroup=1
> > @@ -103,7 +105,7 @@ echo "+io" > $cgdir/cgroup.subtree_control || _fail "subtree control"
> >   echo "read/write"
> >   reset
> >   switch_cg $cgdir/$seq-cg
> > -$XFS_IO_PROG -c "pread 0 $iosize" -c "pwrite 0 $iosize" -c fsync \
> > +$XFS_IO_PROG -c "pread 0 $iosize" -c "pwrite -b $blksize 0 $iosize" -c fsync \
> >   	$SCRATCH_MNT/file >> $seqres.full 2>&1
> >   switch_cg $cgdir
> >   $XFS_IO_PROG -c fsync $SCRATCH_MNT/file
> > @@ -114,9 +116,9 @@ check_cg $cgdir/$seq-cg $iosize $iosize 5% 5%
> >   echo "write -> read/write"
> >   reset
> >   switch_cg $cgdir/$seq-cg
> > -$XFS_IO_PROG -c "pwrite 0 $iosize" $SCRATCH_MNT/file >> $seqres.full 2>&1
> > +$XFS_IO_PROG -c "pwrite -b $blksize 0 $iosize" $SCRATCH_MNT/file >> $seqres.full 2>&1
> >   switch_cg $cgdir/$seq-cg-2
> > -$XFS_IO_PROG -c "pread 0 $iosize" -c "pwrite 0 $iosize" $SCRATCH_MNT/file \
> > +$XFS_IO_PROG -c "pread 0 $iosize" -c "pwrite -b $blksize 0 $iosize" $SCRATCH_MNT/file \
> >   	>> $seqres.full 2>&1
> >   switch_cg $cgdir
> >   $XFS_IO_PROG -c fsync $SCRATCH_MNT/file
> > @@ -134,7 +136,7 @@ reset
> >   switch_cg $cgdir/$seq-cg
> >   $XFS_IO_PROG -c "pread 0 $iosize" $SCRATCH_MNT/file >> $seqres.full 2>&1
> >   switch_cg $cgdir/$seq-cg-2
> > -$XFS_IO_PROG -c "pread 0 $iosize" -c "pwrite 0 $iosize" $SCRATCH_MNT/file \
> > +$XFS_IO_PROG -c "pread 0 $iosize" -c "pwrite -b $blksize 0 $iosize" $SCRATCH_MNT/file \
> >   	>> $seqres.full 2>&1
> >   switch_cg $cgdir
> >   $XFS_IO_PROG -c fsync $SCRATCH_MNT/file
> 
> 

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2024-10-10  6:43 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-09-29 23:50 [PATCH] fstests: generic/563: use fs blocksize to do the writes Qu Wenruo
2024-10-03 19:15 ` Boris Burkov
2024-10-04  9:45 ` Mark Harmstone
2024-10-04  9:52   ` Qu Wenruo
2024-10-04 11:18     ` Mark Harmstone
2024-10-09  9:28 ` Qu Wenruo
2024-10-10  6:43   ` Zorro Lang

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).