[PATCHSET 1/2] fstests: fix fstests for 64k fsblock size

[PATCHSET 1/2] fstests: fix fstests for 64k fsblock size

[Darrick J. Wong]

Hi all,

Various patches to fstests to make 64k fsblock filesystems mostly pass.
I haven't gone back through the remaining failures (mostly quota tests)
to figure out why they fail.

If you're going to start using this code, I strongly recommend pulling
from my git trees, which are linked below.

This has been running on the djcloud for months with no problems.  Enjoy!
Comments and questions are, as always, welcome.

--D

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=fix-64k-blocksize
---
Commits in this patchset:
 * common/xfs: _notrun tests that fail due to block size < sector size
 * xfs/161: adapt the test case for LBS filesystem
---
 common/xfs    |    5 +++++
 tests/xfs/161 |    6 +++++-
 2 files changed, 10 insertions(+), 1 deletion(-)

[PATCH 1/2] common/xfs: _notrun tests that fail due to block size < sector size

[Darrick J. Wong]

From: Darrick J. Wong <djwong@kernel.org>

It makes no sense to fail a test that failed to format a filesystem with
a block size smaller than the sector size since the test preconditions
are not valid.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/xfs |    5 +++++
 1 file changed, 5 insertions(+)


diff --git a/common/xfs b/common/xfs
index 62e3100ee117a7..53d55f9907fbb0 100644
--- a/common/xfs
+++ b/common/xfs
@@ -172,6 +172,11 @@ _try_scratch_mkfs_xfs()
 		mkfs_status=$?
 	fi
 
+	if [ $mkfs_status -ne 0 ] && grep -q '^block size [0-9]* cannot be smaller than sector size' "$tmp.mkfserr"; then
+		errormsg="$(grep '^block size [0-9]* cannot be smaller than sector size' "$tmp.mkfserr" | head -n 1)"
+		_notrun "_scratch_mkfs_xfs: $errormsg"
+	fi
+
 	# output mkfs stdout and stderr
 	cat $tmp.mkfsstd
 	cat $tmp.mkfserr >&2

Re: [PATCH 1/2] common/xfs: _notrun tests that fail due to block size < sector size

[Christoph Hellwig]

On Wed, Oct 16, 2024 at 04:15:16PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> It makes no sense to fail a test that failed to format a filesystem with
> a block size smaller than the sector size since the test preconditions
> are not valid.

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 1/2] common/xfs: _notrun tests that fail due to block size < sector size

[Zorro Lang]

On Wed, Oct 16, 2024 at 04:15:16PM -0700, Darrick J. Wong wrote:
> From: Darrick J. Wong <djwong@kernel.org>
> 
> It makes no sense to fail a test that failed to format a filesystem with
> a block size smaller than the sector size since the test preconditions
> are not valid.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---

Makes sense to me,
Reviewed-by: Zorro Lang <zlang@redhat.com>

>  common/xfs |    5 +++++
>  1 file changed, 5 insertions(+)
> 
> 
> diff --git a/common/xfs b/common/xfs
> index 62e3100ee117a7..53d55f9907fbb0 100644
> --- a/common/xfs
> +++ b/common/xfs
> @@ -172,6 +172,11 @@ _try_scratch_mkfs_xfs()
>  		mkfs_status=$?
>  	fi
>  
> +	if [ $mkfs_status -ne 0 ] && grep -q '^block size [0-9]* cannot be smaller than sector size' "$tmp.mkfserr"; then
> +		errormsg="$(grep '^block size [0-9]* cannot be smaller than sector size' "$tmp.mkfserr" | head -n 1)"
> +		_notrun "_scratch_mkfs_xfs: $errormsg"
> +	fi
> +
>  	# output mkfs stdout and stderr
>  	cat $tmp.mkfsstd
>  	cat $tmp.mkfserr >&2
> 

[PATCH 2/2] xfs/161: adapt the test case for LBS filesystem

[Darrick J. Wong]

From: Pankaj Raghav <p.raghav@samsung.com>

This test fails for >= 64k filesystem block size on a 4k PAGE_SIZE
system(see LBS efforts[1]). Adapt the blksz so that we create more than
one block for the testcase.

Cap the blksz to be at least 64k to retain the same behaviour as before
for smaller filesystem blocksizes.

[1] LBS effort: https://lore.kernel.org/lkml/20230915183848.1018717-1-kernel@pankajraghav.com/

Signed-off-by: Pankaj Raghav <p.raghav@samsung.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 tests/xfs/161 |    6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)


diff --git a/tests/xfs/161 b/tests/xfs/161
index 002ee7d800dcf1..948121c0569484 100755
--- a/tests/xfs/161
+++ b/tests/xfs/161
@@ -37,7 +37,11 @@ _scratch_xfs_db -c 'version' -c 'sb 0' -c 'p' >> $seqres.full
 _scratch_mount >> $seqres.full
 
 
-blksz=$(_get_file_block_size "$SCRATCH_MNT")
+min_blksz=65536
+file_blksz=$(_get_file_block_size "$SCRATCH_MNT")
+blksz=$(( 2 * $file_blksz))
+
+blksz=$(( blksz > min_blksz ? blksz : min_blksz ))
 # Write more than one block to exceed the soft block quota limit via
 # xfs_quota.
 filesz=$(( 2 * $blksz))

Re: [PATCH 2/2] xfs/161: adapt the test case for LBS filesystem

[Christoph Hellwig]

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 2/2] xfs/161: adapt the test case for LBS filesystem

[Zorro Lang]

On Wed, Oct 16, 2024 at 04:15:32PM -0700, Darrick J. Wong wrote:
> From: Pankaj Raghav <p.raghav@samsung.com>
> 
> This test fails for >= 64k filesystem block size on a 4k PAGE_SIZE
> system(see LBS efforts[1]). Adapt the blksz so that we create more than
> one block for the testcase.
> 
> Cap the blksz to be at least 64k to retain the same behaviour as before
> for smaller filesystem blocksizes.
> 
> [1] LBS effort: https://lore.kernel.org/lkml/20230915183848.1018717-1-kernel@pankajraghav.com/
> 
> Signed-off-by: Pankaj Raghav <p.raghav@samsung.com>
> Reviewed-by: Darrick J. Wong <djwong@kernel.org>
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  tests/xfs/161 |    6 +++++-
>  1 file changed, 5 insertions(+), 1 deletion(-)
> 
> 
> diff --git a/tests/xfs/161 b/tests/xfs/161
> index 002ee7d800dcf1..948121c0569484 100755
> --- a/tests/xfs/161
> +++ b/tests/xfs/161
> @@ -37,7 +37,11 @@ _scratch_xfs_db -c 'version' -c 'sb 0' -c 'p' >> $seqres.full
>  _scratch_mount >> $seqres.full
>  
>  
> -blksz=$(_get_file_block_size "$SCRATCH_MNT")
> +min_blksz=65536
> +file_blksz=$(_get_file_block_size "$SCRATCH_MNT")
> +blksz=$(( 2 * $file_blksz))
> +
> +blksz=$(( blksz > min_blksz ? blksz : min_blksz ))

Just to be curious, is there any machine with pagesize bigger than 65536 :)

Reviewed-by: Zorro Lang <zlang@redhat.com>

>  # Write more than one block to exceed the soft block quota limit via
>  # xfs_quota.
>  filesz=$(( 2 * $blksz))
> 

Re: [PATCHSET 1/2] fstests: fix fstests for 64k fsblock size

[Pankaj Raghav (Samsung)]

On Wed, Oct 16, 2024 at 04:15:05PM -0700, Darrick J. Wong wrote:
> Hi all,
> 
> Various patches to fstests to make 64k fsblock filesystems mostly pass.

Thanks for resending xfs/161.

> I haven't gone back through the remaining failures (mostly quota tests)
> to figure out why they fail.
I am on it. I will send the patches soon.

--
Pankaj

[PATCH 0/2] fstest changes for LBS

[Pankaj Raghav (Samsung)]

From: Pankaj Raghav <p.raghav@samsung.com>

Some tests need to be adapted to for LBS[1] based on the filesystem
blocksize. These are generic changes where it uses the filesystem
blocksize instead of assuming it.

There are some more generic test cases that are failing due to logdev
size requirement that changes with filesystem blocksize. I will address
them in a separate series.

[1] https://lore.kernel.org/lkml/20230915183848.1018717-1-kernel@pankajraghav.com/

Pankaj Raghav (2):
  xfs/558: scale blk IO size based on the filesystem blksz
  xfs/161: adapt the test case for LBS filesystem

 tests/xfs/161 | 9 +++++++--
 tests/xfs/558 | 7 ++++++-
 2 files changed, 13 insertions(+), 3 deletions(-)


base-commit: c46ca4d1f6c0c45f9a3ea18bc31ba5ae89e02c70
-- 
2.43.0

[PATCH 2/2] xfs/161: adapt the test case for LBS filesystem

[Pankaj Raghav (Samsung)]

From: Pankaj Raghav <p.raghav@samsung.com>

This test fails for >= 64k filesystem block size on a 4k PAGE_SIZE
system(see LBS efforts[1]). Adapt the blksz so that we create more than
one block for the testcase.

Cap the blksz to be at least 64k to retain the same behaviour as before
for smaller filesystem blocksizes.

[1] LBS effort: https://lore.kernel.org/lkml/20230915183848.1018717-1-kernel@pankajraghav.com/

Signed-off-by: Pankaj Raghav <p.raghav@samsung.com>
---
 tests/xfs/161 | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/tests/xfs/161 b/tests/xfs/161
index 486fa6ca..f7b03f0e 100755
--- a/tests/xfs/161
+++ b/tests/xfs/161
@@ -38,9 +38,14 @@ _qmount_option "usrquota"
 _scratch_xfs_db -c 'version' -c 'sb 0' -c 'p' >> $seqres.full
 _scratch_mount >> $seqres.full
 
+min_blksz=65536
+file_blksz=$(_get_file_block_size "$SCRATCH_MNT")
+blksz=$(( 2 * $file_blksz))
+
+blksz=$(( blksz > min_blksz ? blksz : min_blksz ))
 # Force the block counters for uid 1 and 2 above zero
-_pwrite_byte 0x61 0 64k $SCRATCH_MNT/a >> $seqres.full
-_pwrite_byte 0x61 0 64k $SCRATCH_MNT/b >> $seqres.full
+_pwrite_byte 0x61 0 $blksz $SCRATCH_MNT/a >> $seqres.full
+_pwrite_byte 0x61 0 $blksz $SCRATCH_MNT/b >> $seqres.full
 sync
 chown 1 $SCRATCH_MNT/a
 chown 2 $SCRATCH_MNT/b
-- 
2.43.0

Re: [PATCH 2/2] xfs/161: adapt the test case for LBS filesystem

[Darrick J. Wong]

On Mon, Jan 22, 2024 at 12:17:51PM +0100, Pankaj Raghav (Samsung) wrote:
> From: Pankaj Raghav <p.raghav@samsung.com>
> 
> This test fails for >= 64k filesystem block size on a 4k PAGE_SIZE
> system(see LBS efforts[1]). Adapt the blksz so that we create more than
> one block for the testcase.

How does this fail, specifically?  And, uh, what block sizes > 64k were
tested?

--D

> Cap the blksz to be at least 64k to retain the same behaviour as before
> for smaller filesystem blocksizes.
> 
> [1] LBS effort: https://lore.kernel.org/lkml/20230915183848.1018717-1-kernel@pankajraghav.com/
> 
> Signed-off-by: Pankaj Raghav <p.raghav@samsung.com>
> ---
>  tests/xfs/161 | 9 +++++++--
>  1 file changed, 7 insertions(+), 2 deletions(-)
> 
> diff --git a/tests/xfs/161 b/tests/xfs/161
> index 486fa6ca..f7b03f0e 100755
> --- a/tests/xfs/161
> +++ b/tests/xfs/161
> @@ -38,9 +38,14 @@ _qmount_option "usrquota"
>  _scratch_xfs_db -c 'version' -c 'sb 0' -c 'p' >> $seqres.full
>  _scratch_mount >> $seqres.full
>  
> +min_blksz=65536
> +file_blksz=$(_get_file_block_size "$SCRATCH_MNT")
> +blksz=$(( 2 * $file_blksz))
> +
> +blksz=$(( blksz > min_blksz ? blksz : min_blksz ))
>  # Force the block counters for uid 1 and 2 above zero
> -_pwrite_byte 0x61 0 64k $SCRATCH_MNT/a >> $seqres.full
> -_pwrite_byte 0x61 0 64k $SCRATCH_MNT/b >> $seqres.full
> +_pwrite_byte 0x61 0 $blksz $SCRATCH_MNT/a >> $seqres.full
> +_pwrite_byte 0x61 0 $blksz $SCRATCH_MNT/b >> $seqres.full
>  sync
>  chown 1 $SCRATCH_MNT/a
>  chown 2 $SCRATCH_MNT/b
> -- 
> 2.43.0
> 
> 

Re: [PATCH 2/2] xfs/161: adapt the test case for LBS filesystem

[Pankaj Raghav]

On 22/01/2024 17:57, Darrick J. Wong wrote:
> On Mon, Jan 22, 2024 at 12:17:51PM +0100, Pankaj Raghav (Samsung) wrote:
>> From: Pankaj Raghav <p.raghav@samsung.com>
>>
>> This test fails for >= 64k filesystem block size on a 4k PAGE_SIZE
>> system(see LBS efforts[1]). Adapt the blksz so that we create more than
>> one block for the testcase.
> And, uh, what block sizes > 64k were tested?

I thought I removed >= 64k and put just 64k before I sent the patches as we
don't allow FSB > 64k, for now. Hypothetically, due to the hardcoded 64k blksz, we might
face the same issue for > 64k FSB as well.
> How does this fail, specifically?

This is the output I get when I set the block size to be 64k:

QA output created by 161
Expected timer expiry (0) to be after now (1705944360).
Running xfs_repair to upgrade filesystem.
Adding large timestamp support to filesystem.
FEATURES: BIGTIME:YES
Expected uid 1 expiry (0) to be after now (1705944361).
Expected uid 2 expiry (0) to be after uid 1 (0).
Expected uid 2 expiry (0) to be after 2038.
Expected uid 1 expiry (0) to be after now (1705944361).
Expected uid 2 expiry (0) to be after uid 1 (0).
Expected uid 2 expiry (0) to be after 2038.
grace2 expiry has value of 0
grace2 expiry is NOT in range 7956915737 .. 7956915747
grace2 expiry after remount has value of 0
grace2 expiry after remount is NOT in range 7956915737 .. 7956915747

Seeing the comment: Force the block counters for uid 1 and 2 above zero,
I added the changes which fixed the issues for 64k FSB.

> --D
> 
>> Cap the blksz to be at least 64k to retain the same behaviour as before
>> for smaller filesystem blocksizes.
>>
>> [1] LBS effort: https://lore.kernel.org/lkml/20230915183848.1018717-1-kernel@pankajraghav.com/
>>
>> Signed-off-by: Pankaj Raghav <p.raghav@samsung.com>
>> ---
>>  tests/xfs/161 | 9 +++++++--
>>  1 file changed, 7 insertions(+), 2 deletions(-)
>>
>> diff --git a/tests/xfs/161 b/tests/xfs/161
>> index 486fa6ca..f7b03f0e 100755
>> --- a/tests/xfs/161
>> +++ b/tests/xfs/161
>> @@ -38,9 +38,14 @@ _qmount_option "usrquota"
>>  _scratch_xfs_db -c 'version' -c 'sb 0' -c 'p' >> $seqres.full
>>  _scratch_mount >> $seqres.full
>>  
>> +min_blksz=65536
>> +file_blksz=$(_get_file_block_size "$SCRATCH_MNT")
>> +blksz=$(( 2 * $file_blksz))
>> +
>> +blksz=$(( blksz > min_blksz ? blksz : min_blksz ))
>>  # Force the block counters for uid 1 and 2 above zero
>> -_pwrite_byte 0x61 0 64k $SCRATCH_MNT/a >> $seqres.full
>> -_pwrite_byte 0x61 0 64k $SCRATCH_MNT/b >> $seqres.full
>> +_pwrite_byte 0x61 0 $blksz $SCRATCH_MNT/a >> $seqres.full
>> +_pwrite_byte 0x61 0 $blksz $SCRATCH_MNT/b >> $seqres.full
>>  sync
>>  chown 1 $SCRATCH_MNT/a
>>  chown 2 $SCRATCH_MNT/b
>> -- 
>> 2.43.0
>>
>>

Re: [PATCH 2/2] xfs/161: adapt the test case for LBS filesystem

[Pankaj Raghav]

On 22/01/2024 17:57, Darrick J. Wong wrote:
> On Mon, Jan 22, 2024 at 12:17:51PM +0100, Pankaj Raghav (Samsung) wrote:
>> From: Pankaj Raghav <p.raghav@samsung.com>
>>
>> This test fails for >= 64k filesystem block size on a 4k PAGE_SIZE
>> system(see LBS efforts[1]). Adapt the blksz so that we create more than
>> one block for the testcase.
> 
> How does this fail, specifically?  And, uh, what block sizes > 64k were
> tested?
> 
> --D
> 
>> Cap the blksz to be at least 64k to retain the same behaviour as before
>> for smaller filesystem blocksizes.
>>
>> [1] LBS effort: https://lore.kernel.org/lkml/20230915183848.1018717-1-kernel@pankajraghav.com/
>>

I tried the same test on a machine with 64k page size and block size, and I get the same error
and this patch fixes it!

Kernel version: 6.7.1
xfstest version: for-next
PAGE_SIZE: 64k

# Without this patch

ubuntu@xfstest:/mnt/linux/xfstests$ getconf PAGE_SIZE
65536

ubuntu@xfstest:/mnt/linux/xfstests$ sudo ./check -s 64k xfs/161
SECTION       -- 64k
RECREATING    -- xfs on /dev/sdb2
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/aarch64 xfstest 6.7.1-64k #8 SMP Thu Jan 25 13:38:41 UTC 2024
MKFS_OPTIONS  -- -f -f -m reflink=1,rmapbt=1, -i sparse=1, -b size=64k, /dev/sdb3
MOUNT_OPTIONS -- /dev/sdb3 /mnt/scratch

xfs/161 6s ... - output mismatch (see /mnt/linux/xfstests/results/xfstest/6.7.1-64k/64k/xfs/161.out.bad)
    --- tests/xfs/161.out	2024-01-25 15:36:48.869401419 +0000
    +++ /mnt/linux/xfstests/results/xfstest/6.7.1-64k/64k/xfs/161.out.bad	2024-01-25
15:59:47.702340351 +0000
    @@ -1,6 +1,15 @@
     QA output created by 161
    +Expected timer expiry (0) to be after now (1706198386).
     Running xfs_repair to upgrade filesystem.
     Adding large timestamp support to filesystem.
     FEATURES: BIGTIME:YES
    -grace2 expiry is in range
    -grace2 expiry after remount is in range
    ...
    (Run 'diff -u /mnt/linux/xfstests/tests/xfs/161.out
/mnt/linux/xfstests/results/xfstest/6.7.1-64k/64k/xfs/161.out.bad'  to see the entire diff)
Ran: xfs/161
Failures: xfs/161
Failed 1 of 1 tests

SECTION       -- 64k
=========================
Ran: xfs/161
Failures: xfs/161
Failed 1 of 1 tests


# With this patch:

ubuntu@xfstest:/mnt/linux/xfstests$ sudo ./check -s 64k xfs/161
SECTION       -- 64k
RECREATING    -- xfs on /dev/sdb2
FSTYP         -- xfs (non-debug)
PLATFORM      -- Linux/aarch64 xfstest 6.7.1-64k #8 SMP Thu Jan 25 13:38:41 UTC 2024
MKFS_OPTIONS  -- -f -f -m reflink=1,rmapbt=1, -i sparse=1, -b size=64k, /dev/sdb3
MOUNT_OPTIONS -- /dev/sdb3 /mnt/scratch

xfs/161 6s ...  6s
Ran: xfs/161
Passed all 1 tests

SECTION       -- 64k
=========================
Ran: xfs/161
Passed all 1 tests

>> Signed-off-by: Pankaj Raghav <p.raghav@samsung.com>
>> ---
>>  tests/xfs/161 | 9 +++++++--
>>  1 file changed, 7 insertions(+), 2 deletions(-)
>>
>> diff --git a/tests/xfs/161 b/tests/xfs/161
>> index 486fa6ca..f7b03f0e 100755
>> --- a/tests/xfs/161
>> +++ b/tests/xfs/161
>> @@ -38,9 +38,14 @@ _qmount_option "usrquota"
>>  _scratch_xfs_db -c 'version' -c 'sb 0' -c 'p' >> $seqres.full
>>  _scratch_mount >> $seqres.full
>>  
>> +min_blksz=65536
>> +file_blksz=$(_get_file_block_size "$SCRATCH_MNT")
>> +blksz=$(( 2 * $file_blksz))
>> +
>> +blksz=$(( blksz > min_blksz ? blksz : min_blksz ))
>>  # Force the block counters for uid 1 and 2 above zero
>> -_pwrite_byte 0x61 0 64k $SCRATCH_MNT/a >> $seqres.full
>> -_pwrite_byte 0x61 0 64k $SCRATCH_MNT/b >> $seqres.full
>> +_pwrite_byte 0x61 0 $blksz $SCRATCH_MNT/a >> $seqres.full
>> +_pwrite_byte 0x61 0 $blksz $SCRATCH_MNT/b >> $seqres.full
>>  sync
>>  chown 1 $SCRATCH_MNT/a
>>  chown 2 $SCRATCH_MNT/b
>> -- 
>> 2.43.0
>>
>>