fstest failure due to filesystem size for 16k, 32k and 64k FSB

fstest failure due to filesystem size for 16k, 32k and 64k FSB

[Pankaj Raghav]

As I pointed out in my previous thread [1], there are some testcases
in fstests that are failing for FSB 16k, 32k and 64k due to the filesystem
**size** under test. These are failures **upstream** and not due to the ongoing
LBS work.

fstests creates a lot of tiny filesystems to perform some tests. Even though
the minimum fs size allowed to create XFS filesystem is 300 MB, we have special
condition in mkfs to allow smaller filesystems for fstest[2] (This took some time
to figure out as I was splitting my hair how fstest is able to create XFS on top of
25MB images).

The problem comes when we have FSB 16k, 32k and 64k. As we will
require more log space when we have this feature enabled, some test cases are failing
with the following error message:

max log size XXX smaller than min log size YYY, filesystem is too small

Most test cases run without this error message with **rmapbt disabled** for 16k and 64k (see
the test matrix below).

What should be the approach to solve this issue? 2 options that I had in my mind:

1. Similar to [2], we could add a small hack in mkfs xfs to ignore the log space
requirement while running fstests for these profiles.

2. Increase the size of filesystem under test to accommodate these profiles. It could
even be a conditional increase in filesystem size if the FSB > 16k to reduce the impact
on existing FS test time for 4k FSB.

Let me know what would be the best way to move forward.

Here are the results:

Test environment:
kernel Release: 6.8.0-rc1
xfsprogs: 6.5.0
Architecture: aarch64
Page size: 64k

Test matrix:

| Test        | 32k rmapbt=0 | 32k rmapbt=1 | 64k rmapbt=0 | 64k rmapbt=1 |
| --------    | ---------    | ---------    | ---------    | ---------    |
| generic/042 |     fail     |     fail     |     fail     |     fail     |
| generic/081 |     fail     |     fail     |     pass     |     fail     |
| generic/108 |     fail     |     fail     |     pass     |     fail     |
| generic/455 |     fail     |     fail     |     pass     |     fail     |
| generic/457 |     fail     |     fail     |     pass     |     fail     |
| generic/482 |     fail     |     fail     |     pass     |     fail     |
| generic/704 |     fail     |     fail     |     pass     |     fail     |
| generic/730 |     fail     |     fail     |     pass     |     fail     |
| generic/731 |     fail     |     fail     |     pass     |     fail     |
| shared/298  |     pass     |     pass     |     pass     |     fail     |

16k fails only on generic/042 for both rmapbt=0 and rmapbt=1


[1] https://lore.kernel.org/all/7964c404-bc9d-47ef-97f1-aaaba7d7aee9@samsung.com/
[2] xfsprogs commit: 6e0ed3d19c54603f0f7d628ea04b550151d8a262
-- 
Regards,
Pankaj

Re: fstest failure due to filesystem size for 16k, 32k and 64k FSB

[Darrick J. Wong]

On Tue, Jan 30, 2024 at 02:18:01PM +0100, Pankaj Raghav wrote:
> As I pointed out in my previous thread [1], there are some testcases
> in fstests that are failing for FSB 16k, 32k and 64k due to the filesystem
> **size** under test. These are failures **upstream** and not due to the ongoing
> LBS work.
> 
> fstests creates a lot of tiny filesystems to perform some tests. Even though
> the minimum fs size allowed to create XFS filesystem is 300 MB, we have special
> condition in mkfs to allow smaller filesystems for fstest[2] (This took some time
> to figure out as I was splitting my hair how fstest is able to create XFS on top of
> 25MB images).
> 
> The problem comes when we have FSB 16k, 32k and 64k. As we will
> require more log space when we have this feature enabled, some test cases are failing
> with the following error message:
> 
> max log size XXX smaller than min log size YYY, filesystem is too small
> 
> Most test cases run without this error message with **rmapbt disabled** for 16k and 64k (see
> the test matrix below).
> 
> What should be the approach to solve this issue? 2 options that I had in my mind:
> 
> 1. Similar to [2], we could add a small hack in mkfs xfs to ignore the log space
> requirement while running fstests for these profiles.
> 
> 2. Increase the size of filesystem under test to accommodate these profiles. It could
> even be a conditional increase in filesystem size if the FSB > 16k to reduce the impact
> on existing FS test time for 4k FSB.
> 
> Let me know what would be the best way to move forward.
> 
> Here are the results:
> 
> Test environment:
> kernel Release: 6.8.0-rc1
> xfsprogs: 6.5.0
> Architecture: aarch64
> Page size: 64k
> 
> Test matrix:
> 
> | Test        | 32k rmapbt=0 | 32k rmapbt=1 | 64k rmapbt=0 | 64k rmapbt=1 |
> | --------    | ---------    | ---------    | ---------    | ---------    |
> | generic/042 |     fail     |     fail     |     fail     |     fail     |
> | generic/081 |     fail     |     fail     |     pass     |     fail     |
> | generic/108 |     fail     |     fail     |     pass     |     fail     |
> | generic/455 |     fail     |     fail     |     pass     |     fail     |
> | generic/457 |     fail     |     fail     |     pass     |     fail     |
> | generic/482 |     fail     |     fail     |     pass     |     fail     |
> | generic/704 |     fail     |     fail     |     pass     |     fail     |
> | generic/730 |     fail     |     fail     |     pass     |     fail     |
> | generic/731 |     fail     |     fail     |     pass     |     fail     |
> | shared/298  |     pass     |     pass     |     pass     |     fail     |

I noticed test failures on these tests when running djwong-wtf:
generic/042
generic/081
generic/108
generic/219
generic/305
generic/326
generic/562
generic/704
xfs/093
xfs/113
xfs/161
xfs/262
xfs/508
xfs/604
xfs/709

Still sorting through all of them, but a large portion of them are the
same failure to format due to minimum log size constraints.  I'd bump
them up to ~500M (or whatever makes them work) since upstream doesn't
really support small filesystems anymore.

--D

> 
> 16k fails only on generic/042 for both rmapbt=0 and rmapbt=1
> 
> 
> [1] https://lore.kernel.org/all/7964c404-bc9d-47ef-97f1-aaaba7d7aee9@samsung.com/
> [2] xfsprogs commit: 6e0ed3d19c54603f0f7d628ea04b550151d8a262
> -- 
> Regards,
> Pankaj
> 

Re: fstest failure due to filesystem size for 16k, 32k and 64k FSB

[Pankaj Raghav]

>> What should be the approach to solve this issue? 2 options that I had in my mind:
>>
>> 1. Similar to [2], we could add a small hack in mkfs xfs to ignore the log space
>> requirement while running fstests for these profiles.
>>
>> 2. Increase the size of filesystem under test to accommodate these profiles. It could
>> even be a conditional increase in filesystem size if the FSB > 16k to reduce the impact
>> on existing FS test time for 4k FSB.
>>
>> Let me know what would be the best way to move forward.
>>
>> Here are the results:
>>
>> Test environment:
>> kernel Release: 6.8.0-rc1
>> xfsprogs: 6.5.0
>> Architecture: aarch64
>> Page size: 64k
>>
>> Test matrix:
>>
>> | Test        | 32k rmapbt=0 | 32k rmapbt=1 | 64k rmapbt=0 | 64k rmapbt=1 |
>> | --------    | ---------    | ---------    | ---------    | ---------    |
>> | generic/042 |     fail     |     fail     |     fail     |     fail     |
>> | generic/081 |     fail     |     fail     |     pass     |     fail     |
>> | generic/108 |     fail     |     fail     |     pass     |     fail     |
>> | generic/455 |     fail     |     fail     |     pass     |     fail     |
>> | generic/457 |     fail     |     fail     |     pass     |     fail     |
>> | generic/482 |     fail     |     fail     |     pass     |     fail     |
>> | generic/704 |     fail     |     fail     |     pass     |     fail     |
>> | generic/730 |     fail     |     fail     |     pass     |     fail     |
>> | generic/731 |     fail     |     fail     |     pass     |     fail     |
>> | shared/298  |     pass     |     pass     |     pass     |     fail     |
> 
> I noticed test failures on these tests when running djwong-wtf:
> generic/042
> generic/081
> generic/108
> generic/219
> generic/305
> generic/326
> generic/562
> generic/704
> xfs/093
> xfs/113
> xfs/161
> xfs/262
> xfs/508
> xfs/604
> xfs/709
> 

Ok, there are some more tests that I didn't catch. I will check them out.

> Still sorting through all of them, but a large portion of them are the
> same failure to format due to minimum log size constraints.  I'd bump
> them up to ~500M (or whatever makes them work) since upstream doesn't
> really support small filesystems anymore.

Thanks for the reply. So we can have a small `if` conditional block for xfs
to have fs size = 500M in generic test cases.

We do this irrespective of filesystem blocksizes right? If we do that, then we can
remove the special conditional that allows tiny filesystems for fstests in mkfs
as well.

--
Pankaj

Re: fstest failure due to filesystem size for 16k, 32k and 64k FSB

[Darrick J. Wong]

On Tue, Jan 30, 2024 at 09:34:23PM +0100, Pankaj Raghav wrote:
> >> What should be the approach to solve this issue? 2 options that I had in my mind:
> >>
> >> 1. Similar to [2], we could add a small hack in mkfs xfs to ignore the log space
> >> requirement while running fstests for these profiles.
> >>
> >> 2. Increase the size of filesystem under test to accommodate these profiles. It could
> >> even be a conditional increase in filesystem size if the FSB > 16k to reduce the impact
> >> on existing FS test time for 4k FSB.
> >>
> >> Let me know what would be the best way to move forward.
> >>
> >> Here are the results:
> >>
> >> Test environment:
> >> kernel Release: 6.8.0-rc1
> >> xfsprogs: 6.5.0
> >> Architecture: aarch64
> >> Page size: 64k
> >>
> >> Test matrix:
> >>
> >> | Test        | 32k rmapbt=0 | 32k rmapbt=1 | 64k rmapbt=0 | 64k rmapbt=1 |
> >> | --------    | ---------    | ---------    | ---------    | ---------    |
> >> | generic/042 |     fail     |     fail     |     fail     |     fail     |
> >> | generic/081 |     fail     |     fail     |     pass     |     fail     |
> >> | generic/108 |     fail     |     fail     |     pass     |     fail     |
> >> | generic/455 |     fail     |     fail     |     pass     |     fail     |
> >> | generic/457 |     fail     |     fail     |     pass     |     fail     |
> >> | generic/482 |     fail     |     fail     |     pass     |     fail     |
> >> | generic/704 |     fail     |     fail     |     pass     |     fail     |
> >> | generic/730 |     fail     |     fail     |     pass     |     fail     |
> >> | generic/731 |     fail     |     fail     |     pass     |     fail     |
> >> | shared/298  |     pass     |     pass     |     pass     |     fail     |
> > 
> > I noticed test failures on these tests when running djwong-wtf:
> > generic/042
> > generic/081
> > generic/108
> > generic/219
> > generic/305
> > generic/326
> > generic/562
> > generic/704
> > xfs/093
> > xfs/113
> > xfs/161
> > xfs/262
> > xfs/508
> > xfs/604
> > xfs/709
> > 
> 
> Ok, there are some more tests that I didn't catch. I will check them out.
> 
> > Still sorting through all of them, but a large portion of them are the
> > same failure to format due to minimum log size constraints.  I'd bump
> > them up to ~500M (or whatever makes them work) since upstream doesn't
> > really support small filesystems anymore.
> 
> Thanks for the reply. So we can have a small `if` conditional block for xfs
> to have fs size = 500M in generic test cases.

I'd suggest creating a helper where you pass in the fs size you want and
it rounds that up to the minimum value.  That would then get passed to
_scratch_mkfs_sized or _scsi_debug_get_dev.

(testing this as we speak...)

> We do this irrespective of filesystem blocksizes right? If we do that, then we can
> remove the special conditional that allows tiny filesystems for fstests in mkfs
> as well.

I dunno.  In the ideal world we'd figure out the fsblock size, but
divining that from the MKFS_OPTIONS is hard fugly string parsing.

--D

> 
> --
> Pankaj
> 
> 
> 

Re: fstest failure due to filesystem size for 16k, 32k and 64k FSB

[Pankaj Raghav (Samsung)]

> > 
> > Thanks for the reply. So we can have a small `if` conditional block for xfs
> > to have fs size = 500M in generic test cases.
> 
> I'd suggest creating a helper where you pass in the fs size you want and
> it rounds that up to the minimum value.  That would then get passed to
> _scratch_mkfs_sized or _scsi_debug_get_dev.
> 
> (testing this as we speak...)

I would be more than happy if you send a patch for
this but I also know you are pretty busy, so let me know if you want me
to send a patch for this issue.

You had something like this in mind?

diff --git a/common/config b/common/config
index c9771ff9..216e8e06 100644
--- a/common/config
+++ b/common/config
@@ -168,6 +168,7 @@ export XFS_SPACEMAN_PROG="$(type -P xfs_spaceman)"
 export XFS_SCRUB_PROG="$(type -P xfs_scrub)"
 export XFS_PARALLEL_REPAIR_PROG="$(type -P xfs_prepair)"
 export XFS_PARALLEL_REPAIR64_PROG="$(type -P xfs_prepair64)"
+export MIN_XFS_FS_SZ_MB=600
 export __XFSDUMP_PROG="$(type -P xfsdump)"
 if [ -n "$__XFSDUMP_PROG" ]; then
 	export XFSDUMP_PROG="$__XFSDUMP_PROG -e"
diff --git a/common/rc b/common/rc
index 524ffa02..8d148105 100644
--- a/common/rc
+++ b/common/rc
@@ -916,6 +916,22 @@ _check_minimal_fs_size()
 	fi
 }
 
+# takes size in MB
+_get_optimal_fs_size()
+{
+	local fssize=$1
+
+	case "$FSTYP" in
+	xfs)
+		[ $MIN_XFS_FS_SZ_MB -gt "$fssize" ] && fssize=$MIN_XFS_FS_SZ_MB
+		;;
+	*)
+		;;
+	esac
+
+	echo "$fssize"
+}
+
 # Create fs of certain size on scratch device
 # _scratch_mkfs_sized <size in bytes> [optional blocksize]
 _scratch_mkfs_sized()
diff --git a/tests/generic/081 b/tests/generic/081
index 22ac94de..650c38be 100755
--- a/tests/generic/081
+++ b/tests/generic/081
@@ -62,13 +62,15 @@ snapname=snap_$seq
 mnt=$TEST_DIR/mnt_$seq
 mkdir -p $mnt
 
-# make sure there's enough disk space for 256M lv, test for 300M here in case
+size=$(_get_optimal_fs_size 300)
 # lvm uses some space for metadata
-_scratch_mkfs_sized $((300 * 1024 * 1024)) >>$seqres.full 2>&1
+lvmsize=$((size - 50))
+# make sure there's enough disk space for 256M lv, test for 300M here in case
+_scratch_mkfs_sized $(($size * 1024 * 1024)) >>$seqres.full 2>&1
 $LVM_PROG vgcreate -f $vgname $SCRATCH_DEV >>$seqres.full 2>&1
 # We use yes pipe instead of 'lvcreate --yes' because old version of lvm
 # (like 2.02.95 in RHEL6) don't support --yes option
-yes | $LVM_PROG lvcreate -L 256M -n $lvname $vgname >>$seqres.full 2>&1
+yes | $LVM_PROG lvcreate -L "$lvmsize"M -n $lvname $vgname >>$seqres.full 2>&1
 # wait for lvcreation to fully complete
 $UDEV_SETTLE_PROG >>$seqres.full 2>&1
 
diff --git a/tests/generic/108 b/tests/generic/108
index efe66ba5..a17ee435 100755
--- a/tests/generic/108
+++ b/tests/generic/108
@@ -45,8 +45,11 @@ vgname=vg_$seq
 physical=`blockdev --getpbsz $SCRATCH_DEV`
 logical=`blockdev --getss $SCRATCH_DEV`
 
+size=$(_get_optimal_fs_size 300)
+# lvm uses some space for metadata
+lvmsize=$((size - 50))
 # _get_scsi_debug_dev returns a scsi debug device with 128M in size by default
-SCSI_DEBUG_DEV=`_get_scsi_debug_dev ${physical:-512} ${logical:-512} 0 300`
+SCSI_DEBUG_DEV=`_get_scsi_debug_dev ${physical:-512} ${logical:-512} 0 $size`
 test -b "$SCSI_DEBUG_DEV" || _notrun "Failed to initialize scsi debug device"
 echo "SCSI debug device $SCSI_DEBUG_DEV" >>$seqres.full
 
@@ -55,7 +58,7 @@ $LVM_PROG pvcreate -f $SCSI_DEBUG_DEV $SCRATCH_DEV >>$seqres.full 2>&1
 $LVM_PROG vgcreate -f $vgname $SCSI_DEBUG_DEV $SCRATCH_DEV >>$seqres.full 2>&1
 # We use yes pipe instead of 'lvcreate --yes' because old version of lvm
 # (like 2.02.95 in RHEL6) don't support --yes option
-yes | $LVM_PROG lvcreate -i 2 -I 4m -L 275m -n $lvname $vgname \
+yes | $LVM_PROG lvcreate -i 2 -I 4m -L "$lvmsize"m -n $lvname $vgname \
 	>>$seqres.full 2>&1
 # wait for lv creation to fully complete
 $UDEV_SETTLE_PROG >>$seqres.full 2>&1

Re: fstest failure due to filesystem size for 16k, 32k and 64k FSB

[Darrick J. Wong]

On Wed, Jan 31, 2024 at 03:05:48PM +0100, Pankaj Raghav (Samsung) wrote:
> > > 
> > > Thanks for the reply. So we can have a small `if` conditional block for xfs
> > > to have fs size = 500M in generic test cases.
> > 
> > I'd suggest creating a helper where you pass in the fs size you want and
> > it rounds that up to the minimum value.  That would then get passed to
> > _scratch_mkfs_sized or _scsi_debug_get_dev.
> > 
> > (testing this as we speak...)
> 
> I would be more than happy if you send a patch for
> this but I also know you are pretty busy, so let me know if you want me
> to send a patch for this issue.
> 
> You had something like this in mind?

Close, but something more like below.  It's not exhaustive; it merely
makes the xfs 64k bs tests pass:

From: Darrick J. Wong <djwong@kernel.org>
Subject: [PATCH] misc: fix test that fail formatting with 64k blocksize

There's a bunch of tests that fail the formatting step when the test run
is configured to use XFS with a 64k blocksize.  This happens because XFS
doesn't really support that combination due to minimum log size
constraints.  Fix the test to format larger devices in that case.

Signed-off-by: Darrick J. Wong <djwong@kernel.org>
---
 common/rc         |   29 +++++++++++++++++++++++++++++
 tests/generic/042 |    9 +--------
 tests/generic/081 |    7 +++++--
 tests/generic/108 |    6 ++++--
 tests/generic/704 |    3 ++-
 tests/generic/730 |    3 ++-
 tests/generic/731 |    3 ++-
 tests/xfs/279     |    7 ++++---
 8 files changed, 49 insertions(+), 18 deletions(-)

diff --git a/common/rc b/common/rc
index 29bb90ca4d..953aad85ae 100644
--- a/common/rc
+++ b/common/rc
@@ -940,6 +940,35 @@ _check_minimal_fs_size()
 	fi
 }
 
+# Round a proposed filesystem size up to the minimium supported size.  The
+# input is in MB and so is the output.
+_small_fs_size_mb()
+{
+	local size="$1"
+	local runner_min_size=0
+	local fs_min_size=0
+
+	case "$FSTYP" in
+	xfs)
+		# xfs no longer supports filesystems smaller than 512m
+		fs_min_size=512
+		;;
+	f2fs)
+		# f2fs-utils 1.9.0 needs at least 38 MB space for f2fs image.
+		# However, f2fs-utils 1.14.0 needs at least 52 MB. Not sure if
+		# it will change again. So just set it 128M.
+		fs_min_size=128
+		;;
+	esac
+	(( size < fs_min_size )) && size="$fs_min_size"
+
+	# If the test runner wanted a minimum size, enforce that here.
+	test -n "$MIN_FSSIZE" && runner_min_size=$((MIN_FSSIZE / 1048576))
+	(( size < runner_min_size)) && size="$runner_min_size"
+
+	echo "$size"
+}
+
 # Create fs of certain size on scratch device
 # _scratch_mkfs_sized <size in bytes> [optional blocksize]
 _scratch_mkfs_sized()
diff --git a/tests/generic/042 b/tests/generic/042
index 5116183f79..63a46d6b2b 100755
--- a/tests/generic/042
+++ b/tests/generic/042
@@ -27,14 +27,7 @@ _crashtest()
 	img=$SCRATCH_MNT/$seq.img
 	mnt=$SCRATCH_MNT/$seq.mnt
 	file=$mnt/file
-	size=25M
-
-	# f2fs-utils 1.9.0 needs at least 38 MB space for f2fs image. However,
-	# f2fs-utils 1.14.0 needs at least 52 MB. Not sure if it will change
-	# again. So just set it 128M.
-	if [ $FSTYP == "f2fs" ]; then
-		size=128M
-	fi
+	size=$(_small_fs_size_mb 25)M
 
 	# Create an fs on a small, initialized image. The pattern is written to
 	# the image to detect stale data exposure.
diff --git a/tests/generic/081 b/tests/generic/081
index 22ac94de53..0996f221d3 100755
--- a/tests/generic/081
+++ b/tests/generic/081
@@ -62,13 +62,16 @@ snapname=snap_$seq
 mnt=$TEST_DIR/mnt_$seq
 mkdir -p $mnt
 
+size=$(_small_fs_size_mb 300)
+lvsize=$((size * 85 / 100))	 # ~256M
+
 # make sure there's enough disk space for 256M lv, test for 300M here in case
 # lvm uses some space for metadata
-_scratch_mkfs_sized $((300 * 1024 * 1024)) >>$seqres.full 2>&1
+_scratch_mkfs_sized $((size * 1024 * 1024)) >>$seqres.full 2>&1
 $LVM_PROG vgcreate -f $vgname $SCRATCH_DEV >>$seqres.full 2>&1
 # We use yes pipe instead of 'lvcreate --yes' because old version of lvm
 # (like 2.02.95 in RHEL6) don't support --yes option
-yes | $LVM_PROG lvcreate -L 256M -n $lvname $vgname >>$seqres.full 2>&1
+yes | $LVM_PROG lvcreate -L ${lvsize}M -n $lvname $vgname >>$seqres.full 2>&1
 # wait for lvcreation to fully complete
 $UDEV_SETTLE_PROG >>$seqres.full 2>&1
 
diff --git a/tests/generic/108 b/tests/generic/108
index efe66ba57f..07703fc8f1 100755
--- a/tests/generic/108
+++ b/tests/generic/108
@@ -44,9 +44,11 @@ vgname=vg_$seq
 
 physical=`blockdev --getpbsz $SCRATCH_DEV`
 logical=`blockdev --getss $SCRATCH_DEV`
+size=$(_small_fs_size_mb 300)
+lvsize=$((size * 91 / 100))
 
 # _get_scsi_debug_dev returns a scsi debug device with 128M in size by default
-SCSI_DEBUG_DEV=`_get_scsi_debug_dev ${physical:-512} ${logical:-512} 0 300`
+SCSI_DEBUG_DEV=`_get_scsi_debug_dev ${physical:-512} ${logical:-512} 0 $size`
 test -b "$SCSI_DEBUG_DEV" || _notrun "Failed to initialize scsi debug device"
 echo "SCSI debug device $SCSI_DEBUG_DEV" >>$seqres.full
 
@@ -55,7 +57,7 @@ $LVM_PROG pvcreate -f $SCSI_DEBUG_DEV $SCRATCH_DEV >>$seqres.full 2>&1
 $LVM_PROG vgcreate -f $vgname $SCSI_DEBUG_DEV $SCRATCH_DEV >>$seqres.full 2>&1
 # We use yes pipe instead of 'lvcreate --yes' because old version of lvm
 # (like 2.02.95 in RHEL6) don't support --yes option
-yes | $LVM_PROG lvcreate -i 2 -I 4m -L 275m -n $lvname $vgname \
+yes | $LVM_PROG lvcreate -i 2 -I 4m -L ${lvsize}m -n $lvname $vgname \
 	>>$seqres.full 2>&1
 # wait for lv creation to fully complete
 $UDEV_SETTLE_PROG >>$seqres.full 2>&1
diff --git a/tests/generic/704 b/tests/generic/704
index c0142a6051..6cc4bb4af0 100755
--- a/tests/generic/704
+++ b/tests/generic/704
@@ -30,8 +30,9 @@ _require_scsi_debug
 _require_test
 _require_block_device $TEST_DEV
 
+size=$(_small_fs_size_mb 256)
 echo "Get a device with 4096 physical sector size and 512 logical sector size"
-SCSI_DEBUG_DEV=`_get_scsi_debug_dev 4096 512 0 256`
+SCSI_DEBUG_DEV=`_get_scsi_debug_dev 4096 512 0 $size`
 blockdev --getpbsz --getss $SCSI_DEBUG_DEV
 
 echo "mkfs and mount"
diff --git a/tests/generic/730 b/tests/generic/730
index 11308cdaa1..988c47e18e 100755
--- a/tests/generic/730
+++ b/tests/generic/730
@@ -27,7 +27,8 @@ _require_test
 _require_block_device $TEST_DEV
 _require_scsi_debug
 
-SCSI_DEBUG_DEV=`_get_scsi_debug_dev 512 512 0 256`
+size=$(_small_fs_size_mb 256)
+SCSI_DEBUG_DEV=`_get_scsi_debug_dev 512 512 0 $size`
 test -b "$SCSI_DEBUG_DEV" || _notrun "Failed to initialize scsi debug device"
 echo "SCSI debug device $SCSI_DEBUG_DEV" >>$seqres.full
 
diff --git a/tests/generic/731 b/tests/generic/731
index e1400d062c..b279e3f7b4 100755
--- a/tests/generic/731
+++ b/tests/generic/731
@@ -27,7 +27,8 @@ _require_block_device $TEST_DEV
 _supported_fs generic
 _require_scsi_debug
 
-SCSI_DEBUG_DEV=`_get_scsi_debug_dev 512 512 0 256`
+size=$(_small_fs_size_mb 256)
+SCSI_DEBUG_DEV=`_get_scsi_debug_dev 512 512 0 $size`
 test -b "$SCSI_DEBUG_DEV" || _notrun "Failed to initialize scsi debug device"
 echo "SCSI debug device $SCSI_DEBUG_DEV" >>$seqres.full
 
diff --git a/tests/xfs/279 b/tests/xfs/279
index 835d187f51..9f366d1e75 100755
--- a/tests/xfs/279
+++ b/tests/xfs/279
@@ -26,6 +26,7 @@ _cleanup()
 _supported_fs xfs
 
 _require_scsi_debug
+size=$(_small_fs_size_mb 128)
 
 # Remove xfs signature so -f isn't needed to re-mkfs
 _wipe_device()
@@ -55,7 +56,7 @@ _check_mkfs()
 (
 echo "==================="
 echo "4k physical 512b logical aligned"
-SCSI_DEBUG_DEV=`_get_scsi_debug_dev 4096 512 0 128`
+SCSI_DEBUG_DEV=`_get_scsi_debug_dev 4096 512 0 $size`
 test -b "$SCSI_DEBUG_DEV" || _notrun "Could not get scsi_debug device"
 # sector size should default to 4k
 _check_mkfs $SCSI_DEBUG_DEV
@@ -68,7 +69,7 @@ _put_scsi_debug_dev
 (
 echo "==================="
 echo "4k physical 512b logical unaligned"
-SCSI_DEBUG_DEV=`_get_scsi_debug_dev 4096 512 1 128`
+SCSI_DEBUG_DEV=`_get_scsi_debug_dev 4096 512 1 $size`
 test -b "$SCSI_DEBUG_DEV" || _notrun "Could not get scsi_debug device"
 # should fail on misalignment
 _check_mkfs $SCSI_DEBUG_DEV
@@ -85,7 +86,7 @@ _put_scsi_debug_dev
 (
 echo "==================="
 echo "hard 4k physical / 4k logical"
-SCSI_DEBUG_DEV=`_get_scsi_debug_dev 4096 4096 0 128`
+SCSI_DEBUG_DEV=`_get_scsi_debug_dev 4096 4096 0 $size`
 test -b "$SCSI_DEBUG_DEV" || _notrun "Could not get scsi_debug device"
 # block size smaller than sector size should fail 
 _check_mkfs -b size=2048 $SCSI_DEBUG_DEV

Re: fstest failure due to filesystem size for 16k, 32k and 64k FSB

[Pankaj Raghav (Samsung)]

On Wed, Jan 31, 2024 at 10:28:58AM -0800, Darrick J. Wong wrote:
> On Wed, Jan 31, 2024 at 03:05:48PM +0100, Pankaj Raghav (Samsung) wrote:
> > > > 
> > > > Thanks for the reply. So we can have a small `if` conditional block for xfs
> > > > to have fs size = 500M in generic test cases.
> > > 
> > > I'd suggest creating a helper where you pass in the fs size you want and
> > > it rounds that up to the minimum value.  That would then get passed to
> > > _scratch_mkfs_sized or _scsi_debug_get_dev.
> > > 
> > > (testing this as we speak...)
> > 
> > I would be more than happy if you send a patch for
> > this but I also know you are pretty busy, so let me know if you want me
> > to send a patch for this issue.
> > 
> > You had something like this in mind?
> 
> Close, but something more like below.  It's not exhaustive; it merely
> makes the xfs 64k bs tests pass:
> 

I still see some errors in generic/081 and generic/108 that have been
modified in your patch with the same issue.

This is the mkfs option I am using:
-m reflink=1,rmapbt=1, -i sparse=1, -b size=64k

And with that:
$ ./check -s 64k generic/042 generic/081 generic/108 generic/704 generic/730 generic/731 xfs/279

...
generic/081.out.bad:
 +max log size 1732 smaller than min log size 2028, filesystem is too small
...
generic/108.out.bad:
+max log size 1876 smaller than min log size 2028, filesystem is too small
...
SECTION       -- 64k
=========================
Ran: generic/042 generic/081 generic/108 generic/704 generic/730 generic/731 xfs/279
Failures: generic/081 generic/108
Failed 2 of 7 tests

**Increasing the size** to 600M fixes all the test in 64k system.

The patch itself including `_small_fs_size_mb()` looks good to me.

> From: Darrick J. Wong <djwong@kernel.org>
> Subject: [PATCH] misc: fix test that fail formatting with 64k blocksize
> 
> There's a bunch of tests that fail the formatting step when the test run
> is configured to use XFS with a 64k blocksize.  This happens because XFS
> doesn't really support that combination due to minimum log size
> constraints.  Fix the test to format larger devices in that case.
> 
> Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> ---
>  common/rc         |   29 +++++++++++++++++++++++++++++
>  tests/generic/042 |    9 +--------
>  tests/generic/081 |    7 +++++--
>  tests/generic/108 |    6 ++++--
>  tests/generic/704 |    3 ++-
>  tests/generic/730 |    3 ++-
>  tests/generic/731 |    3 ++-
>  tests/xfs/279     |    7 ++++---

As I indicated at the start of the thread, we need to also fix:
generic/455 generic/457 generic/482 shared/298

Thanks!
--
Pankaj Raghav

Re: fstest failure due to filesystem size for 16k, 32k and 64k FSB

[Darrick J. Wong]

On Thu, Feb 01, 2024 at 04:44:36PM +0100, Pankaj Raghav (Samsung) wrote:
> On Wed, Jan 31, 2024 at 10:28:58AM -0800, Darrick J. Wong wrote:
> > On Wed, Jan 31, 2024 at 03:05:48PM +0100, Pankaj Raghav (Samsung) wrote:
> > > > > 
> > > > > Thanks for the reply. So we can have a small `if` conditional block for xfs
> > > > > to have fs size = 500M in generic test cases.
> > > > 
> > > > I'd suggest creating a helper where you pass in the fs size you want and
> > > > it rounds that up to the minimum value.  That would then get passed to
> > > > _scratch_mkfs_sized or _scsi_debug_get_dev.
> > > > 
> > > > (testing this as we speak...)
> > > 
> > > I would be more than happy if you send a patch for
> > > this but I also know you are pretty busy, so let me know if you want me
> > > to send a patch for this issue.
> > > 
> > > You had something like this in mind?
> > 
> > Close, but something more like below.  It's not exhaustive; it merely
> > makes the xfs 64k bs tests pass:
> > 
> 
> I still see some errors in generic/081 and generic/108 that have been
> modified in your patch with the same issue.
> 
> This is the mkfs option I am using:
> -m reflink=1,rmapbt=1, -i sparse=1, -b size=64k
> 
> And with that:
> $ ./check -s 64k generic/042 generic/081 generic/108 generic/704 generic/730 generic/731 xfs/279
> 
> ...
> generic/081.out.bad:
>  +max log size 1732 smaller than min log size 2028, filesystem is too small
> ...
> generic/108.out.bad:
> +max log size 1876 smaller than min log size 2028, filesystem is too small
> ...
> SECTION       -- 64k
> =========================
> Ran: generic/042 generic/081 generic/108 generic/704 generic/730 generic/731 xfs/279
> Failures: generic/081 generic/108
> Failed 2 of 7 tests
> 
> **Increasing the size** to 600M fixes all the test in 64k system.

Huh.  Can you send me the mkfs output (or xfs_info after the fact) so I
can compare your setup with mine?  I'm curious about what's affecting
the layout here -- maybe you have -s size=4k or something?

(I don't want to stray too far from the /actual/ mkfs minimum fs size of
300M.)

--D

> 
> The patch itself including `_small_fs_size_mb()` looks good to me.
> 
> > From: Darrick J. Wong <djwong@kernel.org>
> > Subject: [PATCH] misc: fix test that fail formatting with 64k blocksize
> > 
> > There's a bunch of tests that fail the formatting step when the test run
> > is configured to use XFS with a 64k blocksize.  This happens because XFS
> > doesn't really support that combination due to minimum log size
> > constraints.  Fix the test to format larger devices in that case.
> > 
> > Signed-off-by: Darrick J. Wong <djwong@kernel.org>
> > ---
> >  common/rc         |   29 +++++++++++++++++++++++++++++
> >  tests/generic/042 |    9 +--------
> >  tests/generic/081 |    7 +++++--
> >  tests/generic/108 |    6 ++++--
> >  tests/generic/704 |    3 ++-
> >  tests/generic/730 |    3 ++-
> >  tests/generic/731 |    3 ++-
> >  tests/xfs/279     |    7 ++++---
> 
> As I indicated at the start of the thread, we need to also fix:
> generic/455 generic/457 generic/482 shared/298
> 
> Thanks!
> --
> Pankaj Raghav
> 

Re: fstest failure due to filesystem size for 16k, 32k and 64k FSB

[Pankaj Raghav]

>> I still see some errors in generic/081 and generic/108 that have been
>> modified in your patch with the same issue.
>>
>> This is the mkfs option I am using:
>> -m reflink=1,rmapbt=1, -i sparse=1, -b size=64k
>>
>> And with that:
>> $ ./check -s 64k generic/042 generic/081 generic/108 generic/704 generic/730 generic/731 xfs/279
>>
>> ...
>> generic/081.out.bad:
>>  +max log size 1732 smaller than min log size 2028, filesystem is too small
>> ...
>> generic/108.out.bad:
>> +max log size 1876 smaller than min log size 2028, filesystem is too small
>> ...
>> SECTION       -- 64k
>> =========================
>> Ran: generic/042 generic/081 generic/108 generic/704 generic/730 generic/731 xfs/279
>> Failures: generic/081 generic/108
>> Failed 2 of 7 tests
>>
>> **Increasing the size** to 600M fixes all the test in 64k system.
> 
> Huh.  Can you send me the mkfs output (or xfs_info after the fact) so I
> can compare your setup with mine?  I'm curious about what's affecting
> the layout here -- maybe you have -s size=4k or something?
> 
> (I don't want to stray too far from the /actual/ mkfs minimum fs size of
> 300M.)
> 

I am using v6.8-rc2 with xfsprogs 6.5.0 and xfstests v2024.01.14 (with your patch on top)

Using oracle OCI instance with 64k page size support.

config:

[default]
FSTYP=xfs
RESULT_BASE=$PWD/results/$HOST/$(uname -r)
DUMP_CORRUPT_FS=1
CANON_DEVS=yes
RECREATE_TEST_DEV=true
TEST_DEV=/dev/sdb2
TEST_DIR=/mnt/test
SCRATCH_DEV=/dev/sdb3
SCRATCH_MNT=/mnt/scratch
LOGWRITES_DEV=/dev/sdb4

[64k]
MKFS_OPTIONS='-f -m reflink=1,rmapbt=1, -i sparse=1, -b size=64k,'


If I use 600MB and if the tests run, then this is the xfs_info I am getting on
the test device (test device should have the same config as scratch as I use RECREATE_TEST_DEV=true):

[nix-shell:/mnt/linux/xfstests]$ sudo xfs_info /dev/sdb2
meta-data=/dev/sdb2              isize=512    agcount=4, agsize=102400 blks
         =                       sectsz=4096  attr=2, projid32bit=1
         =                       crc=1        finobt=1, sparse=1, rmapbt=1
         =                       reflink=1    bigtime=1 inobtcount=1 nrext64=1
data     =                       bsize=65536  blocks=409600, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=65536  ascii-ci=0, ftype=1
log      =internal log           bsize=65536  blocks=2613, version=2
         =                       sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none                   extsz=65536  blocks=0, rtextents=0

Let me know if you need more information.

--
Pankaj