* [PATCH 1/2] common/rc: add _parse_size_from_string
@ 2022-06-14 5:08 An Long
2022-06-14 5:08 ` [PATCH 2/2] common/rc: fix input value to _scratch_mkfs_sized An Long
2022-06-15 6:05 ` [PATCH 1/2] common/rc: add _parse_size_from_string Zorro Lang
0 siblings, 2 replies; 6+ messages in thread
From: An Long @ 2022-06-14 5:08 UTC (permalink / raw)
To: fstests; +Cc: An Long
Add a helper to convert size value to bytes. This is used to handle
value as bytes, such as 4k to 4096.
Signed-off-by: An Long <lan@suse.com>
---
common/rc | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 48 insertions(+)
diff --git a/common/rc b/common/rc
index 3c072c16..22050bc2 100644
--- a/common/rc
+++ b/common/rc
@@ -1028,6 +1028,54 @@ _check_minimal_fs_size()
fi
}
+# Convert size value to bytes
+# _parse_size_from_string <size>
+_parse_size_from_string()
+{
+ local str=$1
+ local mult=1
+ local size
+ local endchar
+
+ if [[ $str =~ ^[0-9]+[a-zA-Z]$ ]] ; then
+ size=${str:: -1}
+ endchar=${str: -1}
+ case $endchar in
+ e|E)
+ mult=$(($mult * 1024))
+ ;&
+ p|P)
+ mult=$(($mult * 1024))
+ ;&
+ t|T)
+ mult=$(($mult * 1024))
+ ;&
+ g|G)
+ mult=$(($mult * 1024))
+ ;&
+ m|M)
+ mult=$(($mult * 1024))
+ ;&
+ k|K)
+ mult=$(($mult * 1024))
+ ;&
+ b|B)
+ ;;
+ *)
+ echo "unknown size descriptor $endchar"
+ exit 1
+ esac
+ elif [[ $str =~ ^[0-9]+$ ]] ; then
+ size=$str
+ else
+ echo "size value $str is invalid"
+ exit 1
+ fi
+
+ size=$(($size * $mult))
+ echo $size
+}
+
# Create fs of certain size on scratch device
# _scratch_mkfs_sized <size in bytes> [optional blocksize]
_scratch_mkfs_sized()
--
2.35.3
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH 2/2] common/rc: fix input value to _scratch_mkfs_sized
2022-06-14 5:08 [PATCH 1/2] common/rc: add _parse_size_from_string An Long
@ 2022-06-14 5:08 ` An Long
2022-06-15 4:15 ` Zorro Lang
2022-06-15 6:05 ` [PATCH 1/2] common/rc: add _parse_size_from_string Zorro Lang
1 sibling, 1 reply; 6+ messages in thread
From: An Long @ 2022-06-14 5:08 UTC (permalink / raw)
To: fstests; +Cc: An Long
_scratch_mkfs_sized only receive integer number of bytes as a valid
input. But if the MKFS_OPTIONS variable exists, it will use the value of
block size in MKFS_OPTIONS to override input. In case of
MKFS_OPTIONS="-b 4k", would result in blocksize=4 but not 4096. This
will give errors to ext2/3/4 etc, and brings potential bugs to xfs or
btrfs.
In addition, since we can receive various strings, so remove integer
number check.
Signed-off-by: An Long <lan@suse.com>
---
common/rc | 18 ++++++------------
1 file changed, 6 insertions(+), 12 deletions(-)
diff --git a/common/rc b/common/rc
index 22050bc2..026007d3 100644
--- a/common/rc
+++ b/common/rc
@@ -1077,7 +1077,7 @@ _parse_size_from_string()
}
# Create fs of certain size on scratch device
-# _scratch_mkfs_sized <size in bytes> [optional blocksize]
+# _scratch_mkfs_sized <size> [optional blocksize]
_scratch_mkfs_sized()
{
local fssize=$1
@@ -1086,13 +1086,13 @@ _scratch_mkfs_sized()
case $FSTYP in
xfs)
- def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-b ?size= ?+([0-9]+).*/\1/p'`
+ def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-b ?size= ?+([0-9]+[a-zA-Z]?).*/\1/p'`
;;
btrfs)
- def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-s ?+([0-9]+).*/\1/p'`
+ def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-s ?+([0-9]+[a-zA-Z]?).*/\1/p'`
;;
ext2|ext3|ext4|ext4dev|udf|reiser4|ocfs2|reiserfs)
- def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-b ?+([0-9]+).*/\1/p'`
+ def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-b ?+([0-9]+[a-zA-Z]?).*/\1/p'`
;;
jfs)
def_blksz=4096
@@ -1101,14 +1101,8 @@ _scratch_mkfs_sized()
[ -n "$def_blksz" ] && blocksize=$def_blksz
[ -z "$blocksize" ] && blocksize=4096
-
- local re='^[0-9]+$'
- if ! [[ $fssize =~ $re ]] ; then
- _notrun "error: _scratch_mkfs_sized: fs size \"$fssize\" not an integer."
- fi
- if ! [[ $blocksize =~ $re ]] ; then
- _notrun "error: _scratch_mkfs_sized: block size \"$blocksize\" not an integer."
- fi
+ blocksize=$(_parse_size_from_string $blocksize)
+ fssize=$(_parse_size_from_string $fssize)
local blocks=`expr $fssize / $blocksize`
--
2.35.3
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH 2/2] common/rc: fix input value to _scratch_mkfs_sized
2022-06-14 5:08 ` [PATCH 2/2] common/rc: fix input value to _scratch_mkfs_sized An Long
@ 2022-06-15 4:15 ` Zorro Lang
2022-06-16 4:30 ` Long An
0 siblings, 1 reply; 6+ messages in thread
From: Zorro Lang @ 2022-06-15 4:15 UTC (permalink / raw)
To: An Long; +Cc: fstests
On Tue, Jun 14, 2022 at 01:08:43PM +0800, An Long wrote:
> _scratch_mkfs_sized only receive integer number of bytes as a valid
> input. But if the MKFS_OPTIONS variable exists, it will use the value of
> block size in MKFS_OPTIONS to override input. In case of
> MKFS_OPTIONS="-b 4k", would result in blocksize=4 but not 4096. This
Hi An,
Good catch. Can you provide an example that a case fails on this issue, and
then it tests passed after having this patch? I'm wondering why no one notice
that before.
> will give errors to ext2/3/4 etc, and brings potential bugs to xfs or
> btrfs.
>
> In addition, since we can receive various strings, so remove integer
> number check.
>
> Signed-off-by: An Long <lan@suse.com>
> ---
> common/rc | 18 ++++++------------
> 1 file changed, 6 insertions(+), 12 deletions(-)
>
> diff --git a/common/rc b/common/rc
> index 22050bc2..026007d3 100644
> --- a/common/rc
> +++ b/common/rc
> @@ -1077,7 +1077,7 @@ _parse_size_from_string()
> }
>
> # Create fs of certain size on scratch device
> -# _scratch_mkfs_sized <size in bytes> [optional blocksize]
> +# _scratch_mkfs_sized <size> [optional blocksize]
> _scratch_mkfs_sized()
> {
> local fssize=$1
> @@ -1086,13 +1086,13 @@ _scratch_mkfs_sized()
>
> case $FSTYP in
> xfs)
> - def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-b ?size= ?+([0-9]+).*/\1/p'`
> + def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-b ?size= ?+([0-9]+[a-zA-Z]?).*/\1/p'`
> ;;
> btrfs)
> - def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-s ?+([0-9]+).*/\1/p'`
> + def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-s ?+([0-9]+[a-zA-Z]?).*/\1/p'`
> ;;
> ext2|ext3|ext4|ext4dev|udf|reiser4|ocfs2|reiserfs)
> - def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-b ?+([0-9]+).*/\1/p'`
> + def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-b ?+([0-9]+[a-zA-Z]?).*/\1/p'`
> ;;
> jfs)
> def_blksz=4096
> @@ -1101,14 +1101,8 @@ _scratch_mkfs_sized()
>
> [ -n "$def_blksz" ] && blocksize=$def_blksz
> [ -z "$blocksize" ] && blocksize=4096
> -
> - local re='^[0-9]+$'
> - if ! [[ $fssize =~ $re ]] ; then
> - _notrun "error: _scratch_mkfs_sized: fs size \"$fssize\" not an integer."
> - fi
> - if ! [[ $blocksize =~ $re ]] ; then
> - _notrun "error: _scratch_mkfs_sized: block size \"$blocksize\" not an integer."
> - fi
> + blocksize=$(_parse_size_from_string $blocksize)
> + fssize=$(_parse_size_from_string $fssize)
For now, the _parse_size_from_string is only used for this patch, I think these
two patches can be merged into one patch, as it trys to fix one single problem.
Thanks,
Zorro
>
> local blocks=`expr $fssize / $blocksize`
>
> --
> 2.35.3
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 1/2] common/rc: add _parse_size_from_string
2022-06-14 5:08 [PATCH 1/2] common/rc: add _parse_size_from_string An Long
2022-06-14 5:08 ` [PATCH 2/2] common/rc: fix input value to _scratch_mkfs_sized An Long
@ 2022-06-15 6:05 ` Zorro Lang
2022-06-16 4:37 ` Long An
1 sibling, 1 reply; 6+ messages in thread
From: Zorro Lang @ 2022-06-15 6:05 UTC (permalink / raw)
To: An Long; +Cc: fstests
On Tue, Jun 14, 2022 at 01:08:42PM +0800, An Long wrote:
> Add a helper to convert size value to bytes. This is used to handle
> value as bytes, such as 4k to 4096.
>
> Signed-off-by: An Long <lan@suse.com>
> ---
> common/rc | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 48 insertions(+)
>
> diff --git a/common/rc b/common/rc
> index 3c072c16..22050bc2 100644
> --- a/common/rc
> +++ b/common/rc
> @@ -1028,6 +1028,54 @@ _check_minimal_fs_size()
> fi
> }
>
> +# Convert size value to bytes
> +# _parse_size_from_string <size>
> +_parse_size_from_string()
I think this name is better to be shorter, as it might be a lower level common
function. How about _convert_num() or _cvtnum()?
> +{
> + local str=$1
> + local mult=1
> + local size
> + local endchar
> +
> + if [[ $str =~ ^[0-9]+[a-zA-Z]$ ]] ; then
> + size=${str:: -1}
> + endchar=${str: -1}
> + case $endchar in
> + e|E)
> + mult=$(($mult * 1024))
> + ;&
> + p|P)
> + mult=$(($mult * 1024))
> + ;&
> + t|T)
> + mult=$(($mult * 1024))
> + ;&
> + g|G)
> + mult=$(($mult * 1024))
> + ;&
> + m|M)
> + mult=$(($mult * 1024))
> + ;&
> + k|K)
> + mult=$(($mult * 1024))
> + ;&
> + b|B)
> + ;;
> + *)
> + echo "unknown size descriptor $endchar"
> + exit 1
> + esac
> + elif [[ $str =~ ^[0-9]+$ ]] ; then
> + size=$str
> + else
> + echo "size value $str is invalid"
> + exit 1
> + fi
> +
> + size=$(($size * $mult))
^ ^
I think above two '$' isn't necessary, although it doesn't make something
wrong if keep it. Same as all above (($mult * 1024)) .
Thanks,
Zorro
^
> + echo $size
> +}
> +
> # Create fs of certain size on scratch device
> # _scratch_mkfs_sized <size in bytes> [optional blocksize]
> _scratch_mkfs_sized()
> --
> 2.35.3
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 2/2] common/rc: fix input value to _scratch_mkfs_sized
2022-06-15 4:15 ` Zorro Lang
@ 2022-06-16 4:30 ` Long An
0 siblings, 0 replies; 6+ messages in thread
From: Long An @ 2022-06-16 4:30 UTC (permalink / raw)
To: zlang@redhat.com; +Cc: fstests@vger.kernel.org
On Wed, 2022-06-15 at 12:15 +0800, Zorro Lang wrote:
> On Tue, Jun 14, 2022 at 01:08:43PM +0800, An Long wrote:
> > _scratch_mkfs_sized only receive integer number of bytes as a valid
> > input. But if the MKFS_OPTIONS variable exists, it will use the
> > value of
> > block size in MKFS_OPTIONS to override input. In case of
> > MKFS_OPTIONS="-b 4k", would result in blocksize=4 but not 4096.
> > This
>
> Hi An,
>
> Good catch. Can you provide an example that a case fails on this
> issue, and
> then it tests passed after having this patch? I'm wondering why no
> one notice
> that before.
>
For example, I set MKFS_OPTIONS="-b 4k" and then run generic/416 on
ext4 filesystem.
# ./check tests/generic/416
FSTYP -- ext4
PLATFORM -- Linux/x86_64 bogon 5.3.18-57-default #1 SMP Wed Apr 28
10:54:41 UTC 2021 (ba3c2e9)
MKFS_OPTIONS -- -b 4k /dev/loop1
MOUNT_OPTIONS -- -o acl,user_xattr /dev/loop1 /mnt/scratch
generic/416 90s ... [failed, exit status 1]- output mismatch (see
/home/lan/work/xfstests/results//generic/416.out.bad)
--- tests/generic/416.out 2021-06-25 23:18:09.595254118 +0800
+++ /home/lan/work/xfstests/results//generic/416.out.bad 2022-
06-15 16:33:19.584206316 +0800
@@ -1,3 +1,4 @@
QA output created by 416
-wrote 16777216/16777216 bytes at offset 0
-XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+mount: /mnt/scratch: wrong fs type, bad option, bad superblock on
/dev/loop1, missing codepage or helper program, or other error.
+mount -o acl,user_xattr /dev/loop1 /mnt/scratch failed
+(see /home/lan/work/xfstests/results//generic/416.full for
details)
...
(Run 'diff -u /home/lan/work/xfstests/tests/generic/416.out
/home/lan/work/xfstests/results//generic/416.out.bad' to see the
entire diff)
Ran: generic/416
Failures: generic/416
Failed 1 of 1 tests
But the more common result as below:
# ./check tests/generic/416
FSTYP -- ext4
PLATFORM -- Linux/x86_64 bogon 5.3.18-57-default #1 SMP Wed Apr 28
10:54:41 UTC 2021 (ba3c2e9)
MKFS_OPTIONS -- -b 4k /dev/loop1
MOUNT_OPTIONS -- -o acl,user_xattr /dev/loop1 /mnt/scratch
umount: /mnt/scratch: target is busy.
generic/416 77s ... 90s
Ran: generic/416
Passed all 1 tests
Test passed due to _require_scratch_nocheck() ignore the umount error.
However, "mkfs.ext4: invalid block size - 4" was found in the 416.full
Apart from this, the error only occurs when MKFS_OPTIONS is set, and
doesn't affect xfs or btrfs. Since ext4 only supports 4k blocksize on
most platforms, we usually don't set blocksize in MKFS_OPTIONS.
I think that's why this problem hasn't been found before. In fact, I
found this issue when I tested on ppc64 with 64k kernel and 4k
subblocksize fs.
> > will give errors to ext2/3/4 etc, and brings potential bugs to xfs
> > or
> > btrfs.
> >
> > In addition, since we can receive various strings, so remove
> > integer
> > number check.
> >
> > Signed-off-by: An Long <lan@suse.com>
> > ---
> > common/rc | 18 ++++++------------
> > 1 file changed, 6 insertions(+), 12 deletions(-)
> >
> > diff --git a/common/rc b/common/rc
> > index 22050bc2..026007d3 100644
> > --- a/common/rc
> > +++ b/common/rc
> > @@ -1077,7 +1077,7 @@ _parse_size_from_string()
> > }
> >
> > # Create fs of certain size on scratch device
> > -# _scratch_mkfs_sized <size in bytes> [optional blocksize]
> > +# _scratch_mkfs_sized <size> [optional blocksize]
> > _scratch_mkfs_sized()
> > {
> > local fssize=$1
> > @@ -1086,13 +1086,13 @@ _scratch_mkfs_sized()
> >
> > case $FSTYP in
> > xfs)
> > - def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-b ?size=
> > ?+([0-9]+).*/\1/p'`
> > + def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-b ?size=
> > ?+([0-9]+[a-zA-Z]?).*/\1/p'`
> > ;;
> > btrfs)
> > - def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-s ?+([0-
> > 9]+).*/\1/p'`
> > + def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-s ?+([0-
> > 9]+[a-zA-Z]?).*/\1/p'`
> > ;;
> > ext2|ext3|ext4|ext4dev|udf|reiser4|ocfs2|reiserfs)
> > - def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-b ?+([0-
> > 9]+).*/\1/p'`
> > + def_blksz=`echo $MKFS_OPTIONS | sed -rn 's/.*-b ?+([0-
> > 9]+[a-zA-Z]?).*/\1/p'`
> > ;;
> > jfs)
> > def_blksz=4096
> > @@ -1101,14 +1101,8 @@ _scratch_mkfs_sized()
> >
> > [ -n "$def_blksz" ] && blocksize=$def_blksz
> > [ -z "$blocksize" ] && blocksize=4096
> > -
> > - local re='^[0-9]+$'
> > - if ! [[ $fssize =~ $re ]] ; then
> > - _notrun "error: _scratch_mkfs_sized: fs size
> > \"$fssize\" not an integer."
> > - fi
> > - if ! [[ $blocksize =~ $re ]] ; then
> > - _notrun "error: _scratch_mkfs_sized: block size
> > \"$blocksize\" not an integer."
> > - fi
> > + blocksize=$(_parse_size_from_string $blocksize)
> > + fssize=$(_parse_size_from_string $fssize)
>
> For now, the _parse_size_from_string is only used for this patch, I
> think these
> two patches can be merged into one patch, as it trys to fix one
> single problem.
>
There are a total of two logical changes. In particular it also include
a Function update. So it's better to keep two separate patch. But I'll
add patch depends on info to commit message.
Thank you for your time!
> Thanks,
> Zorro
>
> >
> > local blocks=`expr $fssize / $blocksize`
> >
> > --
> > 2.35.3
> >
--
An Long <lan@suse.com>
SUSE QE LSG, QE 2, Beijing
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 1/2] common/rc: add _parse_size_from_string
2022-06-15 6:05 ` [PATCH 1/2] common/rc: add _parse_size_from_string Zorro Lang
@ 2022-06-16 4:37 ` Long An
0 siblings, 0 replies; 6+ messages in thread
From: Long An @ 2022-06-16 4:37 UTC (permalink / raw)
To: zlang@redhat.com; +Cc: fstests@vger.kernel.org
On Wed, 2022-06-15 at 14:05 +0800, Zorro Lang wrote:
> On Tue, Jun 14, 2022 at 01:08:42PM +0800, An Long wrote:
> > Add a helper to convert size value to bytes. This is used to handle
> > value as bytes, such as 4k to 4096.
> >
> > Signed-off-by: An Long <lan@suse.com>
> > ---
> > common/rc | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
> > 1 file changed, 48 insertions(+)
> >
> > diff --git a/common/rc b/common/rc
> > index 3c072c16..22050bc2 100644
> > --- a/common/rc
> > +++ b/common/rc
> > @@ -1028,6 +1028,54 @@ _check_minimal_fs_size()
> > fi
> > }
> >
> > +# Convert size value to bytes
> > +# _parse_size_from_string <size>
> > +_parse_size_from_string()
>
> I think this name is better to be shorter, as it might be a lower
> level common
> function. How about _convert_num() or _cvtnum()?
>
Rename _parse_size_from_string() to _parse_size_tring(). So that as
much as possible to keep function meaning.
> > +{
> > + local str=$1
> > + local mult=1
> > + local size
> > + local endchar
> > +
> > + if [[ $str =~ ^[0-9]+[a-zA-Z]$ ]] ; then
> > + size=${str:: -1}
> > + endchar=${str: -1}
> > + case $endchar in
> > + e|E)
> > + mult=$(($mult * 1024))
> > + ;&
> > + p|P)
> > + mult=$(($mult * 1024))
> > + ;&
> > + t|T)
> > + mult=$(($mult * 1024))
> > + ;&
> > + g|G)
> > + mult=$(($mult * 1024))
> > + ;&
> > + m|M)
> > + mult=$(($mult * 1024))
> > + ;&
> > + k|K)
> > + mult=$(($mult * 1024))
> > + ;&
> > + b|B)
> > + ;;
> > + *)
> > + echo "unknown size descriptor
> > $endchar"
> > + exit 1
> > + esac
> > + elif [[ $str =~ ^[0-9]+$ ]] ; then
> > + size=$str
> > + else
> > + echo "size value $str is invalid"
> > + exit 1
> > + fi
> > +
> > + size=$(($size * $mult))
> ^ ^
> I think above two '$' isn't necessary, although it doesn't make
> something
> wrong if keep it. Same as all above (($mult * 1024)) .
>
> Thanks,
> Zorro
>
> ^
>
Removed.
> > + echo $size
> > +}
> > +
> > # Create fs of certain size on scratch device
> > # _scratch_mkfs_sized <size in bytes> [optional blocksize]
> > _scratch_mkfs_sized()
> > --
> > 2.35.3
> >
--
An Long <lan@suse.com>
SUSE QE LSG, QE 2, Beijing
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2022-06-16 4:37 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-06-14 5:08 [PATCH 1/2] common/rc: add _parse_size_from_string An Long
2022-06-14 5:08 ` [PATCH 2/2] common/rc: fix input value to _scratch_mkfs_sized An Long
2022-06-15 4:15 ` Zorro Lang
2022-06-16 4:30 ` Long An
2022-06-15 6:05 ` [PATCH 1/2] common/rc: add _parse_size_from_string Zorro Lang
2022-06-16 4:37 ` Long An
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox