[PATCH v2 0/1] generic: test i_blocks for truncated large files

[PATCH v2 0/1] generic: test i_blocks for truncated large files

[Pavel Reichl]

thanks for the comments on version #1 and sorry for a late reply
- I was on a vacation.

I updated the test as suggested and worked around the EXT2 fail using 
"_within_tolerance" and 1% as the tolerance...I have a mixed feelings
about this - it's not supported by any math, just a magic number to
avoid false positive. Any thoughts?

Thanks!


Pavel Reichl (1):
  generic: test i_blocks for truncated large files

 tests/generic/698     | 48 +++++++++++++++++++++++++++++++++++++++++++
 tests/generic/698.out |  2 ++
 2 files changed, 50 insertions(+)
 create mode 100755 tests/generic/698
 create mode 100644 tests/generic/698.out

-- 
2.37.3

[PATCH v2 1/1] generic: test i_blocks for truncated large files

[Pavel Reichl]

This is a regression test for an incorrect computation of i_blocks for
truncated files larger than 4 GiB. Bug was filed for exFAT.

Test is based on reproducer provied by Christophe Vu-Brugier as part
of kernel patch-fix submission.

Signed-off-by: Pavel Reichl <preichl@redhat.com>
---
 tests/generic/698     | 48 +++++++++++++++++++++++++++++++++++++++++++
 tests/generic/698.out |  2 ++
 2 files changed, 50 insertions(+)
 create mode 100755 tests/generic/698
 create mode 100644 tests/generic/698.out

diff --git a/tests/generic/698 b/tests/generic/698
new file mode 100755
index 00000000..337df6c5
--- /dev/null
+++ b/tests/generic/698
@@ -0,0 +1,48 @@
+#! /bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2022  Red Hat Inc. All Rights Reserved.
+#
+# FS QA Test 698
+#
+# Verify that i_blocks for truncated files larger than 4 GiB have correct
+# values.
+#
+# This test verifies the problem fixed in kernel with commit
+# 92fba084b79e exfat: fix i_blocks for files truncated over 4 GiB
+#
+. ./common/preamble
+. ./common/filter
+
+_begin_fstest auto
+
+# Override the default cleanup function.
+_cleanup()
+{
+	cd /
+	rm -r -f $tmp.* $junk_dir
+}
+
+_supported_fs generic
+_fixed_by_kernel_commit 92fba084b79e \
+	"exfat: fix i_blocks for files truncated over 4 GiB"
+
+_require_test
+_require_fs_space $TEST_DIR $((5 * 1024 * 1024)) #kB
+
+junk_dir=$TEST_DIR/$seq
+junk_file=$junk_dir/junk
+mkdir -p $junk_dir
+
+$XFS_IO_PROG -f -c "pwrite -W 0 5G" $junk_file > /dev/null
+
+truncate -s $((4 * 1024 * 1024 * 1024)) $junk_file
+
+block_size=`stat -c '%B' $junk_file`
+iblocks_after_truncate=`stat -c '%b' $junk_file`
+iblocks_expected=$((4 * 1024 * 1024 * 1024 / $block_size))
+
+_within_tolerance "Number of allocated blocks after truncate" $iblocks_after_truncate $iblocks_expected 1% -v
+
+status=0
+
+exit
diff --git a/tests/generic/698.out b/tests/generic/698.out
new file mode 100644
index 00000000..cbb02d37
--- /dev/null
+++ b/tests/generic/698.out
@@ -0,0 +1,2 @@
+QA output created by 698
+Number of allocated blocks after truncate is in range
-- 
2.37.3

Re: [PATCH v2 1/1] generic: test i_blocks for truncated large files

[Zorro Lang]

On Tue, Sep 13, 2022 at 02:58:01PM +0200, Pavel Reichl wrote:
> This is a regression test for an incorrect computation of i_blocks for
> truncated files larger than 4 GiB. Bug was filed for exFAT.
> 
> Test is based on reproducer provied by Christophe Vu-Brugier as part
> of kernel patch-fix submission.
> 
> Signed-off-by: Pavel Reichl <preichl@redhat.com>
> ---

This version looks good to me.

Reviewed-by: Zorro Lang <zlang@redhat.com>

>  tests/generic/698     | 48 +++++++++++++++++++++++++++++++++++++++++++
>  tests/generic/698.out |  2 ++
>  2 files changed, 50 insertions(+)
>  create mode 100755 tests/generic/698
>  create mode 100644 tests/generic/698.out
> 
> diff --git a/tests/generic/698 b/tests/generic/698
> new file mode 100755
> index 00000000..337df6c5
> --- /dev/null
> +++ b/tests/generic/698
> @@ -0,0 +1,48 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2022  Red Hat Inc. All Rights Reserved.
> +#
> +# FS QA Test 698
> +#
> +# Verify that i_blocks for truncated files larger than 4 GiB have correct
> +# values.
> +#
> +# This test verifies the problem fixed in kernel with commit
> +# 92fba084b79e exfat: fix i_blocks for files truncated over 4 GiB
> +#
> +. ./common/preamble
> +. ./common/filter
> +
> +_begin_fstest auto
> +
> +# Override the default cleanup function.
> +_cleanup()
> +{
> +	cd /
> +	rm -r -f $tmp.* $junk_dir
> +}
> +
> +_supported_fs generic
> +_fixed_by_kernel_commit 92fba084b79e \
> +	"exfat: fix i_blocks for files truncated over 4 GiB"
> +
> +_require_test
> +_require_fs_space $TEST_DIR $((5 * 1024 * 1024)) #kB
> +
> +junk_dir=$TEST_DIR/$seq
> +junk_file=$junk_dir/junk
> +mkdir -p $junk_dir
> +
> +$XFS_IO_PROG -f -c "pwrite -W 0 5G" $junk_file > /dev/null
> +
> +truncate -s $((4 * 1024 * 1024 * 1024)) $junk_file
> +
> +block_size=`stat -c '%B' $junk_file`
> +iblocks_after_truncate=`stat -c '%b' $junk_file`
> +iblocks_expected=$((4 * 1024 * 1024 * 1024 / $block_size))
> +
> +_within_tolerance "Number of allocated blocks after truncate" $iblocks_after_truncate $iblocks_expected 1% -v
> +
> +status=0
> +
> +exit
> diff --git a/tests/generic/698.out b/tests/generic/698.out
> new file mode 100644
> index 00000000..cbb02d37
> --- /dev/null
> +++ b/tests/generic/698.out
> @@ -0,0 +1,2 @@
> +QA output created by 698
> +Number of allocated blocks after truncate is in range
> -- 
> 2.37.3
> 

Re: [PATCH v2 1/1] generic: test i_blocks for truncated large files

[Darrick J. Wong]

On Tue, Sep 13, 2022 at 02:58:01PM +0200, Pavel Reichl wrote:
> This is a regression test for an incorrect computation of i_blocks for
> truncated files larger than 4 GiB. Bug was filed for exFAT.
> 
> Test is based on reproducer provied by Christophe Vu-Brugier as part
> of kernel patch-fix submission.
> 
> Signed-off-by: Pavel Reichl <preichl@redhat.com>
> ---
>  tests/generic/698     | 48 +++++++++++++++++++++++++++++++++++++++++++
>  tests/generic/698.out |  2 ++
>  2 files changed, 50 insertions(+)
>  create mode 100755 tests/generic/698
>  create mode 100644 tests/generic/698.out
> 
> diff --git a/tests/generic/698 b/tests/generic/698
> new file mode 100755
> index 00000000..337df6c5
> --- /dev/null
> +++ b/tests/generic/698
> @@ -0,0 +1,48 @@
> +#! /bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2022  Red Hat Inc. All Rights Reserved.
> +#
> +# FS QA Test 698
> +#
> +# Verify that i_blocks for truncated files larger than 4 GiB have correct
> +# values.
> +#
> +# This test verifies the problem fixed in kernel with commit
> +# 92fba084b79e exfat: fix i_blocks for files truncated over 4 GiB
> +#
> +. ./common/preamble
> +. ./common/filter
> +
> +_begin_fstest auto
> +
> +# Override the default cleanup function.
> +_cleanup()
> +{
> +	cd /
> +	rm -r -f $tmp.* $junk_dir
> +}
> +
> +_supported_fs generic
> +_fixed_by_kernel_commit 92fba084b79e \
> +	"exfat: fix i_blocks for files truncated over 4 GiB"
> +
> +_require_test
> +_require_fs_space $TEST_DIR $((5 * 1024 * 1024)) #kB
> +
> +junk_dir=$TEST_DIR/$seq
> +junk_file=$junk_dir/junk
> +mkdir -p $junk_dir
> +
> +$XFS_IO_PROG -f -c "pwrite -W 0 5G" $junk_file > /dev/null

I wonder, how long will this test take to write 5GB of data?  Can we
speed it up by using fallocate, at least for the filesystems that
support it?

(Aside from that, the rest looks fine to me...)

--D

> +
> +truncate -s $((4 * 1024 * 1024 * 1024)) $junk_file
> +
> +block_size=`stat -c '%B' $junk_file`
> +iblocks_after_truncate=`stat -c '%b' $junk_file`
> +iblocks_expected=$((4 * 1024 * 1024 * 1024 / $block_size))
> +
> +_within_tolerance "Number of allocated blocks after truncate" $iblocks_after_truncate $iblocks_expected 1% -v
> +
> +status=0
> +
> +exit
> diff --git a/tests/generic/698.out b/tests/generic/698.out
> new file mode 100644
> index 00000000..cbb02d37
> --- /dev/null
> +++ b/tests/generic/698.out
> @@ -0,0 +1,2 @@
> +QA output created by 698
> +Number of allocated blocks after truncate is in range
> -- 
> 2.37.3
> 

Re: [PATCH v2 1/1] generic: test i_blocks for truncated large files

[Zorro Lang]

On Wed, Sep 14, 2022 at 10:59:40AM -0700, Darrick J. Wong wrote:
> On Tue, Sep 13, 2022 at 02:58:01PM +0200, Pavel Reichl wrote:
> > This is a regression test for an incorrect computation of i_blocks for
> > truncated files larger than 4 GiB. Bug was filed for exFAT.
> > 
> > Test is based on reproducer provied by Christophe Vu-Brugier as part
> > of kernel patch-fix submission.
> > 
> > Signed-off-by: Pavel Reichl <preichl@redhat.com>
> > ---
> >  tests/generic/698     | 48 +++++++++++++++++++++++++++++++++++++++++++
> >  tests/generic/698.out |  2 ++
> >  2 files changed, 50 insertions(+)
> >  create mode 100755 tests/generic/698
> >  create mode 100644 tests/generic/698.out
> > 
> > diff --git a/tests/generic/698 b/tests/generic/698
> > new file mode 100755
> > index 00000000..337df6c5
> > --- /dev/null
> > +++ b/tests/generic/698
> > @@ -0,0 +1,48 @@
> > +#! /bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +# Copyright (c) 2022  Red Hat Inc. All Rights Reserved.
> > +#
> > +# FS QA Test 698
> > +#
> > +# Verify that i_blocks for truncated files larger than 4 GiB have correct
> > +# values.
> > +#
> > +# This test verifies the problem fixed in kernel with commit
> > +# 92fba084b79e exfat: fix i_blocks for files truncated over 4 GiB
> > +#
> > +. ./common/preamble
> > +. ./common/filter
> > +
> > +_begin_fstest auto
> > +
> > +# Override the default cleanup function.
> > +_cleanup()
> > +{
> > +	cd /
> > +	rm -r -f $tmp.* $junk_dir
> > +}
> > +
> > +_supported_fs generic
> > +_fixed_by_kernel_commit 92fba084b79e \
> > +	"exfat: fix i_blocks for files truncated over 4 GiB"
> > +
> > +_require_test
> > +_require_fs_space $TEST_DIR $((5 * 1024 * 1024)) #kB
> > +
> > +junk_dir=$TEST_DIR/$seq
> > +junk_file=$junk_dir/junk
> > +mkdir -p $junk_dir
> > +
> > +$XFS_IO_PROG -f -c "pwrite -W 0 5G" $junk_file > /dev/null
> 
> I wonder, how long will this test take to write 5GB of data?  Can we
> speed it up by using fallocate, at least for the filesystems that
> support it?
> 
> (Aside from that, the rest looks fine to me...)

I tried it on one of my testing machine (without high performance), it takes
about 20~30s [1].

The generic/694 is the same with this case. Hi Pavel, If you'd like to speed
it up by detecting and using fallocate, please improve g/694 too. I think we
can have a common helper to get i_blocks for a file, in this helper we can
choose a faster way to do that. For example [PATCH 1/2] brings in this helper
and use it in g/694, then [PATCH 2/2] is this patch.

Thanks,
Zorro


[1]
  # time xfs_io -t -f -c "pwrite 0 5g" /home/file
  wrote 5368709120/5368709120 bytes at offset 0
  5.000 GiB, 1310720 ops; 0:00:23.58 (217.048 MiB/sec and 55564.3920 ops/sec)

  real    0m24.228s
  user    0m0.722s
  sys     0m23.173s

  # time xfs_io -t -f -c "pwrite 0 5g" /home/file
  wrote 5368709120/5368709120 bytes at offset 0
  5.000 GiB, 1310720 ops; 0:00:22.91 (223.438 MiB/sec and 57200.0158 ops/sec)

  real    0m27.538s
  user    0m0.684s
  sys     0m26.313s





> 
> --D
> 
> > +
> > +truncate -s $((4 * 1024 * 1024 * 1024)) $junk_file
> > +
> > +block_size=`stat -c '%B' $junk_file`
> > +iblocks_after_truncate=`stat -c '%b' $junk_file`
> > +iblocks_expected=$((4 * 1024 * 1024 * 1024 / $block_size))
> > +
> > +_within_tolerance "Number of allocated blocks after truncate" $iblocks_after_truncate $iblocks_expected 1% -v
> > +
> > +status=0
> > +
> > +exit
> > diff --git a/tests/generic/698.out b/tests/generic/698.out
> > new file mode 100644
> > index 00000000..cbb02d37
> > --- /dev/null
> > +++ b/tests/generic/698.out
> > @@ -0,0 +1,2 @@
> > +QA output created by 698
> > +Number of allocated blocks after truncate is in range
> > -- 
> > 2.37.3
> > 
>