* [PATCH] btrfs: test whether fallocate(2) can preallocate half of the whole fs space
@ 2016-07-20 7:34 Wang Xiaoguang
2016-07-20 9:25 ` Filipe Manana
2016-07-20 19:00 ` Christoph Hellwig
0 siblings, 2 replies; 5+ messages in thread
From: Wang Xiaoguang @ 2016-07-20 7:34 UTC (permalink / raw)
To: fstests; +Cc: linux-btrfs
Currently in btrfs, there is something wrong with data space reservation.
For example, if we try to preallocate more than haf of whole fs space,
ENOSPC will occur, but indeed fs still has free space to satisfy this
request.
To easily reproduce this bug, this test case needs fs is mixed mode(btrfs
specific), so put this test case in btrfs group, not generic group.
Signed-off-by: Wang Xiaoguang <wangxg.fnst@cn.fujitsu.com>
---
tests/btrfs/127 | 61 +++++++++++++++++++++++++++++++++++++++++++++++++++++
tests/btrfs/127.out | 2 ++
tests/btrfs/group | 1 +
3 files changed, 64 insertions(+)
create mode 100755 tests/btrfs/127
create mode 100644 tests/btrfs/127.out
diff --git a/tests/btrfs/127 b/tests/btrfs/127
new file mode 100755
index 0000000..f95c72f
--- /dev/null
+++ b/tests/btrfs/127
@@ -0,0 +1,61 @@
+#! /bin/bash
+# FS QA Test 127
+#
+# Test whether fallocate(2) can preallocate half of the whole fs space.
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2016 Fujitsu. All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1 # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+ cd /
+ rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+MKFS_OPTIONS="-M"
+_scratch_mkfs_sized $((256 * 1024 * 1024)) >> $seqres.full 2>&1
+_scratch_mount
+
+$XFS_IO_PROG -f -c "falloc 0 128M" $SCRATCH_MNT/testfile | _filter_xfs_io
+
+echo "Silence is golden"
+status=0
+exit
diff --git a/tests/btrfs/127.out b/tests/btrfs/127.out
new file mode 100644
index 0000000..0af84c0
--- /dev/null
+++ b/tests/btrfs/127.out
@@ -0,0 +1,2 @@
+QA output created by 127
+Silence is golden
diff --git a/tests/btrfs/group b/tests/btrfs/group
index a21a80a..4c3ac00 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -129,3 +129,4 @@
124 auto replace
125 auto replace
126 auto quick qgroup
+127 auto quick metadata enospc
--
2.9.0
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH] btrfs: test whether fallocate(2) can preallocate half of the whole fs space
2016-07-20 7:34 [PATCH] btrfs: test whether fallocate(2) can preallocate half of the whole fs space Wang Xiaoguang
@ 2016-07-20 9:25 ` Filipe Manana
2016-07-21 7:22 ` Wang Xiaoguang
2016-07-20 19:00 ` Christoph Hellwig
1 sibling, 1 reply; 5+ messages in thread
From: Filipe Manana @ 2016-07-20 9:25 UTC (permalink / raw)
To: Wang Xiaoguang; +Cc: fstests, linux-btrfs@vger.kernel.org
On Wed, Jul 20, 2016 at 8:34 AM, Wang Xiaoguang
<wangxg.fnst@cn.fujitsu.com> wrote:
> Currently in btrfs, there is something wrong with data space reservation.
> For example, if we try to preallocate more than haf of whole fs space,
> ENOSPC will occur, but indeed fs still has free space to satisfy this
> request.
>
> To easily reproduce this bug, this test case needs fs is mixed mode(btrfs
> specific), so put this test case in btrfs group, not generic group.
Why does it need to be in mixed mode (why is it easier)? As far as I
can see from the corresponding btrfs patch (and its description),
the problem affects non-mixed mode, therefore it could be a generic test.
>
> Signed-off-by: Wang Xiaoguang <wangxg.fnst@cn.fujitsu.com>
> ---
> tests/btrfs/127 | 61 +++++++++++++++++++++++++++++++++++++++++++++++++++++
> tests/btrfs/127.out | 2 ++
> tests/btrfs/group | 1 +
> 3 files changed, 64 insertions(+)
> create mode 100755 tests/btrfs/127
> create mode 100644 tests/btrfs/127.out
>
> diff --git a/tests/btrfs/127 b/tests/btrfs/127
> new file mode 100755
> index 0000000..f95c72f
> --- /dev/null
> +++ b/tests/btrfs/127
> @@ -0,0 +1,61 @@
> +#! /bin/bash
> +# FS QA Test 127
> +#
> +# Test whether fallocate(2) can preallocate half of the whole fs space.
> +#
> +#-----------------------------------------------------------------------
> +# Copyright (c) 2016 Fujitsu. All Rights Reserved.
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
> +#-----------------------------------------------------------------------
> +#
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1 # failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> + cd /
> + rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs btrfs
> +_supported_os Linux
> +_require_scratch
> +
> +MKFS_OPTIONS="-M"
This now overrides any specific mkfs options specified in the command
line, doesn't it?
> +_scratch_mkfs_sized $((256 * 1024 * 1024)) >> $seqres.full 2>&1
> +_scratch_mount
> +
> +$XFS_IO_PROG -f -c "falloc 0 128M" $SCRATCH_MNT/testfile | _filter_xfs_io
No need for the filter here.
> +
> +echo "Silence is golden"
> +status=0
> +exit
> diff --git a/tests/btrfs/127.out b/tests/btrfs/127.out
> new file mode 100644
> index 0000000..0af84c0
> --- /dev/null
> +++ b/tests/btrfs/127.out
> @@ -0,0 +1,2 @@
> +QA output created by 127
> +Silence is golden
> diff --git a/tests/btrfs/group b/tests/btrfs/group
> index a21a80a..4c3ac00 100644
> --- a/tests/btrfs/group
> +++ b/tests/btrfs/group
> @@ -129,3 +129,4 @@
> 124 auto replace
> 125 auto replace
> 126 auto quick qgroup
> +127 auto quick metadata enospc
> --
> 2.9.0
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Filipe David Manana,
"People will forget what you said,
people will forget what you did,
but people will never forget how you made them feel."
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] btrfs: test whether fallocate(2) can preallocate half of the whole fs space
2016-07-20 7:34 [PATCH] btrfs: test whether fallocate(2) can preallocate half of the whole fs space Wang Xiaoguang
2016-07-20 9:25 ` Filipe Manana
@ 2016-07-20 19:00 ` Christoph Hellwig
2016-07-21 7:24 ` Wang Xiaoguang
1 sibling, 1 reply; 5+ messages in thread
From: Christoph Hellwig @ 2016-07-20 19:00 UTC (permalink / raw)
To: Wang Xiaoguang; +Cc: fstests, linux-btrfs
On Wed, Jul 20, 2016 at 03:34:50PM +0800, Wang Xiaoguang wrote:
> Currently in btrfs, there is something wrong with data space reservation.
> For example, if we try to preallocate more than haf of whole fs space,
> ENOSPC will occur, but indeed fs still has free space to satisfy this
> request.
>
> To easily reproduce this bug, this test case needs fs is mixed mode(btrfs
> specific), so put this test case in btrfs group, not generic group.
But the actual test isn't btrfs specific, and other file systems might
have issues in this area as well. Please add it to the generic group
instead.
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] btrfs: test whether fallocate(2) can preallocate half of the whole fs space
2016-07-20 9:25 ` Filipe Manana
@ 2016-07-21 7:22 ` Wang Xiaoguang
0 siblings, 0 replies; 5+ messages in thread
From: Wang Xiaoguang @ 2016-07-21 7:22 UTC (permalink / raw)
To: fdmanana; +Cc: fstests, linux-btrfs@vger.kernel.org
hello,
On 07/20/2016 05:25 PM, Filipe Manana wrote:
> On Wed, Jul 20, 2016 at 8:34 AM, Wang Xiaoguang
> <wangxg.fnst@cn.fujitsu.com> wrote:
>> Currently in btrfs, there is something wrong with data space reservation.
>> For example, if we try to preallocate more than haf of whole fs space,
>> ENOSPC will occur, but indeed fs still has free space to satisfy this
>> request.
>>
>> To easily reproduce this bug, this test case needs fs is mixed mode(btrfs
>> specific), so put this test case in btrfs group, not generic group.
> Why does it need to be in mixed mode (why is it easier)? As far as I
> can see from the corresponding btrfs patch (and its description),
> the problem affects non-mixed mode, therefore it could be a generic test.
In btrfs mixed mode, as you know, data and metadata share the same
btrfs_space_info. In btrfs_fallocate(), if we try to pre-allocate 64M data,
it first makes bytes_may_use += 64M, later btrfs_reserve_extent()
only makes bytes_reserved += 64M, but does not decrease bytes_may_use.
Assume our fs is 128M, now bytes_may_use + bytes_reserved is already
greater than 128M, so any later transaction operations in
btrfs_fallocate() will
fail for ENOSPC reason. But note, byte_may_use will only be decreased in end
of btrfs_fallocate(). So I think this bug is easy to reproduce in mixed
mode :)
For non-mixed mode, still assume fs is 128M. For data's btrfs_space_info,
before we call btrfs_free_reserved_data_space() in end of
btrfs_fallocate(), though
bytes_may_use +=64M, bytes_reserved += 64M, and bytes_may_use +
bytes_reserved > 128M, there is no other process to allocating data
space, so
there will still be no ENOSPC error.
I have rewritten a test case which do write(2) and fallocate(2) in
parallel and
put this test case to generic group, thanks.
Regards,
Xiaoguang Wang
>
>> Signed-off-by: Wang Xiaoguang <wangxg.fnst@cn.fujitsu.com>
>> ---
>> tests/btrfs/127 | 61 +++++++++++++++++++++++++++++++++++++++++++++++++++++
>> tests/btrfs/127.out | 2 ++
>> tests/btrfs/group | 1 +
>> 3 files changed, 64 insertions(+)
>> create mode 100755 tests/btrfs/127
>> create mode 100644 tests/btrfs/127.out
>>
>> diff --git a/tests/btrfs/127 b/tests/btrfs/127
>> new file mode 100755
>> index 0000000..f95c72f
>> --- /dev/null
>> +++ b/tests/btrfs/127
>> @@ -0,0 +1,61 @@
>> +#! /bin/bash
>> +# FS QA Test 127
>> +#
>> +# Test whether fallocate(2) can preallocate half of the whole fs space.
>> +#
>> +#-----------------------------------------------------------------------
>> +# Copyright (c) 2016 Fujitsu. All Rights Reserved.
>> +#
>> +# This program is free software; you can redistribute it and/or
>> +# modify it under the terms of the GNU General Public License as
>> +# published by the Free Software Foundation.
>> +#
>> +# This program is distributed in the hope that it would be useful,
>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>> +# GNU General Public License for more details.
>> +#
>> +# You should have received a copy of the GNU General Public License
>> +# along with this program; if not, write the Free Software Foundation,
>> +# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
>> +#-----------------------------------------------------------------------
>> +#
>> +
>> +seq=`basename $0`
>> +seqres=$RESULT_DIR/$seq
>> +echo "QA output created by $seq"
>> +
>> +here=`pwd`
>> +tmp=/tmp/$$
>> +status=1 # failure is the default!
>> +trap "_cleanup; exit \$status" 0 1 2 3 15
>> +
>> +_cleanup()
>> +{
>> + cd /
>> + rm -f $tmp.*
>> +}
>> +
>> +# get standard environment, filters and checks
>> +. ./common/rc
>> +. ./common/filter
>> +
>> +# remove previous $seqres.full before test
>> +rm -f $seqres.full
>> +
>> +# real QA test starts here
>> +
>> +# Modify as appropriate.
>> +_supported_fs btrfs
>> +_supported_os Linux
>> +_require_scratch
>> +
>> +MKFS_OPTIONS="-M"
> This now overrides any specific mkfs options specified in the command
> line, doesn't it?
>
>> +_scratch_mkfs_sized $((256 * 1024 * 1024)) >> $seqres.full 2>&1
>> +_scratch_mount
>> +
>> +$XFS_IO_PROG -f -c "falloc 0 128M" $SCRATCH_MNT/testfile | _filter_xfs_io
> No need for the filter here.
>
>> +
>> +echo "Silence is golden"
>> +status=0
>> +exit
>> diff --git a/tests/btrfs/127.out b/tests/btrfs/127.out
>> new file mode 100644
>> index 0000000..0af84c0
>> --- /dev/null
>> +++ b/tests/btrfs/127.out
>> @@ -0,0 +1,2 @@
>> +QA output created by 127
>> +Silence is golden
>> diff --git a/tests/btrfs/group b/tests/btrfs/group
>> index a21a80a..4c3ac00 100644
>> --- a/tests/btrfs/group
>> +++ b/tests/btrfs/group
>> @@ -129,3 +129,4 @@
>> 124 auto replace
>> 125 auto replace
>> 126 auto quick qgroup
>> +127 auto quick metadata enospc
>> --
>> 2.9.0
>>
>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe fstests" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] btrfs: test whether fallocate(2) can preallocate half of the whole fs space
2016-07-20 19:00 ` Christoph Hellwig
@ 2016-07-21 7:24 ` Wang Xiaoguang
0 siblings, 0 replies; 5+ messages in thread
From: Wang Xiaoguang @ 2016-07-21 7:24 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: fstests, linux-btrfs
hello,
On 07/21/2016 03:00 AM, Christoph Hellwig wrote:
> On Wed, Jul 20, 2016 at 03:34:50PM +0800, Wang Xiaoguang wrote:
>> Currently in btrfs, there is something wrong with data space reservation.
>> For example, if we try to preallocate more than haf of whole fs space,
>> ENOSPC will occur, but indeed fs still has free space to satisfy this
>> request.
>>
>> To easily reproduce this bug, this test case needs fs is mixed mode(btrfs
>> specific), so put this test case in btrfs group, not generic group.
> But the actual test isn't btrfs specific, and other file systems might
> have issues in this area as well. Please add it to the generic group
> instead.
OK, new version will be sent soon.
ext4 & xfs pass the test, btrfs fails.
Regards,
Xiaoguang Wang
>
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2016-07-21 7:27 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-07-20 7:34 [PATCH] btrfs: test whether fallocate(2) can preallocate half of the whole fs space Wang Xiaoguang
2016-07-20 9:25 ` Filipe Manana
2016-07-21 7:22 ` Wang Xiaoguang
2016-07-20 19:00 ` Christoph Hellwig
2016-07-21 7:24 ` Wang Xiaoguang
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox