Re: ext34_free_inode's mess

Re: ext34_free_inode's mess

[Eric Sandeen]

Dmitry Monakhov wrote:
> I've finally automated my favorite testcase (see attachment), 
> before i've run it by hand.

Thanks!  Feel free to cc: the xfs list since the patch hits
xfstests.  (I added it here)

>  227     |  105 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  227.out |    5 +++
>  group   |    1 +
>  3 files changed, 111 insertions(+), 0 deletions(-)
>  create mode 100755 227
>  create mode 100644 227.out
> 
> diff --git a/227 b/227
> new file mode 100755
> index 0000000..d2b0c7d
> --- /dev/null
> +++ b/227
> @@ -0,0 +1,105 @@
> +#! /bin/bash
> +# FS QA Test No. 227
> +#
> +# Perform fsstress test with parallel dd
> +# This proven to be a good stress test
> +# * Continuous dd retult in ENOSPC condition but only for a limited periods
> +#   of time.
> +# * Fsstress test cover many code paths

just little editor nitpicks: 

+# Perform fsstress test with parallel dd
+# This is proven to be a good stress test
+# * Continuous dd results in ENOSPC condition but only for a limited period
+#   of time.
+# * Fsstress test covers many code paths


> +#
> +#-----------------------------------------------------------------------
> +# Copyright (c) 2010 Dmitry Monakhov.  All Rights Reserved.
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
> +#
> +#-----------------------------------------------------------------------
> +#
> +# creator
> +owner=dmonakhov@openvz.org
> +
> +seq=`basename $0`
> +echo "QA output created by $seq"
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1	# failure is the default!
> +
> +_cleanup()
> +{
> +    rm -f $tmp.*
> +}
> +
> +workout()
> +{
> +    # Disable bash job controll, to prevent message about killed task.

s/controll/control/

> +    set +m
> +
> +    #Timing parameters
> +    nr_iterations=5
> +    kill_tries=20
> +    echo Running fsstress. | tee -a $seq.full
> +
> +####################################################

What is all this for?

FWIW other fsstress tests use an $FSSTRESS_AVOID variable,
where you can set the things you want to avoid easily

> +##    -f unresvsp=0 -f allocsp=0 -f freesp=0 \
> +##    -f setxattr=0 -f attr_remove=0 -f attr_set=0 \
> +## 
> +######################################################
> +    mkdir -p $SCRATCH_MNT/fsstress
> +    # It is reasonable to disable sync, otherwise most of tasks will simply
> +    # stuck in that sync() call.
> +    $FSSTRESS_PROG \
> +	-d $SCRATCH_MNT/fsstress \
> +	-p 100 -f sync=0  -n 9999999 > /dev/null 2>&1 &
> +
> +    echo Running ENOSPC hitters. | tee -a $seq.full
> +    for ((i = 0; i < $nr_iterations; i++))
> +    do
> +	#Open with O_TRUNC and then write until error
> +	#hit ENOSPC each time.
> +	dd if=/dev/zero of=$SCRATCH_MNT/BIG_FILE bs=1M 2> /dev/null
> +    done
> +
> +    for ((i = 0; i < $kill_tries; i++))
> +    do
> +	killall -r -q -TERM fsstress 2> /dev/null
> +	sleep 1
> +    done
> +}
> +
> +trap "_cleanup ; exit \$status" 0 1 2 3 15
> +
> +# get standard environment, filters and checks
> +. ./common.rc
> +. ./common.filter
> +
> +# real QA test starts here
> +_supported_fs generic
> +_supported_os Linux
> +_require_scratch
> +
> +rm -f $seq.full
> +
> +umount $TEST_DEV >/dev/null 2>&1
> +umount $SCRATCH_DEV >/dev/null 2>&1
> +echo "*** MKFS ***"                         >>$seq.full
> +echo ""                                     >>$seq.full
> +_scratch_mkfs    >/dev/null 2>&1 || _fail "mkfs failed"
> +_scratch_mount   >/dev/null 2>&1 || _fail "mount failed"
> +
> +workout
> +umount $SCRATCH_MNT
> +echo 
> +echo Checking filesystem
> +_check_scratch_fs
> +status=$?
> +exit
> diff --git a/227.out b/227.out
> new file mode 100644
> index 0000000..6a7342d
> --- /dev/null
> +++ b/227.out
> @@ -0,0 +1,5 @@
> +QA output created by 227
> +Running fsstress.
> +Running ENOSPC hitters.
> +
> +Checking filesystem
> diff --git a/group b/group
> index 8d4a83a..81a2aa4 100644
> --- a/group
> +++ b/group
> @@ -339,3 +339,4 @@ deprecated
>  223 auto quick
>  224 auto
>  225 auto quick
> +227 rw auto prealloc enospc

Is this prealloc just because fsstress may run resvsp?
FWIW, other fsstress tests aren't in that group, so this is
as little inconsistent.

Thanks for writing an xfstests patch! :)

-Eric

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: ext34_free_inode's mess

[Dmitry Monakhov]

[-- Attachment #1: Type: text/plain, Size: 5870 bytes --]

Eric Sandeen <sandeen@redhat.com> writes:

> Dmitry Monakhov wrote:
>> I've finally automated my favorite testcase (see attachment), 
>> before i've run it by hand.
>
> Thanks!  Feel free to cc: the xfs list since the patch hits
> xfstests.  (I added it here)
>
>>  227     |  105 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>  227.out |    5 +++
>>  group   |    1 +
>>  3 files changed, 111 insertions(+), 0 deletions(-)
>>  create mode 100755 227
>>  create mode 100644 227.out
>> 
>> diff --git a/227 b/227
>> new file mode 100755
>> index 0000000..d2b0c7d
>> --- /dev/null
>> +++ b/227
>> @@ -0,0 +1,105 @@
>> +#! /bin/bash
>> +# FS QA Test No. 227
>> +#
>> +# Perform fsstress test with parallel dd
>> +# This proven to be a good stress test
>> +# * Continuous dd retult in ENOSPC condition but only for a limited periods
>> +#   of time.
>> +# * Fsstress test cover many code paths
>
> just little editor nitpicks: 
>
> +# Perform fsstress test with parallel dd
> +# This is proven to be a good stress test
> +# * Continuous dd results in ENOSPC condition but only for a limited period
> +#   of time.
> +# * Fsstress test covers many code paths
>
>
>> +#
>> +#-----------------------------------------------------------------------
>> +# Copyright (c) 2010 Dmitry Monakhov.  All Rights Reserved.
>> +#
>> +# This program is free software; you can redistribute it and/or
>> +# modify it under the terms of the GNU General Public License as
>> +# published by the Free Software Foundation.
>> +#
>> +# This program is distributed in the hope that it would be useful,
>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> +# GNU General Public License for more details.
>> +#
>> +# You should have received a copy of the GNU General Public License
>> +# along with this program; if not, write the Free Software Foundation,
>> +# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
>> +#
>> +#-----------------------------------------------------------------------
>> +#
>> +# creator
>> +owner=dmonakhov@openvz.org
>> +
>> +seq=`basename $0`
>> +echo "QA output created by $seq"
>> +here=`pwd`
>> +tmp=/tmp/$$
>> +status=1	# failure is the default!
>> +
>> +_cleanup()
>> +{
>> +    rm -f $tmp.*
>> +}
>> +
>> +workout()
>> +{
>> +    # Disable bash job controll, to prevent message about killed task.
>
> s/controll/control/
Ok, will redo and submit it one more time.
>
>> +    set +m
>> +
>> +    #Timing parameters
>> +    nr_iterations=5
>> +    kill_tries=20
>> +    echo Running fsstress. | tee -a $seq.full
>> +
>> +####################################################
>
> What is all this for?
>
> FWIW other fsstress tests use an $FSSTRESS_AVOID variable,
> where you can set the things you want to avoid easily
I've add this when investigated uninit=>init extent bug.
and forgot to remove.
>
>> +##    -f unresvsp=0 -f allocsp=0 -f freesp=0 \
>> +##    -f setxattr=0 -f attr_remove=0 -f attr_set=0 \
>> +## 
>> +######################################################
>> +    mkdir -p $SCRATCH_MNT/fsstress
>> +    # It is reasonable to disable sync, otherwise most of tasks will simply
>> +    # stuck in that sync() call.
>> +    $FSSTRESS_PROG \
>> +	-d $SCRATCH_MNT/fsstress \
>> +	-p 100 -f sync=0  -n 9999999 > /dev/null 2>&1 &
>> +
>> +    echo Running ENOSPC hitters. | tee -a $seq.full
>> +    for ((i = 0; i < $nr_iterations; i++))
>> +    do
>> +	#Open with O_TRUNC and then write until error
>> +	#hit ENOSPC each time.
>> +	dd if=/dev/zero of=$SCRATCH_MNT/BIG_FILE bs=1M 2> /dev/null
>> +    done
>> +
>> +    for ((i = 0; i < $kill_tries; i++))
>> +    do
>> +	killall -r -q -TERM fsstress 2> /dev/null
>> +	sleep 1
>> +    done
>> +}
>> +
>> +trap "_cleanup ; exit \$status" 0 1 2 3 15
>> +
>> +# get standard environment, filters and checks
>> +. ./common.rc
>> +. ./common.filter
>> +
>> +# real QA test starts here
>> +_supported_fs generic
>> +_supported_os Linux
>> +_require_scratch
>> +
>> +rm -f $seq.full
>> +
>> +umount $TEST_DEV >/dev/null 2>&1
>> +umount $SCRATCH_DEV >/dev/null 2>&1
>> +echo "*** MKFS ***"                         >>$seq.full
>> +echo ""                                     >>$seq.full
>> +_scratch_mkfs    >/dev/null 2>&1 || _fail "mkfs failed"
>> +_scratch_mount   >/dev/null 2>&1 || _fail "mount failed"
>> +
>> +workout
>> +umount $SCRATCH_MNT
>> +echo 
>> +echo Checking filesystem
>> +_check_scratch_fs
>> +status=$?
>> +exit
>> diff --git a/227.out b/227.out
>> new file mode 100644
>> index 0000000..6a7342d
>> --- /dev/null
>> +++ b/227.out
>> @@ -0,0 +1,5 @@
>> +QA output created by 227
>> +Running fsstress.
>> +Running ENOSPC hitters.
>> +
>> +Checking filesystem
>> diff --git a/group b/group
>> index 8d4a83a..81a2aa4 100644
>> --- a/group
>> +++ b/group
>> @@ -339,3 +339,4 @@ deprecated
>>  223 auto quick
>>  224 auto
>>  225 auto quick
>> +227 rw auto prealloc enospc
>
> Is this prealloc just because fsstress may run resvsp?
> FWIW, other fsstress tests aren't in that group, so this is
> as little inconsistent.
Ohh. i've miss that.
BTW i've got another more bug (NULL pointer deference)
I'm able to reproduce the bug only on host with 8core HT.
see attachment for more info
Seems that it triggered a code which was never triggered before
fs/ext4/extent.c
3477:   if (unlikely(EXT4_I(inode)->i_flags & EXT4_EOFBLOCKS_FL)) {
                if (unlikely(!eh->eh_entries)) {
                        EXT4_ERROR_INODE(inode,
                                         "eh->eh_entries == 0 ee_block
                                         %d",
                                         ex->ee_block);
########   OOPS here because ex == NULL. ^^^^^^^^^^^^^^          
                        err = -EIO;
                        goto out2;
                }
Continue digging...


[-- Attachment #2: oops-1.tag.gz --]
[-- Type: application/octet-stream, Size: 124244 bytes --]

[-- Attachment #3: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: ext34_free_inode's mess

[Dave Chinner]

On Wed, Apr 14, 2010 at 11:01:16AM -0500, Eric Sandeen wrote:
> Dmitry Monakhov wrote:
> > I've finally automated my favorite testcase (see attachment), 
> > before i've run it by hand.
> 
> Thanks!  Feel free to cc: the xfs list since the patch hits
> xfstests.  (I added it here)
> 
> >  227     |  105 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >  227.out |    5 +++
> >  group   |    1 +
> >  3 files changed, 111 insertions(+), 0 deletions(-)
> >  create mode 100755 227
> >  create mode 100644 227.out
> > 
> > diff --git a/227 b/227
> > new file mode 100755
> > index 0000000..d2b0c7d
> > --- /dev/null
> > +++ b/227
> > @@ -0,0 +1,105 @@
> > +#! /bin/bash
> > +# FS QA Test No. 227
> > +#
> > +# Perform fsstress test with parallel dd
> > +# This proven to be a good stress test
> > +# * Continuous dd retult in ENOSPC condition but only for a limited periods
> > +#   of time.
> > +# * Fsstress test cover many code paths
> 
> just little editor nitpicks: 
> 
> +# Perform fsstress test with parallel dd
> +# This is proven to be a good stress test
> +# * Continuous dd results in ENOSPC condition but only for a limited period
> +#   of time.
> +# * Fsstress test covers many code paths

This is close to the same as test 083:

# Exercise filesystem full behaviour - run numerous fsstress
# processes in write mode on a small filesystem.  NB: delayed
# allocate flushing is quite deadlock prone at the filesystem
# full boundary due to the fact that we will retry allocation
# several times after flushing, before giving back ENOSPC.

That test is not really doing anything XFS specific,
so could easily be modified to run on generic filesystems...

> > +
> > +    #Timing parameters
> > +    nr_iterations=5
> > +    kill_tries=20
> > +    echo Running fsstress. | tee -a $seq.full
> > +
> > +####################################################
> 
> What is all this for?
> 
> FWIW other fsstress tests use an $FSSTRESS_AVOID variable,
> where you can set the things you want to avoid easily
> 
> > +##    -f unresvsp=0 -f allocsp=0 -f freesp=0 \
> > +##    -f setxattr=0 -f attr_remove=0 -f attr_set=0 \
> > +## 
> > +######################################################
> > +    mkdir -p $SCRATCH_MNT/fsstress
> > +    # It is reasonable to disable sync, otherwise most of tasks will simply
> > +    # stuck in that sync() call.
> > +    $FSSTRESS_PROG \
> > +	-d $SCRATCH_MNT/fsstress \
> > +	-p 100 -f sync=0  -n 9999999 > /dev/null 2>&1 &
> > +
> > +    echo Running ENOSPC hitters. | tee -a $seq.full
> > +    for ((i = 0; i < $nr_iterations; i++))
> > +    do
> > +	#Open with O_TRUNC and then write until error
> > +	#hit ENOSPC each time.
> > +	dd if=/dev/zero of=$SCRATCH_MNT/BIG_FILE bs=1M 2> /dev/null
> > +    done

OK, so on a 10GB scratch device, this is going to write 50GB of
data, which at 100MB/s is going to take roughly 10 minutes.
The test should use a limited size filesystems (mkfs_scratch_sized)
to limit the runtime...

FWIW, test 083 spends most of it's runtime at or near ENOSPC, so
once again I wonder if that is not a better test to be using...

> > +workout
> > +umount $SCRATCH_MNT
> > +echo 
> > +echo Checking filesystem
> > +_check_scratch_fs

You don't need to check the scratch fs in the test - that is done by
the test harness after the test completes.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs