Re: [LTP] [PATCH] Add new testcases for ext4 new features - V3

[Subrata Modak <subrata@linux.vnet.ibm.com>]

Miao,

On Tue, 2009-10-27 at 19:05 +0530, Subrata Modak wrote: 
> Hi Andreas,
> 
> Thanks for taking a look and clarifying the issues.
> 
> On Mon, 2009-10-26 at 16:15 -0600, Andreas Dilger wrote:
> > On 2009-10-26, at 07:26, Subrata Modak wrote:
> > > We had recently added EXT4 testcases to LTP. As the author of these
> > > tests mentioned, we saw 2 such failures in subdirectory limit test:
> > >
> > > Ext4 subdir limit test
> > > ext4-subdir-limit    0  TINFO  :  Num of dirs to create: 65537, Dir  
> > > name
> > > len: short name                        Parent dir: mnt_point, Block
> > > size: 1024
> > > mke2fs 1.41.9 (22-Aug-2009)
> > > tune2fs 1.41.9 (22-Aug-2009)
> > > Setting test filesystem flag
> > > /dev/VG1_EXT4/LV1_EXT4: The test_fs flag is set (and ext4 is  
> > > available).
> > > CLEARED.
> > 
> > You don't need to set the test_fs flag for modern ext4.
> 
> Miao,
> 
> Can we change this ?
> 
> > 
> > > /dev/VG1_EXT4/LV1_EXT4: clean, 10/8847360 files, 2297019/141557760
> > > blocks
> > > dumpe2fs 1.41.9 (22-Aug-2009)
> > > ext4-subdir-limit    1  TPASS  :  ext4 subdir limit test pass
> > > ext4-subdir-limit    0  TINFO  :  Num of dirs to create: 65537, Dir  
> > > name
> > > len: long name                         Parent dir: mnt_point, Block
> > > size: 1024
> > > mkdir: No space left on device
> > > ext4-subdir-limit    2  TFAIL  :  failed to create directories - 19524
> > > ext4-subdir-limit    0  TINFO  :  Num of dirs to create: 65537, Dir  
> > > name
> > > len: short name                        Parent dir: mnt_point/sub,  
> > > Block
> > > size: 1024
> > > mke2fs 1.41.9 (22-Aug-2009)
> > > tune2fs 1.41.9 (22-Aug-2009)
> > > Setting test filesystem flag
> > > /dev/VG1_EXT4/LV1_EXT4: The test_fs flag is set (and ext4 is  
> > > available).
> > > CLEARED.
> > > /dev/VG1_EXT4/LV1_EXT4: clean, 12/8847360 files, 2297037/141557760
> > > blocks
> > > dumpe2fs 1.41.9 (22-Aug-2009)
> > > ext4-subdir-limit    3  TPASS  :  ext4 subdir limit test pass
> > > ext4-subdir-limit    0  TINFO  :  Num of dirs to create: 65537, Dir  
> > > name
> > > len: long name                         Parent dir: mnt_point/sub,  
> > > Block
> > > size: 1024
> > > mkdir: No space left on device
> > > ext4-subdir-limit    4  TFAIL  :  failed to create directories - 19483
> > > ext4-subdir-limit    0  TINFO  :  Num of dirs to create: 65537, Dir  
> > > name
> > > len: short name                        Parent dir: mnt_point, Block
> > > size: 2048
> > >
> > > Is this a known issue for EXT4 ?
> > 
> > Creating many long filenames (255 bytes) in a 1kB block filesystem is a
> > known issue that is not expected to be fixed (at least I'm not aware of
> > any work being done in this area).  A vast majority of filesystems
> > today are formatted with 4kB blocks (due to disk size) and the  
> > overhead of
> > managing 1kB block filesystems makes these only marginally used, and it
> > is also very rare to have applications with such large filenames.
> 
> Ok.
> 
> > 
> > > We also uncovered that the tests gives a decent 50% lines and over 61%
> > > function coverage for EXT4 code. Please find the individual coverage
> > > under EXT4 attached.
> > 
> > Interesting to see this.
> 
> And the good news continues. I did a complete run of the tests this
> time. Unlike last time where it was running for more than 48 hours owing
> to huge FS size(680GB), i ran it on a small 135GB partition this time.
> It took around 40 hours completing the entire run which also include the
> online-defrag tests. Guess, the new Code Coverage results are:
> 
> Lines:		56.1%,
> Functions:	66.4%,
> 
> However, it is also accompanied by the following failures in the online-defrag part:
> ========================================================================================
> ext4-online-defrag    0  TINFO  :  defrag type: 1, defrag obj: 3, block size: 1024
> mke2fs 1.41.9 (22-Aug-2009)
> tune2fs 1.41.9 (22-Aug-2009)
> Setting test filesystem flag
> dumpe2fs 1.41.9 (22-Aug-2009)
> dumpe2fs 1.41.9 (22-Aug-2009)
> 69630819+0 records in
> 69630819+0 records out
> 71301958656 bytes (71 GB) copied, 2694.12 s, 26.5 MB/s
> ext4-online-defrag    3  TFAIL  :  e4defrag returned failure
> ext4-online-defrag    0  TINFO  :  defrag type: 2, defrag obj: 1, block size: 1024
> mke2fs 1.41.9 (22-Aug-2009)
> tune2fs 1.41.9 (22-Aug-2009)
> ...
> ext4-online-defrag    0  TINFO  :  defrag type: 2, defrag obj: 3, block size: 1024
> mke2fs 1.41.9 (22-Aug-2009)
> tune2fs 1.41.9 (22-Aug-2009)
> Setting test filesystem flag
> dumpe2fs 1.41.9 (22-Aug-2009)
> dumpe2fs 1.41.9 (22-Aug-2009)
> 69630819+0 records in
> 69630819+0 records out
> 71301958656 bytes (71 GB) copied, 2623.31 s, 27.2 MB/s
> ext4-online-defrag    6  TFAIL  :  e4defrag returned failure
> ext4-online-defrag    0  TINFO  :  defrag type: 3, defrag obj: 1, block size: 1024
> mke2fs 1.41.9 (22-Aug-2009)
> tune2fs 1.41.9 (22-Aug-2009)
> Setting test filesystem flag
> File is not regular file
>  "/dev/VG1_EXT4/LV1_EXT4"
> ext4-online-defrag    7  TFAIL  :  e4defrag returned failure
> ext4-online-defrag    0  TINFO  :  defrag type: 3, defrag obj: 2, block size: 1024
> mke2fs 1.41.9 (22-Aug-2009)
> tune2fs 1.41.9 (22-Aug-2009)
> Setting test filesystem flag
> 40+0 records in
> 40+0 records out
> 41943040 bytes (42 MB) copied, 0.737014 s, 56.9 MB/s
> write: Success
> Filesystem           1K-blocks      Used Available Use% Mounted on
> /dev/sda1             25798712  10581072  13907140  44% /
> udev                   1028576       152   1028424   1% /dev
> /dev/sda2              4127108    126572   3790888   4% /boot
> /dev/mapper/VG_EXT3_TO_EXT4_MIGRATION_TEST1-LV_EXT3_TO_EXT4_MIGRATION_TEST2
>                       10321208    154100   9642820   2% /MY_EXT4
> /dev/mapper/VG_EXT3_TO_EXT4_MIGRATION_TEST1-LV_EXT3_TO_EXT4_MIGRATION_TEST1
>                       10321208    154232   9642688   2% /MY_EXT3
> /dev/mapper/VG1_EXT4-LV1_EXT4
>                      139299998 139259037     40961 100% /root/subrata/ltp/ltp-full-20090930/testcases/bin/mnt_point
> File is not regular file
>  "/dev/VG1_EXT4/LV1_EXT4"
> ext4-online-defrag    8  TFAIL  :  e4defrag returned failure
> ext4-online-defrag    0  TINFO  :  defrag type: 3, defrag obj: 3, block size: 1024
> mke2fs 1.41.9 (22-Aug-2009)
> tune2fs 1.41.9 (22-Aug-2009)
> Setting test filesystem flag
> dumpe2fs 1.41.9 (22-Aug-2009)
> dumpe2fs 1.41.9 (22-Aug-2009)
> 69630819+0 records in
> 69630819+0 records out
> 71301958656 bytes (71 GB) copied, 2544.39 s, 28.0 MB/s
> File is not regular file
>  "/dev/VG1_EXT4/LV1_EXT4"
> ext4-online-defrag    9  TFAIL  :  e4defrag returned failure
> ...
> ext4-online-defrag    0  TINFO  :  defrag type: 3, defrag obj: 2, block size: 4096
> mke2fs 1.41.9 (22-Aug-2009)
> tune2fs 1.41.9 (22-Aug-2009)
> Setting test filesystem flag
> 40+0 records in
> 40+0 records out
> 41943040 bytes (42 MB) copied, 0.306366 s, 137 MB/s
> write: Success
> write: Success
> write: Success
> write: Success
> write: Success
> Filesystem           1K-blocks      Used Available Use% Mounted on
> /dev/sda1             25798712  11441524  13046688  47% /
> udev                   1028576       152   1028424   1% /dev
> /dev/sda2              4127108    126572   3790888   4% /boot
> /dev/mapper/VG_EXT3_TO_EXT4_MIGRATION_TEST1-LV_EXT3_TO_EXT4_MIGRATION_TEST2
>                       10321208    154100   9642820   2% /MY_EXT4
> /dev/mapper/VG_EXT3_TO_EXT4_MIGRATION_TEST1-LV_EXT3_TO_EXT4_MIGRATION_TEST1
>                       10321208    154232   9642688   2% /MY_EXT3
> /dev/mapper/VG1_EXT4-LV1_EXT4
>                      139336680 139295720     40960 100% /root/subrata/ltp/ltp-full-20090930/testcases/bin/mnt_point
> File is not regular file
>  "/dev/VG1_EXT4/LV1_EXT4"
> ext4-online-defrag   17  TFAIL  :  e4defrag returned failure
> ext4-online-defrag    0  TINFO  :  defrag type: 3, defrag obj: 3, block size: 4096
> mke2fs 1.41.9 (22-Aug-2009)
> tune2fs 1.41.9 (22-Aug-2009)
> Setting test filesystem flag
> dumpe2fs 1.41.9 (22-Aug-2009)
> dumpe2fs 1.41.9 (22-Aug-2009)
> 17392985+0 records in
> 17392985+0 records out
> 71241666560 bytes (71 GB) copied, 1019.79 s, 69.9 MB/s
> File is not regular file
>  "/dev/VG1_EXT4/LV1_EXT4"
> ext4-online-defrag   18  TFAIL  :  e4defrag returned failure
> incrementing stop
> ========================================================================================
> 
> Miao,
> 
> Did you see these failure on your machine ?

Your take on this ?

> Some more points are:
> 
>      1. Can you please run these tests against the backdrop of the
>         Makefile changes in LTP to see all the build/install/run are
>         executing fine,

Could you please verify this before this release. Let me know a patch if
you find any discrepancy :-)

> 2. Can you also please put all the utilities version info at the
>         beginning of the test rather than printing them again and
>         again(like mke2fs 1.41.9 (22-Aug-2009),tune2fs 1.41.9
>         (22-Aug-2009),dumpe2fs 1.41.9 (22-Aug-2009),dumpe2fs 1.41.9
>         (22-Aug-2009))

This can go in the next month release as well, if you do not have time
to fix now. Else, i have a bug mouth to eat this as well ;-)

Regards--
Subrata

> 
> Regards--
> Subrata
> 
> > 
> > > Following the various system information on which we ran the test:
> > > 1) uname -a
> > > Linux 2.6.31 #1 SMP Tue Oct 20 22:57:16 IST 2009 i686 i686 i386
> > > GNU/Linux
> > >
> > > 2) grep -i ext4 .config
> > > CONFIG_EXT4_FS=m
> > > CONFIG_EXT4DEV_COMPAT=y
> > > CONFIG_EXT4_FS_XATTR=y
> > > CONFIG_EXT4_FS_POSIX_ACL=y
> > > CONFIG_EXT4_FS_SECURITY=y
> > >
> > > 3) mkfs.ext4 -V
> > > mke2fs 1.41.9 (22-Aug-2009)
> > >        Using EXT2FS Library version 1.41.9
> > > e2fsprogs-1.41.9
> > > e2fsprogs-libs-1.41.9
> > >
> > > 4) LTP used:
> > > Unreleased CVS repository. See the URL below to download:
> > > http://ltp.sourceforge.net/documentation/how-to/ltp.php#_3.1,
> > >
> > > You can also also directly review the attached patch(which added this
> > > test cases):
> > > 26_10_2009-(Miao-Xie<miaox@cn.fujitsu.com>)- 
> > > Add_new_testcases_for_ext4_new_features_V3.patch.regenerated,
> > >
> > > 5) I used a single 680GB LVM partition to test this.
> > >
> > > Kindly let us know your thoughts on this failures.
> > 
> > Cheers, Andreas
> > --
> > Andreas Dilger
> > Sr. Staff Engineer, Lustre Group
> > Sun Microsystems of Canada, Inc.
> > 
> ------------------------------------------------------------------------------
> Come build with us! The BlackBerry(R) Developer Conference in SF, CA
> is the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay 
> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
> http://p.sf.net/sfu/devconference
> _______________________________________________ Ltp-list mailing list Ltp-list@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ltp-list


------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay 
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Ltp-list mailing list
Ltp-list@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ltp-list