* inline_data,metadata_csum a lot of bad csum for directories @ 2014-12-01 14:09 Dmitry Monakhov 2014-12-01 23:32 ` Andreas Dilger 0 siblings, 1 reply; 5+ messages in thread From: Dmitry Monakhov @ 2014-12-01 14:09 UTC (permalink / raw) To: ext4 development; +Cc: Theodore Ts'o [-- Attachment #1: Type: text/plain, Size: 848 bytes --] Ted can you please clarify what is status of inline_data patchset. I've enabled inline_data testing in my test environment I use xfstest-bld for this purpose, but most test generates a lot of incorrect inode's csum for directories(almost any test which makes/rm directories). What I'm doing wrong? Are where any important fix I'm missing? Please post your test matrix My config: e2fsprogs git commit: g210baa5 ext4.git commit: recent tytso/dev ca83cc75a16f67 ./kvm-xfstests -c inline generic/011 generic/013 generic/013 [13:51:44] [ 368.507266] EXT4-fs error (device vdd): ext4_iget:3918: inode #294: comm rm: checksum invalid [ 368.511098] EXT4-fs error (device vdd): ext4_iget:3918: inode #294: comm rm: checksum invalid [ 369.098863] EXT4-fs error (device vdd): ext4_iget:3918: inode #2108:comm rm: checksum invalid [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 472 bytes --] ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: inline_data,metadata_csum a lot of bad csum for directories 2014-12-01 14:09 inline_data,metadata_csum a lot of bad csum for directories Dmitry Monakhov @ 2014-12-01 23:32 ` Andreas Dilger 2014-12-02 3:58 ` Theodore Ts'o 0 siblings, 1 reply; 5+ messages in thread From: Andreas Dilger @ 2014-12-01 23:32 UTC (permalink / raw) To: Dmitry Monakhov; +Cc: ext4 development, Theodore Ts'o On Dec 1, 2014, at 7:09 AM, Dmitry Monakhov <dmonakhov@openvz.org> wrote: > > Ted can you please clarify what is status of inline_data patchset. > I've enabled inline_data testing in my test environment I suspect that since inline_data and metadata_csum were developed in isolation, they don't work well together yet. I believe that Darrick is working on fixing some issues in this area. > I use xfstest-bld for this purpose, but most test generates a lot of incorrect > inode's csum for directories(almost any test which makes/rm > directories). What I'm doing wrong? Are where any important fix I'm missing? > Please post your test matrix > > My config: > e2fsprogs git commit: g210baa5 > ext4.git commit: recent tytso/dev ca83cc75a16f67 > ./kvm-xfstests -c inline generic/011 generic/013 > generic/013 [13:51:44] > [ 368.507266] EXT4-fs error (device vdd): ext4_iget:3918: inode #294: comm rm: checksum invalid > [ 368.511098] EXT4-fs error (device vdd): ext4_iget:3918: inode #294: comm rm: checksum invalid > [ 369.098863] EXT4-fs error (device vdd): ext4_iget:3918: inode #2108:comm rm: checksum invalid Cheers, Andreas ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: inline_data,metadata_csum a lot of bad csum for directories 2014-12-01 23:32 ` Andreas Dilger @ 2014-12-02 3:58 ` Theodore Ts'o 2014-12-02 5:08 ` Darrick J. Wong 0 siblings, 1 reply; 5+ messages in thread From: Theodore Ts'o @ 2014-12-02 3:58 UTC (permalink / raw) To: Andreas Dilger; +Cc: Dmitry Monakhov, ext4 development On Mon, Dec 01, 2014 at 04:32:02PM -0700, Andreas Dilger wrote: > On Dec 1, 2014, at 7:09 AM, Dmitry Monakhov <dmonakhov@openvz.org> wrote: > > > > Ted can you please clarify what is status of inline_data patchset. > > I've enabled inline_data testing in my test environment > > I suspect that since inline_data and metadata_csum were developed > in isolation, they don't work well together yet. I believe that > Darrick is working on fixing some issues in this area. The standard xfstests-bld test configuration is with inline_data and metadata_csum in combination: export FS=ext4 export TEST_DEV=$SM_TST_DEV export TEST_DIR=$SM_TST_MNT export MKFS_OPTIONS="-q -I 256 -O inline_data,metadata_csum,64bit" export EXT_MOUNT_OPTIONS="-o block_validity" TESTNAME="Ext4 4k block w/inline" Are you using the very latest version of e2fsprogs.git#next branch for your tests? If not, that could be why you are getting some test failures. It is absolutely required for any tests involving metdata_csum. I have a xfstests-build 32-bit rootfs.img I can upload for your convenience if that's what you are using. Getting a formally upgraded xfstests-bld rootfs.img with the latest xfstests and some other bug fixes and the latest e2fsprogs was on my todo list; sorry, the metadata csum v3 patches that went in in 3.18-rc3 required an updated e2fsprogs that's not in the released rootfs.img. > > Please post your test matrix > > > > My config: > > e2fsprogs git commit: g210baa5 OK, that should be sufficiently up to date. As far as my test matrix, please see: http://git.kernel.org/cgit/fs/ext2/xfstests-bld.git/tree/kvm-xfstests/test-appliance/files/root/conf with these test versions: e2fsprogs v1.42.12-415-gf0a7814 (Sun, 23 Nov 2014 20:32:08 -0500) fio fio-2.1.13-77-g0f9940a (Wed, 15 Oct 2014 14:16:35 -0600) quota f215e3f (Mon, 2 Dec 2013 23:06:23 -0300) xfsprogs v3.2.1-6-gba24eb7 (Fri, 18 Jul 2014 10:47:11 +1000) xfstests-bld 2a20a9d (Fri, 31 Oct 2014 10:08:53 -0400) xfstests linux-v3.8-523-g481c28f (Tue, 14 Oct 2014 22:59:39 +1100) > > ext4.git commit: recent tytso/dev ca83cc75a16f67 > > ./kvm-xfstests -c inline generic/011 generic/013 > > generic/013 [13:51:44] > > [ 368.507266] EXT4-fs error (device vdd): ext4_iget:3918: inode #294: comm rm: checksum invalid > > [ 368.511098] EXT4-fs error (device vdd): ext4_iget:3918: inode #294: comm rm: checksum invalid > > [ 369.098863] EXT4-fs error (device vdd): ext4_iget:3918: inode #2108:comm rm: checksum invalid Hmm, works for me: ./kvm-xfstests -c inline generic/011 generic/013 BEGIN TEST: Ext4 4k block w/inline Mon Dec 1 22:56:21 EST 2014 Device: /dev/vdd mk2fs options: -q -I 256 -O inline_data,metadata_csum,64bit mount options: -o block_validity FSTYP -- ext4 PLATFORM -- Linux/i686 kvm-xfstests 3.18.0-rc3-00530-g32f3869 MKFS_OPTIONS -- -q -I 256 -O inline_data,metadata_csum,64bit /dev/vdc MOUNT_OPTIONS -- -o acl,user_xattr -o block_validity /dev/vdc /vdc generic/011 2s ... [22:56:24] [22:56:26] 2s generic/013 47s ... [22:56:26] [22:57:21] 55s Ran: generic/011 generic/013 Passed all 2 tests - Ted ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: inline_data,metadata_csum a lot of bad csum for directories 2014-12-02 3:58 ` Theodore Ts'o @ 2014-12-02 5:08 ` Darrick J. Wong 2014-12-05 0:10 ` Darrick J. Wong 0 siblings, 1 reply; 5+ messages in thread From: Darrick J. Wong @ 2014-12-02 5:08 UTC (permalink / raw) To: Theodore Ts'o; +Cc: Andreas Dilger, Dmitry Monakhov, ext4 development On Mon, Dec 01, 2014 at 10:58:13PM -0500, Theodore Ts'o wrote: > On Mon, Dec 01, 2014 at 04:32:02PM -0700, Andreas Dilger wrote: > > On Dec 1, 2014, at 7:09 AM, Dmitry Monakhov <dmonakhov@openvz.org> wrote: > > > > > > Ted can you please clarify what is status of inline_data patchset. > > > I've enabled inline_data testing in my test environment > > > > I suspect that since inline_data and metadata_csum were developed > > in isolation, they don't work well together yet. I believe that > > Darrick is working on fixing some issues in this area. Fixing metadata_csum bugs, yes. There could be plenty more in inline_data -- there are a couple of spots (setting extra_isize in particular) where I'm not entirely sure it's setting the inode checksum before writing the raw inode out to disk. On the plus side, from running generic/011 in a loop I think I've found some sort of strange bug where unlinked inodes fail csum validation, causing e2fsck to complain. A cheap solution is not to bother revalidating an inode if its link count is zero, but I want to know why the kernel isn't doing the right thing with the unlinked inodes. --D > > The standard xfstests-bld test configuration is with inline_data and > metadata_csum in combination: > > export FS=ext4 > export TEST_DEV=$SM_TST_DEV > export TEST_DIR=$SM_TST_MNT > export MKFS_OPTIONS="-q -I 256 -O inline_data,metadata_csum,64bit" > export EXT_MOUNT_OPTIONS="-o block_validity" > TESTNAME="Ext4 4k block w/inline" > > Are you using the very latest version of e2fsprogs.git#next branch for > your tests? If not, that could be why you are getting some test > failures. It is absolutely required for any tests involving > metdata_csum. > > I have a xfstests-build 32-bit rootfs.img I can upload for your > convenience if that's what you are using. Getting a formally upgraded > xfstests-bld rootfs.img with the latest xfstests and some other bug > fixes and the latest e2fsprogs was on my todo list; sorry, the > metadata csum v3 patches that went in in 3.18-rc3 required an updated > e2fsprogs that's not in the released rootfs.img. > > > > Please post your test matrix > > > > > > My config: > > > e2fsprogs git commit: g210baa5 > > OK, that should be sufficiently up to date. As far as my test matrix, > please see: > > http://git.kernel.org/cgit/fs/ext2/xfstests-bld.git/tree/kvm-xfstests/test-appliance/files/root/conf > > with these test versions: > > e2fsprogs v1.42.12-415-gf0a7814 (Sun, 23 Nov 2014 20:32:08 -0500) > fio fio-2.1.13-77-g0f9940a (Wed, 15 Oct 2014 14:16:35 -0600) > quota f215e3f (Mon, 2 Dec 2013 23:06:23 -0300) > xfsprogs v3.2.1-6-gba24eb7 (Fri, 18 Jul 2014 10:47:11 +1000) > xfstests-bld 2a20a9d (Fri, 31 Oct 2014 10:08:53 -0400) > xfstests linux-v3.8-523-g481c28f (Tue, 14 Oct 2014 22:59:39 +1100) > > > > ext4.git commit: recent tytso/dev ca83cc75a16f67 > > > ./kvm-xfstests -c inline generic/011 generic/013 > > > generic/013 [13:51:44] > > > [ 368.507266] EXT4-fs error (device vdd): ext4_iget:3918: inode #294: comm rm: checksum invalid > > > [ 368.511098] EXT4-fs error (device vdd): ext4_iget:3918: inode #294: comm rm: checksum invalid > > > [ 369.098863] EXT4-fs error (device vdd): ext4_iget:3918: inode #2108:comm rm: checksum invalid > > Hmm, works for me: > > ./kvm-xfstests -c inline generic/011 generic/013 > > BEGIN TEST: Ext4 4k block w/inline Mon Dec 1 22:56:21 EST 2014 > Device: /dev/vdd > mk2fs options: -q -I 256 -O inline_data,metadata_csum,64bit > mount options: -o block_validity > FSTYP -- ext4 > PLATFORM -- Linux/i686 kvm-xfstests 3.18.0-rc3-00530-g32f3869 > MKFS_OPTIONS -- -q -I 256 -O inline_data,metadata_csum,64bit /dev/vdc > MOUNT_OPTIONS -- -o acl,user_xattr -o block_validity /dev/vdc /vdc > > generic/011 2s ... [22:56:24] [22:56:26] 2s > generic/013 47s ... [22:56:26] [22:57:21] 55s > Ran: generic/011 generic/013 > Passed all 2 tests > > - Ted > -- > To unsubscribe from this list: send the line "unsubscribe linux-ext4" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: inline_data,metadata_csum a lot of bad csum for directories 2014-12-02 5:08 ` Darrick J. Wong @ 2014-12-05 0:10 ` Darrick J. Wong 0 siblings, 0 replies; 5+ messages in thread From: Darrick J. Wong @ 2014-12-05 0:10 UTC (permalink / raw) To: Theodore Ts'o; +Cc: Andreas Dilger, Dmitry Monakhov, ext4 development On Mon, Dec 01, 2014 at 09:08:32PM -0800, Darrick J. Wong wrote: > On Mon, Dec 01, 2014 at 10:58:13PM -0500, Theodore Ts'o wrote: > > On Mon, Dec 01, 2014 at 04:32:02PM -0700, Andreas Dilger wrote: > > > On Dec 1, 2014, at 7:09 AM, Dmitry Monakhov <dmonakhov@openvz.org> wrote: > > > > > > > > Ted can you please clarify what is status of inline_data patchset. > > > > I've enabled inline_data testing in my test environment > > > > > > I suspect that since inline_data and metadata_csum were developed > > > in isolation, they don't work well together yet. I believe that > > > Darrick is working on fixing some issues in this area. > > Fixing metadata_csum bugs, yes. There could be plenty more in inline_data -- > there are a couple of spots (setting extra_isize in particular) where I'm not > entirely sure it's setting the inode checksum before writing the raw inode out > to disk. > > On the plus side, from running generic/011 in a loop I think I've found some > sort of strange bug where unlinked inodes fail csum validation, causing e2fsck > to complain. A cheap solution is not to bother revalidating an inode if its > link count is zero, but I want to know why the kernel isn't doing the right > thing with the unlinked inodes. Ahah, I introduced a race condition with the crypto API rework in ext4_chksum. I failed to notice that _setkey() modifies s_chksum_driver without any sort of concurrency protection. Soo... I'll send a patch tonight that fixes it. Sorry about that, --D > > --D > > > > > The standard xfstests-bld test configuration is with inline_data and > > metadata_csum in combination: > > > > export FS=ext4 > > export TEST_DEV=$SM_TST_DEV > > export TEST_DIR=$SM_TST_MNT > > export MKFS_OPTIONS="-q -I 256 -O inline_data,metadata_csum,64bit" > > export EXT_MOUNT_OPTIONS="-o block_validity" > > TESTNAME="Ext4 4k block w/inline" > > > > Are you using the very latest version of e2fsprogs.git#next branch for > > your tests? If not, that could be why you are getting some test > > failures. It is absolutely required for any tests involving > > metdata_csum. > > > > I have a xfstests-build 32-bit rootfs.img I can upload for your > > convenience if that's what you are using. Getting a formally upgraded > > xfstests-bld rootfs.img with the latest xfstests and some other bug > > fixes and the latest e2fsprogs was on my todo list; sorry, the > > metadata csum v3 patches that went in in 3.18-rc3 required an updated > > e2fsprogs that's not in the released rootfs.img. > > > > > > Please post your test matrix > > > > > > > > My config: > > > > e2fsprogs git commit: g210baa5 > > > > OK, that should be sufficiently up to date. As far as my test matrix, > > please see: > > > > http://git.kernel.org/cgit/fs/ext2/xfstests-bld.git/tree/kvm-xfstests/test-appliance/files/root/conf > > > > with these test versions: > > > > e2fsprogs v1.42.12-415-gf0a7814 (Sun, 23 Nov 2014 20:32:08 -0500) > > fio fio-2.1.13-77-g0f9940a (Wed, 15 Oct 2014 14:16:35 -0600) > > quota f215e3f (Mon, 2 Dec 2013 23:06:23 -0300) > > xfsprogs v3.2.1-6-gba24eb7 (Fri, 18 Jul 2014 10:47:11 +1000) > > xfstests-bld 2a20a9d (Fri, 31 Oct 2014 10:08:53 -0400) > > xfstests linux-v3.8-523-g481c28f (Tue, 14 Oct 2014 22:59:39 +1100) > > > > > > ext4.git commit: recent tytso/dev ca83cc75a16f67 > > > > ./kvm-xfstests -c inline generic/011 generic/013 > > > > generic/013 [13:51:44] > > > > [ 368.507266] EXT4-fs error (device vdd): ext4_iget:3918: inode #294: comm rm: checksum invalid > > > > [ 368.511098] EXT4-fs error (device vdd): ext4_iget:3918: inode #294: comm rm: checksum invalid > > > > [ 369.098863] EXT4-fs error (device vdd): ext4_iget:3918: inode #2108:comm rm: checksum invalid > > > > Hmm, works for me: > > > > ./kvm-xfstests -c inline generic/011 generic/013 > > > > BEGIN TEST: Ext4 4k block w/inline Mon Dec 1 22:56:21 EST 2014 > > Device: /dev/vdd > > mk2fs options: -q -I 256 -O inline_data,metadata_csum,64bit > > mount options: -o block_validity > > FSTYP -- ext4 > > PLATFORM -- Linux/i686 kvm-xfstests 3.18.0-rc3-00530-g32f3869 > > MKFS_OPTIONS -- -q -I 256 -O inline_data,metadata_csum,64bit /dev/vdc > > MOUNT_OPTIONS -- -o acl,user_xattr -o block_validity /dev/vdc /vdc > > > > generic/011 2s ... [22:56:24] [22:56:26] 2s > > generic/013 47s ... [22:56:26] [22:57:21] 55s > > Ran: generic/011 generic/013 > > Passed all 2 tests > > > > - Ted > > -- > > To unsubscribe from this list: send the line "unsubscribe linux-ext4" in > > the body of a message to majordomo@vger.kernel.org > > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- > To unsubscribe from this list: send the line "unsubscribe linux-ext4" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2014-12-05 0:11 UTC | newest] Thread overview: 5+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2014-12-01 14:09 inline_data,metadata_csum a lot of bad csum for directories Dmitry Monakhov 2014-12-01 23:32 ` Andreas Dilger 2014-12-02 3:58 ` Theodore Ts'o 2014-12-02 5:08 ` Darrick J. Wong 2014-12-05 0:10 ` Darrick J. Wong
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).