Strange behavior when attempting to convert a file to extents using chattr

Strange behavior when attempting to convert a file to extents using chattr

[Peter Uchno]

Hello,
I recently converted an ext3 filesystem to ext4 in the hopes of
performance increases. However, I ran into a snag:

In theory, chattr can be used to convert a file to extents using
"chattr +e foo". However, on my system (Archlinux, kernel 2.6.37.4),
the behavior of this command is unexpected. Given a file baz created
when the filesystem was still ext3 (and thus not using extents),
"chattr +e baz" returns zero but "lsattr baz" reveals the file is not
using extents. Run "chattr +e baz" a second time, the same thing
happens. A third time, however, it comes back with:
    "chattr: Input/output error while setting flags on baz"
and an error in the kernel log:
    attempt to access beyond end of device
    loop1: rw=0, want=204802, limit=204800
The file is still not using extents and remounting the filesystem
doesn't seem to change much. Running e2fsck on the filesystem after
unmounting it comes back with some errors. Letting it fix them and
then mounting the FS up again shows that the file is now using
extents.
I attempted this on an Arch system with both kernels 2.6.37.4 and
2.6.38.4 and was able to recreate it by creating a small loopback
filesystem using dd, making a few files on it, converting the FS to
ext4 using "tune2fs -O extents,uninit_bg,dir_index /tmp/fs.ext; e2fsck
-fDC0 /tmp/fs.ext", and then mounting it again as ext4 and attempting
the conversion to extents with chattr. However, on a debian system
with kernel 2.6.32.5, it works as intended (chattr +e bar; lsattr bar
reveals the file converted to extents). I ran strace on the chattr
command on the Arch system and came up with the following:

http://pastebin.com/EpwhLF3t
for the first attempt at running the command;
http://pastebin.com/qB7DiDgF
for the second try; and
http://pastebin.com/jjAdPtBr
for the final attempt.

After doing so, running e2fsck on the filesystem revealed the following:

http://pastebin.com/83q60Wik

And after mounting the filesystem again, the file is now using extents.

My kernel config, if it is of interest, is at:
http://pastebin.com/uk1Hs53P

I'm using e2fsprogs 1.41.14. The Debian system (where it works
normally) has e2fsprogs 1.41.12.

I'm not sure what would cause the filesystem to behave in this
fashion. Does anyone have any insights?
Regards,
Peter

Re: Strange behavior when attempting to convert a file to extents using chattr

[Eric Sandeen]

On 4/28/11 3:14 PM, Peter Uchno wrote:
> Hello,
> I recently converted an ext3 filesystem to ext4 in the hopes of
> performance increases. However, I ran into a snag:
> 
> In theory, chattr can be used to convert a file to extents using
> "chattr +e foo". However, on my system (Archlinux, kernel 2.6.37.4),
> the behavior of this command is unexpected. Given a file baz created
> when the filesystem was still ext3 (and thus not using extents),
> "chattr +e baz" returns zero but "lsattr baz" reveals the file is not
> using extents. Run "chattr +e baz" a second time, the same thing
> happens. A third time, however, it comes back with:
>     "chattr: Input/output error while setting flags on baz"
> and an error in the kernel log:
>     attempt to access beyond end of device
>     loop1: rw=0, want=204802, limit=204800
> The file is still not using extents and remounting the filesystem
> doesn't seem to change much. Running e2fsck on the filesystem after
> unmounting it comes back with some errors. Letting it fix them and
> then mounting the FS up again shows that the file is now using
> extents.
> I attempted this on an Arch system with both kernels 2.6.37.4 and
> 2.6.38.4 and was able to recreate it by creating a small loopback
> filesystem using dd, making a few files on it, converting the FS to
> ext4 using "tune2fs -O extents,uninit_bg,dir_index /tmp/fs.ext; e2fsck
> -fDC0 /tmp/fs.ext", and then mounting it again as ext4 and attempting
> the conversion to extents with chattr. However, on a debian system
> with kernel 2.6.32.5, it works as intended (chattr +e bar; lsattr bar
> reveals the file converted to extents). I ran strace on the chattr
> command on the Arch system and came up with the following:
> 
> http://pastebin.com/EpwhLF3t
> for the first attempt at running the command;
> http://pastebin.com/qB7DiDgF
> for the second try; and
> http://pastebin.com/jjAdPtBr
> for the final attempt.
> 
> After doing so, running e2fsck on the filesystem revealed the following:
> 
> http://pastebin.com/83q60Wik
> 
> And after mounting the filesystem again, the file is now using extents.
> 
> My kernel config, if it is of interest, is at:
> http://pastebin.com/uk1Hs53P
> 
> I'm using e2fsprogs 1.41.14. The Debian system (where it works
> normally) has e2fsprogs 1.41.12.
> 
> I'm not sure what would cause the filesystem to behave in this
> fashion. Does anyone have any insights?

I can't speak directly to the problem because I haven't investigated, but I can tell you that chattr +e foo is probably almost never tested.

If any other ext4 dev does test it, speak up. :)

And I would strongly suggest that if you want an ext4 filesystem, mkfs.ext4 and migratind data to it is the best route to a consistent, featureful, well-tested setup.

-Eric

> Regards,
> Peter

Re: Strange behavior when attempting to convert a file to extents using chattr

[Ted Ts'o]

On Thu, Apr 28, 2011 at 04:14:59PM -0400, Peter Uchno wrote:
> I recently converted an ext3 filesystem to ext4 in the hopes of
> performance increases. However, I ran into a snag:
> 
> In theory, chattr can be used to convert a file to extents using
> "chattr +e foo". However, on my system (Archlinux, kernel 2.6.37.4),
> the behavior of this command is unexpected. Given a file baz created
> when the filesystem was still ext3 (and thus not using extents),
> "chattr +e baz" returns zero but "lsattr baz" reveals the file is not
> using extents. 

I found the problem; we were failing to set the extents flag at the
conclusion of the migration run.  The following patch should fix
things.  Thanks for reporting the problem, and my apologies for not
catching this further.  (/me makes a mental note to create an xfstests
test for this case.)

						- Ted

>From 74e4e6db38918620bdf8a46e94982ead2b228d43 Mon Sep 17 00:00:00 2001
From: Theodore Ts'o <tytso@mit.edu>
Date: Tue, 3 May 2011 09:34:42 -0400
Subject: [PATCH] ext4: set extents flag when migrating file to use extents

Fix a typo that was introduced in commit 07a038245b (in 2.6.36) which
caused the extents flag not to be set at the conclusion of converting
an inode to use extents.

Reported-by: Peter Uchno <peter.uchno@gmail.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
---
 fs/ext4/migrate.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/fs/ext4/migrate.c b/fs/ext4/migrate.c
index 92816b4..b57b98f 100644
--- a/fs/ext4/migrate.c
+++ b/fs/ext4/migrate.c
@@ -376,7 +376,7 @@ static int ext4_ext_swap_inode_data(handle_t *handle, struct inode *inode,
 	 * We have the extent map build with the tmp inode.
 	 * Now copy the i_data across
 	 */
-	ext4_test_inode_flag(inode, EXT4_INODE_EXTENTS);
+	ext4_set_inode_flag(inode, EXT4_INODE_EXTENTS);
 	memcpy(ei->i_data, tmp_ei->i_data, sizeof(ei->i_data));
 
 	/*
-- 
1.7.3.1