Online ext4 defragmention

Online ext4 defragmention

[Alex Buell]

Can someone confirm if the online ext4 defragmentation ioctls will
be going into 2.6.29? Thanks, I'm planning a migration from a
disparate collection of filesystems to ext4 over the next few months. 

Thanks
-- 
http://www.munted.org.uk

Fearsome grindings.

Re: Online ext4 defragmention

[Theodore Tso]

On Sun, Jan 11, 2009 at 01:44:43PM +0000, Alex Buell wrote:
> Can someone confirm if the online ext4 defragmentation ioctls will
> be going into 2.6.29? Thanks, I'm planning a migration from a
> disparate collection of filesystems to ext4 over the next few months. 

Unfortunately, the defragmentation patches need to a lot of work (to
be honest, largely refactored and almost rewritten) before they are
ready to for mainline yes.  I am also concerned that the current
defrag patches also try too hard to keep blocks in the same block
group, even as a higher priority keeping them non-fragmented.

Also, note that some of the benefits of ext4 only show up if you do a
backup, mkfs, and restore; that's because there are layout changes
that can only take place if you reformat the filesystem.  Finally,
there are some allocation algorithm changes which didn't make the
2.6.29 merge window which I think will make a long-term difference.
So you'll probably want to use 2.6.29 with the ext4 patch set.

So if you want the best performance and fastest fsck times (which I
infer given your query about the defragmentation ioctls), you may want
to consider doing a reformat and restore operation as part of your
ext4 migration, at least for filesystem that you plan to use for
active use.  If the filesystem is just going to be an mp3 archive, for
example, it might not be worth it to do the backup/reformat/restore
path.

Regards,

						- Ted

Re: Online ext4 defragmention

[Mike Snitzer]

On Sun, Jan 11, 2009 at 3:28 PM, Theodore Tso <tytso@mit.edu> wrote:
> On Sun, Jan 11, 2009 at 01:44:43PM +0000, Alex Buell wrote:
>> Can someone confirm if the online ext4 defragmentation ioctls will
>> be going into 2.6.29? Thanks, I'm planning a migration from a
>> disparate collection of filesystems to ext4 over the next few months.
>
> Unfortunately, the defragmentation patches need to a lot of work (to
> be honest, largely refactored and almost rewritten) before they are
> ready to for mainline yes.  I am also concerned that the current
> defrag patches also try too hard to keep blocks in the same block
> group, even as a higher priority keeping them non-fragmented.
>
> Also, note that some of the benefits of ext4 only show up if you do a
> backup, mkfs, and restore; that's because there are layout changes
> that can only take place if you reformat the filesystem.  Finally,
> there are some allocation algorithm changes which didn't make the
> 2.6.29 merge window which I think will make a long-term difference.
> So you'll probably want to use 2.6.29 with the ext4 patch set.

Hi Ted,

Could you elaborate on which allocation patches you're referring to in
the ext4 patch queue (2.6.28-ext4-3?) ?

Thanks,
Mike

Re: Online ext4 defragmention

[Theodore Tso]

On Mon, Jan 12, 2009 at 08:39:33AM -0500, Mike Snitzer wrote:
> 
> Hi Ted,
> 
> Could you elaborate on which allocation patches you're referring to in
> the ext4 patch queue (2.6.28-ext4-3?) ?

They haven't been implemented yet.  They've been discussed on the ext4
mailing list, however.  Basically, the Orlov algorithm was disabled
for flex_bg, and I want to optimize how directory blocks are allocated
to hopefully speed up fsck times a tad bit more.  The design has been
roughly worked out; but it's a matter of me finding time to code and
test it...

						- Ted

Re: Online ext4 defragmention

[Mike Snitzer]

On 1/12/09, Theodore Tso <tytso@mit.edu> wrote:
> On Mon, Jan 12, 2009 at 08:39:33AM -0500, Mike Snitzer wrote:
>  >
>  > Hi Ted,
>  >
>  > Could you elaborate on which allocation patches you're referring to in
>  > the ext4 patch queue (2.6.28-ext4-3?) ?
>
>
> They haven't been implemented yet.  They've been discussed on the ext4
>  mailing list, however.  Basically, the Orlov algorithm was disabled
>  for flex_bg, and I want to optimize how directory blocks are allocated
>  to hopefully speed up fsck times a tad bit more.  The design has been
>  roughly worked out; but it's a matter of me finding time to code and
>  test it...

Ah, that ext4 thread is here:
http://kerneltrap.org/mailarchive/linux-ext4/2008/12/12/4403324

Thanks for clarifying,
Mike