ext2/3/4 online defrag

ext2/3/4 online defrag

[Eric]

[-- Attachment #1: Type: text/plain, Size: 724 bytes --]

Hi all,

This is my first post to the list. I want to add online defrag support
to ext2/3/4 just to scratch an itch.

I've searched the archives and googled around, and I've come to the
following conclusions:
*An offline defrag tool is available but dangerous.
*Some patches exist that provide preliminary support for online defrag
in the ext3 driver, but interest in improving/integrating these patches
is low.
*An implementation of an ext* filesystem driver can work with any
ext2/3/4 filesystem as long as it supports the necessary revision
(GOOD_OLD_REV or DYNAMIC_REV) and feature flags set in the filesystem.

Please let me know if any of my assumptions are incorrect.


Thanks,

Eric Anopolsky


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: ext2/3/4 online defrag

[Andreas Dilger]

On May 09, 2007  21:19 -0700, Eric wrote:
> This is my first post to the list. I want to add online defrag support
> to ext2/3/4 just to scratch an itch.
> 
> I've searched the archives and googled around, and I've come to the
> following conclusions:
> *An offline defrag tool is available but dangerous.

Yes, old and dangerous.

> *Some patches exist that provide preliminary support for online defrag
> in the ext3 driver, but interest in improving/integrating these patches
> is low.

I don't think that is true.  Takashi Sato is actively working on online
defrag, and patches are posted regularly to the linux-ext4 mailing list.

I'd recommend that any work being done be spent in improving those
patches instead of starting from scratch.

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.

Re: ext2/3/4 online defrag

[Takashi Sato]

Hi,

> I've searched the archives and googled around, and I've come to the
> following conclusions:
> *An offline defrag tool is available but dangerous.
> *Some patches exist that provide preliminary support for online defrag
> in the ext3 driver, but interest in improving/integrating these patches
> is low.
> *An implementation of an ext* filesystem driver can work with any
> ext2/3/4 filesystem as long as it supports the necessary revision
> (GOOD_OLD_REV or DYNAMIC_REV) and feature flags set in the filesystem.

I'm working on the development of online defrag for ext4.
The defrag can work effectively for fragmented huge files.
So I'm focussing on ext4 which can store many huge files and
developing the defrag for extent files on ext4.
Any comments are very welcome.

Takashi

Re: ext2/3/4 online defrag

[Eric]

[-- Attachment #1: Type: text/plain, Size: 616 bytes --]

On Thu, 2007-05-10 at 18:20 +0900, Takashi Sato wrote:
> > *Some patches exist that provide preliminary support for online defrag
> > in the ext3 driver, but interest in improving/integrating these patches
> > is low.
>
> I'm working on the development of online defrag for ext4.

This makes sense. I didn't notice that the HTTP list archive was sorting
my search results by relevance instead of date, so it appeared that the
last post about online defrag was made in January.

> Any comments are very welcome.

I think I have a lot of reading to do before I go making comments.


Thanks,

Eric


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: ext2/3/4 online defrag

[Jan Kara]

  Hello,

  Sorry for replying a bit late...

> This is my first post to the list. I want to add online defrag support
> to ext2/3/4 just to scratch an itch.
> 
> I've searched the archives and googled around, and I've come to the
> following conclusions:
> *An offline defrag tool is available but dangerous.
  For original e2defrag - yes. But me (and several other people
independently as I've learnt recently) have written some tools which
should result in something useful. If you're interested, you can join
prefetch-devel@googlegroups.com - it's led by one guy who is doing
defrag and stuff as his google summer of code project.

> *Some patches exist that provide preliminary support for online defrag
> in the ext3 driver, but interest in improving/integrating these patches
> is low.
  The code itself for ext3 is not a big deal. Main problem are kernel
interfaces and I hope to find some time soon to put together some
reasonable proposal for reasonably generic calls...

> *An implementation of an ext* filesystem driver can work with any
> ext2/3/4 filesystem as long as it supports the necessary revision
> (GOOD_OLD_REV or DYNAMIC_REV) and feature flags set in the filesystem.
  Not sure what you mean here...

								Honza

-- 
Jan Kara <jack@suse.cz>
SuSE CR Labs

Re: ext2/3/4 online defrag

[Eric]

[-- Attachment #1: Type: text/plain, Size: 1373 bytes --]

On Thu, 2007-05-17 at 18:11 +0200, Jan Kara wrote:
> But me (and several other people
> independently as I've learnt recently) have written some tools which
> should result in something useful. If you're interested, you can join
> prefetch-devel@googlegroups.com - it's led by one guy who is doing
> defrag and stuff as his google summer of code project.

Is this different from the ext4/extent-based defrag patch that's been
mentioned on this list?

> > *An implementation of an ext* filesystem driver can work with any
> > ext2/3/4 filesystem as long as it supports the necessary revision
> > (GOOD_OLD_REV or DYNAMIC_REV) and feature flags set in the filesystem.
>   Not sure what you mean here...

The "ext2 filesystem"/"ext3 filesystem"/"ext4 filesystem" terminology
was confusing to me when I first started reading about them. In my mind,
it implied that those three filesystems were more different than they
actually are.

I think it would be more accurate to say that they are all essentially
the same filesystem, and that any filesystem driver that can mount a
given filesystem can mount any other ext2/3/4 filesystem of the same
revision with the same feature flags set.

I was asking for confirmation of this assumption, but I've since found a
lot of really good documentation that has cleared up a lot of things.

Thanks,

Eric


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: ext2/3/4 online defrag

[Jan Kara]

> On Thu, 2007-05-17 at 18:11 +0200, Jan Kara wrote:
> > But me (and several other people
> > independently as I've learnt recently) have written some tools which
> > should result in something useful. If you're interested, you can join
> > prefetch-devel@googlegroups.com - it's led by one guy who is doing
> > defrag and stuff as his google summer of code project.
> 
> Is this different from the ext4/extent-based defrag patch that's been
> mentioned on this list?
  Yes, it is different. In particular, it's offline only tool so far...

> > > *An implementation of an ext* filesystem driver can work with any
> > > ext2/3/4 filesystem as long as it supports the necessary revision
> > > (GOOD_OLD_REV or DYNAMIC_REV) and feature flags set in the filesystem.
> >   Not sure what you mean here...
> 
> The "ext2 filesystem"/"ext3 filesystem"/"ext4 filesystem" terminology
> was confusing to me when I first started reading about them. In my mind,
> it implied that those three filesystems were more different than they
> actually are.
> 
> I think it would be more accurate to say that they are all essentially
> the same filesystem, and that any filesystem driver that can mount a
> given filesystem can mount any other ext2/3/4 filesystem of the same
> revision with the same feature flags set.
> 
> I was asking for confirmation of this assumption, but I've since found a
> lot of really good documentation that has cleared up a lot of things.
  Yes, basically it's just a question of a feature set. But for example
current online defrag from Takashi requires extents, which are not
available for ext2 or ext3.

								Honza

-- 
Jan Kara <jack@suse.cz>
SuSE CR Labs