Status of ext4 online defrag?

Status of ext4 online defrag?

[Clemens Eisserer]

Hello,

Does anybody know what the current state of ext4 online defrag is?

Although ext4 does an excellent job in avoiding fragmentation,
sometimes it still happens, e.g. with firefox's sqlite database
which made using firefox horrible because disk-io took ages (and took
also some time until I found out whats the problem):

> [ce@localhost ~]$ filefrag /fc8/home/ce/.mozilla/firefox/default/places.sqlite
> /fc8/home/ce/.mozilla/firefox/default/places.sqlite: 1730 extents found, perfection would be 1 extent

Thats 1730 extents for a 75mb file :-/
Sure I can copy it between my partitions, but I would really prefer a
defrag tool ;)

Thank you in advance, Clemens

Re: Status of ext4 online defrag?

[Diego Calleja]

On Viernes 24 Julio 2009 15:36:53 Clemens Eisserer escribió:
> Hello,
> 
> Does anybody know what the current state of ext4 online defrag is?

It got merged for 2.6.31 (commit 748de6736c1e482e111f9d1b5a5d5b1787600cad)

Re: Status of ext4 online defrag?

[Clemens Eisserer]

>> Does anybody know what the current state of ext4 online defrag is?
>
> It got merged for 2.6.31 (commit 748de6736c1e482e111f9d1b5a5d5b1787600cad)

Thanks, how cool!
I feared already that this feature is dead, glad to hear it is not :)

When compiling e2fsprogs from git, I get:
> e4defrag.c:388: error: conflicting types for ‘fallocate’
> /usr/include/bits/fcntl.h:240: note: previous declaration of ‘fallocate’ was here
> make: *** [e4defrag.o] Error 1

I had to comment out fallocate in fcntl.h, no idea who is wrong here.

Thanks again, Clemens

Re: Status of ext4 online defrag?

[Theodore Tso]

On Fri, Jul 24, 2009 at 11:29:40AM -0400, Clemens Eisserer wrote:
> >> Does anybody know what the current state of ext4 online defrag is?
> >
> > It got merged for 2.6.31 (commit 748de6736c1e482e111f9d1b5a5d5b1787600cad)
> 
> Thanks, how cool!
> I feared already that this feature is dead, glad to hear it is not :)

It's not dead, but it's not fully complete.  There are more patches
that introduce a half-dozen or so new ioctl's that still need review,
and the e4defrag tool is still quite primitive and needs to (a) use
the new ioctls, and (b) be made more intelligent about defragging free
space.

At the moment what e4defrag will do is to ask the filesystem to
allocate new space, and if the new space contains fewer fragments that
the current file, use an ioctl to atomically copy the data to the new
blocks and swap the blocks so that the file being defragged uses the
newly allocated blocks.  If the filesystem's free space is not badly
fragmented, this will tend to defrag individual files, but e4defrag
doesn't currently have any kind of global knowledge of where files
live and any global knowledge of the free space and how to move blocks
around so that the free space will get defragmented.

The new ioctls are designed to give the userspace program the tools to
do this, but (a) we haven't had the time to review the ioctls and new
code so they haven't been merged yet, and (b) the userspace program
hasn't been taught to use these new ioctls.

> When compiling e2fsprogs from git, I get:
> > e4defrag.c:388: error: conflicting types for ‘fallocate’
> > /usr/include/bits/fcntl.h:240: note: previous declaration of ‘fallocate’ was here
> > make: *** [e4defrag.o] Error 1
> 
> I had to comment out fallocate in fcntl.h, no idea who is wrong here.

Eric Sandeen has patches that he sent out in the last two weeks to fix
e4defrag so it will compile on glibc's that define fallocate.  I just
haven't had a chance to get them applied into mainline.  Things like
fixing bugs where off-line resize2fs could corrupt ext4 filesystems
were higher priority.  :-(

The bottom line is that online defrag is something that is happening,
it's just not happening as fast as we would all like, mainly due to
resource issues.

Regards,

					- Ted