* Re: [PATCH 00/11 RESEND] e4defrag: fragmentation score rework and cleanups
@ 2011-06-15 20:31 Tomasz Chmielewski
2011-06-16 5:10 ` Kazuya Mio
0 siblings, 1 reply; 6+ messages in thread
From: Tomasz Chmielewski @ 2011-06-15 20:31 UTC (permalink / raw)
To: k-mio, linux-ext4
> Hi,
>
> This patch set adds the new function get_fragment_score() to libe2p, that
> calculates the fragmentation score. Fragmentation score shows how badly
Out of curiosity: is there a minimal kernel version requirement (one
which can be safely used with e4defrag)?
--
Tomasz Chmielewski
http://wpkg.org
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 00/11 RESEND] e4defrag: fragmentation score rework and cleanups
2011-06-15 20:31 [PATCH 00/11 RESEND] e4defrag: fragmentation score rework and cleanups Tomasz Chmielewski
@ 2011-06-16 5:10 ` Kazuya Mio
2011-06-16 8:18 ` Tomasz Chmielewski
0 siblings, 1 reply; 6+ messages in thread
From: Kazuya Mio @ 2011-06-16 5:10 UTC (permalink / raw)
To: Tomasz Chmielewski; +Cc: linux-ext4
2011/06/16 5:31, Tomasz Chmielewski wrote:
> Out of curiosity: is there a minimal kernel version requirement (one which can be safely used with e4defrag)?
EXT4_IOC_MOVE_EXT ioctl was merged 2.6.31. And then some bugfix were merged
before 2.6.35. I suggest you to build 2.6.35 or later if you try to do
e4defrag.
e4defrag also merged some bugfix like this patch, so you should use
the latest e4defrag.
If you find out something about online defrag, please post a comment.
Regards,
Kazuya Mio
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 00/11 RESEND] e4defrag: fragmentation score rework and cleanups
2011-06-16 5:10 ` Kazuya Mio
@ 2011-06-16 8:18 ` Tomasz Chmielewski
2011-06-17 5:13 ` Kazuya Mio
0 siblings, 1 reply; 6+ messages in thread
From: Tomasz Chmielewski @ 2011-06-16 8:18 UTC (permalink / raw)
To: Kazuya Mio; +Cc: linux-ext4
On 16.06.2011 07:10, Kazuya Mio wrote:
> 2011/06/16 5:31, Tomasz Chmielewski wrote:
>> Out of curiosity: is there a minimal kernel version requirement (one
>> which can be safely used with e4defrag)?
>
> EXT4_IOC_MOVE_EXT ioctl was merged 2.6.31. And then some bugfix were merged
> before 2.6.35. I suggest you to build 2.6.35 or later if you try to do
> e4defrag.
>
> e4defrag also merged some bugfix like this patch, so you should use
> the latest e4defrag.
Thanks for clarification.
> If you find out something about online defrag, please post a comment.
My main usage for e4defrag would be defragmenting virtual guest (like
KVM) file-based images.
These images are typically quite small when created (i.e. qcow2, or a
"raw", sparse file), but grow in time, which leads to serious fragmentation.
I noticed that when using e4defrag on such a file-based image, while the
guest is running (i.e. the file is in use), all IO from that guest to
the disk (disk is this file) is "frozen".
In other words, as soon as we run e4defrag, any guest writes to that
file will only complete if e4defrag finishes. As the images can be quite
big, it can mean guest IO can be frozen even for hours.
Is it a known/intended limitation (at least, it behaved like this when I
tried e4defrag a few months ago)?
In some cases, it can be a serious drawback, potentially leading to
filesystem corruption in the guest - if guest block layer is SCSI, with
default settings (at least in Linux) it may detect that no IO happened
for some time, and just "offline" the disk.
--
Tomasz Chmielewski
http://wpkg.org
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 00/11 RESEND] e4defrag: fragmentation score rework and cleanups
2011-06-16 8:18 ` Tomasz Chmielewski
@ 2011-06-17 5:13 ` Kazuya Mio
2011-06-17 13:08 ` Greg Freemyer
0 siblings, 1 reply; 6+ messages in thread
From: Kazuya Mio @ 2011-06-17 5:13 UTC (permalink / raw)
To: Tomasz Chmielewski; +Cc: linux-ext4
2011/06/16 17:18, Tomasz Chmielewski wrote:
> I noticed that when using e4defrag on such a file-based image, while the
> guest is running (i.e. the file is in use), all IO from that guest to
> the disk (disk is this file) is "frozen".
>
> In other words, as soon as we run e4defrag, any guest writes to that
> file will only complete if e4defrag finishes. As the images can be quite
> big, it can mean guest IO can be frozen even for hours.
>
>
> Is it a known/intended limitation (at least, it behaved like this when I
> tried e4defrag a few months ago)?
Thank you for the feedback. I will try to reproduce this problem later.
Regards,
Kazuya Mio
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 00/11 RESEND] e4defrag: fragmentation score rework and cleanups
2011-06-17 5:13 ` Kazuya Mio
@ 2011-06-17 13:08 ` Greg Freemyer
0 siblings, 0 replies; 6+ messages in thread
From: Greg Freemyer @ 2011-06-17 13:08 UTC (permalink / raw)
To: Kazuya Mio; +Cc: Tomasz Chmielewski, linux-ext4
On Fri, Jun 17, 2011 at 1:13 AM, Kazuya Mio <k-mio@sx.jp.nec.com> wrote:
> 2011/06/16 17:18, Tomasz Chmielewski wrote:
>>
>> I noticed that when using e4defrag on such a file-based image, while the
>> guest is running (i.e. the file is in use), all IO from that guest to
>> the disk (disk is this file) is "frozen".
>>
>> In other words, as soon as we run e4defrag, any guest writes to that
>> file will only complete if e4defrag finishes. As the images can be quite
>> big, it can mean guest IO can be frozen even for hours.
>>
>>
>> Is it a known/intended limitation (at least, it behaved like this when I
>> tried e4defrag a few months ago)?
>
> Thank you for the feedback. I will try to reproduce this problem later.
>
> Regards,
> Kazuya Mio
Kazuya,
When trying to come up with a better plan for handling large files,
you may want to review this msg and Akira's response from last spring:
http://www.kerneltrap.com/mailarchive/linux-ext4/2010/3/30/6899823
Greg
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 00/11 RESEND] e4defrag: fragmentation score rework and cleanups
@ 2011-06-15 6:33 Kazuya Mio
0 siblings, 0 replies; 6+ messages in thread
From: Kazuya Mio @ 2011-06-15 6:33 UTC (permalink / raw)
To: ext4, Theodore Tso
Hi,
This patch set adds the new function get_fragment_score() to libe2p, that
calculates the fragmentation score. Fragmentation score shows how badly
fragmented the file might be. filefrag uses it to make a user understand
the status of the file fragmentation easily. And e4defrag also uses it
to determine smarter whether to call EXT4_IOC_MOVE_EXT ioctl. This improvement
is based on the following Ted's idea.
http://marc.info/?l=linux-ext4&m=129324612305011&w=4
In addition, the 6th to the last is bugfix or cleanup patch.
This patch set is for e2fsprogs git tree. (commit:2d34a25f81)
Kazuya Mio (11):
libe2p: Add new function get_fragment_score()
filefrag: Output fragmentation score
e4defrag: Remove -c option which check the file fragments
e4defrag: Remove old fragmentation score calculation code
e4defrag: Use get_fragment_score() for decision of whether to defrag
e4defrag: Allow user who has read+write permission to defrag
e4defrag: Fix memory leak in file_defrag()
e4defrag: Use blk64_t instead of ext4_fsblk_t
e4defrag: Use EXT2_SUPER_MAGIC instead of EXT4_SUPER_MAGIC
e4defrag: Fix error messages more clearly
e4defrag: Remove unnecessary code
lib/e2p/Makefile.in | 6
lib/e2p/e2p.h | 2
lib/e2p/fragment_score.c | 142 +++++++
misc/Makefile.in | 8
misc/e4defrag.8.in | 30 -
misc/e4defrag.c | 889 +++++------------------------------------------
misc/filefrag.c | 28 +
7 files changed, 288 insertions(+), 817 deletions(-)
Regards,
Kazuya Mio
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2011-06-17 13:08 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-06-15 20:31 [PATCH 00/11 RESEND] e4defrag: fragmentation score rework and cleanups Tomasz Chmielewski
2011-06-16 5:10 ` Kazuya Mio
2011-06-16 8:18 ` Tomasz Chmielewski
2011-06-17 5:13 ` Kazuya Mio
2011-06-17 13:08 ` Greg Freemyer
-- strict thread matches above, loose matches on Subject: below --
2011-06-15 6:33 Kazuya Mio
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).