defragmentation of boot related files

defragmentation of boot related files

[andreas]

hi,

I try to find ways to reduce the start time of programs or the whole 
operating system. The idea to increase disk performance is to gather all 
related files on disk physically close together. Unlike the last patches 
from Kazuya Mio for "relevant file defragmentation" files used by system 
boot are mostly not in the same directory.

First i look at the EXT4_IOC_MOVE_EXT call.
To reduce the amount of disk seeks i want to move the extents from 
several files into a single donor file. To do so i increment the 
move_extent.donor_start by the size of each file. But i got an error.

The error is caused by the lines in the kernel source 
fs/ext4/move_extent.c:998
998	/* Start offset should be same */
999	if (orig_start != donor_start) {
1000		ext4_debug("ext4 move extent: orig and donor's start "
1001		     "offset are not same [ino:orig %lu, donor %lu]\n",
1002                 orig_inode->i_ino, donor_inode->i_ino);
1003            return -EINVAL;
1004    }

Is there a reason why the offset of the original file and the donor file 
must be the same?

As i can see the patch for relevant file defragmentation in e4defrag 
supports only directories. May it be possible to select any desired file?

What would you suggest as the best way to put files physically together?

If I could help, tell me. :)

Regards,

andy

Re: defragmentation of boot related files

[Kazuya Mio]

Hi Andreas,
Thanks for the comments.

2010/08/25 21:00, andreas@rid-net.de wrote:
> Is there a reason why the offset of the original file and the donor file 
> must be the same?

e4defrag creates a donor file whose size is the same of the original file by
fallocate. There is a possibility that the original file will be corrupted
after moving an extent if the offset of the original file and the donor file
are different. So they are checked in the kernel space, but it may be
unnecessary from the point of view of the ioctl.

> As i can see the patch for relevant file defragmentation in e4defrag 
> supports only directories. May it be possible to select any desired file?

That's interesting. I came up with the new interface of e4defrag -r.
What do you think the following implementation idea?

Usage: e4defrag -r directory...| device...
       e4defrag -r base_file move_file...     <--- new

1. Defrag base_file to reduce fragmentation of extents (call EXT4_IOC_MOVE_EXT)
2. Preallocate physical blocks near the data blocks of base_file
3. Move move_file's extents to the blocks that are allocated by (2).
4. Repeat (2) and (3) for all files specified as move_file

Regards,
Kazuya Mio

Re: defragmentation of boot related files

[Greg Freemyer]

2010/8/26 Kazuya Mio <k-mio@sx.jp.nec.com>:
> Hi Andreas,
> Thanks for the comments.
>
> 2010/08/25 21:00, andreas@rid-net.de wrote:
>> Is there a reason why the offset of the original file and the donor file
>> must be the same?
>
> e4defrag creates a donor file whose size is the same of the original file by
> fallocate. There is a possibility that the original file will be corrupted
> after moving an extent if the offset of the original file and the donor file
> are different. So they are checked in the kernel space, but it may be
> unnecessary from the point of view of the ioctl.
>
>> As i can see the patch for relevant file defragmentation in e4defrag
>> supports only directories. May it be possible to select any desired file?
>
> That's interesting. I came up with the new interface of e4defrag -r.
> What do you think the following implementation idea?
>
> Usage: e4defrag -r directory...| device...
>       e4defrag -r base_file move_file...     <--- new
>
> 1. Defrag base_file to reduce fragmentation of extents (call EXT4_IOC_MOVE_EXT)
> 2. Preallocate physical blocks near the data blocks of base_file
> 3. Move move_file's extents to the blocks that are allocated by (2).
> 4. Repeat (2) and (3) for all files specified as move_file
>
> Regards,
> Kazuya Mio

I suspect the original idea would work better because it is more
likely to pack the libs / files into perfectly contiguous block
ranges.

I too have never understood why the donor offsets have to match the
original offsets, although I had previously assumed the issue could be
worked around via making the donor file sparse and only have the block
range of interest allocated.

This use case is a specific example of where it would be beneficial to
eliminate that artificial limitation.

Greg
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: defragmentation of boot related files

[andreas]

hi,

>> Usage: e4defrag -r directory...| device...
>>        e4defrag -r base_file move_file...<--- new
>>

The new interface looks nice. As I see it's easy to implement cause only 
the main-function must be modified. If you have no objections, I will do 
it myself and send you the patch to review it.

> I suspect the original idea would work better because it is more
> likely to pack the libs / files into perfectly contiguous block
> ranges.

I don't know if it makes a difference for the block allocator. Even 
though you know the size of all files to be moved.

For a better result, may it be possible to create a base_file whose 
inode is part of an empty block group?


  Andreas

Re: defragmentation of boot related files

[Andreas Dilger]

On 2010-08-26, at 14:16, andreas@rid-net.de wrote:
>>> Usage: e4defrag -r directory...| device...
>>>       e4defrag -r base_file move_file...<--- new
>>> 
> 
> The new interface looks nice. As I see it's easy to implement cause only the main-function must be modified. If you have no objections, I will do it myself and send you the patch to review it.
> 
>> I suspect the original idea would work better because it is more
>> likely to pack the libs / files into perfectly contiguous block
>> ranges.
> 
> I don't know if it makes a difference for the block allocator. Even though you know the size of all files to be moved.
> 
> For a better result, may it be possible to create a base_file whose inode is part of an empty block group?

There is a /proc/fs/ext4/*/inode_goal that will cause new inodes allocated in the filesystem to be after the specified goal inode.

I suppose it would also be possible to have e4defrag simply take an inode number in the form "<inode_number>" in place of "base_file", as debugfs does.

Cheers, Andreas

Re: defragmentation of boot related files

[Kazuya Mio]

2010/08/27 5:16, andreas@rid-net.de wrote:
>>> Usage: e4defrag -r directory...| device...
>>> e4defrag -r base_file move_file...<--- new
>>>
> 
> The new interface looks nice. As I see it's easy to implement cause only the main-function must be modified.

I think so.

> If you have no objections, I will do it myself and send you the patch to review it.

That would be great.

Regards,
Kazuya Mio