Re: [RFC][PATCH 0/3] ext4: inode preferred block allocation

[Greg Freemyer <greg.freemyer@gmail.com>]

Dmitry added to cc because of use case at the bottom discussing ProjectID

2010/4/14 Kazuya Mio <k-mio@sx.jp.nec.com>:
> Hi,
>
> We implemented new two ioctls to allocate preferred blocks using inode PA.
> The old implementation idea is the following (b).
> http://marc.info/?l=linux-ext4&m=124962738211206&w=4
>
> One is EXT4_IOC_CONTROL_PA that is to create or discard inode PA, and the
> other is EXT4_IOC_GET_PA that is to get inode PA information.
>
> 1. EXT4_IOC_CONTROL_PA
>
>   EXT4_IOC_CONTROL_PA is used to create new inode PA, or to discard all inode
>   PAs in the target inode. This means that we will be able to allocate
>   the blocks we want.
>   We have a plan to add a new feature to e4defrag with these ioctls. This
>   feature improves read throughput when we read the files in the same
>   directory by reallocating target files near their parent directory.
>
> 2. EXT4_IOC_GET_PA
>
>   EXT4_IOC_GET_PA is used to get inode PA information.
>
> Moreover, when we create an inode PA, ext4_mb_new_inode_pa() merges
> contiguous inode PA if possible.
>
> This patch set consists of the following three patches. They can be applied to
> the ext4 patch queue:
> commit: 1dea5b6f540ad056d51d11cda71fa757cb44cbc4
>
> [RFC][PATCH 0/3] ext4: inode preferred block allocation
> [RFC][PATCH 1/3] ext4: add EXT4_IOC_CONTROL_PA to create/discard inode PA
> [RFC][PATCH 2/3] ext4: sort and merge inode PA
> [RFC][PATCH 3/3] ext4: add EXT4_IOC_GET_PA to get inode PA information
>
> Any comments are very welcome.
>
> Best regards,
> Kazuya Mio

Kazuya,

I have a basic understanding how these could be used by e4defrag to
organize stable data blocks / extents, but is the goal to also allow a
working set of dynamic files which allocate new data blocks routinely?

== details / example

Assume I know that files owned by a specific user (such as the apache
daemon) need to be collocated to reduce seek times as pages are
displayed.

After the fact, I can see the e4defrag moving all files with UID
apached into a subset of block groups thus increasing locality and
decreasing seeks.

But what if those files themselves are dynamically being created /
extended and thus allocating new data blocks/extents on the fly.   The
need in that situation would be more along the lines of defining a
preferred block group range for all files with the same UID.  And all
of those files would be provided exactly the same block range.

ie. If we have a 1 TB array, but the heavily used dynamic webserver
pages is only 5 GB, so I want to define a 5 GB block group range to
have those files live in.

Anyway, my main question again is if this patchset is only designed
for after the fact file/data locality organization, or if it is also
designed to support dynamic environments.

fyi: Creating locality groups is the use case I see for Dmitry's
Project ID patchset.  A collection of files that are used together can
be assigned a unique ProjectID and then e4defrag can grow the
knowledge to place them within a locality area on the disk.

But I also can see that new files within a directory would inherit the
ProjectID from the directory, and the data blocks allocated from the
correct locality area from the get go.

Dmitry, I haven't studied your patchset, but does it allow for
ProjectID inheritance from the parent directory?

Thanks
Greg
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html