Tool to view extent metadata

Tool to view extent metadata

[Manuel Benitez]

I'm currently evaluating the ext4 allocator and one thing I've come
across is the lack of a tool that displays the exact structure of the
extents making up a file. I've found plenty of tools that will tell me
how many contiguous segments a file contains, but nothing so far to
let me see the actual makeup of the extents that map the inode to the
blocks that comprise the file. Have I just missed something obvious,
or would this be something worth me spending some time doing?

If so, one option would be to either modify the stat command or add a
similar command in debugfs to show the extents from the roots through
the root down to the leafs. Anyone have preferences or opinions?

Thanks,
-Ricky

Re: Tool to view extent metadata

[Andreas Dilger]

On Jul 09, 2009  15:17 -0700, Manuel Benitez wrote:
> I'm currently evaluating the ext4 allocator and one thing I've come
> across is the lack of a tool that displays the exact structure of the
> extents making up a file. I've found plenty of tools that will tell me
> how many contiguous segments a file contains, but nothing so far to
> let me see the actual makeup of the extents that map the inode to the
> blocks that comprise the file. Have I just missed something obvious,
> or would this be something worth me spending some time doing?
> 
> If so, one option would be to either modify the stat command or add a
> similar command in debugfs to show the extents from the roots through
> the root down to the leafs. Anyone have preferences or opinions?

The "filefrag" tool in recent e2fsprogs should provide such information,
using the FIEMAP ioctl to extract extent information from the kernel.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.

Re: Tool to view extent metadata

[Manuel Benitez]

Andreas,

Thank you for the reply. I wrote a program that uses FIEMAP to see
what I could get from it. It gives the same information that you can
extract from FIBMAP except that it is much more efficient (one ioctl
call per extent rather than one call per block). Unfortunately, it
does not give much insight into the structure of the metadata, so
there is no way to tell how efficiently the metadata represents the
file (depth of extent tree, number of internal and leaf nodes, etc.).

-Ricky

On Thu, Jul 9, 2009 at 3:55 PM, Andreas Dilger<adilger@sun.com> wrote:
> On Jul 09, 2009  15:17 -0700, Manuel Benitez wrote:
>> I'm currently evaluating the ext4 allocator and one thing I've come
>> across is the lack of a tool that displays the exact structure of the
>> extents making up a file. I've found plenty of tools that will tell me
>> how many contiguous segments a file contains, but nothing so far to
>> let me see the actual makeup of the extents that map the inode to the
>> blocks that comprise the file. Have I just missed something obvious,
>> or would this be something worth me spending some time doing?
>>
>> If so, one option would be to either modify the stat command or add a
>> similar command in debugfs to show the extents from the roots through
>> the root down to the leafs. Anyone have preferences or opinions?
>
> The "filefrag" tool in recent e2fsprogs should provide such information,
> using the FIEMAP ioctl to extract extent information from the kernel.
>
> Cheers, Andreas
> --
> Andreas Dilger
> Sr. Staff Engineer, Lustre Group
> Sun Microsystems of Canada, Inc.
>
>
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Tool to view extent metadata

[Eric Sandeen]

Manuel Benitez wrote:
> Andreas,
> 
> Thank you for the reply. I wrote a program that uses FIEMAP to see
> what I could get from it. It gives the same information that you can
> extract from FIBMAP except that it is much more efficient (one ioctl
> call per extent rather than one call per block). Unfortunately, it
> does not give much insight into the structure of the metadata, so
> there is no way to tell how efficiently the metadata represents the
> file (depth of extent tree, number of internal and leaf nodes, etc.).

adding something to debugfs should not be hard either; look at how it
prints out the BLOCKS: when you stat a file, for extent-format files
it'd be nice to have a better representation.  If you wanted to write a
patch for this it'd be great.  :)

-Eric

Re: Tool to view extent metadata

[Andreas Dilger]

On Jul 13, 2009  08:03 -0700, Manuel Benitez wrote:
> Thank you for the reply. I wrote a program that uses FIEMAP to see
> what I could get from it. It gives the same information that you can
> extract from FIBMAP except that it is much more efficient (one ioctl
> call per extent rather than one call per block). Unfortunately, it
> does not give much insight into the structure of the metadata, so
> there is no way to tell how efficiently the metadata represents the
> file (depth of extent tree, number of internal and leaf nodes, etc.).

It would also be possible to extend the FIEMAP interface to return
metadata by adding a new FIEMAP_FLAG_METADATA and then have the
metadata "extents" be marked with a new FIEMAP_EXTENT_METADATA.

In my previous musings on this topic, I thought it would make sense
to have the "extent" of e.g. an index block be the the logical range
of the file that this index block covers.  The inode itself might be
marked with FIEMAP_EXTENT_INODE|FIEMAP_EXTENT_METADATA and cover the
entire logical range of the file.

This would essentially allow the caller to generate a tree of the
metadata by heirarchical ordering of the extent ranges.


> On Thu, Jul 9, 2009 at 3:55 PM, Andreas Dilger<adilger@sun.com> wrote:
> > On Jul 09, 2009  15:17 -0700, Manuel Benitez wrote:
> >> I'm currently evaluating the ext4 allocator and one thing I've come
> >> across is the lack of a tool that displays the exact structure of the
> >> extents making up a file. I've found plenty of tools that will tell me
> >> how many contiguous segments a file contains, but nothing so far to
> >> let me see the actual makeup of the extents that map the inode to the
> >> blocks that comprise the file. Have I just missed something obvious,
> >> or would this be something worth me spending some time doing?
> >>
> >> If so, one option would be to either modify the stat command or add a
> >> similar command in debugfs to show the extents from the roots through
> >> the root down to the leafs. Anyone have preferences or opinions?
> >
> > The "filefrag" tool in recent e2fsprogs should provide such information,
> > using the FIEMAP ioctl to extract extent information from the kernel.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Tool to view extent metadata

[Aneesh Kumar K.V]

On Thu, Jul 09, 2009 at 03:17:00PM -0700, Manuel Benitez wrote:
> I'm currently evaluating the ext4 allocator and one thing I've come
> across is the lack of a tool that displays the exact structure of the
> extents making up a file. I've found plenty of tools that will tell me
> how many contiguous segments a file contains, but nothing so far to
> let me see the actual makeup of the extents that map the inode to the
> blocks that comprise the file. Have I just missed something obvious,
> or would this be something worth me spending some time doing?
> 
> If so, one option would be to either modify the stat command or add a
> similar command in debugfs to show the extents from the roots through
> the root down to the leafs. Anyone have preferences or opinions?
> 


tst_exents in lib/ext2fs/ provide a debugfs similar interface. There are
commands for inode, root etc

-aneesh