Re: Meeting

Re: Meeting

[Somdeep Dey]

[-- Attachment #1.1: Type: text/plain, Size: 7090 bytes --]

Hi,

We've now resumed working on the xfs_fsr utility as discussed before, after
our exams.

The first task that we undertook was to use fiemap to get file extent
mappings and tried to correlate the output with the information obtained
from xfs_bmap. For this we used the two underlying structures fiemap
and fiemap_extent. We're now trying to use the available free space mapping
patches to get free spaces in the file system.

We also wanted to ask about the current status of the rmap, as we'll
be required to define the interfaces that query it, as a key component of
our
work.

Regards,
A-DRS.


On Mon, Dec 1, 2014 at 1:01 AM, Swapnil Pimpale <pimpale.swapnil@gmail.com>
wrote:

> Just say that you have resumed work on the project after the exam.
>
> On Sun, Nov 30, 2014 at 2:16 PM, Somdeep Dey <somdeepdey10@gmail.com>
> wrote:
>
>> This is the final draft :
>> Hi,
>> We were occupied with our exams for the last two months.
>> We've now resumed working on the xfs_fsr utility as discussed before.
>>
>> The first task that we undertook was to use fiemap to get file extent
>> mappings and tried to correlate the output with the information obtained
>> from xfs_bmap. For this we used the two underlying structures fiemap
>> and fiemap_extent. We're now trying to use the available free space
>> mapping
>> patches to get free spaces in the file system.
>>
>> We also wanted to ask about the current status of the rmap, as we'll
>> be required to define the interfaces that query it, as a key component of
>> our
>> work.
>>
>> Regards,
>> A-DRS.
>>
>> On Sun, Nov 30, 2014 at 11:55 PM, Somdeep Dey <somdeepdey10@gmail.com>
>> wrote:
>>
>>> Hi,
>>> We were occupied with our exams for the last two months.
>>> We've now resumed working on the xfs_fsr utility as discussed before.
>>>
>>> The first task that we undertook was to use fiemap to get file extent
>>> mappings and tried to correlate the output with the information obtained
>>> from xfs_bmap. For this we used the two underlying structures fiemap
>>> and fiemap_extent. We're now trying to use the available free space
>>> mapping
>>> patches to get free spaces in the file system.
>>>
>>> We also wanted to ask about the current status of the rmap, as we'll
>>> be required to define the interfaces that query it, as a key component
>>> of our
>>> work.
>>>
>>> Regards,
>>> A-DRS.
>>>
>>> On Sun, Nov 30, 2014 at 9:37 PM, Nafisa Mandliwala <
>>> nafisa.mandliwala@gmail.com> wrote:
>>>
>>>> Umm, no need to apologize.
>>>>
>>>> What you showed me on Thursday was the standard fiemap ioctl that gives
>>>> the extents used by a file.
>>>> Say that you're now trying to use Dave's freespace mapping patches to
>>>> get free spaces in the file system.
>>>> Also, you're going to *define* the interfaces needed to query the
>>>> rmap, not just query them.
>>>>
>>>> Looks fine otherwise.
>>>>
>>>> On Sun, Nov 30, 2014 at 9:16 PM, Somdeep Dey <somdeepdey10@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>> This is the mail we drafted for Dave :
>>>>>
>>>>> Hi,
>>>>> We'd like to apologise for our unavailability over the last two months
>>>>> given the occurrence of our exams. We've now resumed working on the
>>>>> xfs_fsr utility as discussed before.
>>>>>
>>>>> The first task that we undertook was to use fiemap to get file extent
>>>>> mappings and tried to correlate the output with the information
>>>>> obtained
>>>>> from xfs_bmap. For this we used the two underlying structures fiemap
>>>>> and fiemap_extent.
>>>>>
>>>>> We also wanted to ask about the current status of the rmap, as we'll
>>>>> be required to query the interfaces of that as a key component of our
>>>>> work.
>>>>>
>>>>> Regards,
>>>>> A-DRS.
>>>>>
>>>>>
>>>>> Do tell us if this will suffice.
>>>>> Thanks,
>>>>> A-DRS.
>>>>>
>>>>> On Sun, Nov 30, 2014 at 9:05 PM, Somdeep Dey <somdeepdey10@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>> This is the mail we drafted for Dave :
>>>>>>
>>>>>> Hi,
>>>>>> We'd like to apologise for our unavailability over the last two months
>>>>>> given the occurrence of our exams. We've now resumed working on the
>>>>>> xfs_fsr utility as discussed before.
>>>>>>
>>>>>> The first task that we undertook was to use fiemap to get file extent
>>>>>> mappings and tried to correlate the output with the information
>>>>>> obtained
>>>>>> from xfs_bmap. For this we used the two underlying structures fiemap
>>>>>> and fiemap_extent.
>>>>>>
>>>>>> We also wanted to ask about the current status of the rmap, as we'll
>>>>>> be required to query the interfaces of that as a key component of our
>>>>>> work.
>>>>>>
>>>>>> Regards,
>>>>>> A-DRS.
>>>>>>
>>>>>>
>>>>>> Do tell us if this will suffice.
>>>>>> Thanks,
>>>>>> A-DRS.
>>>>>>
>>>>>> On Thu, Nov 27, 2014 at 9:39 PM, Nafisa Mandliwala <
>>>>>> nafisa.mandliwala@gmail.com> wrote:
>>>>>>
>>>>>>> Guys, you can find the xfs_alloc.h, xfs_alloc.c and xfs_iops.c in
>>>>>>> linux-x.x.x/fs/xfs/
>>>>>>> Please check if it has the changes we're looking for.
>>>>>>>
>>>>>>> I couldn't find anything on rmap, you should write a mail asap.
>>>>>>>
>>>>>>> On Thu, Nov 27, 2014 at 2:40 AM, Somdeep Dey <somdeepdey10@gmail.com
>>>>>>> > wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>> We ran the code for fiemap on files, that included  the following
>>>>>>>> three types :
>>>>>>>> 1) A non-fragmented file
>>>>>>>> 2) A fragmented file before using xfs_fsr
>>>>>>>> 3) The defragmeneted equivalent of the above file after using
>>>>>>>> xfs_fsr
>>>>>>>>
>>>>>>>>
>>>>>>>> Thereafter we examined the data by printing the fiemap structure
>>>>>>>> and fiemap_extents struct. The logs for the process followed is also
>>>>>>>> attached here.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> A-DRS.
>>>>>>>>
>>>>>>>> --
>>>>>>>> You received this message because you are subscribed to the Google
>>>>>>>> Groups "ADRScube" group.
>>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>>> send an email to adrscube+unsubscribe@googlegroups.com.
>>>>>>>> To view this discussion on the web visit
>>>>>>>> https://groups.google.com/d/msgid/adrscube/CAJAKVEFdkSEMW84%3DTvRNqOQdNMz6nS_WRVGN-2MX5se9NUaV1A%40mail.gmail.com
>>>>>>>> <https://groups.google.com/d/msgid/adrscube/CAJAKVEFdkSEMW84%3DTvRNqOQdNMz6nS_WRVGN-2MX5se9NUaV1A%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>>>>>> .
>>>>>>>>
>>>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>  --
>> You received this message because you are subscribed to the Google Groups
>> "ADRScube" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to adrscube+unsubscribe@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/adrscube/CAJAKVEE2Lu%3DMReV9fdg0L4OvWAymJJv9qteNDm_1oHyNNcdKVw%40mail.gmail.com
>> <https://groups.google.com/d/msgid/adrscube/CAJAKVEE2Lu%3DMReV9fdg0L4OvWAymJJv9qteNDm_1oHyNNcdKVw%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
>         Swapnil
>

[-- Attachment #1.2: Type: text/html, Size: 13454 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Meeting

[Dave Chinner]

On Mon, Dec 01, 2014 at 01:29:47AM +0530, Somdeep Dey wrote:
> Hi,
> 
> We've now resumed working on the xfs_fsr utility as discussed before, after
> our exams.
> 
> The first task that we undertook was to use fiemap to get file extent
> mappings and tried to correlate the output with the information obtained
> from xfs_bmap. For this we used the two underlying structures fiemap
> and fiemap_extent. We're now trying to use the available free space mapping
> patches to get free spaces in the file system.
> 
> We also wanted to ask about the current status of the rmap, as we'll
> be required to define the interfaces that query it, as a key component of
> our
> work.

The rmap design is slowly being thrashed out. Brian and I had a
discussion about it on IRC a couple of weeks ago (below).

I'm relatively close to having a proof of concept for single-owner
rmap btrees that works...

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

>From #xfs on freenode.net

[13/11/14 10:07] <dchinner_> foster: still around?
[13/11/14 10:10] <foster> dchinner_: yep
[13/11/14 10:27] <dchinner_> foster: been prototyping reverse mapping btree code over the past couple of days
[13/11/14 10:28] <dchinner_> couple of interesting issues have come up that need discussion
[13/11/14 10:28] <dchinner_> I think I have solutions to them, but I'm sure there are other ways of solving the problems
[13/11/14 10:28] <dchinner_> basically I want the rmap btree for two purposes
[13/11/14 10:29] <dchinner_> 1) to keep owner information so we can do block-to-owner lookups efficiently
[13/11/14 10:29] <dchinner_> e.g. to identify the files corrupted by bad sectors
[13/11/14 10:29] <dchinner_> found during a media scan
[13/11/14 10:31] <dchinner_> or to provide sufficient redundant information for an online rebuild of a corrupted free space btree
[13/11/14 10:32] <foster> so a btree that maps extents to inodes or something of that nature?
[13/11/14 10:32] <dchinner_> exactly
[13/11/14 10:32] <dchinner_> per-ag btree
[13/11/14 10:32] <dchinner_> that contains { start block, length, owner }
[13/11/14 10:32] <dchinner_> records
[13/11/14 10:32] <foster> ok
[13/11/14 10:33] <dchinner_> that's relatively easy to do
[13/11/14 10:33] <dchinner_> The patches I've written do that.
[13/11/14 10:33] <dchinner_> (not that it does anything other than compile yet)
[13/11/14 10:33] <dchinner_> however, there is a second reason for having a reverse mapping tree
[13/11/14 10:34] <dchinner_> it's for reference counting extents shared between inodes
[13/11/14 10:34] <foster> ah, reflink?
[13/11/14 10:34] <dchinner_> i.e. to implement reflink semantics
[13/11/14 10:34] <dchinner_> *nod*
[13/11/14 10:35] <dchinner_> this doesn't affect how the ramp btree interacts with the rest of the allocation/freeing code
[13/11/14 10:35] <dchinner_> but it does affect the "extent owner" tracking
[13/11/14 10:35] <dchinner_> i.e. we can now have multiple owners of an extent
[13/11/14 10:36] <dchinner_> so that btree record now becomes {stat, len, refcount, owner, owner, .... owner }
[13/11/14 10:36] <foster> yeah
[13/11/14 10:36] <dchinner_> and we can't do that with the generic btree infrastructure because it's all based around fixed length records
[13/11/14 10:38] <dchinner_> I've come up with a way of using fixed length records to implement this variable length shared rmap record
[13/11/14 10:38] <dchinner_> which uses the high bit of the start block number to distinguish between the types of records
[13/11/14 10:39] <dchinner_> and, in some cases, also uses the high bit of the extent length field to hold more information again.
[13/11/14 10:40] <dchinner_> but the issue is that it's quite complicated
[13/11/14 10:40] <dchinner_> and there's some interesting warts around records that span multiple btree blocks
[13/11/14 10:41] <dchinner_> because they've been shared across hundreds of owners
[13/11/14 10:43] <dchinner_> I can't see any obvious way of tracking owner information another way when we have shared extents
[13/11/14 10:44] <dchinner_> it's an 1:N mapping
[13/11/14 10:44] <foster> this information that's encoded in the record indicates the length of the record, or some kind of record chaining method..?
[13/11/14 10:44] <dchinner_> both ;)
[13/11/14 10:45] <foster> heh, ok
[13/11/14 10:45] <dchinner_> the first record becomes { start block, length, refcount, owner records}
[13/11/14 10:45] <dchinner_> and so a shared extent record looks like:
[13/11/14 10:46] <dchinner_> {{ master extent record}, {owner record }, {owner record }, .... {owner record}}
[13/11/14 10:46] <dchinner_> when an owner record is simply {owner #1, owner #2}
[13/11/14 10:47] <dchinner_> so both the master record and the owner record are teh same size (16 bytes)
[13/11/14 10:48] <dchinner_> so you can see how it can be problematic when a btree block only contains owner records
[13/11/14 10:48] <dchinner_> there's no start/len information, and so it's problematic for indexing that block in the higher levels of the btree
[13/11/14 10:49] <dchinner_> as the higher levels need to point to the master records....
[13/11/14 10:49] <foster> I'm missing how the master record refers to the owner record
[13/11/14 10:50] <foster> does it, or it's simply followed by the owner records?
[13/11/14 10:50] <dchinner_> owner records always follow the master record
[13/11/14 10:50] <foster> ok
[13/11/14 10:50] <dchinner_> right
[13/11/14 10:51] <dchinner_> So what I'm wondering is whether you think this is way too complex
[13/11/14 10:51] <dchinner_> or whether we might do better to have some other representation
[13/11/14 10:52] <dchinner_> such as keeping owner records in a different tree
[13/11/14 10:53] <dchinner_> or even not keeping them at all for shared extents
[13/11/14 10:53] <foster> sounds somewhat hairy at first, my first reaction is to think about whether there's some kind of level of indirection
[13/11/14 10:53] <foster> but i obviously haven't thought about this much at all
[13/11/14 10:54] <dchinner_> right, and I'm trying not to expose you to allthe gruesome details of what I've come up with ;)
[13/11/14 10:54] <dchinner_> just enough to describe the problem
[13/11/14 10:54] <foster> understood, i think i get the gist of it
[13/11/14 10:54] <foster> effectively creating first order/second order records within the tree
[13/11/14 10:55] <dchinner_> right
[13/11/14 10:55] <foster> or chaining or whatever the best terminology is ;)
[13/11/14 10:56] <dchinner_> hmmm, which triggers me immediately to think of an interesting btree extension
[13/11/14 10:57] <foster> hmm, a second tree is an interesting thought
[13/11/14 10:57] <foster> or some kind of magic/hidden owner inode that handles shared records
[13/11/14 10:57] <dchinner_> which, at first glance, makes it very similar to the directory btree structure....
[13/11/14 10:59] <dchinner_> need to think about that more....
[13/11/14 11:02] <dchinner_> (basically adding another level below the current leaf level of the btree that only holds owner records)
[13/11/14 11:06] <foster> interesting, though i'm not familiar enough with the on-disk dir structure to reason about off hand

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Meeting

[Somdeep Dey]

[-- Attachment #1.1: Type: text/plain, Size: 8118 bytes --]

Hi,

Good to hear about the progress related to rmap. We'll continue
working here on our side.

As we mentioned previously, we've familiarized ourselves
with the fiemap interface and developed an understanding
of how the ioctl works.

We also went through different discussions on the mailing
list related to fiemap, as well as obtained the latest patches.

While working on the rmap, would it be possible for you to
give us some essential details about the required fiemap
interface, so that we can obtain a clearer understanding of
 what we are required to do.


Regards,
A-DRS.

On Mon, Dec 1, 2014 at 10:01 AM, Dave Chinner <david@fromorbit.com> wrote:

> On Mon, Dec 01, 2014 at 01:29:47AM +0530, Somdeep Dey wrote:
> > Hi,
> >
> > We've now resumed working on the xfs_fsr utility as discussed before,
> after
> > our exams.
> >
> > The first task that we undertook was to use fiemap to get file extent
> > mappings and tried to correlate the output with the information obtained
> > from xfs_bmap. For this we used the two underlying structures fiemap
> > and fiemap_extent. We're now trying to use the available free space
> mapping
> > patches to get free spaces in the file system.
> >
> > We also wanted to ask about the current status of the rmap, as we'll
> > be required to define the interfaces that query it, as a key component of
> > our
> > work.
>
> The rmap design is slowly being thrashed out. Brian and I had a
> discussion about it on IRC a couple of weeks ago (below).
>
> I'm relatively close to having a proof of concept for single-owner
> rmap btrees that works...
>
> Cheers,
>
> Dave.
> --
> Dave Chinner
> david@fromorbit.com
>
> From #xfs on freenode.net
>
> [13/11/14 10:07] <dchinner_> foster: still around?
> [13/11/14 10:10] <foster> dchinner_: yep
> [13/11/14 10:27] <dchinner_> foster: been prototyping reverse mapping
> btree code over the past couple of days
> [13/11/14 10:28] <dchinner_> couple of interesting issues have come up
> that need discussion
> [13/11/14 10:28] <dchinner_> I think I have solutions to them, but I'm
> sure there are other ways of solving the problems
> [13/11/14 10:28] <dchinner_> basically I want the rmap btree for two
> purposes
> [13/11/14 10:29] <dchinner_> 1) to keep owner information so we can do
> block-to-owner lookups efficiently
> [13/11/14 10:29] <dchinner_> e.g. to identify the files corrupted by bad
> sectors
> [13/11/14 10:29] <dchinner_> found during a media scan
> [13/11/14 10:31] <dchinner_> or to provide sufficient redundant
> information for an online rebuild of a corrupted free space btree
> [13/11/14 10:32] <foster> so a btree that maps extents to inodes or
> something of that nature?
> [13/11/14 10:32] <dchinner_> exactly
> [13/11/14 10:32] <dchinner_> per-ag btree
> [13/11/14 10:32] <dchinner_> that contains { start block, length, owner }
> [13/11/14 10:32] <dchinner_> records
> [13/11/14 10:32] <foster> ok
> [13/11/14 10:33] <dchinner_> that's relatively easy to do
> [13/11/14 10:33] <dchinner_> The patches I've written do that.
> [13/11/14 10:33] <dchinner_> (not that it does anything other than compile
> yet)
> [13/11/14 10:33] <dchinner_> however, there is a second reason for having
> a reverse mapping tree
> [13/11/14 10:34] <dchinner_> it's for reference counting extents shared
> between inodes
> [13/11/14 10:34] <foster> ah, reflink?
> [13/11/14 10:34] <dchinner_> i.e. to implement reflink semantics
> [13/11/14 10:34] <dchinner_> *nod*
> [13/11/14 10:35] <dchinner_> this doesn't affect how the ramp btree
> interacts with the rest of the allocation/freeing code
> [13/11/14 10:35] <dchinner_> but it does affect the "extent owner" tracking
> [13/11/14 10:35] <dchinner_> i.e. we can now have multiple owners of an
> extent
> [13/11/14 10:36] <dchinner_> so that btree record now becomes {stat, len,
> refcount, owner, owner, .... owner }
> [13/11/14 10:36] <foster> yeah
> [13/11/14 10:36] <dchinner_> and we can't do that with the generic btree
> infrastructure because it's all based around fixed length records
> [13/11/14 10:38] <dchinner_> I've come up with a way of using fixed length
> records to implement this variable length shared rmap record
> [13/11/14 10:38] <dchinner_> which uses the high bit of the start block
> number to distinguish between the types of records
> [13/11/14 10:39] <dchinner_> and, in some cases, also uses the high bit of
> the extent length field to hold more information again.
> [13/11/14 10:40] <dchinner_> but the issue is that it's quite complicated
> [13/11/14 10:40] <dchinner_> and there's some interesting warts around
> records that span multiple btree blocks
> [13/11/14 10:41] <dchinner_> because they've been shared across hundreds
> of owners
> [13/11/14 10:43] <dchinner_> I can't see any obvious way of tracking owner
> information another way when we have shared extents
> [13/11/14 10:44] <dchinner_> it's an 1:N mapping
> [13/11/14 10:44] <foster> this information that's encoded in the record
> indicates the length of the record, or some kind of record chaining
> method..?
> [13/11/14 10:44] <dchinner_> both ;)
> [13/11/14 10:45] <foster> heh, ok
> [13/11/14 10:45] <dchinner_> the first record becomes { start block,
> length, refcount, owner records}
> [13/11/14 10:45] <dchinner_> and so a shared extent record looks like:
> [13/11/14 10:46] <dchinner_> {{ master extent record}, {owner record },
> {owner record }, .... {owner record}}
> [13/11/14 10:46] <dchinner_> when an owner record is simply {owner #1,
> owner #2}
> [13/11/14 10:47] <dchinner_> so both the master record and the owner
> record are teh same size (16 bytes)
> [13/11/14 10:48] <dchinner_> so you can see how it can be problematic when
> a btree block only contains owner records
> [13/11/14 10:48] <dchinner_> there's no start/len information, and so it's
> problematic for indexing that block in the higher levels of the btree
> [13/11/14 10:49] <dchinner_> as the higher levels need to point to the
> master records....
> [13/11/14 10:49] <foster> I'm missing how the master record refers to the
> owner record
> [13/11/14 10:50] <foster> does it, or it's simply followed by the owner
> records?
> [13/11/14 10:50] <dchinner_> owner records always follow the master record
> [13/11/14 10:50] <foster> ok
> [13/11/14 10:50] <dchinner_> right
> [13/11/14 10:51] <dchinner_> So what I'm wondering is whether you think
> this is way too complex
> [13/11/14 10:51] <dchinner_> or whether we might do better to have some
> other representation
> [13/11/14 10:52] <dchinner_> such as keeping owner records in a different
> tree
> [13/11/14 10:53] <dchinner_> or even not keeping them at all for shared
> extents
> [13/11/14 10:53] <foster> sounds somewhat hairy at first, my first
> reaction is to think about whether there's some kind of level of indirection
> [13/11/14 10:53] <foster> but i obviously haven't thought about this much
> at all
> [13/11/14 10:54] <dchinner_> right, and I'm trying not to expose you to
> allthe gruesome details of what I've come up with ;)
> [13/11/14 10:54] <dchinner_> just enough to describe the problem
> [13/11/14 10:54] <foster> understood, i think i get the gist of it
> [13/11/14 10:54] <foster> effectively creating first order/second order
> records within the tree
> [13/11/14 10:55] <dchinner_> right
> [13/11/14 10:55] <foster> or chaining or whatever the best terminology is
> ;)
> [13/11/14 10:56] <dchinner_> hmmm, which triggers me immediately to think
> of an interesting btree extension
> [13/11/14 10:57] <foster> hmm, a second tree is an interesting thought
> [13/11/14 10:57] <foster> or some kind of magic/hidden owner inode that
> handles shared records
> [13/11/14 10:57] <dchinner_> which, at first glance, makes it very similar
> to the directory btree structure....
> [13/11/14 10:59] <dchinner_> need to think about that more....
> [13/11/14 11:02] <dchinner_> (basically adding another level below the
> current leaf level of the btree that only holds owner records)
> [13/11/14 11:06] <foster> interesting, though i'm not familiar enough with
> the on-disk dir structure to reason about off hand
>

[-- Attachment #1.2: Type: text/html, Size: 10128 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Meeting

[Dave Chinner]

On Sun, Dec 07, 2014 at 05:27:16PM +0530, Somdeep Dey wrote:
> Hi,
> 
> Good to hear about the progress related to rmap. We'll continue
> working here on our side.
> 
> As we mentioned previously, we've familiarized ourselves
> with the fiemap interface and developed an understanding
> of how the ioctl works.
> 
> We also went through different discussions on the mailing
> list related to fiemap, as well as obtained the latest patches.
> 
> While working on the rmap, would it be possible for you to
> give us some essential details about the required fiemap
> interface, so that we can obtain a clearer understanding of
>  what we are required to do.

Turn FS_IOC_FIEMAPFS into an XFS specific ioctl that uses the fiemap
plumbing. Here's the original kernel patch that I wrote that
implemented FS_IOC_FIEMAPFS. This was the original fiemap extension
patches:

http://oss.sgi.com/archives/xfs/2012-10/msg00363.html

And I mentioned that it needed to be converted to FS_IOC_FIEMAPFS
due to review suggesting that it should be separate from file based
fiemap commands. I never posted those patches; I just forward ported
them to a current 3.18-rc7+for-next XFS tree and pushed them to the
fiemapfs fiemapfs branch in my kernel tree at:

git://git.kernel.org/pub/scm/linux/kernel/git/dgc/linux-xfs.git

The original xfs_spaceman tool that I wrote to call the fiemap
interface and make use of it is here:

http://oss.sgi.com/archives/xfs/2012-10/msg00366.html

I just updated it to the 3.2.2 code base and pushed it to the
spaceman branch in this tree:

git://git.kernel.org/pub/scm/fs/xfs/xfsprogs-dev.git

This code mostly works:

$ uname -a
Linux test4 3.18.0-rc7-dgc+ #629 SMP Mon Dec 8 09:20:09 EST 2014 x86_64 GNU/Linux
$ sudo xfs_spaceman -V
xfs_spaceman version 3.2.2
$ sudo xfs_db -V
xfs_db version 3.2.2
$ sudo xfs_db -r -c "freesp" /dev/vdc
   from      to extents  blocks    pct
      1       1    2018    2018   0.00
      2       3      23      64   0.00
4194304 8388607      64 536595213   0.40
134217728 268435455     500 133680330088  99.60
$ sudo xfs_spaceman -c "freesp" /mnt/scratch
   from      to extents  blocks    pct
      1       1      18      18   0.00
      2       3      23      64   0.00
4194304 8388607      64 536595213   0.40
134217728 268435455     500 133680330088  99.60
$

It looks like FS_IOC_FIEMAPFS isn't accounting blocks in teh AGFL
(4 per AG, and there are 500 AGs in that filesystem), so that
will need to be added to the kernel code that iterates the free
space.

Christoph has (more recently) suggested that this should be
implemented as an XFS specific ioctl (XFS_IOC_FIEMAPFS) that makes
use of all the existing fiemap infrastructure to implement it. That
means we can review it and push it as we need, and that makes the
process much simpler.

This means the kernel patches need to change - the ioctl
infrastructure for XFS_IOC_FIEMAPFS needs to be added to
fs/xfs/xfs_ioctl.c and the new ioctl definition and flags added to
fs/xfs/xfs_fs.h rather than to include/linux/....  i.e. the first
patch needs to be reworked to do this. It can also call the
xfs_fs_fiemapfs() function implemented in the second patch directly
rather than through an operations vector.

The change to the userspace code should just be to use the new ioctl
definition and flags, as the rest of the code is unchanged. I
strongly suggest that you work on the kernel patches to get the API
and ioctl code correct ASAP so we can get this into the tree ASAP;
that will make your life easier if you don't have to run patched
kernels just to test everything you are doing in userspace. Not to
mention having the code already committed upstream will look really
good in your final reports. :)

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs