about XFS_IOC_RESVSP

about XFS_IOC_RESVSP

[Joe Hsu]

        I pre-allocate blocks for a file. Then I use "open" and "write" system
calls to generate the content for that file. After the file is really
written to the disk,
I want to set all the extent(s) of that file unwritten. Then I will
write new content to
the same file. Is that possible? That means:
I want to dis-care the content just written and use the same allocated blocks.
In another words, I want to restore the state of the file to the state
when it was
first pre-allocated.

        Why am I doing this? Why not just over-write it? When doing
partial over-writing,
some blocks may be read for partial update before they are written
out. This hurts
some IO performance and If I can, I would prefer to dis-care old
content of the file and
use same (pre)allocated blocks. (In my case, I am doing intensive IO.)

        Any one can give me some hints? Thanks.

-- 
The sun is shinny but the ice is slippery.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: about XFS_IOC_RESVSP

[Eric Sandeen]

Joe Hsu wrote:
>         I pre-allocate blocks for a file. Then I use "open" and "write" system
> calls to generate the content for that file. After the file is really
> written to the disk,
> I want to set all the extent(s) of that file unwritten. Then I will
> write new content to
> the same file. Is that possible? That means:
> I want to dis-care the content just written and use the same allocated blocks.
> In another words, I want to restore the state of the file to the state
> when it was
> first pre-allocated.
> 
>         Why am I doing this? Why not just over-write it? When doing
> partial over-writing,
> some blocks may be read for partial update before they are written
> out. This hurts
> some IO performance and If I can, I would prefer to dis-care old
> content of the file and
> use same (pre)allocated blocks. (In my case, I am doing intensive IO.)
> 
>         Any one can give me some hints? Thanks.
> 

Do you really need the exact same blocks?  What if you just truncate to
0 & re-allocate?

-Eric

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: about XFS_IOC_RESVSP

[Joe Hsu]

Well, I have multiple processes running concurrently, and each writes
on its own files( the sort of files I mentioned), while at the same
time, I have other programs doing normal but light I/O to other files
on the same xfs partition.

Once I thought maybe I can pre-allocate these special files within a
directory, which has fixed allocation groups(I guess that means fixed
sets of blocks), and then I can try to make 'truncate to 0 and
pre-allocate' requests sequential for the running processes. But, XFS
seems to have this feature, I cannot find how to do that.

2009/5/22 Eric Sandeen <sandeen@sandeen.net>:
> Joe Hsu wrote:
> Do you really need the exact same blocks?  What if you just truncate to
> 0 & re-allocate?
>
> -Eric
>



-- 
The sun is shinny but the ice is slippery.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: about XFS_IOC_RESVSP

[Joe Hsu]

      After days of testing(I only ftruncate to 0 and re-preallocate
files if needed),
fragmentation become much more serious, sigh

2009/5/22 Joe Hsu <nagual.hsu@gmail.com>:
>        I pre-allocate blocks for a file. Then I use "open" and "write" system
> calls to generate the content for that file. After the file is really
> written to the disk,
> I want to set all the extent(s) of that file unwritten. Then I will
> write new content to
> the same file. Is that possible? That means:
> I want to dis-care the content just written and use the same allocated blocks.
> In another words, I want to restore the state of the file to the state
> when it was
> first pre-allocated.
>
>        Why am I doing this? Why not just over-write it? When doing
> partial over-writing,
> some blocks may be read for partial update before they are written
> out. This hurts
> some IO performance and If I can, I would prefer to dis-care old
> content of the file and
> use same (pre)allocated blocks. (In my case, I am doing intensive IO.)
>
>        Any one can give me some hints? Thanks.
>
> --
> The sun is shinny but the ice is slippery.
>



-- 
The sun is shinny but the ice is slippery.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: about XFS_IOC_RESVSP

[Eric Sandeen]

Joe Hsu wrote:
> Well, I have multiple processes running concurrently, and each writes
>  on its own files( the sort of files I mentioned), while at the same 
> time, I have other programs doing normal but light I/O to other files
> on the same xfs partition.
> 
> Once I thought maybe I can pre-allocate these special files within a 
> directory, which has fixed allocation groups(I guess that means fixed
> sets of blocks), and then I can try to make 'truncate to 0 and 
> pre-allocate' requests sequential for the running processes. But, XFS
> seems to have this feature, I cannot find how to do that.

There is no interface to re-mark existing blocks as unwritten, I'm
afraid.  It sounds like an interesting interface, but it's not there
today AFAIK.

You said:

> Why am I doing this? Why not just over-write it? When doing partial 
> over-writing, some blocks may be read for partial update before they
>  are written out. This hurts some IO performance

I guess it's not possible for you to do whole-block IO instead?  Or even
pad out the writes to block boundaries if needed?

and:

> After days of testing(I only ftruncate to 0 and re-preallocate files 
> if needed), fragmentation become much more serious, sigh

It's interesting that it's so bad, I'd have hoped that if you free a
contiguous chunk of blocks and then immediately reallocate them on the
same inode, that they'd get preallocated nicely.... How bad is it?

-Eric

> 2009/5/22 Eric Sandeen <sandeen@sandeen.net>:
>> Joe Hsu wrote: Do you really need the exact same blocks?  What if 
>> you just truncate to 0 & re-allocate?
>> 
>> -Eric
>> 
> 
> 
> 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs