Re: Contiguous file sequences

Daire Byrne
2010-09-27T16:29:39Z

Eric,

On Mon, Sep 27, 2010 at 5:08 AM, Eric Sandeen <sandeen@sandeen.net> wrote:
> Daire Byrne wrote:
>>> Why is this the goal, what are you trying to achieve?
>>
>> I am essentially trying to play back a large frame sequence and trying
>> to minimise seeks as it can lead to sporadic slowdowns on a SATA based
>> RAID.
>
> Ok - and you've really seen allocation patterns that cause the playback
> to slow down?  xfs_bmap information for a few sequential files that were
> this far off would be interesting to see.
>
> Are you certain that it's seekiness causing the problem?  A great way
> to visualize it would be to use the seekwatcher application while you
> run a problematic file sequence.

I'm certain that the seekiness is the culprit. The image files are
pretty big and require 400MB/s+ speeds to play them back at full rate.
I can play a sequence which is aligned perfectly on disk just fine
(readahead) but when seeks are required between frames the framerate
drops noticeably. I'm using SATA disks which probably doesn't help
matters.

>>> You can't specify a starting block for any given file I'm afraid.
>>
>> Somebody pointed me at this which looks fairly promising:
>>
>>   http://oss.sgi.com/archives/xfs/2006-07/msg01005.html
>
> Yeah, that never got merged, but I think it still could be.
>
> It's only half your battle though, you need to find that contiguous
> space first, then specify the start block for it with the interface
> above.

I played around with the patch and I think I have a way to do what I
want using something like:

# allocate a big file that all the frames can fit into and hope it is contiguous
BLOCK=`xfs_io -f -c "resvsp 0 $TOTALSIZE" -c "freesp $FRAMESIZE 0" -c
"pwrite 0 1" -c "bmap" $DIR/test.0 | grep "0: \[" | sed 's/\../ /g' |
cut -f5 -d" "`
for x in `seq 1 $FRAMES`; do
allocnear $DIR/test.$x $BLOCK
BLOCK=`xfs_io -f -c "bmap" $DIR/test.$x | grep "0: \[" | sed
's/\../ /g' | cut -f5 -d" "`
dd if=/dev/zero of=$DIR/test.$x bs=1M count=13 conv=notrunc,nocreat
sync
done

where "allocnear" just creates a new file with the near block hint. It
isn't pretty atm but it does a better job of allocating files without
any block gaps between them. FYI the allocation patch is bypassed on
newer kernels and is useless without modification thanks to:

http://www.mail-archive.com/ocfs2-devel@oss.oracle.com/msg04387.html

>> I'm still trying to get my head around how I would actually write a
>> userspace app/script to use it but I think it should allow me to do
>> what I want. It would be good if I could script it through xfs_io. I'd
>> really like a script where I could point it at a directory and it
>> would do something like:
>>
>>   1. count total space used by file sequence
>>   2. find start block for that much contiguous space on disk (or as
>> much of it as possible)
>>   3. allocate the files using the start block one after another on disk
>>
>>>> Another option might be to create a single contiguous large file,
>>>> concatenate all the images into it and then split it up on disk using
>>>> offsets but I don't think such a thing is even possible? I always know
>>>> the image sequence size beforehand, all images are exactly the same
>>>> size and I can control/freeze the filesystem access if needed.
>>>>
>>>> Anybody got any suggestions? It *seems* like something that should be
>>>> possible and would be useful.
>>> This would be pretty low-level control of the allocator by userspace.
>>>
>>> I'll just go back and ask what problem you're trying to solve?  There
>>> may be a better (i.e. currently existing) solution.
>>
>> The "realtime" option is sometimes suggested as a way to do sequence
>> streaming but I'd really rather avoid that. It seems to me like the
>> option to allocate a sequence of files end on end in a known chunk of
>> contiguous space is something that would be useful in the normal
>> operating mode.
>
> It would be, but it's not there now.  Also, without some more complexity
> it'd still probably end up being a best effort rather than a guarantee,
> but some hints from userspace might be better than nothing.

I'm pretty sure I can do what I need to do now. Just a case of writing
a userspace application to "defrag" a directory of images now ....

Thanks for the feedback,

Daire