Re: [PATCH 17/17] pnfs/blocklayout: allocate separate pages for the layoutcommit payload

[Peng Tao <bergwolf@gmail.com>]

On Thu, Aug 7, 2014 at 7:27 PM, Christoph Hellwig <hch@lst.de> wrote:
> On Thu, Aug 07, 2014 at 07:05:03PM +0800, Peng Tao wrote:
>> On Thu, Aug 7, 2014 at 3:23 PM, Christoph Hellwig <hch@lst.de> wrote:
>> > Instead of overflowing the XDR send buffer with our extent list allocate
>> > pages and pre-encode the layoutupdate payload into them.  We optimistically
>> > allocate a single page use alloc_page and only switch to vmalloc when we
>> > have more extents outstanding.  Currently there is only a single testcase
>> > (xfstests generic/113) which can reproduce large enough extent lists for
>> > this to occur.
>> >
>> depending on how badly the extents are fragmented, it might worth
>> starting an async layoutcommit within blocklayout client when bl_count
>> reaches certain limit, so that we don't send an unlimited amount of
>> extents in one layoutcommit.
>
> I though about this, but this only reduces the problem and we'd still
> need to be able to commit all extents for a data integrity operation,
> so we'd still need someting like the above.
>
yeah, I wasn't against your patch. What I suggested is a way to limit
the amount of commit extents such that we don't end up sending more
that one RPC, whose size is still limited even if we allocate large
enough XDR buffer.

> The other option I considered and prototyped was to send multiple on the
> wire layoutcommit calls from pnfs_layoutcommit_inode, but that did prove
> to be even more cumbersome.