Re: [PATCH v2] Btrfs: improve multi-thread buffer read

[Chris Mason <chris.mason@fusionio.com>]

On Thu, Jul 12, 2012 at 08:52:15PM -0600, Liu Bo wrote:
> On 07/12/2012 02:04 PM, Chris Mason wrote:
> > I like this patch, its a  big improvement for just a little timing
> > change.  Instead of doing the kmalloc of this struct, can you please
> > change it to put a pagevec on the stack.
> > 
> > The model would be:
> > 
> > add a page to the pagevec array
> > if pagevec full
> > 	launch all the readpages
> > 
> > This lets you avoid the kmalloc, and it is closer to how we solve
> > similar problems in other parts of the kernel.
> > 
> 
> 
> Yeah, but there is something different.
> 
> Actually my first attempt is doing this with struct pagevec, but pagevec has
> a PAGEVEC_SIZE, which is limited to 14.
> 
> That means that at the worst case, we batch only 14 pages in a bio to submit.
> 
> However, a bio is able to contains at most 128 pages with my devices, that's the
> reason why I turn to kmalloc another struct.
> 
> Here is some performance number:
>           w/o patch      w pvec patch     w kmalloc patch
>         -------------   --------------  ---------------
> READ:      745MB/s         880MB/s          987MB/s
> 
> 
> So what do you think about it?  I'm ok with both.

Lets go with the pagevec.  This particular workload (multiple threads
hammering big reads on the same range) is very specific.  The pagevec
gets us half way there, but won't add extra cost in any other workloads.

-chris