Re: RE: RE: poor domU VBD performance.

[Jens Axboe <axboe@suse.de>]

On Wed, Mar 30 2005, Kurt Garloff wrote:
> Hi Ian,
> 
> On Tue, Mar 29, 2005 at 07:09:50PM +0100, Ian Pratt wrote:
> > We'd really appreciate your help on this, or from someone else at SuSE
> > who actually understands the Linux block layer?
> 
> I'm Cc'ing Jens ...
>  
> > In the 2.6 blkfront driver, what scheduler should we be registering
> > with? What should we be setting as max_sectors? Are there other
> > parameters we should be setting that we aren't? (block size?)
> 
> I think noop is a good choice for secondary domains, as you don't
> want to be too clever there, otherwise you stack a clever scheduler
> on top of a clever scheduler. noop basically only does front- and
> backmerging to make the request sizes larger.
> 
> But you probably should initialize the readahead sectors.
> 
> Please test attached patch.
> 
> It fixed the problem for me, but my testing was very limited,
> I only had a small loopback mounted root fs to test with quickly.
> 
> Note that initializing to 256 (128k) would be OK as well (and might 
> be the better default); it seems to be set to 256 (128k) by default, 
> but it's not ... If you explicitly set it to 256, the performance 
> still increases tremendously.
> 
> > In the blkback driver that actually issues the IO's in dom0, is there
> > something we should be doing to cause IOs to get batched? In 2.4 we used
> > a task_queue to push the IO through to the disk having queued it with
> > generic_make_request(). In 2.6 we're currently using submit_bio() and
> > just hoping that batching happens.
> 
> I don't think the blkback driver does anything wrong here.
> 
> Regards,
> -- 
> Kurt Garloff, Director SUSE Labs, Novell Inc.

> From: Kurt Garloff <garloff@suse.de>
> Subject: Initialize readahead in vbd Q init code
> 
> The domU read performance is poor without readahead, so
> better make sure we initialize this value.
> 
> Signed-off-by: Kurt Garloff <garloff@suse.de>
> 
> Index: linux-2.6.11/drivers/xen/blkfront/vbd.c
> ===================================================================
> --- linux-2.6.11.orig/drivers/xen/blkfront/vbd.c
> +++ linux-2.6.11/drivers/xen/blkfront/vbd.c
> @@ -268,8 +268,11 @@ static struct gendisk *xlvbd_get_gendisk
>              xlbd_blk_queue, BLKIF_MAX_SEGMENTS_PER_REQUEST);
>  
>          /* Make sure buffer addresses are sector-aligned. */
>          blk_queue_dma_alignment(xlbd_blk_queue, 511);
> +
> +	/* Set readahead */
> +	blk_queue_max_sectors(xlbd_blk_queue, 512);

This isn't read-ahead, it's the max request size setting. The actual
read-ahead setting is in q->backing_dev_info.ra_pages.

There is a helper function for this type of stacking,
blk_queue_stack_limits(). You call it after setting up your own queue:

        blk_queue_stack_limits(my_queue, bottom_queue);

I'll check the xen block driver to see if there's anything else that
sticks out.

-- 
Jens Axboe