Re: [patch] max-sectors-2.6.9-rc1-bk14-A0

[Ingo Molnar <mingo@elte.hu>]

* Andrew Morton <akpm@osdl.org> wrote:

> > the attached patch introduces two new /sys/block values:
> > 
> >    /sys/block/*/queue/max_hw_sectors_kb
> >    /sys/block/*/queue/max_sectors_kb
> > 
> >  max_hw_sectors_kb is the maximum that the driver can handle and is
> >  readonly. max_sectors_kb is the current max_sectors value and can be
> >  tuned by root. PAGE_SIZE granularity is enforced.
> > 
> >  It's all locking-safe and all affected layered drivers have been updated
> >  as well. The patch has been in testing for a couple of weeks already as
> >  part of the voluntary-preempt patches and it works just fine - people
> >  use it to reduce IDE IRQ handling latencies.
> 
> Could you remind us what the cause of the latency is, and its
> duration?
>
> (Am vaguely surprised that it's an issue at, what, 32 pages?  Is
> something sucky happening?)

yes, we are touching and completing 32 (or 64?) completely cache-cold
structures: the page and the bio which are on two separate cachelines a
pop. We also call into the mempool code for every bio completed. With
the default max_sectors people reported hardirq latencies up to 1msec or
more. You can see a trace of a 600+usec latency at:

  http://krustophenia.net/testresults.php?dataset=2.6.8-rc4-bk3-O7#/var/www/2.6.8-rc4-bk3-O7/ide_irq_latency_trace.txt

here it's ~8 usecs per page completion - with 64 pages this completion
activity alone is 512 usecs. So people want to have a way to tune down
the maximum overhead in hardirq handlers. Users of the VP patches have
reported good results (== no significant performance impact) with
max_sectors at 32KB (8 pages) or even 16KB (4 pages).

	Ingo