Re: [patch] max-sectors-2.6.9-rc1-bk14-A0

[Andrew Morton <akpm@osdl.org>]

Ingo Molnar <mingo@elte.hu> wrote:
>
> * Andrew Morton <akpm@osdl.org> wrote:
> 
>  > > the attached patch introduces two new /sys/block values:
>  > > 
>  > >    /sys/block/*/queue/max_hw_sectors_kb
>  > >    /sys/block/*/queue/max_sectors_kb
>  > > 
>  > >  max_hw_sectors_kb is the maximum that the driver can handle and is
>  > >  readonly. max_sectors_kb is the current max_sectors value and can be
>  > >  tuned by root. PAGE_SIZE granularity is enforced.
>  > > 
>  > >  It's all locking-safe and all affected layered drivers have been updated
>  > >  as well. The patch has been in testing for a couple of weeks already as
>  > >  part of the voluntary-preempt patches and it works just fine - people
>  > >  use it to reduce IDE IRQ handling latencies.
>  > 
>  > Could you remind us what the cause of the latency is, and its
>  > duration?
>  >
>  > (Am vaguely surprised that it's an issue at, what, 32 pages?  Is
>  > something sucky happening?)
> 
>  yes, we are touching and completing 32 (or 64?) completely cache-cold
>  structures: the page and the bio which are on two separate cachelines a
>  pop. We also call into the mempool code for every bio completed. With
>  the default max_sectors people reported hardirq latencies up to 1msec or
>  more. You can see a trace of a 600+usec latency at:
> 
>    http://krustophenia.net/testresults.php?dataset=2.6.8-rc4-bk3-O7#/var/www/2.6.8-rc4-bk3-O7/ide_irq_latency_trace.txt
> 
>  here it's ~8 usecs per page completion - with 64 pages this completion
>  activity alone is 512 usecs. So people want to have a way to tune down
>  the maximum overhead in hardirq handlers. Users of the VP patches have
>  reported good results (== no significant performance impact) with
>  max_sectors at 32KB (8 pages) or even 16KB (4 pages).

Still sounds a bit odd.  How many cachelines can that CPU fetch in 8 usecs?
Several tens at least?