Re: [patch] max-sectors-2.6.9-rc1-bk14-A0

Andrew Morton
2004-09-08T11:45:57Z

Ingo Molnar <mingo@elte.hu> wrote:
>
> * Andrew Morton <akpm@osdl.org> wrote:
>
> > > the attached patch introduces two new /sys/block values:
> > >
> > > /sys/block/*/queue/max_hw_sectors_kb
> > > /sys/block/*/queue/max_sectors_kb
> > >
> > > max_hw_sectors_kb is the maximum that the driver can handle and is
> > > readonly. max_sectors_kb is the current max_sectors value and can be
> > > tuned by root. PAGE_SIZE granularity is enforced.
> > >
> > > It's all locking-safe and all affected layered drivers have been updated
> > > as well. The patch has been in testing for a couple of weeks already as
> > > part of the voluntary-preempt patches and it works just fine - people
> > > use it to reduce IDE IRQ handling latencies.
> >
> > Could you remind us what the cause of the latency is, and its
> > duration?
> >
> > (Am vaguely surprised that it's an issue at, what, 32 pages? Is
> > something sucky happening?)
>
> yes, we are touching and completing 32 (or 64?) completely cache-cold
> structures: the page and the bio which are on two separate cachelines a
> pop. We also call into the mempool code for every bio completed. With
> the default max_sectors people reported hardirq latencies up to 1msec or
> more. You can see a trace of a 600+usec latency at:
>
> http://krustophenia.net/testresults.php?dataset=2.6.8-rc4-bk3-O7#/var/www/2.6.8-rc4-bk3-O7/ide_irq_latency_trace.txt
>
> here it's ~8 usecs per page completion - with 64 pages this completion
> activity alone is 512 usecs. So people want to have a way to tune down
> the maximum overhead in hardirq handlers. Users of the VP patches have
> reported good results (== no significant performance impact) with
> max_sectors at 32KB (8 pages) or even 16KB (4 pages).

Still sounds a bit odd. How many cachelines can that CPU fetch in 8 usecs?
Several tens at least?