Re: request baset device mapper in Linux

["Jun'ichi Nomura" <j-nomura@ce.jp.nec.com>]

Hi Lukas,

On 10/05/11 19:35, Lukas Hejtmanek wrote:
> On Wed, Oct 05, 2011 at 05:13:36PM +0900, Jun'ichi Nomura wrote:
>>> yes, 3GB/s and only kwapd0 and kswapd1 is running, no kworker or ksoftirqd..
>>
>> Hmm.. did you find any difference in your profile this time?
> 
> not sure what do you mean.

With SLES 2.6.32.36-0.5-default kernel, you found ksoftirqd
spent most of the time in __blk_recalc_rq_segments, using
sysprof/oprofile. That's why my patch was effective for it.

My question is whether you see such difference between
no-multipath and multipath, in profile data of 3.0.3 (without my patch).

>> I'm trying to reproduce it myself but no success so far
>> (perhaps disks are not fast enough to saturate CPU on my test machine).
> 
> hmm, I have 80 SAS 2.0 disks and two E5640 @ 2.67GHz cpus. 
>  
>> As ksoftirqd in top implies your CPU4 gets too much I/O completions,
>> 'rq_affnity = 2' for both dm and SCSI devices might be a solution.
>> It'll distribute block completion softirqs to submitters and possibly
>> reduce the loads of the 1st CPU in the socket.
>> (See the commit below. It's a new feature of 3.1. Not available in 3.0...)
> 
> So what next? Should I try 3.1 kernel with this patch applied?

Please try 3.1 (without my patch) + 'rq_affinity = 2',
on both multipath and no-multipath.

If you still see performance difference and ksoftirqd
spends most of the time in __blk_recalc_rq_segments,
try 3.1 + my patch + 'rq_affinity = 2'.

Thanks,
-- 
Jun'ichi Nomura, NEC Corporation