Re: [PATCH (block.git) 0/2] IO CPU affinity update:

Alan D. Brunelle
2008-03-17T20:57:48Z

Jens Axboe wrote:
> On Mon, Mar 17 2008, Alan D. Brunelle wrote:
>> Hi Jens -
>>
>> Two patches:
>>
>> 1. Adds in the IRQ saving to generic_smp_call_function_single_interrupt (as you had suggested).
>> 2. Ensures a single IPI generated to get a remote function call handler going.
>>
>> So far it is working better than before on the 4-way IA64 w/ the mkfs/untar/make test suite - after 22 runs:
>>
>> Part RQ MIN AVG MAX Dev
>> ----- -- ------ ------ ------ ------
>> mkfs 0 18.786 19.253 19.655 0.241
>> mkfs 1 18.639 19.182 19.786 0.293
>>
>> untar 0 17.140 17.486 18.250 0.322
>> untar 1 16.951 17.494 18.274 0.350
>>
>> make 0 22.927 24.310 34.339 2.287
>> make 1 22.863 23.788 24.189 0.333
>>
>> comb 0 59.478 61.049 70.320 2.142
>> comb 1 59.875 60.463 61.305 0.458
>>
>> psys 0 3.96% 4.14% 4.39% 0.100
>> psys 1 3.60% 3.85% 4.19% 0.176
>>
>> So we're seeing reduced time (~1.0%) and reduced %sys to do it (7.0%).
>> The tighter deviations for make with rq=1 may be interesting... :-)
>>
>> I've compiled & booted the patches for x86_64 - rq=1 is working on
>> that platform too.
>
> This is starting to look pretty good! Thanks a lot for these results,
> and the ->activated optimizations. I had a feeling the unstable results
> were something like this, missing ipi's.
>

Jens: FYI: I am still seeing infrequent hangs on the x86_64-based platform. It happened again today after my patch was added, I did get <alt><sysrq><W> to work this time, and the threads that were stuck were waiting for IOs to complete. I believe at some point you were thinking of hacking in a block IO dump magic key as well - is that there yet?

I'll get to the ia64 testing tomorrow - have to run now, spent most of the day looking at what could cause stuff to be missed on x86_64. The code looks solid to me, but this hang needs to be figured out.

Lastly, I did do a run on a 16-way ia64 (with my patches) and again it ran fine.

Alan