Re: [PATCH] vfs: Avoid IPI storm due to bh LRU invalidation

[Hannes Reinecke <hare@suse.de>]

On 02/06/2012 04:42 PM, Srivatsa S. Bhat wrote:
> On 02/06/2012 07:25 PM, Jan Kara wrote:
> 
>> When discovery of lots of disks happen in parallel, we call
>> invalidate_bh_lrus() once for each disk from partitioning code resulting in a
>> storm of IPIs and causing a softlockup detection to fire (it takes several
>> *minutes* for a machine to execute all the invalidate_bh_lrus() calls).
>>
>> Fix the issue by allowing only single invalidation to run using a mutex and let
>> waiters for mutex figure out whether someone invalidated LRUs for them while
>> they were waiting.
>>
>> Signed-off-by: Jan Kara <jack@suse.cz>
>> ---
>>  fs/buffer.c |   23 ++++++++++++++++++++++-
>>  1 files changed, 22 insertions(+), 1 deletions(-)
>>
>>   I feel this is slightly hacky approach but it works. If someone has better
>> idea, please speak up.
>>
> 
> 
> Something related that you might be interested in:
> https://lkml.org/lkml/2012/2/5/109
> 
> (This is part of Gilad's patchset that tries to reduce cross-CPU IPI
> interference.)
> 
Yes, but this is only part of the equation.

When booting a machine with lots of disks chances are that each CPU
_will_ have LRU BHs attached to it (due to partitions table
reading). However, these LRU BHs have nothing to do with the device
in question. So we wouldn't even need to send IPIs here.
Sadly we seem to lack the facilities to figure that out
(I'm not an expert in that area to tell for sure :-).

So the best we can hope for is to serialise the IPIs to not overload
system with tons of IPIs.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		      zSeries & Storage
hare@suse.de			      +49 911 74053 688
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: J. Hawn, J. Guild, F. Imendörffer, HRB 16746 (AG Nürnberg)