Re: [PATCH]IPC locks breaking down with RCU

[Manfred Spraul <manfred@colorfullife.com>]

Hugh wrote:

>I had toyed with the idea of never
>freeing entries once allocated, which is a similarly simple solution;
>

Not possible, the structure size for ipc sem depends on the number of 
semaphores in the semaphore set.

Probably the best approach is to add a "deleted" flag into the ipc_id 
structure, and to check that flag after acquiring the spinlock. And 
perform the actual free operations for the ipc element in a rcu callback.
At which context do the rcu callbacks run? The semaphore sets are 
allocated with vmalloc for large sets, and that function is only 
permitted from process context, not from bh or irq context. According to 
a comment in rcupdate.c, rcu_process_callbacks runs in a tasklet, i.e. 
at bh context.

>I'm happy to be overruled by someone who understands these cacheline
>bounce issues better than we do, but nobody else has spoken up yet.
>  
>
Are there any good documents about cacheline bouncing, or rules how to 
reduce it?

For example, should a spinlock and the data it protects be in the same 
cacheline, or in different cachelines?
I guess that "same cacheline" means that only one cacheline is 
transfered if a cpu acquires the spinlock and touches the data.
But OTHO a spinning cpu would probably force the cacheline into shared 
state, and that'll slow down the data access for the cpu that owns the 
spinlock.

--
    Manfred