Re: per cpun+ spin locks coexistence?

[Eric Dumazet <dada1@cosmosbay.com>]

Peter Teoh a écrit :
> Thanks for the explanation, much apologies for this newbie discussion.
>   But I still find it inexplicable:
>
> On Mon, Mar 17, 2008 at 4:20 AM, Johannes Weiner <hannes@saeurebad.de> wrote:
>   
>>  A per-cpu variable is basically an array the size of the number of
>>  possible CPUs in the system.  get_cpu_var() checks what current CPU we
>>  are running on and gets the array-element corresponding to this CPU.
>>
>>  So, really oversimplified, get_cpu_var(foo) translates to something like
>>  foo[smp_processor_id()].
>>
>>     
>
> Ok, so calling get_cpu_var() always return the array-element for the
> current CPU, and since by design, only the current CPU can
> modify/write to this array element (this is my assumption - correct?),
> and the other CPU will just read it (using the per_cpu construct).
> So far correct?   So why do u still need to spin_lock() to lock other
> CPU from accessing - the other CPU will always just READ it, so just
> go ahead and let them read it.   Seemed like it defeats the purpose of
> get_cpu_var()'s design?
>
> But supposed u really want to put a spin_lock(), just to be sure
> nobody is even reading it, or modifying it, so then what is the
> original purpose of get_cpu_var() - is it not to implement something
> that can be parallelized among different CPU, without affecting each
> other, and using no locks?
>
> The dual use of spin_lock+get_cpu_var() confuses me here :-).   (not
> the per_cpu(), which I agree is supposed to be callabe from all the
> different CPU, for purpose of enumeration or data collection).
>
>   
You are right Peter, that fs/file.c contains some leftover from previous 
implementation of defer queue,
that was using a timer.

So we can probably provide a patch that :

- Use spin_lock() & spin_unlock() instead of spin_lock_bh() & 
spin_unlock_bh() in free_fdtable_work()
since we dont anymore use a softirq (timer) to reschedule the workqueue.

( this timer was deleted by the following patch :
http://readlist.com/lists/vger.kernel.org/linux-kernel/50/251040.html


But, you cannot avoid use of spin_lock()/spin_unlock() because 
schedule_work() makes no garantee that the work will be done by this cpu.

(free_fdtable_work() can be called to flush the fd defer queue of CPU X 
on behalf CPU Y,   with X != Y .
You then can have a corruption because CPU X is inside 
free_fdtable_rcu() and CPU Y is inside free_fdtable_work() )

So both spin_lock() and get_cpu_var() are necessary :

- One to get the precpu data for optimal performance on SMP (but not 
mandatory)
- One to protect the data from corruption on SMP.