Re: [PATCH] netfilter: use per-CPU r**ursive lock {XV}

[Stephen Hemminger <shemminger@vyatta.com>]

On Mon, 27 Apr 2009 19:44:57 +0200
Peter Zijlstra <peterz@infradead.org> wrote:

> On Sun, 2009-04-26 at 14:57 -0700, Stephen Hemminger wrote:
> > On Sun, 26 Apr 2009 14:56:46 -0400
> > Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca> wrote:
> > 
> > > * Eric Dumazet (dada1@cosmosbay.com) wrote:
> > > > From: Stephen Hemminger <shemminger@vyatta.com>
> > > > 
> > > > > Epilogue due to master Jarek. Lockdep carest not about the locking
> > > > > doth bestowed. Therefore no keys are needed.
> > > > > 
> > > > > Signed-off-by: Stephen Hemminger <shemminger@vyatta.com>
> > > > 
> > > > So far, so good, should be ready for inclusion now, nobody complained :)
> > > > 
> > > > I include the final patch, merge of your last two patches.
> > > > 
> > > > David, could you please review it once again and apply it if it's OK ?
> > > 
> > > > Thanks to all for your help and patience
> > > > 
> > > > [PATCH] netfilter: use per-CPU recursive lock {XV}
> > > 
> > > Hi Eric, 
> > > 
> > > Suitable name would probably be :
> > > 
> > 
> > But Linus is trying to delude himself.
> > 
> > This usage is recursive even if he doesn't like the terminology.
> > The same CPU has to be able to reacquire the read lock without deadlocking.
> > If reader/writer locks were implemented in a pure writer gets priority
> > method, then this code would break!  So yes read locks can be used recursively
> > now in Linux, but if the were implemented differently then this code
> > would break.  For example, the -rt kernel turns all read/write locks into
> > mutexs, so the -rt kernel developers will have to address this.
> 
> A recursive lock has the property:
> 
> lock()
> {
>   if (lock->owner == current) {
>     lock->depth++;
>     return;
>   }
> 
>   /* regular lock stuff */
> }
> 
> unlock()
> {
>   if (!--lock->depth)
>     /* regular unlock */
> }

Only on Linux, and only because you look at locking from
the point of view of the magic variable "current" process
point of view.

> non of the linux kernel locking primitives have this -- with the
> possible exception of the cpu-hotplug lock.
> 
> What rwlock_t has, is reader bias to the point where you can utterly
> starve writers, with the side effect that you can obtain multiple read
> ownerships without causing a deadlock.

But what happens when this side effect disappears?
 
> This is not what is called a recursive lock. A recursive lock would have
> each owner only once, this rwlock_t thing is simply so unfair that it
> can have unlimited owners, including multiple copies of the same one.
> 
> rwsem has fifo fairness, and therefore can deadlock in this scenario,
> suppose thread A does a read, thread B tries a write and blocks, then
> thread A recurses and tries to obtain another read ownership --
> deadlock, as the FIFO fairness will demand the second read ownership
> will wait on the pending writer, which will wait on the outstanding read
> owner.
> 
> Now if rwsem were a fifo-fair recursive lock, the above would not
> deadlock, since it would detect that the task already had (read)
> ownership and simply increment the depth, instead of trying to acquire a
> second ownership.
> 
> This is all common and well understood terminology, not something Linus
> invented just to harras you with.

In Documentation/ ?  online ?  Where is the definition? The only reference
I se is indirectly in DocBook/kernel-locking.tmpl.

> Generally speaking we do not condone recursive locking strategies -- and
> afaik reiserfs (as per the BKL) and the network code (as per abusing
> rwlock_t unfairness) are the only offenders.
> 
> Like Linus stated, recursive locking is generally poor taste and
> indicates you basically gave up on trying to find a proper locking
> scheme. We should very much work towards getting rid of these
> abberations instead of adding new ones.

The people complaining about naming never seem to be the ones providing
workable suggestions or patches.

> Linus is very much right on what he said, and you calling him delusional
> only high-lights your ignorance on the issue.
> 
> [ PS. -rt implements rwlock_t as a proper recursive lock (either a mutex
>   or a full multi-owner reader-writer lock with PI fairness) so if
>   anybody abuses rwlock_t unfairness in a way that is not strict owner
>   recursive we have a problem. ]

Name it "dog's breath locking" for all I care. I am not bothering
with arguments over names; there is real work to do elsewhere.