From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
	id S1756489AbbHDNmt (ORCPT <rfc822;w@1wt.eu>);
	Tue, 4 Aug 2015 09:42:49 -0400
Received: from mx1.redhat.com ([209.132.183.28]:59166 "EHLO mx1.redhat.com"
	rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
	id S1756447AbbHDNms (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
	Tue, 4 Aug 2015 09:42:48 -0400
Date: Tue, 4 Aug 2015 15:40:43 +0200
From: Oleg Nesterov <oleg@redhat.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Arnd Bergmann <arnd@arndb.de>, Ingo Molnar <mingo@redhat.com>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        "Paul E.McKenney" <paulmck@linux.vnet.ibm.com>,
        Waiman Long <Waiman.Long@hp.com>, linux-kernel@vger.kernel.org
Subject: Re: qrwlock && read-after-read
Message-ID: <20150804134043.GA24252@redhat.com>
References: <20150804130053.GA22608@redhat.com> <20150804131036.GQ25159@twins.programming.kicks-ass.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20150804131036.GQ25159@twins.programming.kicks-ass.net>
User-Agent: Mutt/1.5.18 (2008-05-17)
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 08/04, Peter Zijlstra wrote:
>
> On Tue, Aug 04, 2015 at 03:00:53PM +0200, Oleg Nesterov wrote:
> > I am working on the (off-topic) bug report which motivated me to
> > look at locking/qrwlock.c and it seems to me there is a problem
> > with the queued rwlocks.
> >
> > Unless I am totally confused read-after-read is no longer valid,
> > write_lock() stops the new readers. And lockdep doesn't know this,
> > read_lock()->rwlock_acquire_read() doesn't match the reality. The
> > code doing
> >
> > 	read_lock(X);
> > 	read_lock(X);
> >
> > can deadlock if another CPU does write_lock(X) in between. This
> > was fine before rwlock_t was changed to use qrwlock.
> >
> > A nested read_lock() in interrupt should be fine though, and this
> > is because queue_read_lock_slowpath() "ignores" _QW_WAITING if
> > in_interrupt().
> >
> > This means that rwlock_t has the really strange semantics imho,
> > and again, it is not lockdep-friendly.
> >
> > What do you think we can/should do? Or did I misread this code?
>
> Fix lockdep, although that's non trivial from what I remember.
>
> These (new) semantics were very much on purpose and suggested by Linus
> IIRC.

Hmm, OK.

Lets fix the lockdep annotaions?

Oleg.

--- x/include/linux/rwlock_api_smp.h
+++ x/include/linux/rwlock_api_smp.h
@@ -146,7 +146,7 @@ static inline int __raw_write_trylock(rw
 static inline void __raw_read_lock(rwlock_t *lock)
 {
 	preempt_disable();
-	rwlock_acquire_read(&lock->dep_map, 0, 0, _RET_IP_);
+	lock_acquire(..., /* read */ in_interrupt() 2 : 1, ...);
 	LOCK_CONTENDED(lock, do_raw_read_trylock, do_raw_read_lock);
 }