From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
	id S1751317AbcGNSJg (ORCPT <rfc822;w@1wt.eu>);
	Thu, 14 Jul 2016 14:09:36 -0400
Received: from mx1.redhat.com ([209.132.183.28]:58621 "EHLO mx1.redhat.com"
	rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
	id S1751098AbcGNSJd (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
	Thu, 14 Jul 2016 14:09:33 -0400
Date: Thu, 14 Jul 2016 20:09:52 +0200
From: Oleg Nesterov <oleg@redhat.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Tejun Heo <tj@kernel.org>, John Stultz <john.stultz@linaro.org>,
        Ingo Molnar <mingo@redhat.com>, lkml <linux-kernel@vger.kernel.org>,
        Dmitry Shmidt <dimitrysh@google.com>,
        Rom Lemarchand <romlem@google.com>, Colin Cross <ccross@google.com>,
        Todd Kjos <tkjos@google.com>,
        "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Subject: Re: Severe performance regression w/ 4.4+ on Android due to cgroup
	locking changes
Message-ID: <20160714180951.GA16454@redhat.com>
References: <CALAqxLV2H06M2AxP2kK7pGHjLgQAaGdHB9HJEGZjH_Rouy4WhQ@mail.gmail.com> <20160713182102.GJ4065@mtj.duckdns.org> <20160713183347.GK4065@mtj.duckdns.org> <CALAqxLU76kVm8b5GNnSk25-Xk6D1efiWtwNJbY1th_pcN9Ks=g@mail.gmail.com> <20160713201823.GB29670@mtj.duckdns.org> <20160713202657.GW30154@twins.programming.kicks-ass.net> <20160713203944.GC29670@mtj.duckdns.org> <20160713205102.GZ30909@twins.programming.kicks-ass.net> <20160714131809.GO30927@twins.programming.kicks-ass.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20160714131809.GO30927@twins.programming.kicks-ass.net>
User-Agent: Mutt/1.5.18 (2008-05-17)
X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Thu, 14 Jul 2016 18:09:32 +0000 (UTC)
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 07/14, Peter Zijlstra wrote:
>
> The below is a compile tested only first draft so far. I'll go give it
> some runtime next.

So I will wait for the new version, but at first glance this matches the
code I already reviewed in the past (at least, tried hard to review ;)
and it looks correct.

Just a couple of almost cosmetic nits, feel free to ignore.

> --- a/include/linux/percpu-rwsem.h
> +++ b/include/linux/percpu-rwsem.h
> @@ -10,29 +10,107 @@
>  
>  struct percpu_rw_semaphore {
>  	struct rcu_sync		rss;
> -	unsigned int __percpu	*fast_read_ctr;
> +	unsigned int __percpu	*refcount;
>  	struct rw_semaphore	rw_sem;
> -	atomic_t		slow_read_ctr;
> -	wait_queue_head_t	write_waitq;
> +	wait_queue_head_t	writer;
> +	int			state;
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

I think that this "int state" and "enum { readers_slow, readers_block }"
just add a bit of complication/confusion.

All we need is the simple "bool readers_block" in percpu_rw_semaphore,
no?

> +void percpu_down_write(struct percpu_rw_semaphore *sem)
>  {
> +	down_write(&sem->rw_sem);
> +
> +	/* Notify readers to take the slow path. */
> +	rcu_sync_enter(&sem->rss);

I'd suggest to call rcu_sync_enter() before down_write(). This can help
when we wait for another writer which holds this lock.

Oleg.