From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755069AbbCGDxJ (ORCPT ); Fri, 6 Mar 2015 22:53:09 -0500 Received: from g1t5424.austin.hp.com ([15.216.225.54]:53342 "EHLO g1t5424.austin.hp.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750954AbbCGDxF (ORCPT ); Fri, 6 Mar 2015 22:53:05 -0500 Message-ID: <1425700382.2475.347.camel@j-VirtualBox> Subject: Re: softlockups in multi_cpu_stop From: Jason Low To: Ming Lei Cc: Davidlohr Bueso , Linus Torvalds , Ingo Molnar , Sasha Levin , Peter Zijlstra , LKML , Dave Jones , jason.low2@hp.com Date: Fri, 06 Mar 2015 19:53:02 -0800 In-Reply-To: References: <54F41516.6060608@oracle.com> <54F98F1F.3080107@oracle.com> <20150306123233.GA9972@gmail.com> <1425662342.19505.41.camel@stgolabs.net> <1425668223.2475.94.camel@j-VirtualBox> <1425670188.2475.113.camel@j-VirtualBox> <1425676346.2475.135.camel@j-VirtualBox> <1425680137.19505.63.camel@stgolabs.net> <20150307095526.5ffb1bf0@tom-ThinkPad-T410> <1425694046.19505.71.camel@stgolabs.net> <1425696991.2475.329.camel@j-VirtualBox> <1425698224.2475.339.camel@j-VirtualBox> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.2.3-0ubuntu6 Content-Transfer-Encoding: 7bit Mime-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Sat, 2015-03-07 at 11:39 +0800, Ming Lei wrote: > On Sat, Mar 7, 2015 at 11:17 AM, Jason Low wrote: > > On Sat, 2015-03-07 at 11:08 +0800, Ming Lei wrote: > >> On Sat, Mar 7, 2015 at 10:56 AM, Jason Low wrote: > >> > On Sat, 2015-03-07 at 10:10 +0800, Ming Lei wrote: > >> >> On Sat, Mar 7, 2015 at 10:07 AM, Davidlohr Bueso wrote: > >> >> > On Sat, 2015-03-07 at 09:55 +0800, Ming Lei wrote: > >> >> >> On Fri, 06 Mar 2015 14:15:37 -0800 > >> >> >> Davidlohr Bueso wrote: > >> >> >> > >> >> >> > On Fri, 2015-03-06 at 13:12 -0800, Jason Low wrote: > >> >> >> > > In owner_running() there are 2 conditions that would make it return > >> >> >> > > false: if the owner changed or if the owner is not running. However, > >> >> >> > > that patch continues spinning if there is a "new owner" but it does not > >> >> >> > > take into account that we may want to stop spinning if the owner is not > >> >> >> > > running (due to getting rescheduled). > >> >> >> > > >> >> >> > So you're rationale is that we're missing this need_resched: > >> >> >> > > >> >> >> > while (owner_running(sem, owner)) { > >> >> >> > /* abort spinning when need_resched */ > >> >> >> > if (need_resched()) { > >> >> >> > rcu_read_unlock(); > >> >> >> > return false; > >> >> >> > } > >> >> >> > } > >> >> >> > > >> >> >> > Because the owner_running() would return false, right? Yeah that makes > >> >> >> > sense, as missing a resched is a bug, as opposed to our heuristics being > >> >> >> > so painfully off. > >> >> >> > > >> >> >> > Sasha, Ming (Cc'ed), does this address the issues you guys are seeing? > >> >> >> > >> >> >> For the xfstest lockup, what matters is that the owner isn't running, since > >> >> >> the following simple change does fix the issue: > >> >> > > >> >> > I much prefer Jason's approach, which should also take care of the > >> >> > issue, as it includes the !owner->on_cpu stop condition to stop > >> >> > spinning. > >> >> > >> >> But the check on owner->on_cpu should be moved outside the loop > >> >> because new owner can be scheduled out too, right? > >> > > >> > We should keep the owner->on_cpu check inside the loop, otherwise we > >> > could continue spinning if the owner is not running. > >> > >> So how about checking in this way outside the loop for avoiding the spin? > >> > >> if (owner) > >> return owner->on_cpu; > > > > So these owner->on_cpu checks outside of the loop "fixes" the issue as > > well, but I don't see the benefit of needing to guess why we break out > > of the spin loop (which may make things less readable) and checking > > owner->on_cpu duplicate times when one check is enough. > > I mean moving the check on owner->on_cpu outside loop, so there is > only one check for both new and old owner. If it is inside loop, > the check is only on old owner. > > That is correct to keep it inside loop if you guys are sure new > owner can't be scheduled out, but better to add comment why > it can't, looks no one explained yet. The new owner can get rescheduled. And if there's a new owner, then the spinner goes to rwsem_spin_on_owner() again and checks the new owner's on_cpu.