From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751714AbcFYPR2 (ORCPT ); Sat, 25 Jun 2016 11:17:28 -0400 Received: from mail-io0-f173.google.com ([209.85.223.173]:32853 "EHLO mail-io0-f173.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751450AbcFYPR0 (ORCPT ); Sat, 25 Jun 2016 11:17:26 -0400 Date: Sat, 25 Jun 2016 23:21:30 +0800 From: Boqun Feng To: Peter Zijlstra Cc: Pan Xinhui , linux-kernel@vger.kernel.org, mingo@redhat.com, dave@stgolabs.net, will.deacon@arm.com, Waiman.Long@hpe.com, benh@kernel.crashing.org Subject: Re: [PATCH] locking/osq: Drop the overload of osq lock Message-ID: <20160625152130.GA2452@insomnia> References: <1466876523-33437-1-git-send-email-xinhui.pan@linux.vnet.ibm.com> <20160625142447.GK30154@twins.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="h31gzZEtNLTqOjlF" Content-Disposition: inline In-Reply-To: <20160625142447.GK30154@twins.programming.kicks-ass.net> User-Agent: Mutt/1.6.1 (2016-04-27) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org --h31gzZEtNLTqOjlF Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Jun 25, 2016 at 04:24:47PM +0200, Peter Zijlstra wrote: > On Sat, Jun 25, 2016 at 01:42:03PM -0400, Pan Xinhui wrote: > > An over-committed guest with more vCPUs than pCPUs has a heavy overload > > in osq_lock(). > >=20 > > This is because vCPU A hold the osq lock and yield out, vCPU B wait > > per_cpu node->locked to be set. IOW, vCPU B wait vCPU A to run and > > unlock the osq lock. Even there is need_resched(), it did not help on > > such scenario. > >=20 > > To fix such bad issue, add a threshold in one while-loop of osq_lock(). > > The value of threshold is somehow equal to SPIN_THRESHOLD. >=20 > Blergh, virt ... >=20 > So yes, lock holder preemption sucks. You would also want to limit the > immediate spin on owner. >=20 > Also; I really hate these random number spin-loop thresholds. >=20 > Is it at all possible to get feedback from your LPAR stuff that the vcpu > was preempted? Because at that point we can add do something like: >=20 Good point! >=20 > int vpc =3D vcpu_preempt_count(); >=20 > ... >=20 > for (;;) { >=20 > /* the big spin loop */ >=20 > if (need_resched() || vpc !=3D vcpu_preempt_count()) So on PPC, we have lppaca::yield_count to detect when an vcpu is preempted, if the yield_count is even, the vcpu is running, otherwise it is preempted(__spin_yield() is a user of this). Therefore it makes more sense we if (need_resched() || vcpu_is_preempted(old)) here, and implement vcpu_is_preempted() on PPC as bool vcpu_is_preempted(int cpu) { return !!(be32_to_cpu(lppaca_of(cpu).yield_count) & 1) } Thoughts? Regards, Boqun > /* bail */ >=20 > } >=20 >=20 > With a default implementation like: >=20 > static inline int vcpu_preempt_count(void) > { > return 0; > } >=20 > So the compiler can make it all go away. >=20 >=20 > But on virt muck it would stop spinning the moment the vcpu gets > preempted, which is the right moment I'm thinking. --h31gzZEtNLTqOjlF Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQEcBAABCAAGBQJXbqF1AAoJEEl56MO1B/q4IvcH/3Bth2MVCWKoKg40uxKVwW21 eNpJ+dG5Irar9SxJhoXlollezlVdP0EMIT/H+J/C1FBTIGCoIyhrLoh1ccyXWSBX Ysm2jmLiOOf525Z5pK6bxQik49dV25wvbVx03a/8t/ZafxRtenf/Itq9SUJjQVhi 5VRe8fZyXCbhUgS/rLBSVBOm7tDAHmYBVKHOIBVmkNuOBLPRfd+aoNnyLQra3lqd ON5zr5u0w+v8XsCkcvAeVcsrmOVNj6dIORWFZ223F6wSPY1QCjFd3ZPBcUPGbepQ REzKPJy3KCX4NCbvD+k0bb4F4/D3oJ2a+GOx209yNjEB6+iY47S4JcWqw07p+t0= =nlDi -----END PGP SIGNATURE----- --h31gzZEtNLTqOjlF--