From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751217AbdGaRXu (ORCPT ); Mon, 31 Jul 2017 13:23:50 -0400 Received: from mail-wr0-f171.google.com ([209.85.128.171]:37875 "EHLO mail-wr0-f171.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750940AbdGaRXs (ORCPT ); Mon, 31 Jul 2017 13:23:48 -0400 Message-ID: <1501521826.5348.12.camel@gmail.com> Subject: Re: wake_wide mechanism clarification From: Mike Galbraith To: Josef Bacik Cc: Joel Fernandes , Peter Zijlstra , LKML , Juri Lelli , Dietmar Eggemann , Patrick Bellasi , Brendan Jackman , Chris Redpath , Michael Wang , Matt Fleming Date: Mon, 31 Jul 2017 19:23:46 +0200 In-Reply-To: <20170731144806.GA7791@li70-116.members.linode.com> References: <20170630142815.GA9743@destiny> <1498842140.15161.66.camel@gmail.com> <1501340845.7706.168.camel@gmail.com> <20170731122149.GA7539@li70-116.members.linode.com> <1501508545.6867.32.camel@gmail.com> <20170731144806.GA7791@li70-116.members.linode.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.20.5 Mime-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, 2017-07-31 at 14:48 +0000, Josef Bacik wrote: > On Mon, Jul 31, 2017 at 03:42:25PM +0200, Mike Galbraith wrote: > > On Mon, 2017-07-31 at 12:21 +0000, Josef Bacik wrote: > > > > > > I've been working in this area recently because of a cpu imbalance problem. > > > Wake_wide() definitely makes it so we're waking affine way too often, but I > > > think messing with wake_waide to solve that problem is the wrong solution. This > > > is just a heuristic to see if we should wake affine, the simpler the better. I > > > solved the problem of waking affine too often like this > > > > > > https://marc.info/?l=linux-kernel&m=150003849602535&w=2 > > > > Wait a minute, that's not quite fair :)  Wake_wide() can't be blamed > > for causing too frequent affine wakeups when what it does is filter > > some out.  While it may not reject aggressively enough for you (why you > > bent it up to be very aggressive), seems the problem from your loads > > POV is the scheduler generally being too eager to bounce. > > > > Yeah sorry, I hate this stuff because it's so hard to talk about without mixing > up different ideas. I should say the scheduler in general prefers to wake > affine super hard, and wake_wide() is conservative in it's filtering of this > behavior. The rest still holds true, I think tinkering with it is just hard and > the wrong place to do it, it's a good first step, and we can be smarter further > down. Yeah, it's hard, and yeah, bottom line remains unchanged. > > I've also played with rate limiting migration per task, but it had > > negative effects too: when idle/periodic balance pulls buddies apart, > > rate limiting inhibits them quickly finding each other again, making > > undoing all that hard load balancer work a throughput win.  Sigh. > > > > That's why I did the HZ thing, we don't touch the task for HZ to let things > settle out, and then allow affine wakeups after that. I kinda like the way you did it better than what I tried, but until a means exists to _target_ the win, it's gonna be rob Peter to pay Paul, swap rolls, repeat endlessly. -Mike