From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754230AbbE1LyA (ORCPT ); Thu, 28 May 2015 07:54:00 -0400 Received: from mail-wi0-f182.google.com ([209.85.212.182]:38900 "EHLO mail-wi0-f182.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753956AbbE1Lxo (ORCPT ); Thu, 28 May 2015 07:53:44 -0400 Date: Thu, 28 May 2015 13:53:39 +0200 From: Ingo Molnar To: Josef Bacik Cc: riel@redhat.com, mingo@redhat.com, peterz@infradead.org, linux-kernel@vger.kernel.org, kernel-team@fb.com, Arnaldo Carvalho de Melo Subject: Re: [PATCH] sched: prefer an idle cpu vs an idle sibling for BALANCE_WAKE Message-ID: <20150528115339.GB29228@gmail.com> References: <1432675865-378571-1-git-send-email-jbacik@fb.com> <55662460.2050501@fb.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <55662460.2050501@fb.com> User-Agent: Mutt/1.5.23 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org * Josef Bacik wrote: > On 05/26/2015 05:31 PM, Josef Bacik wrote: > >At Facebook we have a pretty heavily multi-threaded application that is > >sensitive to latency. We have been pulling forward the old SD_WAKE_IDLE code > >because it gives us a pretty significant performance gain (like 20%). It turns > >out this is because there are cases where the scheduler puts our task on a busy > >CPU when there are idle CPU's in the system. We verify this by reading the > >cpu_delay_req_avg_us from the scheduler netlink stuff. With our crappy patch we > >get much lower numbers vs baseline. > > > >SD_BALANCE_WAKE is supposed to find us an idle cpu to run on, however it is just > >looking for an idle sibling, preferring affinity over all else. This is not > >helpful in all cases, and SD_BALANCE_WAKE's job is to find us an idle cpu, not > >garuntee affinity. Fix this by first trying to find an idle sibling, and then > >if the cpu is not idle fall through to the logic to find an idle cpu. With this > >patch we get slightly better performance than with our forward port of > >SD_WAKE_IDLE. Thanks, > > > > I rigged up a test script to run the perf bench sched tests and give me the > numbers. Here are the numbers > > 4.0 > > Messaging: 56.934 Total runtime in seconds > Pipe: 105620.762 ops/sec > > 4.0 + my patch > > Messaging: 47.374 > Pipe: 113691.199 Btw., with perf bench you don't really need much extra scripting, something like this should give you pretty good numbers plus an stddev estimate: perf stat --null --repeat 10 perf bench sched messaging -l 10000 on my box this gives: 4.391469643 seconds time elapsed ( +- 2.81% ) you can adjust the -l value to move the runtime up/down to a value that you think runs long enough to give stable results. Thanks, Ingo