From mboxrd@z Thu Jan 1 00:00:00 1970 From: Sergei Shtylyov Subject: Re: [RFC PATCH] sched/numa: do load balance between remote nodes Date: Wed, 06 Jun 2012 14:53:08 +0400 Message-ID: <4FCF3694.206@mvista.com> References: <1338965571-9812-1-git-send-email-alex.shi@intel.com> Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <1338965571-9812-1-git-send-email-alex.shi@intel.com> Sender: linux-sh-owner@vger.kernel.org List-ID: Content-Type: text/plain; charset="us-ascii"; format="flowed" To: Alex Shi Cc: a.p.zijlstra@chello.nl, anton@samba.org, benh@kernel.crashing.org, cmetcalf@tilera.com, dhowells@redhat.com, davem@davemloft.net, fenghua.yu@intel.com, hpa@zytor.com, ink@jurassic.park.msu.ru, linux-alpha@vger.kernel.org, linux-ia64@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mips@linux-mips.org, linuxppc-dev@lists.ozlabs.org, linux-sh@vger.kernel.org, mattst88@gmail.com, paulus@samba.org, lethal@linux-sh.org, ralf@linux-mips.org, rth@twiddle.net, sparclinux@vger.kernel.org, tony.luck@intel.com, x86@kernel.org, sivanich@sgi.com, greg.pearson@hp.com, kamezawa.hiroyu@jp.fujitsu.com, bob.picco@oracle.com, chris.mason@oracle.com, torvalds@linux-foundation.org, akpm@linux-foundation.org, mingo@kernel.org, pjt@google.com, tglx@linutronix.de, seto.hidetoshi@jp.fujitsu.com, ak@linux.intel.com, arjan.van.de.ven@intel.com Hello. On 06-06-2012 10:52, Alex Shi wrote: > commit cb83b629b Please also specify that commit's summary in parens. > remove the NODE sched domain and check if the node > distance in SLIT table is farther than REMOTE_DISTANCE, if so, it will > lose the load balance chance at exec/fork/wake_affine points. > But actually, even the node distance is farther than REMOTE_DISTANCE, > Modern CPUs also has QPI like connections, that make memory access is "Is" not needed here. > not too slow between nodes. So above losing on NUMA machine make a > huge performance regression on benchmark: hackbench, tbench, netperf > and oltp etc. > This patch will recover the scheduler behavior to old mode on all my > Intel platforms: NHM EP/EX, WSM EP, SNB EP/EP4S, and so remove the > perfromance regressions. (all of them just has 2 kinds distance, 10 21) > Signed-off-by: Alex Shi WBR, Sergei From mboxrd@z Thu Jan 1 00:00:00 1970 From: Sergei Shtylyov Date: Wed, 06 Jun 2012 10:53:08 +0000 Subject: Re: [RFC PATCH] sched/numa: do load balance between remote nodes Message-Id: <4FCF3694.206@mvista.com> List-Id: References: <1338965571-9812-1-git-send-email-alex.shi@intel.com> In-Reply-To: <1338965571-9812-1-git-send-email-alex.shi@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: Alex Shi Cc: a.p.zijlstra@chello.nl, anton@samba.org, benh@kernel.crashing.org, cmetcalf@tilera.com, dhowells@redhat.com, davem@davemloft.net, fenghua.yu@intel.com, hpa@zytor.com, ink@jurassic.park.msu.ru, linux-alpha@vger.kernel.org, linux-ia64@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mips@linux-mips.org, linuxppc-dev@lists.ozlabs.org, linux-sh@vger.kernel.org, mattst88@gmail.com, paulus@samba.org, lethal@linux-sh.org, ralf@linux-mips.org, rth@twiddle.net, sparclinux@vger.kernel.org, tony.luck@intel.com, x86@kernel.org, sivanich@sgi.com, greg.pearson@hp.com, kamezawa.hiroyu@jp.fujitsu.com, bob.picco@oracle.com, chris.mason@oracle.com, torvalds@linux-foundation.org, akpm@linux-foundation.org, mingo@kernel.org, pjt@google.com, tglx@linutronix.de, seto.hidetoshi@jp.fujitsu.com, ak@linux.intel.com, arjan.van.de.ven@intel.com Hello. On 06-06-2012 10:52, Alex Shi wrote: > commit cb83b629b Please also specify that commit's summary in parens. > remove the NODE sched domain and check if the node > distance in SLIT table is farther than REMOTE_DISTANCE, if so, it will > lose the load balance chance at exec/fork/wake_affine points. > But actually, even the node distance is farther than REMOTE_DISTANCE, > Modern CPUs also has QPI like connections, that make memory access is "Is" not needed here. > not too slow between nodes. So above losing on NUMA machine make a > huge performance regression on benchmark: hackbench, tbench, netperf > and oltp etc. > This patch will recover the scheduler behavior to old mode on all my > Intel platforms: NHM EP/EX, WSM EP, SNB EP/EP4S, and so remove the > perfromance regressions. (all of them just has 2 kinds distance, 10 21) > Signed-off-by: Alex Shi WBR, Sergei From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-lb0-f179.google.com (mail-lb0-f179.google.com [209.85.217.179]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority" (not verified)) by ozlabs.org (Postfix) with ESMTPS id 29B40B6FC4 for ; Wed, 6 Jun 2012 20:53:45 +1000 (EST) Received: by lban1 with SMTP id n1so4469279lba.38 for ; Wed, 06 Jun 2012 03:53:42 -0700 (PDT) Message-ID: <4FCF3694.206@mvista.com> Date: Wed, 06 Jun 2012 14:53:08 +0400 From: Sergei Shtylyov MIME-Version: 1.0 To: Alex Shi Subject: Re: [RFC PATCH] sched/numa: do load balance between remote nodes References: <1338965571-9812-1-git-send-email-alex.shi@intel.com> In-Reply-To: <1338965571-9812-1-git-send-email-alex.shi@intel.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Cc: linux-mips@linux-mips.org, linux-ia64@vger.kernel.org, linux-sh@vger.kernel.org, dhowells@redhat.com, paulus@samba.org, hpa@zytor.com, sparclinux@vger.kernel.org, mingo@kernel.org, sivanich@sgi.com, x86@kernel.org, greg.pearson@hp.com, chris.mason@oracle.com, arjan.van.de.ven@intel.com, mattst88@gmail.com, pjt@google.com, fenghua.yu@intel.com, seto.hidetoshi@jp.fujitsu.com, a.p.zijlstra@chello.nl, cmetcalf@tilera.com, ak@linux.intel.com, ink@jurassic.park.msu.ru, anton@samba.org, tglx@linutronix.de, kamezawa.hiroyu@jp.fujitsu.com, rth@twiddle.net, tony.luck@intel.com, torvalds@linux-foundation.org, linux-kernel@vger.kernel.org, ralf@linux-mips.org, lethal@linux-sh.org, linux-alpha@vger.kernel.org, bob.picco@oracle.com, akpm@linux-foundation.org, linuxppc-dev@lists.ozlabs.org, davem@davemloft.net List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Hello. On 06-06-2012 10:52, Alex Shi wrote: > commit cb83b629b Please also specify that commit's summary in parens. > remove the NODE sched domain and check if the node > distance in SLIT table is farther than REMOTE_DISTANCE, if so, it will > lose the load balance chance at exec/fork/wake_affine points. > But actually, even the node distance is farther than REMOTE_DISTANCE, > Modern CPUs also has QPI like connections, that make memory access is "Is" not needed here. > not too slow between nodes. So above losing on NUMA machine make a > huge performance regression on benchmark: hackbench, tbench, netperf > and oltp etc. > This patch will recover the scheduler behavior to old mode on all my > Intel platforms: NHM EP/EX, WSM EP, SNB EP/EP4S, and so remove the > perfromance regressions. (all of them just has 2 kinds distance, 10 21) > Signed-off-by: Alex Shi WBR, Sergei