From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755100AbbFSRQr (ORCPT ); Fri, 19 Jun 2015 13:16:47 -0400 Received: from e28smtp07.in.ibm.com ([122.248.162.7]:47846 "EHLO e28smtp07.in.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753653AbbFSRQp (ORCPT ); Fri, 19 Jun 2015 13:16:45 -0400 X-Helo: d28dlp01.in.ibm.com X-MailFrom: srikar@linux.vnet.ibm.com X-RcptTo: linux-kernel@vger.kernel.org Date: Fri, 19 Jun 2015 22:46:33 +0530 From: Srikar Dronamraju To: Rik van Riel Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, mingo@kernel.org, mgorman@suse.de Subject: Re: [PATCH] sched,numa: document and fix numa_preferred_nid setting Message-ID: <20150619171633.GC16576@linux.vnet.ibm.com> Reply-To: Srikar Dronamraju References: <20150616155450.62ec234b@cuia.usersys.redhat.com> <20150618155547.GA16576@linux.vnet.ibm.com> <5582EC99.8040005@redhat.com> <20150618164140.GB16576@linux.vnet.ibm.com> <5582F944.6080204@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline In-Reply-To: <5582F944.6080204@redhat.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 15061917-0025-0000-0000-0000056AD418 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org > > OK, so we are looking at two multi-threaded processes > on a 4 node system, and waiting for them to converge? > > It may make sense to add my patch in with your patch > 1/4 from last week, as well as the correct part of > your patch 4/4, and see how they all work together. > Tested specjbb and autonumabenchmark on 4 kernels. Plain 4.1.0-rc7-tip (i) tip + only Rik's patch (ii) tip + Rik's ++ (iii) tip + Srikar's ++ (iv) (i) = Plain 4.1.0-rc7-tip = tip = 4.1.0-rc7 (b7ca96b) (ii) = tip + only Rik's patch = (i) + Rik's fix numa_preferred_nid setting (iii) = tip + Rik's ++ (iii) = (ii) + Srikar's numa hotness + correct nid for evaluating task weight (iv) = tip + Srikar's ++ (iv) = (i) + Srikar's numa hotness + correct nid for evaluating task weight + numa_has_capacity fix + always update preferred node Plain 4.1.0-rc7-tip (i) Testcase: Min Max Avg StdDev elapsed_numa01: 858.85 949.18 915.64 33.06 elapsed_numa02: 23.09 29.89 26.43 2.18 Testcase: Min Max Avg StdDev system_numa01: 1516.72 1855.08 1686.24 113.95 system_numa02: 63.69 79.06 70.35 5.87 Testcase: Min Max Avg StdDev user_numa01: 73284.76 80818.21 78060.88 2773.60 user_numa02: 1690.18 2071.07 1821.64 140.25 Testcase: Min Max Avg StdDev total_numa01: 74801.50 82572.60 79747.12 2875.61 total_numa02: 1753.87 2142.77 1891.99 143.59 tip + only Rik's patch (ii) Testcase: Min Max Avg StdDev %Change elapsed_numa01: 665.26 877.47 776.77 79.23 15.83% elapsed_numa02: 24.59 31.30 28.17 2.48 -5.56% Testcase: Min Max Avg StdDev %Change system_numa01: 659.57 1220.99 942.36 234.92 60.92% system_numa02: 44.62 86.01 64.64 14.24 6.64% Testcase: Min Max Avg StdDev %Change user_numa01: 56280.95 75908.81 64993.57 7764.30 17.21% user_numa02: 1790.35 2155.02 1916.12 132.57 -4.38% Testcase: Min Max Avg StdDev %Change total_numa01: 56940.50 77128.20 65935.92 7993.49 17.91% total_numa02: 1834.97 2227.03 1980.76 136.51 -3.99% tip + Rik's ++ (iii) Testcase: Min Max Avg StdDev %Change elapsed_numa01: 630.60 860.06 760.07 74.33 18.09% elapsed_numa02: 21.92 34.42 27.72 4.49 -3.75% Testcase: Min Max Avg StdDev %Change system_numa01: 474.31 1379.49 870.12 296.35 59.16% system_numa02: 63.74 120.25 86.69 20.69 -13.59% Testcase: Min Max Avg StdDev %Change user_numa01: 53004.03 68125.84 61697.01 5011.38 24.02% user_numa02: 1650.82 2278.71 1941.26 224.59 -5.25% Testcase: Min Max Avg StdDev %Change total_numa01: 53478.30 69505.30 62567.12 5288.18 24.72% total_numa02: 1714.56 2398.96 2027.95 238.08 -5.67% tip + Srikar's ++ (iv) Testcase: Min Max Avg StdDev %Change elapsed_numa01: 690.74 919.49 782.67 78.51 14.46% elapsed_numa02: 21.78 29.57 26.02 2.65 1.39% Testcase: Min Max Avg StdDev %Change system_numa01: 659.12 1041.19 870.15 143.13 78.38% system_numa02: 52.20 78.73 64.18 11.28 7.84% Testcase: Min Max Avg StdDev %Change user_numa01: 56410.39 71492.31 62514.78 5444.90 21.75% user_numa02: 1594.27 1934.40 1754.37 126.41 3.48% Testcase: Min Max Avg StdDev %Change total_numa01: 57069.50 72509.90 63384.94 5567.71 22.57% total_numa02: 1647.85 2010.87 1818.55 136.88 3.65% 5 interations of Specjbb on 4 node, 24 core powerpc machine. Ran 1 instance per system. For specjbb (higher bops per JVM is better) Plain 4.1.0-rc7-tip (i) Metric: Min Max Avg StdDev bopsperJVM: 265519.00 272466.00 269377.80 2391.04 tip + only Rik's patch (ii) Metric: Min Max Avg StdDev %Change bopsperJVM: 263393.00 269660.00 266920.20 2792.07 -0.91% tip + Rik's ++ (iii) Metric: Min Max Avg StdDev %Change bopsperJVM: 264298.00 271236.00 266818.20 2579.62 -0.94% tip + Srikar's ++ (iv) Metric: Min Max Avg StdDev %Change bopsperJVM: 266774.00 272434.00 269839.60 2083.19 +0.17% So fix for numa_has_capacity and always setting preferred node based on fault stats seems to help autonuma and specjbb. -- Thanks and Regards Srikar Dronamraju -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in Please read the FAQ at http://www.tux.org/lkml/