From mboxrd@z Thu Jan 1 00:00:00 1970 From: Madhavan Srinivasan Subject: Re: [PATCH V4 0/2] mm: FAULT_AROUND_ORDER patchset performance data for powerpc Date: Tue, 20 May 2014 07:36:42 +0530 Message-ID: <537AB8B2.3040000@linux.vnet.ibm.com> References: <1399541296-18810-1-git-send-email-maddy@linux.vnet.ibm.com> <537479E7.90806@linux.vnet.ibm.com> <87wqdik4n5.fsf@rustcorp.com.au> <53797511.1050409@linux.vnet.ibm.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: linuxppc-dev-bounces+glppd-linuxppc64-dev=m.gmane.org@lists.ozlabs.org Sender: "Linuxppc-dev" To: Hugh Dickins Cc: linux-arch@vger.kernel.org, riel@redhat.com, x86@kernel.org, dave.hansen@intel.com, peterz@infradead.org, Rusty Russell , linux-kernel@vger.kernel.org, linux-mm@kvack.org, ak@linux.intel.com, paulus@samba.org, mgorman@suse.de, akpm@linux-foundation.org, linuxppc-dev@lists.ozlabs.org, mingo@kernel.org, "Kirill A. Shutemov" List-Id: linux-arch.vger.kernel.org T24gVHVlc2RheSAyMCBNYXkgMjAxNCAwNDo1MyBBTSwgSHVnaCBEaWNraW5zIHdyb3RlOgo+IE9u IE1vbiwgMTkgTWF5IDIwMTQsIE1hZGhhdmFuIFNyaW5pdmFzYW4gd3JvdGU6Cj4+IE9uIE1vbmRh eSAxOSBNYXkgMjAxNCAwNTo0MiBBTSwgUnVzdHkgUnVzc2VsbCB3cm90ZToKPj4+IEh1Z2ggRGlj a2lucyA8aHVnaGRAZ29vZ2xlLmNvbT4gd3JpdGVzOgo+Pj4+IE9uIFRodSwgMTUgTWF5IDIwMTQs IE1hZGhhdmFuIFNyaW5pdmFzYW4gd3JvdGU6Cj4+Pj4+Cj4+Pj4+IEhpIEluZ28sCj4+Pj4+Cj4+ Pj4+IAlEbyB5b3UgaGF2ZSBhbnkgY29tbWVudHMgZm9yIHRoZSBsYXRlc3QgdmVyc2lvbiBvZiB0 aGUgcGF0Y2hzZXQuIElmCj4+Pj4+IG5vdCwga2luZGx5IGNhbiB5b3UgcGljayBpdCB1cCBhcyBp cy4KPj4+Pj4KPj4+Pj4KPj4+Pj4gV2l0aCByZWdhcmRzCj4+Pj4+IE1hZGR5Cj4+Pj4+Cj4+Pj4+ PiBLaXJpbGwgQS4gU2h1dGVtb3Ygd2l0aCA4YzZlNTBiMDI5IGNvbW1pdCBpbnRyb2R1Y2VkCj4+ Pj4+PiB2bV9vcHMtPm1hcF9wYWdlcygpIGZvciBtYXBwaW5nIGVhc3kgYWNjZXNzaWJsZSBwYWdl cyBhcm91bmQKPj4+Pj4+IGZhdWx0IGFkZHJlc3MgaW4gaG9wZSB0byByZWR1Y2UgbnVtYmVyIG9m IG1pbm9yIHBhZ2UgZmF1bHRzLgo+Pj4+Pj4KPj4+Pj4+IFRoaXMgcGF0Y2ggY3JlYXRlcyBpbmZy YXN0cnVjdHVyZSB0byBtb2RpZnkgdGhlIEZBVUxUX0FST1VORF9PUkRFUgo+Pj4+Pj4gdmFsdWUg dXNpbmcgbW0vS2NvbmZpZy4gVGhpcyB3aWxsIGVuYWJsZSBhcmNoaXRlY3R1cmUgbWFpbnRhaW5l cnMKPj4+Pj4+IHRvIGRlY2lkZSBvbiBzdWl0YWJsZSBGQVVMVF9BUk9VTkRfT1JERVIgdmFsdWUg YmFzZWQgb24KPj4+Pj4+IHBlcmZvcm1hbmNlIGRhdGEgZm9yIHRoYXQgYXJjaGl0ZWN0dXJlLiBG aXJzdCBwYXRjaCBhbHNvIGRlZmF1bHRzCj4+Pj4+PiBGQVVMVF9BUk9VTkRfT1JERVIgS2NvbmZp ZyBlbGVtZW50IHRvIDQuIFNlY29uZCBwYXRjaCBsaXN0Cj4+Pj4+PiBvdXQgdGhlIHBlcmZvcm1h bmNlIG51bWJlcnMgZm9yIHBvd2VycGMgKHBsYXRmb3JtIHBzZXJpZXMpIGFuZAo+Pj4+Pj4gaW5p dGlhbGl6ZSB0aGUgZmF1bHQgYXJvdW5kIG9yZGVyIHZhcmlhYmxlIGZvciBwc2VyaWVzIHBsYXRm b3JtIG9mCj4+Pj4+PiBwb3dlcnBjLgo+Pj4+Cj4+Pj4gU29ycnkgZm9yIG5vdCBjb21tZW50aW5n IGVhcmxpZXIgLSBqdXN0IHJlbWluZGVkIGJ5IHRoaXMgcGluZyB0byBJbmdvLgo+Pj4+Cj4+Pj4g SSBkaWRuJ3Qgc3R1ZHkgeW91ciBudW1iZXJzLCBidXQgbm93aGVyZSBkaWQgSSBzZWUgd2hhdCBQ QUdFX1NJWkUgeW91IHVzZS4KPj4+Pgo+Pj4+IGFyY2gvcG93ZXJwYy9LY29uZmlnIHN1Z2dlc3Rz IHRoYXQgUG93ZXIgc3VwcG9ydHMgYmFzZSBwYWdlIHNpemUgb2YKPj4+PiA0aywgMTZrLCA2NGsg b3IgMjU2ay4KPj4+Pgo+Pj4+IEkgd291bGQgZXhwZWN0IHlvdXIgb3B0aW1hbCBmYXVsdF9hcm91 bmRfb3JkZXIgdG8gZGVwZW5kIHZlcnkgbXVjaCBvbgo+Pj4+IHRoZSBiYXNlIHBhZ2Ugc2l6ZS4K Pj4+Cj4+PiBJdCB3YXMgNjRrLCB3aGljaCBpcyB3aGF0IFBQQzY0IHVzZXMgb24gYWxsIHRoZSBt YWpvciBkaXN0cmlidXRpb25zLgo+Pj4gWW91IHJlYWxseSBvbmx5IGdldCBhIGNob2ljZSBvZiA0 ayBhbmQgNjRrIHdpdGggNjQgYml0IHBvd2VyLgo+Pj4KPj4gVGhpcyBpcyB0cnVlLiBQUEM2NCBz dXBwb3J0IG11bHRpcGxlIHBhZ2VzaXplIGFuZCB5ZXMgdGhlIGRlZmF1bHQgcGFnZQo+PiBzaXpl IG9mIDY0aywgaXMgdGFrZW4gYXMgYmFzZSBwYWdlc2l6ZSBmb3IgdGhlIHRlc3RzLgo+Pgo+Pj4+ IFBlcmhhcHMgZmF1bHRfYXJvdW5kX3NpemUgd291bGQgcHJvdmlkZSBhIG1vcmUgdXNlZnVsIGRl ZmF1bHQ/Cj4+Pgo+Pj4gVGhhdCBzZWVtcyB0byBmaXQuICBXaXRoIDRrIHBhZ2VzIGFuZCBvcmRl ciA0LCB5b3UncmUgYXNraW5nIGZvciA2NGsuCj4+PiBNYWRkeSdzIHJlc3VsdCBzaG93cyA2NGsg aXMgYWxzbyByZWFzb25hYmxlIGZvciA2NGsgcGFnZXMuCj4+Pgo+Pj4gUGVyaGFwcyB3ZSB0cnkg dG8gZ2VuZXJhbGl6ZSBmcm9tIHR3byBkYXRhIHBvaW50cyAoYSBzbGlnaHQgaW1wcm92ZW1lbnQK Pj4+IG92ZXIgZG9pbmcgaXQgZnJvbSAxISksIGVnOgo+Pj4KPj4+IC8qIDQgc2VlbXMgZ29vZCBm b3IgNGstcGFnZSB4ODYsIDAgc2VlbXMgZ29vZCBmb3IgNjRrIHBhZ2UgcHBjNjQsIHNvOiAqLwo+ Pj4gdW5zaWduZWQgaW50IGZhdWx0X2Fyb3VuZF9vcmRlciBfX3JlYWRfbW9zdGx5ID0KPj4+ICAg ICAgICAgKDE2IC0gUEFHRV9TSElGVCA8IDAgPyAwIDogMTYgLSBQQUdFX1NISUZUKTsKPiAKPiBS dXN0eSdzIGJpbW9kYWwgYW5zd2VyIGRvZXNuJ3Qgc2VlbSB0aGUgcmlnaHQgc3RhcnRpbmcgcG9p bnQgdG8gbWUuCj4gCj4gU2hvdWxkbid0IEZBVUxUX0FST1VORF9PUkRFUiBhbmQgZmF1bHRfYXJv dW5kX29yZGVyIGJlIGNoYW5nZWQgdG8gYmUKPiB0aGUgb3JkZXIgb2YgdGhlIGZhdWx0LWFyb3Vu ZCBzaXplIGluIGJ5dGVzLCBhbmQgZmF1bHRfYXJvdW5kX3BhZ2VzKCkKPiB1c2UgMVVMIDw8IChm YXVsdF9hcm91bmRfb3JkZXIgLSBQQUdFX1NISUZUKQo+IC0gd2hlbiB0aGF0IGRvZXNuJ3Qgd3Jh cCwgb2YgY291cnNlIQo+IAo+IFRoYXQgd291bGQgYXQgbGVhc3QgaGF2ZSBhIGJldHRlciBjaGFu Y2Ugb2YgYmVpbmcgYXBwcm9wcmlhdGUgZm9yCj4gYXJjaGl0ZWN0dXJlcyB3aXRoIDhrIGFuZCAx NmsgcGFnZXMgKEl0YW5pdW0gc3ByaW5ncyB0byBtaW5kKS4KPiAKPiBOb3QgbmVjZXNzYXJpbHkg cmlnaHQgZm9yIHRoZW0sIHNpbmNlIGVhY2ggYXJjaGl0ZWN0dXJlIG1heSBoYXZlCj4gZGlmZmVy ZW50IGZhdWx0aW5nIG92ZXJoZWFkczsgYnV0IGEgYmV0dGVyIGNoYW5jZSBvZiBiZWluZyByaWdo dAo+IHRoYW4gYmxpbmRseSBhc3N1bWluZyA0ayBvciA2NGsgcGFnZXMgZm9yIGV2ZXJ5b25lLgo+ IAo+IEknZCBiZSBnbGFkIHRvIHNlZSB0aGF0IGNoYW5nZSBnbyBpbnRvIHYzLjE1OiB3aGF0IGRv IHlvdSB0aGluaywKPiBLaXJpbGwsIGFyZSB3ZSB0b28gbGF0ZSB0byBtYWtlIHN1Y2ggYSBjaGFu Z2Ugbm93Pwo+IE9yIGRvIHlvdSBzZWUgc29tZSBvYmplY3Rpb24gdG8gaXQ/Cj4gCj4+IFRoaXMg bWF5IGJlIHJpZ2h0LiBCdXQgdGhlc2UgYXJlIHRoZSBjb25jZXJucywgd2lsbCBub3QgdGhpcyBt YWtlIG90aGVyCj4+IGFyY2ggdG8gcGljayBkZWZhdWx0IHdpdGhvdXQgYW55IHR1bmluZwo+IAo+ IFdhc24ndCBGQVVMVF9BUk9VTkRfT1JERVIgNCBjaG9zZW4gc29sZWx5IG9uIHRoZSBiYXNpcyBv ZiB4ODYgNGsgcGFnZXM/Cj4gRGlkIG90aGVyIGFyY2hpdGVjdHVyZXMsIHdpdGggb3RoZXIgcGFn ZSBzaXplcywgYmFjayB0aGF0IGRlZmF1bHQ/Cj4gQ2xlYXJseSBub3QgcG93ZXJwYy4KCk9rLgoK PiAKPj4gYW5kIGFsc28gdGhpcyB3aWxsIHJlbW92ZSB0aGUKPj4gY29tcGlsZSB0aW1lIG9wdGlv biB0byBkaXNhYmxlIHRoZSBmZWF0dXJlPwo+IAo+IENvbXBpbGUgdGltZSBvcHRpb24gbWVhbmlu ZyB5b3VyIEZBVUxUX0FST1VORF9PUkRFUiBpbiBtbS9LY29uZmlnCj4gZm9yIHYzLjE2Pwo+IAo+ IEknbSBub3Qgc3VyZSB3aGV0aGVyIFJ1c3R5IHdhcyBhcmd1aW5nIGFnYWluc3QgdGhhdCBvciBu b3QgSSB0aGluawoKPiB3ZSBhcmUgYWxsIHRocmVlIGNvbmNlcm5lZCB0byBoYXZlIGEgbW9yZSBz ZW5zaWJsZSBkZWZhdWx0IHRoYW4gd2hhdCdzCj4gdGhlcmUgYXQgcHJlc2VudC4gIEkgZG9uJ3Qg ZmVlbCB2ZXJ5IHN0cm9uZ2x5IGFib3V0IHlvdXIgS2NvbmZpZwoKQWRkZWQgaXQgYXMgb25lIHdh eSB0byByZXNldCBvciBkaXNhYmxlIHRoZSBkZWZhdWx0IHZhbHVlLiBCdXQgdGhlbiBJCmd1ZXNz IHdlIGRlY2lkZWQgb24gaGF2aW5nIEZBVUxUX0FST1VORF9PUkRFUiBhcyBhIHZhcmlhYmxlIHdo aWNoIGlzCm1vcmUgaW1wb3J0YW50IHRoYW4gS2NvbmZpZyBvcHRpb24uCgo+IG9wdGlvbjogSSd2 ZSBubyBvYmplY3Rpb24sIGlmIGl0IHdlcmUgdG8gZGVmYXVsdCB0byBieXRlIG9yZGVyIDE2Lgo+ IAoKVGhhbmtzIGZvciByZXZpZXcKV2l0aCByZWdhcmRzCk1hZGR5Cgo+IEh1Z2gKPiAKCl9fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCkxpbnV4cHBjLWRldiBt YWlsaW5nIGxpc3QKTGludXhwcGMtZGV2QGxpc3RzLm96bGFicy5vcmcKaHR0cHM6Ly9saXN0cy5v emxhYnMub3JnL2xpc3RpbmZvL2xpbnV4cHBjLWRldg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from e23smtp06.au.ibm.com ([202.81.31.148]:32894 "EHLO e23smtp06.au.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752500AbaETCHD (ORCPT ); Mon, 19 May 2014 22:07:03 -0400 Received: from /spool/local by e23smtp06.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 20 May 2014 12:07:00 +1000 Message-ID: <537AB8B2.3040000@linux.vnet.ibm.com> Date: Tue, 20 May 2014 07:36:42 +0530 From: Madhavan Srinivasan MIME-Version: 1.0 Subject: Re: [PATCH V4 0/2] mm: FAULT_AROUND_ORDER patchset performance data for powerpc References: <1399541296-18810-1-git-send-email-maddy@linux.vnet.ibm.com> <537479E7.90806@linux.vnet.ibm.com> <87wqdik4n5.fsf@rustcorp.com.au> <53797511.1050409@linux.vnet.ibm.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Sender: linux-arch-owner@vger.kernel.org List-ID: To: Hugh Dickins Cc: Rusty Russell , "Kirill A. Shutemov" , linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, x86@kernel.org, benh@kernel.crashing.org, paulus@samba.org, akpm@linux-foundation.org, riel@redhat.com, mgorman@suse.de, ak@linux.intel.com, peterz@infradead.org, mingo@kernel.org, dave.hansen@intel.com Message-ID: <20140520020642.4sTPpIZmtzUjfnNBe6X9aKWxofLKliFR66Qc10xpWuk@z> On Tuesday 20 May 2014 04:53 AM, Hugh Dickins wrote: > On Mon, 19 May 2014, Madhavan Srinivasan wrote: >> On Monday 19 May 2014 05:42 AM, Rusty Russell wrote: >>> Hugh Dickins writes: >>>> On Thu, 15 May 2014, Madhavan Srinivasan wrote: >>>>> >>>>> Hi Ingo, >>>>> >>>>> Do you have any comments for the latest version of the patchset. If >>>>> not, kindly can you pick it up as is. >>>>> >>>>> >>>>> With regards >>>>> Maddy >>>>> >>>>>> Kirill A. Shutemov with 8c6e50b029 commit introduced >>>>>> vm_ops->map_pages() for mapping easy accessible pages around >>>>>> fault address in hope to reduce number of minor page faults. >>>>>> >>>>>> This patch creates infrastructure to modify the FAULT_AROUND_ORDER >>>>>> value using mm/Kconfig. This will enable architecture maintainers >>>>>> to decide on suitable FAULT_AROUND_ORDER value based on >>>>>> performance data for that architecture. First patch also defaults >>>>>> FAULT_AROUND_ORDER Kconfig element to 4. Second patch list >>>>>> out the performance numbers for powerpc (platform pseries) and >>>>>> initialize the fault around order variable for pseries platform of >>>>>> powerpc. >>>> >>>> Sorry for not commenting earlier - just reminded by this ping to Ingo. >>>> >>>> I didn't study your numbers, but nowhere did I see what PAGE_SIZE you use. >>>> >>>> arch/powerpc/Kconfig suggests that Power supports base page size of >>>> 4k, 16k, 64k or 256k. >>>> >>>> I would expect your optimal fault_around_order to depend very much on >>>> the base page size. >>> >>> It was 64k, which is what PPC64 uses on all the major distributions. >>> You really only get a choice of 4k and 64k with 64 bit power. >>> >> This is true. PPC64 support multiple pagesize and yes the default page >> size of 64k, is taken as base pagesize for the tests. >> >>>> Perhaps fault_around_size would provide a more useful default? >>> >>> That seems to fit. With 4k pages and order 4, you're asking for 64k. >>> Maddy's result shows 64k is also reasonable for 64k pages. >>> >>> Perhaps we try to generalize from two data points (a slight improvement >>> over doing it from 1!), eg: >>> >>> /* 4 seems good for 4k-page x86, 0 seems good for 64k page ppc64, so: */ >>> unsigned int fault_around_order __read_mostly = >>> (16 - PAGE_SHIFT < 0 ? 0 : 16 - PAGE_SHIFT); > > Rusty's bimodal answer doesn't seem the right starting point to me. > > Shouldn't FAULT_AROUND_ORDER and fault_around_order be changed to be > the order of the fault-around size in bytes, and fault_around_pages() > use 1UL << (fault_around_order - PAGE_SHIFT) > - when that doesn't wrap, of course! > > That would at least have a better chance of being appropriate for > architectures with 8k and 16k pages (Itanium springs to mind). > > Not necessarily right for them, since each architecture may have > different faulting overheads; but a better chance of being right > than blindly assuming 4k or 64k pages for everyone. > > I'd be glad to see that change go into v3.15: what do you think, > Kirill, are we too late to make such a change now? > Or do you see some objection to it? > >> This may be right. But these are the concerns, will not this make other >> arch to pick default without any tuning > > Wasn't FAULT_AROUND_ORDER 4 chosen solely on the basis of x86 4k pages? > Did other architectures, with other page sizes, back that default? > Clearly not powerpc. Ok. > >> and also this will remove the >> compile time option to disable the feature? > > Compile time option meaning your FAULT_AROUND_ORDER in mm/Kconfig > for v3.16? > > I'm not sure whether Rusty was arguing against that or not I think > we are all three concerned to have a more sensible default than what's > there at present. I don't feel very strongly about your Kconfig Added it as one way to reset or disable the default value. But then I guess we decided on having FAULT_AROUND_ORDER as a variable which is more important than Kconfig option. > option: I've no objection, if it were to default to byte order 16. > Thanks for review With regards Maddy > Hugh >