From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932520Ab2KVUtz (ORCPT ); Thu, 22 Nov 2012 15:49:55 -0500 Received: from mail-ee0-f46.google.com ([74.125.83.46]:48082 "EHLO mail-ee0-f46.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756768Ab2KVUtu (ORCPT ); Thu, 22 Nov 2012 15:49:50 -0500 Date: Wed, 21 Nov 2012 23:46:53 +0100 From: Ingo Molnar To: Linus Torvalds Cc: David Rientjes , Mel Gorman , Linux Kernel Mailing List , linux-mm , Peter Zijlstra , Paul Turner , Lee Schermerhorn , Christoph Lameter , Rik van Riel , Andrew Morton , Andrea Arcangeli , Thomas Gleixner , Johannes Weiner , Hugh Dickins Subject: Re: [PATCH 00/27] Latest numa/core release, v16 Message-ID: <20121121224653.GA4164@gmail.com> References: <20121120060014.GA14065@gmail.com> <20121120074445.GA14539@gmail.com> <20121120090637.GA14873@gmail.com> <20121121171047.GA28875@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org * Linus Torvalds wrote: > On Wed, Nov 21, 2012 at 7:10 AM, Ingo Molnar wrote: > > > > Because scalability slowdowns are often non-linear. > > Only if you hold locks or have other non-cpu-private activity. > > Which the vsyscall code really shouldn't have. Yeah, the faults accessing any sort of thread shared cache line was my main thinking - the vsyscall faults are so hidden, and David's transaction score was so low that I could not exclude some extremely high page fault rate (which would not get reported by anything other than a strange blip on the profile). I was thinking of a hundred thousand vsyscall page faults per second as a possibility - SPECjbb measures time for every transaction. So this was just a "maybe-that-has-an-effect" blind theory of mine - and David's testing did not confirm it so we know it was a bad idea. I basically wanted to see a profile from David that looked as flat as mine - that would have excluded a handful of unknown unknowns. > That said, it might be worth removing the > "prefetchw(&mm->mmap_sem)" from the VM fault path. Partly > because software prefetches have never ever worked on any > reasonable hardware, and partly because it could seriously > screw up things like the vsyscall stuff. Yeah, I was wondering about that one too ... > I think we only turn prefetchw into an actual prefetch > instruction on 3DNOW hardware. Which is the *old* AMD chips. I > don't think even the Athlon does that. > > Anyway, it might be interesting to see a instruction-level > annotated profile of do_page_fault() or whatever Yes. > > So with CONFIG_NUMA_BALANCING=y we are taking a higher page > > fault rate, in exchange for a speedup. > > The thing is, so is autonuma. > > And autonuma doesn't show any of these problems. [...] AutoNUMA regresses on this workload, at least on my box: v3.7 AutoNUMA | numa/core-v16 [ vs. v3.7] ----- -------- | ------------- ----------- | [ SPECjbb transactions/sec ] | [ higher is better ] | | SPECjbb single-1x32 524k 507k | 638k +21.7% It regresses by 3.3% over mainline. [I have not measured a THP-disabled number for AutoNUMA.] Maybe it does not regress on David's box - I have just re-checked all of David's mails and AFAICS he has not reported AutoNUMA SPECjbb performance. > Why are you ignoring that fact? I'm not :-( Thanks, Ingo