From: Ingo Molnar <mingo@kernel.org>
To: Mel Gorman <mgorman@suse.de>
Cc: Rik van Riel <riel@redhat.com>,
Thomas Gleixner <tglx@linutronix.de>,
linux-kernel@vger.kernel.org, linux-mm@kvack.org,
Peter Zijlstra <a.p.zijlstra@chello.nl>,
Paul Turner <pjt@google.com>,
Lee Schermerhorn <Lee.Schermerhorn@hp.com>,
Christoph Lameter <cl@linux.com>,
Andrew Morton <akpm@linux-foundation.org>,
Andrea Arcangeli <aarcange@redhat.com>,
Linus Torvalds <torvalds@linux-foundation.org>,
Johannes Weiner <hannes@cmpxchg.org>,
Hugh Dickins <hughd@google.com>
Subject: Re: [GIT TREE] Unified NUMA balancing tree, v3
Date: Mon, 10 Dec 2012 21:07:55 +0100 [thread overview]
Message-ID: <20121210200755.GA15097@gmail.com> (raw)
In-Reply-To: <20121210192828.GL1009@suse.de>
* Mel Gorman <mgorman@suse.de> wrote:
> On Mon, Dec 10, 2012 at 08:15:45PM +0100, Ingo Molnar wrote:
> >
> > * Rik van Riel <riel@redhat.com> wrote:
> >
> > > On 12/10/2012 01:22 PM, Thomas Gleixner wrote:
> > >
> > > > So autonuma and numacore are basically on the same page,
> > > > with a slight advantage for numacore in the THP enabled
> > > > case. balancenuma is closer to mainline than to
> > > > autonuma/numacore.
> > >
> > > Indeed, when the system is fully loaded, numacore does very
> > > well.
> >
> > Note that the latest (-v3) code also does well in under-loaded
> > situations:
> >
> > http://lkml.org/lkml/2012/12/7/331
> >
> > Here's the 'perf bench numa' comparison to 'balancenuma':
> >
> > balancenuma | NUMA-tip
> > [test unit] : -v10 | -v3
> > ------------------------------------------------------------
> > 2x1-bw-process : 6.136 | 9.647: 57.2%
> > 3x1-bw-process : 7.250 | 14.528: 100.4%
> > 4x1-bw-process : 6.867 | 18.903: 175.3%
> > 8x1-bw-process : 7.974 | 26.829: 236.5%
> > 8x1-bw-process-NOTHP : 5.937 | 22.237: 274.5%
> > 16x1-bw-process : 5.592 | 29.294: 423.9%
> > 4x1-bw-thread : 13.598 | 19.290: 41.9%
> > 8x1-bw-thread : 16.356 | 26.391: 61.4%
> > 16x1-bw-thread : 24.608 | 29.557: 20.1%
> > 32x1-bw-thread : 25.477 | 30.232: 18.7%
> > 2x3-bw-thread : 8.785 | 15.327: 74.5%
> > 4x4-bw-thread : 6.366 | 27.957: 339.2%
> > 4x6-bw-thread : 6.287 | 27.877: 343.4%
> > 4x8-bw-thread : 5.860 | 28.439: 385.3%
> > 4x8-bw-thread-NOTHP : 6.167 | 25.067: 306.5%
> > 3x3-bw-thread : 8.235 | 21.560: 161.8%
> > 5x5-bw-thread : 5.762 | 26.081: 352.6%
> > 2x16-bw-thread : 5.920 | 23.269: 293.1%
> > 1x32-bw-thread : 5.828 | 18.985: 225.8%
> > numa02-bw : 29.054 | 31.431: 8.2%
> > numa02-bw-NOTHP : 27.064 | 29.104: 7.5%
> > numa01-bw-thread : 20.338 | 28.607: 40.7%
> > numa01-bw-thread-NOTHP : 18.528 | 21.119: 14.0%
> > ------------------------------------------------------------
> >
> > More than half of these testcases are under-loaded situations.
> >
> > > The main issues that have been observed with numacore are when
> > > the system is only partially loaded. Something strange seems
> > > to be going on that causes performance regressions in that
> > > situation.
> >
> > I haven't seen such reports with -v3 yet, which is what Thomas
> > tested. Mel has not tested -v3 yet AFAICS.
> >
>
> Yes, I have. The drop I took and the results I posted to you
> were based on a tip/master pull from December 9th. v3 was
> released on December 7th and your release said to test based
> on tip/master. The results are here
> https://lkml.org/lkml/2012/12/9/108 . Look at the columns
> marked numafix-20121209 which is tip/master with a bodge on
> top to remove the "if (p->nr_cpus_allowed !=
> num_online_cpus())" check.
Ah, indeed - I saw those results but the 'numafix' tag threw me
off.
Looks like at least in terms of AutoNUMA-benchmark numbers you
measured the best-ever results with the -v3 tree? That aspect is
obviously good news.
This part isn't:
> > If there are any such instances left then I'll investigate,
> > but right now it's looking pretty good.
>
> If you had read that report, you would know that I didn't have
> results for specjbb with THP enabled due to the JVM crashing
> with null pointer exceptions.
Hm, it's the unified tree where most of the mm/ bits are the
AutoNUMA bits from your tree. (It does not match 100%, because
your tree has an ancient version of key memory usage statistics
that the scheduler needs for its convergence model. I'll take a
look at the differences.)
Given how well the unified kernel performs, and given that the
segfaults occur on your box, would you be willing to debug this
a bit and help me out fixing the bug? Thanks!
Ingo
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2012-12-10 20:08 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-12-07 0:19 [GIT TREE] Unified NUMA balancing tree, v3 Ingo Molnar
2012-12-07 0:19 ` [PATCH 1/9] numa, sched: Fix NUMA tick ->numa_shared setting Ingo Molnar
2012-12-07 0:19 ` [PATCH 2/9] numa, sched: Add tracking of runnable NUMA tasks Ingo Molnar
2012-12-07 0:19 ` [PATCH 3/9] numa, sched: Implement wake-cpu migration support Ingo Molnar
2012-12-07 0:19 ` [PATCH 4/9] numa, mm, sched: Implement last-CPU+PID hash tracking Ingo Molnar
2012-12-07 0:19 ` [PATCH 5/9] numa, mm, sched: Fix NUMA affinity tracking logic Ingo Molnar
2012-12-07 0:19 ` [PATCH 6/9] numa, mm: Fix !THP, 4K-pte "2M-emu" NUMA fault handling Ingo Molnar
2012-12-07 0:19 ` [PATCH 7/9] numa, sched: Improve staggered convergence Ingo Molnar
2012-12-07 0:19 ` [PATCH 8/9] numa, sched: Improve directed convergence Ingo Molnar
2012-12-07 0:19 ` [PATCH 9/9] numa, sched: Streamline and fix numa_allow_migration() use Ingo Molnar
2012-12-10 18:22 ` [GIT TREE] Unified NUMA balancing tree, v3 Thomas Gleixner
2012-12-10 18:41 ` Rik van Riel
2012-12-10 19:15 ` Ingo Molnar
2012-12-10 19:28 ` Mel Gorman
2012-12-10 20:07 ` Ingo Molnar [this message]
2012-12-10 20:10 ` Ingo Molnar
2012-12-10 21:03 ` Ingo Molnar
2012-12-10 22:19 ` Mel Gorman
2012-12-10 19:32 ` Mel Gorman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20121210200755.GA15097@gmail.com \
--to=mingo@kernel.org \
--cc=Lee.Schermerhorn@hp.com \
--cc=a.p.zijlstra@chello.nl \
--cc=aarcange@redhat.com \
--cc=akpm@linux-foundation.org \
--cc=cl@linux.com \
--cc=hannes@cmpxchg.org \
--cc=hughd@google.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mgorman@suse.de \
--cc=pjt@google.com \
--cc=riel@redhat.com \
--cc=tglx@linutronix.de \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).