From: Mel Gorman <mgorman@suse.de>
To: riel@redhat.com
Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org,
peterz@infradead.org, mingo@redhat.com, chegu_vinod@hp.com
Subject: Re: [PATCH 6/9] numa,sched: normalize faults_cpu stats and weigh by CPU use
Date: Tue, 28 Jan 2014 10:01:31 +0000 [thread overview]
Message-ID: <20140128100131.GS4963@suse.de> (raw)
In-Reply-To: <1390860228-21539-7-git-send-email-riel@redhat.com>
On Mon, Jan 27, 2014 at 05:03:45PM -0500, riel@redhat.com wrote:
> From: Rik van Riel <riel@redhat.com>
>
> Tracing the code that decides the active nodes has made it abundantly clear
> that the naive implementation of the faults_from code has issues.
>
> Specifically, the garbage collector in some workloads will access orders
> of magnitudes more memory than the threads that do all the active work.
> This resulted in the node with the garbage collector being marked the only
> active node in the group.
>
> This issue is avoided if we weigh the statistics by CPU use of each task in
> the numa group, instead of by how many faults each thread has occurred.
>
> To achieve this, we normalize the number of faults to the fraction of faults
> that occurred on each node, and then multiply that fraction by the fraction
> of CPU time the task has used since the last time task_numa_placement was
> invoked.
>
> This way the nodes in the active node mask will be the ones where the tasks
> from the numa group are most actively running, and the influence of eg. the
> garbage collector and other do-little threads is properly minimized.
>
> On a 4 node system, using CPU use statistics calculated over a longer interval
> results in about 1% fewer page migrations with two 32-warehouse specjbb runs
> on a 4 node system, and about 5% fewer page migrations, as well as 1% better
> throughput, with two 8-warehouse specjbb runs, as compared with the shorter
> term statistics kept by the scheduler.
>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Mel Gorman <mgorman@suse.de>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: Chegu Vinod <chegu_vinod@hp.com>
> Signed-off-by: Rik van Riel <riel@redhat.com>
Major changes are related to the weight calculations to avoid overflow
and the avg runtime is calculated based on a longer runtime than the v4
version. Both seem sane so
Acked-by: Mel Gorman <mgorman@suse>
--
Mel Gorman
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
WARNING: multiple messages have this Message-ID (diff)
From: Mel Gorman <mgorman@suse.de>
To: riel@redhat.com
Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org,
peterz@infradead.org, mingo@redhat.com, chegu_vinod@hp.com
Subject: Re: [PATCH 6/9] numa,sched: normalize faults_cpu stats and weigh by CPU use
Date: Tue, 28 Jan 2014 10:01:31 +0000 [thread overview]
Message-ID: <20140128100131.GS4963@suse.de> (raw)
In-Reply-To: <1390860228-21539-7-git-send-email-riel@redhat.com>
On Mon, Jan 27, 2014 at 05:03:45PM -0500, riel@redhat.com wrote:
> From: Rik van Riel <riel@redhat.com>
>
> Tracing the code that decides the active nodes has made it abundantly clear
> that the naive implementation of the faults_from code has issues.
>
> Specifically, the garbage collector in some workloads will access orders
> of magnitudes more memory than the threads that do all the active work.
> This resulted in the node with the garbage collector being marked the only
> active node in the group.
>
> This issue is avoided if we weigh the statistics by CPU use of each task in
> the numa group, instead of by how many faults each thread has occurred.
>
> To achieve this, we normalize the number of faults to the fraction of faults
> that occurred on each node, and then multiply that fraction by the fraction
> of CPU time the task has used since the last time task_numa_placement was
> invoked.
>
> This way the nodes in the active node mask will be the ones where the tasks
> from the numa group are most actively running, and the influence of eg. the
> garbage collector and other do-little threads is properly minimized.
>
> On a 4 node system, using CPU use statistics calculated over a longer interval
> results in about 1% fewer page migrations with two 32-warehouse specjbb runs
> on a 4 node system, and about 5% fewer page migrations, as well as 1% better
> throughput, with two 8-warehouse specjbb runs, as compared with the shorter
> term statistics kept by the scheduler.
>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Mel Gorman <mgorman@suse.de>
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: Chegu Vinod <chegu_vinod@hp.com>
> Signed-off-by: Rik van Riel <riel@redhat.com>
Major changes are related to the weight calculations to avoid overflow
and the avg runtime is calculated based on a longer runtime than the v4
version. Both seem sane so
Acked-by: Mel Gorman <mgorman@suse>
--
Mel Gorman
SUSE Labs
next prev parent reply other threads:[~2014-01-28 10:01 UTC|newest]
Thread overview: 37+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-01-27 22:03 [PATCH v5 0/9] numa,sched,mm: pseudo-interleaving for automatic NUMA balancing riel
2014-01-27 22:03 ` riel
2014-01-27 22:03 ` [PATCH 1/9] numa,sched,mm: remove p->numa_migrate_deferred riel
2014-01-27 22:03 ` riel
2014-01-28 19:23 ` [tip:sched/numa] sched/numa, mm: Remove p->numa_migrate_deferred tip-bot for Rik van Riel
2014-01-27 22:03 ` [PATCH 2/9] rename p->numa_faults to numa_faults_memory riel
2014-01-27 22:03 ` riel
2014-01-28 19:23 ` [tip:sched/numa] sched/numa: Rename p-> numa_faults " tip-bot for Rik van Riel
2014-01-27 22:03 ` [PATCH 3/9] numa,sched: track from which nodes NUMA faults are triggered riel
2014-01-27 22:03 ` riel
2014-01-28 19:23 ` [tip:sched/numa] sched/numa: Track " tip-bot for Rik van Riel
2014-01-27 22:03 ` [PATCH 4/9] numa,sched: build per numa_group active node mask from numa_faults_cpu statistics riel
2014-01-27 22:03 ` riel
2014-01-28 19:24 ` [tip:sched/numa] sched/numa: Build " tip-bot for Rik van Riel
2014-01-27 22:03 ` [PATCH 5/9] numa,sched,mm: use active_nodes nodemask to limit numa migrations riel
2014-01-27 22:03 ` riel
2014-01-28 9:58 ` Mel Gorman
2014-01-28 9:58 ` Mel Gorman
2014-01-28 19:24 ` [tip:sched/numa] sched/numa, mm: Use " tip-bot for Rik van Riel
2014-01-27 22:03 ` [PATCH 6/9] numa,sched: normalize faults_cpu stats and weigh by CPU use riel
2014-01-27 22:03 ` riel
2014-01-28 10:01 ` Mel Gorman [this message]
2014-01-28 10:01 ` Mel Gorman
2014-01-28 19:24 ` [tip:sched/numa] sched/numa: Normalize " tip-bot for Rik van Riel
2014-01-27 22:03 ` [PATCH 7/9] numa,sched: do statistics calculation using local variables only riel
2014-01-27 22:03 ` riel
2014-01-28 19:24 ` [tip:sched/numa] sched/numa: Do " tip-bot for Rik van Riel
2014-01-27 22:03 ` [PATCH 8/9] numa,sched: rename variables in task_numa_fault riel
2014-01-27 22:03 ` riel
2014-01-28 19:24 ` [tip:sched/numa] sched/numa: Rename variables in task_numa_fault( ) tip-bot for Rik van Riel
2014-01-27 22:03 ` [PATCH 9/9] numa,sched: turn some magic numbers into defines riel
2014-01-27 22:03 ` riel
2014-01-28 19:25 ` [tip:sched/numa] sched/numa: Turn some magic numbers into #defines tip-bot for Rik van Riel
2014-01-28 0:35 ` [PATCH v5 0/9] numa,sched,mm: pseudo-interleaving for automatic NUMA balancing Chegu Vinod
2014-01-28 0:35 ` Chegu Vinod
-- strict thread matches above, loose matches on Subject: below --
2014-01-21 22:20 [PATCH v4 0/9] " riel
2014-01-21 22:20 ` [PATCH 6/9] numa,sched: normalize faults_cpu stats and weigh by CPU use riel
2014-01-21 22:20 ` riel
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20140128100131.GS4963@suse.de \
--to=mgorman@suse.de \
--cc=chegu_vinod@hp.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=riel@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.