From: Mel Gorman <mgorman@suse.de>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com>,
Ingo Molnar <mingo@kernel.org>,
Andrea Arcangeli <aarcange@redhat.com>,
Johannes Weiner <hannes@cmpxchg.org>,
Linux-MM <linux-mm@kvack.org>,
LKML <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH 05/16] sched: Select a preferred node with the most numa hinting faults
Date: Thu, 11 Jul 2013 13:53:03 +0100 [thread overview]
Message-ID: <20130711125303.GB2355@suse.de> (raw)
In-Reply-To: <20130711112359.GG25631@dyad.programming.kicks-ass.net>
On Thu, Jul 11, 2013 at 01:23:59PM +0200, Peter Zijlstra wrote:
> On Thu, Jul 11, 2013 at 10:46:49AM +0100, Mel Gorman wrote:
> > --- a/kernel/sched/core.c
> > +++ b/kernel/sched/core.c
> > @@ -1593,6 +1593,7 @@ static void __sched_fork(struct task_struct *p)
> > p->numa_scan_seq = p->mm ? p->mm->numa_scan_seq : 0;
> > p->numa_migrate_seq = p->mm ? p->mm->numa_scan_seq - 1 : 0;
> > p->numa_scan_period = sysctl_numa_balancing_scan_delay;
> > + p->numa_preferred_nid = -1;
>
> (1)
>
> > p->numa_work.next = &p->numa_work;
> > p->numa_faults = NULL;
> > #endif /* CONFIG_NUMA_BALANCING */
> > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> > index 904fd6f..c0bee41 100644
> > --- a/kernel/sched/fair.c
> > +++ b/kernel/sched/fair.c
> > @@ -793,7 +793,8 @@ unsigned int sysctl_numa_balancing_scan_delay = 1000;
> >
> > static void task_numa_placement(struct task_struct *p)
> > {
> > - int seq;
> > + int seq, nid, max_nid = 0;
>
> Should you not start with max_nid = -1?
>
For consistency, yes.
> > + unsigned long max_faults = 0;
> >
> > if (!p->mm) /* for example, ksmd faulting in a user's mm */
> > return;
> > @@ -802,7 +803,19 @@ static void task_numa_placement(struct task_struct *p)
> > return;
> > p->numa_scan_seq = seq;
> >
> > - /* FIXME: Scheduling placement policy hints go here */
> > + /* Find the node with the highest number of faults */
> > + for (nid = 0; nid < nr_node_ids; nid++) {
> > + unsigned long faults = p->numa_faults[nid];
> > + p->numa_faults[nid] >>= 1;
> > + if (faults > max_faults) {
> > + max_faults = faults;
> > + max_nid = nid;
> > + }
> > + }
> > +
>
> It is rather unlikely; but suppose the entire ->numa_faults[] array is 0, you'd
> somehow end up selecting max_nid := 0. Which seems inconsistent with \1.
>
This happens more often than you might imagine due to THP false sharing
getting count as shared. If the entire numa_faults[] array is 0 then
max_faults == 0 and it will not update preferred_nid due to the
"max_faults" check below.
> > + /* Update the tasks preferred node if necessary */
> > + if (max_faults && max_nid != p->numa_preferred_nid)
> > + p->numa_preferred_nid = max_nid;
> > }
> >
--
Mel Gorman
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
WARNING: multiple messages have this Message-ID (diff)
From: Mel Gorman <mgorman@suse.de>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com>,
Ingo Molnar <mingo@kernel.org>,
Andrea Arcangeli <aarcange@redhat.com>,
Johannes Weiner <hannes@cmpxchg.org>,
Linux-MM <linux-mm@kvack.org>,
LKML <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH 05/16] sched: Select a preferred node with the most numa hinting faults
Date: Thu, 11 Jul 2013 13:53:03 +0100 [thread overview]
Message-ID: <20130711125303.GB2355@suse.de> (raw)
In-Reply-To: <20130711112359.GG25631@dyad.programming.kicks-ass.net>
On Thu, Jul 11, 2013 at 01:23:59PM +0200, Peter Zijlstra wrote:
> On Thu, Jul 11, 2013 at 10:46:49AM +0100, Mel Gorman wrote:
> > --- a/kernel/sched/core.c
> > +++ b/kernel/sched/core.c
> > @@ -1593,6 +1593,7 @@ static void __sched_fork(struct task_struct *p)
> > p->numa_scan_seq = p->mm ? p->mm->numa_scan_seq : 0;
> > p->numa_migrate_seq = p->mm ? p->mm->numa_scan_seq - 1 : 0;
> > p->numa_scan_period = sysctl_numa_balancing_scan_delay;
> > + p->numa_preferred_nid = -1;
>
> (1)
>
> > p->numa_work.next = &p->numa_work;
> > p->numa_faults = NULL;
> > #endif /* CONFIG_NUMA_BALANCING */
> > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> > index 904fd6f..c0bee41 100644
> > --- a/kernel/sched/fair.c
> > +++ b/kernel/sched/fair.c
> > @@ -793,7 +793,8 @@ unsigned int sysctl_numa_balancing_scan_delay = 1000;
> >
> > static void task_numa_placement(struct task_struct *p)
> > {
> > - int seq;
> > + int seq, nid, max_nid = 0;
>
> Should you not start with max_nid = -1?
>
For consistency, yes.
> > + unsigned long max_faults = 0;
> >
> > if (!p->mm) /* for example, ksmd faulting in a user's mm */
> > return;
> > @@ -802,7 +803,19 @@ static void task_numa_placement(struct task_struct *p)
> > return;
> > p->numa_scan_seq = seq;
> >
> > - /* FIXME: Scheduling placement policy hints go here */
> > + /* Find the node with the highest number of faults */
> > + for (nid = 0; nid < nr_node_ids; nid++) {
> > + unsigned long faults = p->numa_faults[nid];
> > + p->numa_faults[nid] >>= 1;
> > + if (faults > max_faults) {
> > + max_faults = faults;
> > + max_nid = nid;
> > + }
> > + }
> > +
>
> It is rather unlikely; but suppose the entire ->numa_faults[] array is 0, you'd
> somehow end up selecting max_nid := 0. Which seems inconsistent with \1.
>
This happens more often than you might imagine due to THP false sharing
getting count as shared. If the entire numa_faults[] array is 0 then
max_faults == 0 and it will not update preferred_nid due to the
"max_faults" check below.
> > + /* Update the tasks preferred node if necessary */
> > + if (max_faults && max_nid != p->numa_preferred_nid)
> > + p->numa_preferred_nid = max_nid;
> > }
> >
--
Mel Gorman
SUSE Labs
next prev parent reply other threads:[~2013-07-11 12:53 UTC|newest]
Thread overview: 58+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-07-11 9:46 [PATCH 0/16] Basic scheduler support for automatic NUMA balancing V4 Mel Gorman
2013-07-11 9:46 ` Mel Gorman
2013-07-11 9:46 ` [PATCH 01/16] mm: numa: Document automatic NUMA balancing sysctls Mel Gorman
2013-07-11 9:46 ` Mel Gorman
2013-07-11 9:46 ` [PATCH 02/16] sched: Track NUMA hinting faults on per-node basis Mel Gorman
2013-07-11 9:46 ` Mel Gorman
2013-07-11 9:46 ` [PATCH 03/16] mm: numa: Account for THP numa hinting faults on the correct node Mel Gorman
2013-07-11 9:46 ` Mel Gorman
2013-07-11 9:46 ` [PATCH 04/16] mm: numa: Do not migrate or account for hinting faults on the zero page Mel Gorman
2013-07-11 9:46 ` Mel Gorman
2013-07-11 11:21 ` Peter Zijlstra
2013-07-11 11:21 ` Peter Zijlstra
2013-07-11 12:42 ` Mel Gorman
2013-07-11 12:42 ` Mel Gorman
2013-07-11 9:46 ` [PATCH 05/16] sched: Select a preferred node with the most numa hinting faults Mel Gorman
2013-07-11 9:46 ` Mel Gorman
2013-07-11 11:23 ` Peter Zijlstra
2013-07-11 11:23 ` Peter Zijlstra
2013-07-11 12:53 ` Mel Gorman [this message]
2013-07-11 12:53 ` Mel Gorman
2013-07-11 9:46 ` [PATCH 06/16] sched: Update NUMA hinting faults once per scan Mel Gorman
2013-07-11 9:46 ` Mel Gorman
2013-07-11 9:46 ` [PATCH 07/16] sched: Favour moving tasks towards the preferred node Mel Gorman
2013-07-11 9:46 ` Mel Gorman
2013-07-11 9:46 ` [PATCH 08/16] sched: Reschedule task on preferred NUMA node once selected Mel Gorman
2013-07-11 9:46 ` Mel Gorman
2013-07-11 12:30 ` Peter Zijlstra
2013-07-11 12:30 ` Peter Zijlstra
2013-07-11 13:03 ` Mel Gorman
2013-07-11 13:03 ` Mel Gorman
2013-07-11 13:11 ` Peter Zijlstra
2013-07-11 13:11 ` Peter Zijlstra
2013-07-11 14:09 ` Mel Gorman
2013-07-11 14:09 ` Mel Gorman
2013-07-12 10:14 ` Peter Zijlstra
2013-07-12 10:14 ` Peter Zijlstra
2013-07-12 10:28 ` Mel Gorman
2013-07-12 10:28 ` Mel Gorman
2013-07-11 9:46 ` [PATCH 09/16] sched: Add infrastructure for split shared/private accounting of NUMA hinting faults Mel Gorman
2013-07-11 9:46 ` Mel Gorman
2013-07-11 9:46 ` [PATCH 10/16] sched: Increase NUMA PTE scanning when a new preferred node is selected Mel Gorman
2013-07-11 9:46 ` Mel Gorman
2013-07-11 9:46 ` [PATCH 11/16] sched: Check current->mm before allocating NUMA faults Mel Gorman
2013-07-11 9:46 ` Mel Gorman
2013-07-11 9:46 ` [PATCH 12/16] sched: Set the scan rate proportional to the size of the task being scanned Mel Gorman
2013-07-11 9:46 ` Mel Gorman
2013-07-11 9:46 ` [PATCH 13/16] mm: numa: Scan pages with elevated page_mapcount Mel Gorman
2013-07-11 9:46 ` Mel Gorman
2013-07-11 9:46 ` [PATCH 14/16] sched: Remove check that skips small VMAs Mel Gorman
2013-07-11 9:46 ` Mel Gorman
2013-07-11 9:46 ` [PATCH 15/16] sched: Set preferred NUMA node based on number of private faults Mel Gorman
2013-07-11 9:46 ` Mel Gorman
2013-07-11 9:47 ` [PATCH 16/16] sched: Select least loaded CPU on preferred NUMA node Mel Gorman
2013-07-11 9:47 ` Mel Gorman
2013-07-11 12:39 ` Peter Zijlstra
2013-07-11 12:39 ` Peter Zijlstra
2013-07-11 13:24 ` Mel Gorman
2013-07-11 13:24 ` Mel Gorman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20130711125303.GB2355@suse.de \
--to=mgorman@suse.de \
--cc=aarcange@redhat.com \
--cc=hannes@cmpxchg.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mingo@kernel.org \
--cc=peterz@infradead.org \
--cc=srikar@linux.vnet.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.