From: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
To: Tharindu Rukshan Bamunuarachchi <btharindu@gmail.com>
Cc: Tim Pepper <lnxninja@linux.vnet.ibm.com>, linux-numa@vger.kernel.org
Subject: Re: NUMA page allocation from next Node
Date: Mon, 01 Nov 2010 10:59:13 -0400 [thread overview]
Message-ID: <1288623553.3229.24.camel@useless.localdomain> (raw)
In-Reply-To: <AANLkTi=s47jwWEDkUpUQGxZjf=oojyDUj74HKBhi-Tnv@mail.gmail.com>
On Mon, 2010-11-01 at 19:48 +0530, Tharindu Rukshan Bamunuarachchi
wrote:
> Tim,
>
> I found that default value for "zone_reclaim_mode" is zero in HP
> machine. But It is one in IBM.
> Why does it set 1 or 0 in different hardware
Because the SLIT on the IBM platform has distances > 20. Looks like IBM
is populating the SLIT on those platforms with "real" values. The HP
bios is not supplying a slit, letting the remote distances default to
20. That is the threshold for setting zone_reclaim_mode. A patch was
submitted recently to bump the threshold to ~30. Now that vendors are
starting to populate the SLIT with values > 20, we've begun to see the
behavior that you experienced.
Regards,
Lee
>
> On Sat, Oct 30, 2010 at 1:22 AM, Tim Pepper <lnxninja@linux.vnet.ibm.com> wrote:
> >
> > It would be interesting to see the output of "numactl --hardware" for each
> > of these scenarios.
> >
>
> 1. SLES11 + IBM HW
> After consuming all memory in node1, it shows following ...
>
> available: 2 nodes (0-1)
> node 0 cpus: 0 1 2 3 4 5
> node 0 size: 24564 MB
> node 0 free: 23025 MB
> node 1 cpus: 6 7 8 9 10 11
> node 1 size: 24576 MB
> node 1 free: 16 MB
> node distances:
> node 0 1
> 0: 10 21
> 1: 21 10
>
>
> 2. SLES 11 + HP
>
> available: 2 nodes (0-1)
> node 0 cpus: 0 2 4 6
> node 0 size: 24565 MB
> node 0 free: 19929 MB
> node 1 cpus: 1 3 5 7
> node 1 size: 24575 MB
> node 1 free: 3043 MB
> node distances:
> node 0 1
> 0: 10 20
> 1: 20 10
>
> available: 2 nodes (0-1)
> node 0 cpus: 0 2 4 6
> node 0 size: 24565 MB
> node 0 free: 19912 MB
> node 1 cpus: 1 3 5 7
> node 1 size: 24575 MB
> node 1 free: 335 MB
> node distances:
> node 0 1
> 0: 10 20
> 1: 20 10
>
> available: 2 nodes (0-1)
> node 0 cpus: 0 2 4 6
> node 0 size: 24565 MB
> node 0 free: 17066 MB
> node 1 cpus: 1 3 5 7
> node 1 size: 24575 MB
> node 1 free: 16 MB
> node distances:
> node 0 1
> 0: 10 20
> 1: 20 10
>
> available: 2 nodes (0-1)
> node 0 cpus: 0 2 4 6
> node 0 size: 24565 MB
> node 0 free: 10468 MB
> node 1 cpus: 1 3 5 7
> node 1 size: 24575 MB
> node 1 free: 16 MB
> node distances:
> node 0 1
> 0: 10 20
> 1: 20 10
>
>
>
> > --
> > Tim Pepper <lnxninja@linux.vnet.ibm.com>
> > IBM Linux Technology Center
> >
> --
> To unsubscribe from this list: send the line "unsubscribe linux-numa" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
next prev parent reply other threads:[~2010-11-01 14:59 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-10-26 16:27 NUMA page allocation from next Node Tharindu Rukshan Bamunuarachchi
[not found] ` <20101027213652.GA12345@sgi.com>
2010-10-29 2:05 ` Tharindu Rukshan Bamunuarachchi
[not found] ` <20101029033058.GB555@www.lurndal.org>
2010-10-29 6:58 ` Tharindu Rukshan Bamunuarachchi
[not found] ` <AANLkTikDtKc7RdAWJagqCf7T0JKscfe0Hd0ojc8g7yYo@mail.gmail.com>
2010-10-29 7:06 ` Tharindu Rukshan Bamunuarachchi
2010-10-29 8:49 ` Andi Kleen
2010-10-29 9:16 ` Tharindu Rukshan Bamunuarachchi
2010-10-29 19:52 ` Tim Pepper
2010-10-29 20:30 ` Lee Schermerhorn
2010-11-01 13:55 ` Tharindu Rukshan Bamunuarachchi
2010-11-01 14:18 ` Tharindu Rukshan Bamunuarachchi
2010-11-01 14:59 ` Lee Schermerhorn [this message]
2010-11-01 18:00 ` Andi Kleen
2010-11-02 0:49 ` Tharindu Rukshan Bamunuarachchi
2010-11-01 17:59 ` Andi Kleen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1288623553.3229.24.camel@useless.localdomain \
--to=lee.schermerhorn@hp.com \
--cc=btharindu@gmail.com \
--cc=linux-numa@vger.kernel.org \
--cc=lnxninja@linux.vnet.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).