about the meaning of numactl -i

about the meaning of numactl -i

[Alfredo Buttari]

Hi all,
I have a simple question regarding the functioning of the numactl
command with the -i flag.

Does it mean that, if I have, e.g., multiple allocations in my
executable then these allocations are interleaved on the numa nodes in
a round-robin fashion?

or does it mean that if I have, e.g., a single big allocation in my
executable the corresponding memory area will be split into multiple,
non-contiguous regions which are then interleaved on the numa nodes in
rr fashion?

or both of the above?


Best regards

Alfredo Buttari

Re: about the meaning of numactl -i

[Andi Kleen]

On Thu, Apr 14, 2011 at 03:40:58PM +0200, Alfredo Buttari wrote:
> Hi all,
> I have a simple question regarding the functioning of the numactl
> command with the -i flag.
> 
> Does it mean that, if I have, e.g., multiple allocations in my
> executable then these allocations are interleaved on the numa nodes in
> a round-robin fashion?
> 
> or does it mean that if I have, e.g., a single big allocation in my
> executable the corresponding memory area will be split into multiple,
> non-contiguous regions which are then interleaved on the numa nodes in
> rr fashion?
> 
> or both of the above?

It depends on whether you apply it to a process or to a shm mapping.
For a process it would the first, for a shm mapping the later.
Also for a process this happens in first touch fault-order.

-Andi

-- 
ak@linux.intel.com -- Speaking for myself only.

Re: about the meaning of numactl -i

[Alfredo Buttari]

Andi,

On Thu, Apr 14, 2011 at 7:29 PM, Andi Kleen <andi@firstfloor.org> wrote:
> Also for a process this happens in first touch fault-order.


does this mean that memory is allocated in rr fashion but the actual
node is only chosen at the moment where the memory is touched for the
first time?

best regards


-- 
-----------------------------------------
Alfredo Buttari, PhD
CNRS-IRIT
2 rue Camichel, 31071 Toulouse, France
http://buttari.perso.enseeiht.fr

Re: about the meaning of numactl -i

[Lee Schermerhorn]

On Thu, 2011-04-14 at 15:40 +0200, Alfredo Buttari wrote:
> Hi all,
> I have a simple question regarding the functioning of the numactl
> command with the -i flag.
> 
> Does it mean that, if I have, e.g., multiple allocations in my
> executable then these allocations are interleaved on the numa nodes in
> a round-robin fashion?
> 
> or does it mean that if I have, e.g., a single big allocation in my
> executable the corresponding memory area will be split into multiple,
> non-contiguous regions which are then interleaved on the numa nodes in
> rr fashion?
> 
> or both of the above?

Alfredo:

Memory interleaving mode [-i] behaves like your first alternative.  See
Documentation/vm/numa_memory_policy.txt in the kernel source tree or
where ever your distro installs it.  [kernel-doc-*.rpm, linux-doc*.deb]

Lee

Re: about the meaning of numactl -i

[Alfredo Buttari]

Dear Lee,
thanks a lot for pointing me to this document. I've been looking for
something like this for some time.

best regards

alfredo



On Wed, Jun 22, 2011 at 3:26 PM, Lee Schermerhorn
<Lee.Schermerhorn@hp.com> wrote:
> Alfredo:
>
> Memory interleaving mode [-i] behaves like your first alternative.  See
> Documentation/vm/numa_memory_policy.txt in the kernel source tree or
> where ever your distro installs it.  [kernel-doc-*.rpm, linux-doc*.deb]
>
> Lee



-- 
-----------------------------------------
Alfredo Buttari, PhD
CNRS-IRIT
2 rue Camichel, 31071 Toulouse, France
http://buttari.perso.enseeiht.fr