Interleaved allocation

Interleaved allocation

[Martin Ichilevici de Oliveira]

[-- Attachment #1: Type: text/plain, Size: 861 bytes --]

Hello,

I'm trying to understand what's the exact behavior of the
interleaved allocation in libnuma. I have two questions, the first
regarding numa_alloc_interleaved() and the second regarding numactl.
Suppose I'm running in a 4-node machine.

1. What happens when I try to allocate 100 pages with a call like this:

numa_alloc_interleaved(100 * numa_pagesize())

I can think of two possible allocation schemes:
a. Pages 0-24 are allocated to node 0, pages 25-49 are allocated to
node 1, pages 50-74 are allocated on node 2 and pages 75-99 are
allocated on node 3.

b. Pages 0,4,8... are allocated on node 0, pages 1,5,9... are
allocated on node 1 and so on

2. What's the effect of using numactl --interleaved=0,1,2,3 -- ./myprog?
I would assume it "substitutes" any malloc's to
numa_alloc_interleaved(), but I'm not sure if it's just that.

Thank you,
Martin

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]

Re: Interleaved allocation

[Andi Kleen]

Martin Ichilevici de Oliveira <iomartin@iomartin.net> writes:

> 1. What happens when I try to allocate 100 pages with a call like this:
>
> numa_alloc_interleaved(100 * numa_pagesize())
>
> I can think of two possible allocation schemes:
> a. Pages 0-24 are allocated to node 0, pages 25-49 are allocated to
> node 1, pages 50-74 are allocated on node 2 and pages 75-99 are
> allocated on node 3.
>
> b. Pages 0,4,8... are allocated on node 0, pages 1,5,9... are
> allocated on node 1 and so on

It's approximately (b)

> 2. What's the effect of using numactl --interleaved=0,1,2,3 -- ./myprog?
> I would assume it "substitutes" any malloc's to
> numa_alloc_interleaved(), but I'm not sure if it's just that.

Any page allocated by the process is interleaved in access order
(which is different from what numa_alloc_interleaved does)

-Andi

-- 
ak@linux.intel.com -- Speaking for myself only

Re: Interleaved allocation

[Martin Ichilevici de Oliveira]

[-- Attachment #1: Type: text/plain, Size: 1019 bytes --]

On Tue, Dec 09, 2014 at 02:03:16PM -0800, Andi Kleen wrote:
> Martin Ichilevici de Oliveira <iomartin@iomartin.net> writes:
> 
> > 1. What happens when I try to allocate 100 pages with a call like this:
> >
> > numa_alloc_interleaved(100 * numa_pagesize())
> >
> > I can think of two possible allocation schemes:
> > a. Pages 0-24 are allocated to node 0, pages 25-49 are allocated to
> > node 1, pages 50-74 are allocated on node 2 and pages 75-99 are
> > allocated on node 3.
> >
> > b. Pages 0,4,8... are allocated on node 0, pages 1,5,9... are
> > allocated on node 1 and so on
> 
> It's approximately (b)
> 
> > 2. What's the effect of using numactl --interleaved=0,1,2,3 -- ./myprog?
> > I would assume it "substitutes" any malloc's to
> > numa_alloc_interleaved(), but I'm not sure if it's just that.
> 
> Any page allocated by the process is interleaved in access order
> (which is different from what numa_alloc_interleaved does)

Andi,

Thank you for clarifying these up.

Martin

[-- Attachment #2: Type: application/pgp-signature, Size: 819 bytes --]