powerpc,numa: Memory hotplug to memory-less nodes ?

powerpc,numa: Memory hotplug to memory-less nodes ?

[Bharata B Rao]

Hi,

While developing memory hotplug support in QEMU for PoweKVM, I
realized that guest kernel has specific checks to prevent hot addition
of memory to a memory-less node.

I am referring to arch/powerpc/mm/numa.c:hot_add_scn_to_nid() which
has explicit checks to ensure that it returns a nid that has some some
memory (NODE_DATA(nid)->node_spanned_pages) even when user wants to
hotplug to a node that currently has zero memory.

Is this limitation by design ?

Regards,
Bharata.
-- 
http://raobharata.wordpress.com/

Re: powerpc,numa: Memory hotplug to memory-less nodes ?

[Bharata B Rao]

So will it be correct to say that memory hotplug to memory-less node
isn't supported by PowerPC kernel ? Should I enforce the same in QEMU
for PowerKVM ?

On Mon, Jun 22, 2015 at 10:18 AM, Bharata B Rao <bharata.rao@gmail.com> wrote:
> Hi,
>
> While developing memory hotplug support in QEMU for PoweKVM, I
> realized that guest kernel has specific checks to prevent hot addition
> of memory to a memory-less node.
>
> I am referring to arch/powerpc/mm/numa.c:hot_add_scn_to_nid() which
> has explicit checks to ensure that it returns a nid that has some some
> memory (NODE_DATA(nid)->node_spanned_pages) even when user wants to
> hotplug to a node that currently has zero memory.
>
> Is this limitation by design ?
>
> Regards,
> Bharata.
> --
> http://raobharata.wordpress.com/



-- 
http://raobharata.wordpress.com/

Re: powerpc,numa: Memory hotplug to memory-less nodes ?

[Nathan Fontenot]

On 06/23/2015 11:01 PM, Bharata B Rao wrote:
> So will it be correct to say that memory hotplug to memory-less node
> isn't supported by PowerPC kernel ? Should I enforce the same in QEMU
> for PowerKVM ?
>

I'm not sure if that is correct. It appears that we initialize all online
nodes, even those without spanned_pages, at boot time. This occurs
in setup_node_data() called from initmem_init().

Looking at this I would think that we could add memory to any online node
even if it does not have any spanned_pages. I think an interesting test
we be to check for the node being online instead of checking to see if
it has any memory.

-Nathan

> On Mon, Jun 22, 2015 at 10:18 AM, Bharata B Rao <bharata.rao@gmail.com> wrote:
>> Hi,
>>
>> While developing memory hotplug support in QEMU for PoweKVM, I
>> realized that guest kernel has specific checks to prevent hot addition
>> of memory to a memory-less node.
>>
>> I am referring to arch/powerpc/mm/numa.c:hot_add_scn_to_nid() which
>> has explicit checks to ensure that it returns a nid that has some some
>> memory (NODE_DATA(nid)->node_spanned_pages) even when user wants to
>> hotplug to a node that currently has zero memory.
>>
>> Is this limitation by design ?
>>
>> Regards,
>> Bharata.
>> --
>> http://raobharata.wordpress.com/
> 
> 
> 

Re: powerpc,numa: Memory hotplug to memory-less nodes ?

[Nishanth Aravamudan]

On 24.06.2015 [07:13:36 -0500], Nathan Fontenot wrote:
> On 06/23/2015 11:01 PM, Bharata B Rao wrote:
> > So will it be correct to say that memory hotplug to memory-less node
> > isn't supported by PowerPC kernel ? Should I enforce the same in QEMU
> > for PowerKVM ?
> >
> 
> I'm not sure if that is correct. It appears that we initialize all online
> nodes, even those without spanned_pages, at boot time. This occurs
> in setup_node_data() called from initmem_init().
> 
> Looking at this I would think that we could add memory to any online node
> even if it does not have any spanned_pages. I think an interesting test
> we be to check for the node being online instead of checking to see if
> it has any memory.

I see no *technical* reason we should't be able to hotplug to an
initially memoryless node. I'm not sure it happens in practice under
PowerVM (where we have far less control of the topology anyways). One
aspect of this that I have on my todo list is seeing what SLUB does when
a node goes from memoryless to populated -- as during boot memoryless
nodes get a 'useless' per node structure (early_kmem_cache_node_alloc).

I can look at this a bit under KVM maybe later this week myself to see
what happens in a guest.

-Nish

Re: powerpc,numa: Memory hotplug to memory-less nodes ?

[Reza Arbab]

On Mon, Jun 22, 2015 at 10:18:43AM +0530, Bharata B Rao wrote:
>While developing memory hotplug support in QEMU for PoweKVM, I
>realized that guest kernel has specific checks to prevent hot addition
>of memory to a memory-less node.
>
>I am referring to arch/powerpc/mm/numa.c:hot_add_scn_to_nid() which
>has explicit checks to ensure that it returns a nid that has some some
>memory (NODE_DATA(nid)->node_spanned_pages) even when user wants to
>hotplug to a node that currently has zero memory.
>
>Is this limitation by design ?

Sorry to resurrect such an old thread, but did anything come of this?

I've recently bumped into the same limitation. Could we consider 
removing it?

(I couldn't find the original distribution list for this thread. Please 
add whomever is appropriate to the Cc:)

-- 
Reza Arbab