Re: [RFC PATCH]: ACPI: Automatically online hot-added memory

[ykzhao <yakui.zhao@intel.com>]

On Thu, 2010-03-11 at 10:18 +0800, Prarit Bhargava wrote:
> >
> > Yes. The nehalem processor has the integrated memory controller. But it
> > is not required that the hot-added memory should be onlined before
> > bringing up CPU.
> >     I do the following memory-hotplug test on one Machine.
> >     a. Before hot plugging memory, four CPUs socket are installed and
> > all the logical CPU are brought up. (Only one node has the memory)
> >     b. The memory is hot-plugged and then the memory is onlined so that
> > it can be accessed by the system.
> >
> > In the above testing case the CPU is brought up before onlining the
> > hot-added memory. And the test shows that it can work well.
> >   
> 
> That doesn't work when you have multiple nodes AFAICT.  The cpus do not 
> come into service because of a lack of memory on the node....  per node 
> allocations will fail.

In the test the system has multiple nodes. The reason is that the cpu
without memory can turn to other node and allocate the memory.

> 
> Just curious, exactly what did you test with?  2.6.33 (or newer)?

I test it on 2.6.32 kernel.

> 
> >   
> >> ie) with new processors it is possible that an entire node which 
> >> consists of memory and cpus comes and goes with the socket enable and 
> >> disable.
> >>
> >> The cpu bringup code does local node allocations for the cpu.  If the 
> >> memory connected to the node (which is "behind" the socket) isn't 
> >> online, then these allocations fail, and then the cpu bringup fails.
> >>     
> >
> > If the CPU can't allocate the memory from its own node, it can turn to
> > other node and see whether the memory can be allocated. And this depends
> > on the NUMA allocation policy.
> >   
> 
> Maybe that could work, but I haven't gotten that to work.  Even if it 
> does work, it's a HUGE performance hit :(.  I can't imagine incurring an 
> extra hop just to get to per_cpu memory.  I'd rather bring the memory on 
> the local node up first.
> 
> I think it is much better to bring the memory up first.
> 
> Either way, it's a nice feature to have.
> 
> P.
> 
> >    
> >   
> >> P.
> >>
> >>     
> >
> >