Re: [Patch 0/3] Modify how per_cpu allocations are done. -V8

[Robin Holt <holt@sgi.com>]

Tony,

Can you give me some direction on this?  I think the first two patches
are probably in the ball park of what we should take.  The third (MCA
stack allocations) should probably be dropped.  I would like to get one
final (hopefully) set out before the merge window opens.

Thanks,
Robin

On Wed, Mar 19, 2008 at 08:40:42PM -0500, Robin Holt wrote:
> On Tue, Mar 18, 2008 at 10:46:59AM -0700, Luck, Tony wrote:
> > > Have you had a chance to test this revised patchset on your tiger box?
> > 
> > Yes.  The new patchset still builds cleanly for all my test configs, and
> > this time it boots on my Tiger.
> > 
> > I'm still wondering about the direction you've taken though.  One
> > of the goals of this patchset is to reduce boot-time memory
> > allocation by making sure that we don't allocate per-cpu resources
> > for cpus that will never exist.  But not all systems provide
> > enough information to determine this reliably (and Linux doesn't
> > get around to looking at that information until after we need to
> > do these allocations) so you've added the "early_cpu_possible_map"
> > which in some cases has to err on the safe side in how many cpus
> > it thinks may ever exist ... and so we may allocate resources for
> > some non-existant cpus.
> 
> I am not sure where to go with this.  I have done some more pondering
> today and now I feel further from a solution.  The code now does limit
> the scope of our overallocation.  In the case where the ACPI tables
> have described the cpu's numa placement, we don't overallocate at all.
> In all other cases, we overshoot by far fewer.  With a defconfig kernel
> on a zx1, the allocation drops from 512 per_cpu areas to 32.  I agree it
> is not an ideal solution, but I fail to see a better solution.  Is there a
> different table I should be walking to discover the actual number of cpus?
> 
> > The existing MCA part of the code appears to be more conservative
> > than the code that you are adding.  It allocates for cpu0 using
> > bootmem_alloc, and then for other cpus on an as-needed basis using
> > alloc_pages_node().  The current code is very ugly ... needing
> > an "__init refok" function, and not dealing well with possible
> > allocation failures.  A clean-up is definitely needed, but can't
> > we still maintain the alloc-on-demand part (perhaps moving it
> > from being run by the new cpu itself to some pre-bring-up-code
> > that will be run by the cpu that is going to initiate bringing
> > the cpu online ... which would make the error handling path
> > easier).
> 
> I think I am going to give up on that patch entirely.  My swag at this at
> least ensured there was no way the allocation could fail, but it does
> not make any of the over-allocation issues better.  Unless you feel
> there is merit here, I will drop that patch.
> 
> Thanks,
> Robin
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ia64" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html