Re: [RFC v3 0/5] Transparent on-demand struct page initialization embedded in the buddy allocator

[Ingo Molnar <mingo@kernel.org>]

* Linus Torvalds <torvalds@linux-foundation.org> wrote:

> On Tue, Aug 13, 2013 at 4:10 PM, Nathan Zimmer <nzimmer@sgi.com> wrote:
> >
> > The only mm structure we are adding to is a new flag in page->flags. 
> > That didn't seem too much.
> 
> I don't agree.
> 
> I see only downsides, and no upsides. Doing the same thing *without* the 
> downsides seems straightforward, so I simply see no reason for any extra 
> flags or tests at runtime.

The code as presented clearly looks more involved and neither simple nor 
zero-cost - I was hoping for a much more simple approach.

I see three solutions:

 - Speed up the synchronous memory init code: live migrate to the node 
   being set up via set_cpus_allowed(), to make sure the init is always 
   fast and local.

   Pros: if it solves the problem then mem init is still synchronous, 
   deterministic and essentially equivalent to what we do today - so 
   relatively simple and well-tested, with no 'large machine' special
   path.

   Cons: it might not be enough and we might not have scheduling
   enabled on the affected nodes yet.

 - Speed up the synchronous memory init code by paralellizing the key, 
   most expensive initialization portion of setting up the page head 
   arrays to per node, via SMP function-calls.

   Pros: by far the fastest synchronous option. (It will also test the
   power budget and the mains fuses right during bootup.)

   Cons: more complex and depends on SMP cross-calls being available at
   mem init time. Not necessarily hotplug friendly.

 - Avoid the problem by punting to async mem init.

   Pros: it gets us to a minimal working system quickly and leaves the 
   memory code relatively untouched.

   Disadvantages: makes memory state asynchronous and non-deterministic.
   Stats either fluctuate shortly after bootup or have to be faked.

Thanks,

	Ingo