Re: [PATCH 07/10] bootmem: add free_bootmem_late

[Ingo Molnar <mingo@elte.hu>]

* Pekka Enberg <penberg@cs.helsinki.fi> wrote:

> On Wed, Oct 28, 2009 at 2:12 PM, Tejun Heo <tj@kernel.org> wrote:
> >> We're doing it before scheduler init now but I haven't put any effort
> >> into moving it earlier than that yet. I don't see any fundamental
> >> reason we can't do that but the practical problem is that we're going
> >> to affect architecture specific boot code which is really hard to
> >> test.
> >
> > Thanks for the explanation. ?It would be really great if we can pull
> > that off someday. ?This should be doable architecture-by-architecture,
> > right? ?You can, for example, first convert x86 and then make bootmem
> > allocator thin wrapper around the slab allocator. ?After all archs
> > have been converted, the wrappers can be dropped.
> 
> I am not sure how you we could do that.
> 
> The main challenge in initializing slab earlier is the various 
> implicit dependencies between different parts of the kernel boot code. 
> If we initialize slab earlier, we also need to initialize page 
> allocator earlier which requires page table setup, mem_init(), and so 
> on. Unfortunately architectures don't do boot-time setup in "standard" 
> places which means that for some architectures mem_init() might need 
> traps but for other architectures we can't just enable traps earlier 
> unless we do something else before that as well.
> 
> So I think we need to untagle the mess in init/main.c some more first 
> before we try to initialize slab earlier.

Page tables is the main dependency. x86 boots with a limited set of page 
tables, the real ones are set up later.

We'd need to see what bootmem allocations are done before page table 
init in practice. I think i did such tests a few years ago and i think 
it's rather limited (if it happens at all).

If that's mapped out we can just convert x86 to an 'emulated' bootmem 
allocator: buddy and slab is set up right when pagetables are set up, 
and bootmem can just use kmalloc.

If that works and is without nasty surprises then we can repeat the same 
for other architectures as well. (x86-only code could switch away from 
bootmem at that point as well)

( It's not without mm/page_alloc.c challenges: we'd need new helpers to 
  be able to bootstrap the buddy straight out of arch code, without any 
  bootmem data structures. )

The elimination of bootmem would be an awesome simplification of our 
memory bootstrap design, and universal kmalloc would be very nice for 
platform code as well.

	Ingo