[U-Boot] Early malloc() summary

[Marek Vasut <marex@denx.de>]

Dear Graeme Russ,

> Hi Marek,
> 
> On Tue, Aug 14, 2012 at 10:37 PM, Marek Vasut <marex@denx.de> wrote:
> > Dear Tomas Hlavacek,
> > 
> >> Hello Marek,
> >> 
> >> On Sun, Aug 12, 2012 at 1:16 AM, Marek Vasut <marex@denx.de> wrote:
> >> > So ... we should aim for firing up the real mallocator as soon as
> >> > possible and maybe implement discontigmem (sparsemem) into it, so we
> >> > don't have to bother with relocating pointers maybe?
> >> > 
> >> > The only problem I see is platforms where the memory disappears.
> >> 
> >> I doubt that on ARM for instance you can set off real mallocator that
> >> early without completely rewriting it.
> > 
> > Esp. on ARM, I won't see much of a problem. You usually have some small
> > SRAM where such allocator could run.
> > 
> >> The idea of having one complex
> >> mallocator working in the same manner in board_init_f and board_init_r
> >> stages, being able to operate on all platforms using their nifty
> >> memory-management/model features
> > 
> > Do you need them? You usually need only a piece of RW memory.
> > 
> >> and being seamless to users is really
> >> tempting. But do we need/want to introduce such deep rewrites?
> > 
> > Deep rewrites? You'd only need to implement sparsemem into it, which
> > might be list of RW memory areas instead of one memory area.
> 
> OK, stop right there (please) - This gets into a highly architecturally
> specific implementations of malloc. Let's not go there OK :)

Not really, today you give the mallocator one slab of memory on which it 
operates ... making it so it'd operate on a list of such slabs shouldn't be 
_that_ hard.

> >> What
> >> would be the justification? I would say we should stick with the
> >> Wolfgang's opinion: Create small and efficient early_malloc for DM and
> >> prepare it for future extensions and possible reuses.
> > 
> > dm_malloc you mean? I'm not happy about it, maybe Graeme can pour in some
> > crazy juice in our direction again?
> 
> I don't like the idea of dm_malloc() either, but it may be the only way to
> get this past Wolfgang in the initial pass...

Hehe >:-)

> >> >> Switching the focus back to DM, I really would like to ask to delay
> >> >> alls uch activities until DM has been done (or at least has
> >> >> stabilized so far that we can affort the luxury of thinking about
> >> >> the next version with fancy extensions).
> >> > 
> >> > We still need to handle the pre-reloc drivers somehow, you know ...
> >> > but I still believe we can pull the DM internals in three people and
> >> > leave Thomas to do proper malloc stuff ...
> >> 
> >> Yes, this is the main question: Should I hack malloc() function or
> >> does it make sense to have both early_malloc() and malloc() exposed to
> >> DM cores/drivers?
> > 
> > This is indeed the main question -- ideas ?
> > 
> >> The first is better from the point of view of drivers - when you ask
> >> for memory, you get it. But you have to check yourself whether you
> >> need to relocate your pointers or not, though we can provide
> >> "relocation chain" you can register your relocation routine into to
> >> facilitate it. The later makes sense because this makes it explicit
> >> that whenever you use early_malloc() you are responsible for
> >> relocating your data on your own (again, we can provide some facility
> >> for ir).
> 
> And there is the crux of it. Two failure scenarios:
> 
>  1) Write a driver which uses malloc() and fail to implement a relocation
>     helper - Driver blows up after relocation
> 
>  2) Write a driver using malloc() which you never thought to use prior to
>     relocation and it blows up because someone used it pre-relocation or
>     in SPL and didn't convert it to use early_malloc()
> 
> Neither can be picked up by at build time...

Yes.

> >> There is a third path possible: We can provide early_malloc() and say
> >> wrapped_malloc() which can be the third function "give me memory, I do
> >> not care whether it is early or not". So drivers and/or DM can choose
> >> to use malloc routines working in early-only, late-only or both.
> 
> Third path is dm_malloc() - Although ugly, it has a few nicities...
> 
>  1) It wraps malloc() and early_malloc() around a gd->flags & GD_FLG_RELOC
>     test
>  2) We can pass a pointer to a driver_core struct (or whatever struct it
>     is that holds the 'reloc' helper function pointer). We can't pick up
>     misuse at compile time, but dm_malloc() can print a meaningful message
>     if it is called pre-relocation with no relocation function. (We should
>     add a flag to indicate that no relocation helper is required which may
>     be the case for very simple drivers)
>  3) We can see right away when driver developers forget to use it
>  4) It will (hopefully) get past Wolfgang

:-D

>  5) It can be implemented 'right now'
>  6) We can always come back later and replace it since the usage will be
>     consistent
> 
> So my vote would be for:
> 
> void *early_malloc(size_t bytes)
> {
>    ...blah...
> }
> 
> void *dm_malloc(struct driver_core *drv, size_t bytes)
> {
>   if (gd->flags & GD_FLG_RELOC) {
>     return malloc(bytes);
>   } else {
>     if (!drv->reloc && !(drv->flags & NO_RELOC_FUNC))
>       puts("ARRRRRGGGGGG - Early malloc with no reloc function!!!!");
>     return early_malloc(bytes);
>   }
> }
> 
> Let's leave it at that for the time being - my other thought of registering
> ealry_malloc relocation helpers can wait until someone other than DM needs
> to use early_malloc(). Until then, DM can deal with managing the calls to
> the relocation functions.

Let's wait for Thomas on that ... I'm fine with anything that works ;-)

> Hows that sound?
> 
> Regards,
> 
> Graeme