memblock versus bootmem (relevant for sparc32?)

memblock versus bootmem (relevant for sparc32?)

[Sam Ravnborg]

Hi Ben, Yinghai, Ingo.

While doing some sparc related tasks I noticed that sparc64
uses memblock - whereas sparc32 does not.

I am aware that memblock was called lmb in the older days and
that sparc64 had lmb support which was converted to memblock support.

But so far I have failed to find any information on what the benefit
of memblock is and how it compared to for example bootmem.
I can see that x86 no longer uses bootmem - as the only arch.

But I also noticed that mm/nobootmem.c contains several functions
named bla_x86_bla(). So I continued to be a bit confused.

Do there exist any writeup of what memblock is?

Also does it make sense to consider introducing memblock for sparc32.
And if yes - what is the benefit and what does it replace?

And would it make sense to drop bootmem on sparc64?

I know - lots of questions!

	Sam

Re: memblock versus bootmem (relevant for sparc32?)

[Benjamin Herrenschmidt]

On Sat, 2011-04-23 at 19:42 +0200, Sam Ravnborg wrote:
> Hi Ben, Yinghai, Ingo.
> 
> While doing some sparc related tasks I noticed that sparc64
> uses memblock - whereas sparc32 does not.
> 
> I am aware that memblock was called lmb in the older days and
> that sparc64 had lmb support which was converted to memblock support.
> 
> But so far I have failed to find any information on what the benefit
> of memblock is and how it compared to for example bootmem.
> I can see that x86 no longer uses bootmem - as the only arch.

Mostly memblock works earlier and handles both the actual physical
memory layout and allocation within that memory. Archs like powerpc
still use memblock, then bootmem, then page alloc.

x86 eventually removed the bootmem step in the middle, which is a good
thing, unfortunately...

> But I also noticed that mm/nobootmem.c contains several functions
> named bla_x86_bla(). So I continued to be a bit confused.

 ... as you noticed, this is all quite horrible with x86 junk in generic
places etc... and so I haven't yet looked at doing that on powerpc as
well (though I'd like to at some stage).
 
> Do there exist any writeup of what memblock is?

Other than the code ? :-) It's a very simple region list management
which works on two lists, one represents the physical memory in the
system and the other represents reserved areas and can be used as an
allocator (or to reserve "FW" regions etc...).

It initially worked entirely on static arrays (it used to be that way so
it can be used very early during boot), but it now grew the ability to
re-allocate its own arrays so that it can be used more generically as an
allocator instead of bootmem.

It's probably slower than bootmem tho, but on the other hand it
shouldn't be on any critical path.

> Also does it make sense to consider introducing memblock for sparc32.
> And if yes - what is the benefit and what does it replace?

I will let DaveM reply here :-)

> And would it make sense to drop bootmem on sparc64?
> 
> I know - lots of questions!

Cheers,
Ben.

> 	Sam

Re: memblock versus bootmem (relevant for sparc32?)

[Sam Ravnborg]

Hi Ben.

On Sun, Apr 24, 2011 at 09:20:46AM +1000, Benjamin Herrenschmidt wrote:
> On Sat, 2011-04-23 at 19:42 +0200, Sam Ravnborg wrote:
> > Hi Ben, Yinghai, Ingo.
> > 
> > While doing some sparc related tasks I noticed that sparc64
> > uses memblock - whereas sparc32 does not.
> > 
> > I am aware that memblock was called lmb in the older days and
> > that sparc64 had lmb support which was converted to memblock support.
> > 
> > But so far I have failed to find any information on what the benefit
> > of memblock is and how it compared to for example bootmem.
> > I can see that x86 no longer uses bootmem - as the only arch.
> 
> Mostly memblock works earlier and handles both the actual physical
> memory layout and allocation within that memory. Archs like powerpc
> still use memblock, then bootmem, then page alloc.
> 
> x86 eventually removed the bootmem step in the middle, which is a good
> thing, unfortunately...
> 
> > But I also noticed that mm/nobootmem.c contains several functions
> > named bla_x86_bla(). So I continued to be a bit confused.
> 
>  ... as you noticed, this is all quite horrible with x86 junk in generic
> places etc... and so I haven't yet looked at doing that on powerpc as
> well (though I'd like to at some stage).
>  
> > Do there exist any writeup of what memblock is?
> 
> Other than the code ? :-) It's a very simple region list management
> which works on two lists, one represents the physical memory in the
> system and the other represents reserved areas and can be used as an
> allocator (or to reserve "FW" regions etc...).
> 
> It initially worked entirely on static arrays (it used to be that way so
> it can be used very early during boot), but it now grew the ability to
> re-allocate its own arrays so that it can be used more generically as an
> allocator instead of bootmem.
> 
> It's probably slower than bootmem tho, but on the other hand it
> shouldn't be on any critical path.

Seems that my initial impression is OK. But I was confused by the x86
specific parts.

> 
> > Also does it make sense to consider introducing memblock for sparc32.
> > And if yes - what is the benefit and what does it replace?

I have digged a bit deeper in the code now. sparc32 has a local implementation
that handle the availabe memory which it then handed over to bootmem.
It would be a nice cleanup to get this local implementation migrated
over to memblock. Both in terms of using generic code but also
to be more aligned with sparc64.

Replacing bootmem seems like a bigger task - that will wait.

	Sam

Re: memblock versus bootmem (relevant for sparc32?)

[Yinghai Lu]

On 04/24/2011 01:22 AM, Sam Ravnborg wrote:
> Hi Ben.
> 
> On Sun, Apr 24, 2011 at 09:20:46AM +1000, Benjamin Herrenschmidt wrote:
>> On Sat, 2011-04-23 at 19:42 +0200, Sam Ravnborg wrote:
>>> Hi Ben, Yinghai, Ingo.
>>>
>>> While doing some sparc related tasks I noticed that sparc64
>>> uses memblock - whereas sparc32 does not.
>>>
>>> I am aware that memblock was called lmb in the older days and
>>> that sparc64 had lmb support which was converted to memblock support.
>>>
>>> But so far I have failed to find any information on what the benefit
>>> of memblock is and how it compared to for example bootmem.
>>> I can see that x86 no longer uses bootmem - as the only arch.
>>
>> Mostly memblock works earlier and handles both the actual physical
>> memory layout and allocation within that memory. Archs like powerpc
>> still use memblock, then bootmem, then page alloc.
>>
>> x86 eventually removed the bootmem step in the middle, which is a good
>> thing, unfortunately...
>>
>>> But I also noticed that mm/nobootmem.c contains several functions
>>> named bla_x86_bla(). So I continued to be a bit confused.
>>
>>  ... as you noticed, this is all quite horrible with x86 junk in generic
>> places etc... and so I haven't yet looked at doing that on powerpc as
>> well (though I'd like to at some stage).
>>  
>>> Do there exist any writeup of what memblock is?
>>
>> Other than the code ? :-) It's a very simple region list management
>> which works on two lists, one represents the physical memory in the
>> system and the other represents reserved areas and can be used as an
>> allocator (or to reserve "FW" regions etc...).
>>
>> It initially worked entirely on static arrays (it used to be that way so
>> it can be used very early during boot), but it now grew the ability to
>> re-allocate its own arrays so that it can be used more generically as an
>> allocator instead of bootmem.
>>
>> It's probably slower than bootmem tho, but on the other hand it
>> shouldn't be on any critical path.
> 
> Seems that my initial impression is OK. But I was confused by the x86
> specific parts.

that three references _x86_ in nobootmem.c

mm/nobootmem.c: memblock_x86_reserve_range(addr, addr + size, "BOOTMEM");
mm/nobootmem.c: memblock_x86_free_range(physaddr, physaddr + size);
mm/nobootmem.c: memblock_x86_free_range(addr, addr + size);


and difference to memblock_reserve/free is debug printing, and do not panic if start=end.

void __init memblock_x86_reserve_range(u64 start, u64 end, char *name)
{
        if (start = end)
                return;

        if (WARN_ONCE(start > end, "memblock_x86_reserve_range: wrong range [%#llx, %#llx)\n", start, end))
                return;

        memblock_dbg("    memblock_x86_reserve_range: [%#010llx-%#010llx] %16s\n", start, end - 1, name);

        memblock_reserve(start, end - start);
}

void __init memblock_x86_free_range(u64 start, u64 end)
{
        if (start = end)
                return;

        if (WARN_ONCE(start > end, "memblock_x86_free_range: wrong range [%#llx, %#llx)\n", start, end))
                return;

        memblock_dbg("       memblock_x86_free_range: [%#010llx-%#010llx]\n", start, end - 1);

        memblock_free(start, end - start);
}


> 
>>
>>> Also does it make sense to consider introducing memblock for sparc32.
>>> And if yes - what is the benefit and what does it replace?
> 
> I have digged a bit deeper in the code now. sparc32 has a local implementation
> that handle the availabe memory which it then handed over to bootmem.
> It would be a nice cleanup to get this local implementation migrated
> over to memblock. Both in terms of using generic code but also
> to be more aligned with sparc64.
> 
> Replacing bootmem seems like a bigger task - that will wait.

will need to move get_free_all_memory_range() and related funcs from arch/x86/mm/memblock.c to mm/memblock.c or mm/nobootmem.c

replacing bootmem with memblock in arch code should be simple.


Thanks

Yinghai

Re: memblock versus bootmem (relevant for sparc32?)

[Sam Ravnborg]

> > 
> > Replacing bootmem seems like a bigger task - that will wait.
> 
> will need to move get_free_all_memory_range() and related funcs
> from arch/x86/mm/memblock.c to mm/memblock.c or mm/nobootmem.c

It would be great if you could look at this - then it would be more obvious what is needed
for other archs to support nobootmem.

> 
> replacing bootmem with memblock in arch code should be simple.

This is also my impression after looking at it.

As for sparc32 I will give it a try - but it will take a while as I need to finish
some other stuff first.

	Sam