use bootmem in platform code on MIPS

use bootmem in platform code on MIPS

[Manuel Lauss]

Hello,

I'd like to use bootmem to reserve large chunks of RAM (at a particular physical
address; for Au1200 MAE, CIM and framebuffer, and later Au1300 OpenGL block)
but it seems that it can't be done:  Doing __alloc_bootmem() in
plat_mem_setup() is
too early, while an arch_initcall() is too late because by then the
slab allocator is
already up and handing out random addresses and/or refusing allocations larger
than a few MBytes.

Is there another callback I could use which would allow me to use bootmem (short
of abusing plat_smp_setup)?

Would a separate callback like this be an acceptable solution?

--- a/arch/mips/kernel/setup.c
+++ b/arch/mips/kernel/setup.c
@@ -487,6 +487,10 @@ static void __init arch_mem_init(char **cmdline_p)
        }

        bootmem_init();
+
+       if (plat_bootmem_init)
+               plat_bootmem_init();
+
        sparse_init();
        paging_init();
 }

Thanks,
     Manuel Lauss

Re: use bootmem in platform code on MIPS

[Ralf Baechle]

On Tue, Apr 27, 2010 at 02:14:32PM +0200, Manuel Lauss wrote:

> I'd like to use bootmem to reserve large chunks of RAM (at a particular physical
> address; for Au1200 MAE, CIM and framebuffer, and later Au1300 OpenGL block)
> but it seems that it can't be done:  Doing __alloc_bootmem() in
> plat_mem_setup() is
> too early, while an arch_initcall() is too late because by then the
> slab allocator is
> already up and handing out random addresses and/or refusing allocations larger
> than a few MBytes.

The maximum is actually configurable.  CONFIG_FORCE_MAX_ZONEORDER defaults
to 11 which means with 4kB pages you get 8MB maximum allocation - more for
larger pages.

CONFIG_FORCE_MAX_ZONEORDER is a tradeoff though.  A smaller value will give
slightly better performance and safe a bit of memory but I can't really
quantify these numbers - I assume it's a small difference.

It may actually be preferable to never tell the bootmem allocator about the
memory you need for these devices that is bypass the mm code entirely.

> Is there another callback I could use which would allow me to use bootmem (short
> of abusing plat_smp_setup)?
> 
> Would a separate callback like this be an acceptable solution?

Certainly better than using plat_smp_setup which would require enabling
SMP support for no good reason at all.

I know we will eventually have to add another platform hooks to run after
bootmem_init.  The name of plat_mem_setup() already shows what this hook
originally was meant for but it ended up as the everything-and-the-kitchen-
sink hook for platform-specific early initialization.  I just dislike
conditional hooks.  Let's add a call to a new hook function and fix whatever
breaks or think about what other hooks needs there should be.

  Ralf

Re: use bootmem in platform code on MIPS

[Manuel Lauss]

On Wed, Apr 28, 2010 at 5:54 PM, Ralf Baechle <ralf@linux-mips.org> wrote:
> On Tue, Apr 27, 2010 at 02:14:32PM +0200, Manuel Lauss wrote:
>
>> I'd like to use bootmem to reserve large chunks of RAM (at a particular physical
>> address; for Au1200 MAE, CIM and framebuffer, and later Au1300 OpenGL block)
>> but it seems that it can't be done:  Doing __alloc_bootmem() in
>> plat_mem_setup() is
>> too early, while an arch_initcall() is too late because by then the
>> slab allocator is
>> already up and handing out random addresses and/or refusing allocations larger
>> than a few MBytes.
>
> The maximum is actually configurable.  CONFIG_FORCE_MAX_ZONEORDER defaults
> to 11 which means with 4kB pages you get 8MB maximum allocation - more for
> larger pages.

I already had to modify it for large display resolutions.


> CONFIG_FORCE_MAX_ZONEORDER is a tradeoff though.  A smaller value will give
> slightly better performance and safe a bit of memory but I can't really
> quantify these numbers - I assume it's a small difference.
>
> It may actually be preferable to never tell the bootmem allocator about the
> memory you need for these devices that is bypass the mm code entirely.

Do you mean by not adding the whole RAM area with add_memory_region()?
Can I give the memory back later (if it's not required)?  Right now I think with
bootmem that is actually possible.


>> Is there another callback I could use which would allow me to use bootmem (short
>> of abusing plat_smp_setup)?
>>
>> Would a separate callback like this be an acceptable solution?
>
> Certainly better than using plat_smp_setup which would require enabling
> SMP support for no good reason at all.
>
> I know we will eventually have to add another platform hooks to run after
> bootmem_init.  The name of plat_mem_setup() already shows what this hook
> originally was meant for but it ended up as the everything-and-the-kitchen-
> sink hook for platform-specific early initialization.  I just dislike

The comment above arch_mem_init() too mentions a separate function.


> conditional hooks.  Let's add a call to a new hook function and fix whatever
> breaks or think about what other hooks needs there should be.

Okay, I'll cook something up.

Thank you,
        Manuel Lauss

Re: use bootmem in platform code on MIPS

[David Daney]

On 04/28/2010 09:28 AM, Manuel Lauss wrote:
> On Wed, Apr 28, 2010 at 5:54 PM, Ralf Baechle<ralf@linux-mips.org>  wrote:
>> On Tue, Apr 27, 2010 at 02:14:32PM +0200, Manuel Lauss wrote:
>>
>>> I'd like to use bootmem to reserve large chunks of RAM (at a particular physical
>>> address; for Au1200 MAE, CIM and framebuffer, and later Au1300 OpenGL block)
>>> but it seems that it can't be done:  Doing __alloc_bootmem() in
>>> plat_mem_setup() is
>>> too early, while an arch_initcall() is too late because by then the
>>> slab allocator is
>>> already up and handing out random addresses and/or refusing allocations larger
>>> than a few MBytes.
>>
>> The maximum is actually configurable.  CONFIG_FORCE_MAX_ZONEORDER defaults
>> to 11 which means with 4kB pages you get 8MB maximum allocation - more for
>> larger pages.
>
> I already had to modify it for large display resolutions.

You also have to modify it for huge pages combined with larger pages.

I have:

config FORCE_MAX_ZONEORDER
	int "Maximum zone order"
	range 13 64 if SYS_SUPPORTS_HUGETLBFS && PAGE_SIZE_32KB
	default "13" if SYS_SUPPORTS_HUGETLBFS && PAGE_SIZE_32KB
	range 12 64 if SYS_SUPPORTS_HUGETLBFS && PAGE_SIZE_16KB
	default "12" if SYS_SUPPORTS_HUGETLBFS && PAGE_SIZE_16KB
	range 11 64
	default "11"
	help
	  The kernel memory allocator divides physically contiguous memory
	  blocks into "zones", where each zone is a power of two number of
	  pages.  This option selects the largest power of two that the kernel
	  keeps in the memory allocator.  If you need to allocate very large
	  blocks of physically contiguous memory, then you may need to
	  increase this value.

	  This config option is actually maximum order plus one. For example,
	  a value of 11 means that the largest free memory block is 2^10 pages.

	  The page size is not necessarily 4KB.  Keep this in mind
	  when choosing a value for this option.




>
>
>> CONFIG_FORCE_MAX_ZONEORDER is a tradeoff though.  A smaller value will give
>> slightly better performance and safe a bit of memory but I can't really
>> quantify these numbers - I assume it's a small difference.
>>
>> It may actually be preferable to never tell the bootmem allocator about the
>> memory you need for these devices that is bypass the mm code entirely.
>
> Do you mean by not adding the whole RAM area with add_memory_region()?
> Can I give the memory back later (if it's not required)?  Right now I think with
> bootmem that is actually possible.
>
>
>>> Is there another callback I could use which would allow me to use bootmem (short
>>> of abusing plat_smp_setup)?
>>>
>>> Would a separate callback like this be an acceptable solution?
>>
>> Certainly better than using plat_smp_setup which would require enabling
>> SMP support for no good reason at all.
>>
>> I know we will eventually have to add another platform hooks to run after
>> bootmem_init.  The name of plat_mem_setup() already shows what this hook
>> originally was meant for but it ended up as the everything-and-the-kitchen-
>> sink hook for platform-specific early initialization.  I just dislike
>
> The comment above arch_mem_init() too mentions a separate function.
>
>
>> conditional hooks.  Let's add a call to a new hook function and fix whatever
>> breaks or think about what other hooks needs there should be.
>
> Okay, I'll cook something up.
>
> Thank you,
>          Manuel Lauss
>
>