Re: Need some help in understanding sparsemem.

[Minchan Kim <minchan.kim@gmail.com>]

On Tue, Jul 6, 2010 at 3:07 PM, KAMEZAWA Hiroyuki
<kamezawa.hiroyu@jp.fujitsu.com> wrote:
> On Tue, 6 Jul 2010 10:41:06 +0530
> naren.mehra@gmail.com wrote:
>
>> Hi,
>>
>> I am trying to understand the sparsemem implementation in linux for
>> NUMA/multiple node systems.
>>
>> From the available documentation and the sparsemem patches, I am able
>> to make out that sparsemem divides memory into different sections and
>> if the whole section contains a hole then its marked as invalid
>> section and if some pages in a section form a hole then those pages
>> are marked reserved. My issue is that this classification, I am not
>> able to map it to the code.
>>
>> e.g. from arch specific code, we call memory_present()  to prepare a
>> list of sections in a particular node. but unable to find where
>> exactly some sections are marked invalid because they contain a hole.
>>
>> Can somebody tell me where in the code are we identifying sections as
>> invalid and where we are marking pages as reserved.
>>
>
> As you wrote, memory_present() is just for setting flags
> "SECTION_MARKED_PRESENT". If a section contains both of valid pages and
> holes, the section itself is marked as SECTION_MARKED_PRESENT.
>
> This memory_present() is called in very early stage. The function which allocates
> mem_map(array of struct page) is sparse_init(). It's called somewhere after
> memory_present().
> (In x86, it's called by paging_init(), in ARM, it's called by bootmem_init()).
>
> After sparse_init(), mem_maps are allocated. (depends on config..plz see codes.)
> But, here, mem_map is not initialized.
> This is because initialization logic of memmap doesn't depend on
> FLATMEM/DISCONTIGMEM/SPARSEMEM.
>
> After sprase_init(), mem_map is allocated. It's not encouraged to detect a section
> is valid or invalid but you can use pfn_valid() to check there are memmap or not.
> (*) pfn_valid(pfn) is not for detecting there is memory but for detecting
>    there is memmap.
>
> Initializing mem_map is done by free_area_init_node(). This function initializes
> memory range regitered by add_active_range() (see mm/page_alloc.c)
> (*)There are architecutures which doesn't use add_active_range(), but this function
>   is for generic use.
>
> After free_area_init_node(), all mem_map are initialized as PG_reserved and
> NODE_DATA(nid)->star_pfn, etc..are available.
>
> When PG_reserved is cleared is at free_all_bootmem(). If you want to keep pages
> as Reserved (because of holes), OR, don't register memory hole as bootmem.
> Then, pages will be kept as Reserved.
>
> clarification:
>  memory_present().... prepare for section[] and mark up PRESENT.
>  sparse_init()   .... allocates mem_map. but just allocates it.
>  free_area_init_node() .... initizalize mem_map at el.
>  free_all_bootmem() .... make pages available and put into buddy allocator.
>
>  pfn_valid() ... useful for checking there are mem_map.

Kame explained greatly.
I want to elaborate on pfn_valid but it's off-topic. ;)

The pfn_valid isn't enough on ARM if you walk whole memmap.
That's because ARM frees memmap on hole to save the memory by
free_unused_memmap_node.

In such case, you have to use memmap_valid_within.

-- 
Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>