[RFC] calc_memmap_size() isn't accurate and one suggestion to improve

[RFC] calc_memmap_size() isn't accurate and one suggestion to improve

[Wei Yang]

[-- Attachment #1: Type: text/plain, Size: 1834 bytes --]

Hi, masters,

# What I found

I found the function calc_memmap_size() may not be that accurate to get the
pages for memmap.

The reason is:

> memmap is allocated on a node base,
> while the calculation is on a zone base

This applies both to SPARSEMEM and FLATMEM.

For example, on my laptop with 6G memory, all the memmap space is allocated
from ZONE_NORMAL.

# My suggestion 

Current code path is:

    sparse_init()                          <- memmap allocated
    zone_sizes_init()
        free_area_init_nodes()
	    calculate_node_totalpages()
	    free_area_init_core()          <- where we do the calculation

From the code snippet, memmap is already allocated in memblock, which
means we can get the information by comparing memblock.memory and
memblock.reserved.

My suggestion is to record this information in pg_data_t in
calculate_node_totalpages(), which is already doing the calculation on each
zone's spanned_pages and present_pages.

# Other solutions came to my mind

1. Assume all the memmap is allocated from the highest zone.

Pro:

Easy to calculate

Cor:

Not good to do this assumption. How to set the boundary. And there is the case
memory is allocated bottom-up.

2. Record the memmap area for each allocation

Pro:

Accurate and exact the size and zone index is recorded.

Cor:

Too expensive, especially when VMEMMAP && !ALLOC_MEM_MAP_TOGETHER. There would
be too many.

# Look for you comment

This code path applies to most of the arch, while I am not 100% for sure this
applies to all the arch. If isn't, this change may not be a good one.

The solution looks good to me, while I may miss some corner case or some
important facts.

Willing to hear from you :-)

-- 
Wei Yang
Help you, Help me

-- 
Wei Yang
Help you, Help me

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [RFC] calc_memmap_size() isn't accurate and one suggestion to improve

[Michal Hocko]

On Tue 28-03-17 09:11:37, Wei Yang wrote:
> Hi, masters,
> 
> # What I found
> 
> I found the function calc_memmap_size() may not be that accurate to get the
> pages for memmap.
> 
> The reason is:
> 
> > memmap is allocated on a node base,
> > while the calculation is on a zone base
> 
> This applies both to SPARSEMEM and FLATMEM.
> 
> For example, on my laptop with 6G memory, all the memmap space is allocated
> from ZONE_NORMAL.

Please try to be more specific. Why is this a problem? Are you trying to
fix some bad behavior or you want to make it more optimal?

I am sorry I didn't look closer into your proposal but I am quite busy
and other people are probably in a similar situation. If you want to get
a proper feedback please try to state the problem and be explicit if it
is user observable.
-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [RFC] calc_memmap_size() isn't accurate and one suggestion to improve

[Wei Yang]

[-- Attachment #1: Type: text/plain, Size: 1158 bytes --]

On Mon, Apr 03, 2017 at 11:18:19AM +0200, Michal Hocko wrote:
>On Tue 28-03-17 09:11:37, Wei Yang wrote:
>> Hi, masters,
>> 
>> # What I found
>> 
>> I found the function calc_memmap_size() may not be that accurate to get the
>> pages for memmap.
>> 
>> The reason is:
>> 
>> > memmap is allocated on a node base,
>> > while the calculation is on a zone base
>> 
>> This applies both to SPARSEMEM and FLATMEM.
>> 
>> For example, on my laptop with 6G memory, all the memmap space is allocated
>> from ZONE_NORMAL.
>
>Please try to be more specific. Why is this a problem? Are you trying to
>fix some bad behavior or you want to make it more optimal?
>
>I am sorry I didn't look closer into your proposal but I am quite busy
>and other people are probably in a similar situation. If you want to get
>a proper feedback please try to state the problem and be explicit if it
>is user observable.

Michal

Glad to hear from you.

Sure, let me do more investigation on this and try some experiment to see
whether this change is observable.

Have a nice day~

>-- 
>Michal Hocko
>SUSE Labs

-- 
Wei Yang
Help you, Help me

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]