Re: [External] Re: [PATCH v19 7/8] mm: hugetlb: add a kernel parameter hugetlb_free_vmemmap

[David Hildenbrand <david@redhat.com>]

On 19.03.21 13:36, David Hildenbrand wrote:
> On 19.03.21 13:15, Muchun Song wrote:
>> On Fri, Mar 19, 2021 at 4:59 PM Oscar Salvador <osalvador@suse.de> wrote:
>>>
>>> On Mon, Mar 15, 2021 at 05:20:14PM +0800, Muchun Song wrote:
>>>> --- a/arch/x86/mm/init_64.c
>>>> +++ b/arch/x86/mm/init_64.c
>>>> @@ -34,6 +34,7 @@
>>>>    #include <linux/gfp.h>
>>>>    #include <linux/kcore.h>
>>>>    #include <linux/bootmem_info.h>
>>>> +#include <linux/hugetlb.h>
>>>>
>>>>    #include <asm/processor.h>
>>>>    #include <asm/bios_ebda.h>
>>>> @@ -1557,7 +1558,8 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node,
>>>>    {
>>>>         int err;
>>>>
>>>> -     if (end - start < PAGES_PER_SECTION * sizeof(struct page))
>>>> +     if ((is_hugetlb_free_vmemmap_enabled()  && !altmap) ||
>>>> +         end - start < PAGES_PER_SECTION * sizeof(struct page))
>>>>                 err = vmemmap_populate_basepages(start, end, node, NULL);
>>>>         else if (boot_cpu_has(X86_FEATURE_PSE))
>>>>                 err = vmemmap_populate_hugepages(start, end, node, altmap);
>>>
>>> I've been thinking about this some more.
>>>
>>> Assume you opt-in the hugetlb-vmemmap feature, and assume you pass a valid altmap
>>> to vmemmap_populate.
>>> This will lead to use populating the vmemmap array with hugepages.
>>
>> Right.
>>
>>>
>>> What if then, a HugeTLB gets allocated and falls within that memory range (backed
>>> by hugetpages)?
>>
>> I am not sure whether we can allocate the HugeTLB pages from there.
>> Will only device memory pass a valid altmap parameter to
>> vmemmap_populate()? If yes, can we allocate HugeTLB pages from
>> device memory? Sorry, I am not an expert on this.
> 
> I think, right now, yes. System RAM that's applicable for HugePages
> never uses an altmap. But Oscar's patch will change that, maybe before
> your series might get included from what I've been reading. [1]
> 
> [1] https://lkml.kernel.org/r/20210319092635.6214-1-osalvador@suse.de
> 
>>
>>
>>> AFAIK, this will get us in trouble as currently the code can only operate on memory
>>> backed by PAGE_SIZE pages, right?
>>>
>>> I cannot remember, but I do not think nothing prevents that from happening?
>>> Am I missing anything?
>>
>> Maybe David H is more familiar with this.
>>
>> Hi David,
>>
>> Do you have some suggestions on this?
> 
> There has to be some way to identify whether we can optimize specific
> vmemmap pages or should just leave them alone. altmap vs. !altmap.
> 
> Unfortunately, there is no easy way to detect that - e.g.,
> PageReserved() applies also to boot memory.
> 
> We could go back to setting a special PageType for these vmemmap pages,
> indicating "this is a page allocated from an altmap, don't touch it".
> 

With SPARSEMEM we can use

PageReserved(page) && early_section(): vmemmap from bootmem

PageReserved(page) && !early_section(): vmemmap from altmap

!PageReserved(page): vmemmap from buddy

But it's a bit shaky :)

-- 
Thanks,

David / dhildenb