Request for clarification regarding huge pages and remapping of virtual to physical address

Request for clarification regarding huge pages and remapping of virtual to physical address

[sunil bhargo]

[-- Attachment #1: Type: text/plain, Size: 577 bytes --]

Hi,
We have an application which relies on virtual address to physical address mapping to remain static. The buffer is allocated using malloc. It is in user space and we were using the mlock thinking that as it would prevent the swapping the va->pa address mapping would be static. But it seems to be a faulty assumption.
If the memory is allocated from huge pages by reserving space upfront at the kernel boot up and then mmap’ing using MAP_HUGETLB, is it assured that virtual address to physical mapping won’t be remapped and will be static ?
Thanks,Sunil Bhargo 

[-- Attachment #2: Type: text/html, Size: 1249 bytes --]

Re: Request for clarification regarding huge pages and remapping of virtual to physical address

[David Hildenbrand]

On 17.04.24 11:11, sunil bhargo wrote:
> Hi,
> 
> We have an application which relies on virtual address to physical 
> address mapping to remain static. The buffer is allocated using 
> malloc. It is in user space and we were using the mlock thinking that as 
> it would prevent the swapping the va->pa address mapping would be 
> static. But it seems to be a faulty assumption.
> 
> If the memory is allocated from huge pages by reserving space upfront at 
> the kernel boot up and then mmap’ing using MAP_HUGETLB, is it assured 
> that virtual address to physical mapping won’t be remapped and will be 
> static ?

Unless you have a kernel that has page migration disabled, no.

You could use secretmem (which is currently not migratabale/swappable/ 
...), but that has very limited capabilities.

A "hack" would be to register that (ordinary) memory using iouring as a 
fixed buffer. That way, it will be longterm pinned and can neither get 
migrated nor swapped out.

-- 
Cheers,

David / dhildenb

Re: Request for clarification regarding huge pages and remapping of virtual to physical address

[sunil bhargo]

[-- Attachment #1: Type: text/plain, Size: 1331 bytes --]

 David,Thanks for your response. So if i get it right then reserving the memory at boot time with/without the huge page, the page migration can happen ? 

Thanks,Sunil Bhargo

    On Wednesday, April 17, 2024 at 02:47:30 PM GMT+5:30, David Hildenbrand <david@redhat.com> wrote:  
 
 On 17.04.24 11:11, sunil bhargo wrote:
> Hi,
> 
> We have an application which relies on virtual address to physical 
> address mapping to remain static. The buffer is allocated using 
> malloc. It is in user space and we were using the mlock thinking that as 
> it would prevent the swapping the va->pa address mapping would be 
> static. But it seems to be a faulty assumption.
> 
> If the memory is allocated from huge pages by reserving space upfront at 
> the kernel boot up and then mmap’ing using MAP_HUGETLB, is it assured 
> that virtual address to physical mapping won’t be remapped and will be 
> static ?

Unless you have a kernel that has page migration disabled, no.

You could use secretmem (which is currently not migratabale/swappable/ 
...), but that has very limited capabilities.

A "hack" would be to register that (ordinary) memory using iouring as a 
fixed buffer. That way, it will be longterm pinned and can neither get 
migrated nor swapped out.

-- 
Cheers,

David / dhildenb

  

[-- Attachment #2: Type: text/html, Size: 2789 bytes --]

Re: Request for clarification regarding huge pages and remapping of virtual to physical address

[David Hildenbrand]

On 17.04.24 12:09, sunil bhargo wrote:
> David,
> Thanks for your response. So if i get it right then reserving the memory 
> at boot time with/without the huge page, the page migration can happen ?

I *think* even hugetlb folios allocated/reserved during early boot can 
be migrated in memory later. The chance of that happening is small, 
though. Or I might be wrong :)

Migration of gigantic huge pages (e.g., 1 GiB) is even more unlikely to 
happen.

-- 
Cheers,

David / dhildenb