Transparent Hugepage Nit

Transparent Hugepage Nit

[John McCorquodale]

Suppose a hugepage-aligned mmap(MAP_ANONYMOUS) mapping has been madvise()d
HUGEPAGE.  If a subeqeuent call to mremap() grows the mapping and has to
move the mapping, the hugepage-alignment is not preserved in the choice of 
new address (in 3.7.2).

I can workaround this by doing a 1-hugepage-oversized remap to find a new 
aligned address and then size it back down MREMAP_FIXED, but that's probably 
a lot of frags to 4k pages and back that aren't necessary.

Should it not be the case that mremap(MAYMOVE) on something advised hugepage
ALWAYS chooses a hugepage-aligned address?  This would be handy when doing the
initial allocation too: mmap, madvise, mremap (to the same size) to get
alignment.

Cheers,

-mcq

Re: Transparent Hugepage Nit

[Andi Kleen]

John McCorquodale <mcq@rockgeek.org> writes:

> Suppose a hugepage-aligned mmap(MAP_ANONYMOUS) mapping has been madvise()d
> HUGEPAGE.  If a subeqeuent call to mremap() grows the mapping and has to
> move the mapping, the hugepage-alignment is not preserved in the choice of 
> new address (in 3.7.2).
>
> I can workaround this by doing a 1-hugepage-oversized remap to find a new 
> aligned address and then size it back down MREMAP_FIXED, but that's probably 
> a lot of frags to 4k pages and back that aren't necessary.
>
> Should it not be the case that mremap(MAYMOVE) on something advised hugepage
> ALWAYS chooses a hugepage-aligned address?  This would be handy when doing the
> initial allocation too: mmap, madvise, mremap (to the same size) to get
> alignment.

The hole searching currently doesn't know anything about transparent
huge pages. There were some discussions on fixing it. But it's
essentially a trade off between memory fragmentation and huge page
optimization: aggressively aligning to 2MB can lose address space
in holes.

Usually if the program uses large enough mappings and enough memory 
it shouldn't be a problem.

-Andi
-- 
ak@linux.intel.com -- Speaking for myself only

Re: Transparent Hugepage Nit

[Rob Landley]

On 01/15/2013 03:26:58 PM, Andi Kleen wrote:
> John McCorquodale <mcq@rockgeek.org> writes:
> 
> > Suppose a hugepage-aligned mmap(MAP_ANONYMOUS) mapping has been  
> madvise()d
> > HUGEPAGE.  If a subeqeuent call to mremap() grows the mapping and  
> has to
> > move the mapping, the hugepage-alignment is not preserved in the  
> choice of
> > new address (in 3.7.2).
> >
> > I can workaround this by doing a 1-hugepage-oversized remap to find  
> a new
> > aligned address and then size it back down MREMAP_FIXED, but that's  
> probably
> > a lot of frags to 4k pages and back that aren't necessary.
> >
> > Should it not be the case that mremap(MAYMOVE) on something advised  
> hugepage
> > ALWAYS chooses a hugepage-aligned address?  This would be handy  
> when doing the
> > initial allocation too: mmap, madvise, mremap (to the same size) to  
> get
> > alignment.
> 
> The hole searching currently doesn't know anything about transparent
> huge pages. There were some discussions on fixing it. But it's
> essentially a trade off between memory fragmentation and huge page
> optimization: aggressively aligning to 2MB can lose address space
> in holes.
> 
> Usually if the program uses large enough mappings and enough memory
> it shouldn't be a problem.

Possibly we should just document that doing mremap() on a hugepage  
loses the hugepageness, and consider it pilot error to do that?

Rob