Re: Hugepages for shm page cache (defrag)

["Radosław Smogura" <mail@smogura.eu>]

[-- Attachment #1: Type: Text/Plain, Size: 2382 bytes --]

Andi Kleen <andi@firstfloor.org> Thursday 07 of July 2011 07:28:59
> Radosław Smogura <mail@smogura.eu> writes:
> > Hello,
> > 
> > This is may first try with Linux patch, so please do not blame me too
> > much. Actually I started with small idea to add MAP_HUGTLB for /dev/shm
> > but it grew up in something more like support for huge pages in page
> > cache, but according to documentation to submit alpha-work too, I
> > decided to send this.
> 
> Shouldn't this be rather integrated with the normal transparent huge
> pages? It seems odd to develop parallel infrastructure.
> 
> -Andi
Hello,

I send as attachment some preview of work, last time I putted many work to 
make code less messy, etc, but it is still messy and in alpha stage.

=== Info about changes and how it's works ===
Current work is focused to give support for huge pages for tmpfs, but I 
designed it for private mappings too (my dream will be if I will be able to 
map execution portions of glibc as huge). I think there will be no big change 
to expose THP for general file system.

I putted some effort to do not create new type of huge pages, so I do not use 
longer special flag for managing this.

PMD is establish (from historical reasons) from pte fault, so I unmaps pte's 
from pmd (I think I have memory leak here).

Splitting of huge page for page cache still need some work, and it has 
different concept. Mainly I use currently compound_lock to prohibit concurrent 
splitting etc. Splitting acquires compound lock, then unmaps all pmds, and 
split occurs.

In splitting there is usage of new ref-counting. I may be wrong, but in other 
cases we will need to manage ref-count bit for compound pages (this what is 
included /mm.h, swap.c/ is slightly different from my previous send; I still 
think if I should use irqsave lock for compounds). I think included construct 
removes all "under us" risks.

I don't want to waste Your time for reviewing this work, but I will be glad if 
you will find some time. One thing I worried is about pde locking is it needed 
to do pmd locking, like for pte (or any other kind of locking? I have some 
ideas)?

I hope this time here is no missing files.
Regards,
Radosław Smogura
P. S. In one of mails in CC there is putted wrongly my e-mail address and 
name, please remove it to avoid unknown recipient.

[-- Attachment #2: defrag-preview-20110807.patch.gz --]
[-- Type: application/x-gzip, Size: 20948 bytes --]