[PATCH] fix hugepages leak due to pagetable page sharing.

[Larry Woodman <lwoodman@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 1359 bytes --]

The shared page table code for hugetlb memory on x86 and x86_64
is causing a leak.  When a user of hugepages exits using this code
the system leaks some of the hugepages.

-------------------------------------------------------
Part of /proc/meminfo just before database startup:
HugePages_Total:  5500
HugePages_Free:   5500
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Just before shutdown:
HugePages_Total:  5500
HugePages_Free:   4475
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

After shutdown:
HugePages_Total:  5500
HugePages_Free:   4988
HugePages_Rsvd:     
0 Hugepagesize:     2048 kB
----------------------------------------------------------

The problem occurs durring a fork, in copy_hugetlb_page_range(). It 
locates the dst_pte using
huge_pte_alloc().  Since huge_pte_alloc() calls huge_pmd_share() it will 
share the pmd page
if can, yet the main loop in copy_hugetlb_page_range() does a get_page() 
on every hugepage.
This is a violation of the shared hugepmd pagetable protocol and creates 
additional referenced
to the hugepages causing a leak when the unmap of the VMA occurs.   We 
can skip the entire
replication of the ptes when the hugepage pagetables are shared.
The attached patch skips copying the ptes and the get_page() calls if 
the hugetlbpage pagetable
is shared.

Signed-off-by: Larry Woodman <lwoodman@redhat.com>

[-- Attachment #2: linux-shared.patch --]
[-- Type: text/plain, Size: 487 bytes --]

--- linux-2.6.23/mm/hugetlb.c.orig	2008-01-16 12:05:41.496448000 -0500
+++ linux-2.6.23/mm/hugetlb.c	2008-01-17 10:27:21.740353000 -0500
@@ -377,6 +377,11 @@ int copy_hugetlb_page_range(struct mm_st
 		dst_pte = huge_pte_alloc(dst, addr);
 		if (!dst_pte)
 			goto nomem;
+
+		/* if the pagetables are shared dont copy or take references */
+		if(dst_pte == src_pte)
+			continue;
+
 		spin_lock(&dst->page_table_lock);
 		spin_lock(&src->page_table_lock);
 		if (!pte_none(*src_pte)) {