Re: Process-shared futexes on hugepages puts the kernel in an infinite loop in 2.6.32.11; is this fixed now?

[Peter Zijlstra <peterz@infradead.org>]

On Fri, 2010-04-16 at 23:45 +0800, r6144 wrote:
> Hello all,
> 
> I'm having an annoying kernel bug regarding huge pages in Fedora 12:
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=552257
> 
> Basically I want to use huge pages in a multithreaded number crunching
> program, which happens to use process-shared semaphores (because fftw
> does it).  The futex for the semaphore ends up lying on a huge page, and
> I then get an endless loop in get_futex_key(), apparently because the
> anonymous huge page containing the futex does not have a page->mapping.
> A test case is provided in the above link.
> 
> I reported the bug to Fedora bugzilla months ago, but haven't received
> any feedback yet. 

No, it works much better if you simply mail LKML and CC people who work
on the code in question ;-)

>  The Fedora kernel is based on 2.6.32.11, and a
> cursory glance at the 2.6.34-rc3 source does not yield any relevant
> change.
> 
> So, could anyone tell me if the current mainline kernel might act better
> in this respect, before I get around to compiling it?

Right, so I had a quick chat with Mel, and it appears MAP_PRIVATE
hugetlb pages don't have their page->mapping set.

I guess something like the below might work, but I'd really rather not
add hugetlb knowledge to futex.c. Does anybody else have a better idea?
Maybe create something similar to an anon_vma for hugetlb pages?

---
 kernel/futex.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/futex.c b/kernel/futex.c
index e7a35f1..b0f1b2d 100644
--- a/kernel/futex.c
+++ b/kernel/futex.c
@@ -252,7 +252,7 @@ again:
 
 	page = compound_head(page);
 	lock_page(page);
-	if (!page->mapping) {
+	if (!page->mapping && !PageHuge(page)) {
 		unlock_page(page);
 		put_page(page);
 		goto again;
@@ -265,7 +265,7 @@ again:
 	 * it's a read-only handle, it's expected that futexes attach to
 	 * the object not the particular process.
 	 */
-	if (PageAnon(page)) {
+	if (PageAnon(page) || (PageHuge(page) && !page->mapping)) {
 		key->both.offset |= FUT_OFF_MMSHARED; /* ref taken on mm */
 		key->private.mm = mm;
 		key->private.address = address;