Transparent Hugepage Support #22

Transparent Hugepage Support #22

[Andrea Arcangeli]

http://git.kernel.org/?p=linux/kernel/git/andrea/aa.git;a=shortlog

first: git clone git://git.kernel.org/pub/scm/linux/kernel/git/andrea/aa.git
or first: git clone --reference linux-2.6 git://git.kernel.org/pub/scm/linux/kernel/git/andrea/aa.git
later: git fetch; git checkout -f origin/master; git diff a41f0dcfdbebeac21b42e152d3ed9f4bf20070a3

The tree is rebased and git pull won't work.

http://www.kernel.org/pub/linux/kernel/people/andrea/patches/v2.6/2.6.34-rc5/transparent_hugepage-22
http://www.kernel.org/pub/linux/kernel/people/andrea/patches/v2.6/2.6.34-rc5/transparent_hugepage-22.gz

This fixes the swapops.h crash in migrate.c:migration_entry_wait
(Kame/Mel if you could test if you can still reproduce your memory
compaction crash on aa.git let me know, this fix is here
http://git.kernel.org/?p=linux/kernel/git/andrea/aa.git;a=patch;h=6efa1dfa5152ef8d7f26beb188d6877525a9dd03). It
also fixes potential memory corruption when swapping out hugepage
backed kvm (host_level wasn't read inside the mmu notifier protected
critical section of the page fault). It cleanups some bits in
khugepaged and notably it's removing
transparent_hugepage/khugepaged/enabled. Now khugepaged is started
automatically if transparent_hugepage/enabled is set to
"always|madvise" and it exits if it's set to "never" (I found it too
confusing to have it separated and an unnecessary annoyance having to
change two files instead of just one). I removed the dependency on
embedded to set transparent hugepage support to N at compile time, and
I leave the default to N to reduce as much as possible the risk while
merging the feature.

All reports I have says it's very stable (the only two open issues in
migrate vs execve and in the kvm patch should be fixed in #22).

There is no easy way this can work with the new anon-vma code until I
adapt it significantly (as an example see how wait_split_huge_page is
implemented right now
http://git.kernel.org/?p=linux/kernel/git/andrea/aa.git;a=blob_plain;f=include/linux/huge_mm.h;hb=HEAD),
and I'll have to consider if it's possible to move completely away
from the anon-vma lock and use compound_lock or something else. For
now this has to run on the old stable anon-vma code to be stable.

I added more details to the generic document.

http://git.kernel.org/?p=linux/kernel/git/andrea/aa.git;a=blob_plain;f=Documentation/vm/transhuge.txt;hb=HEAD

If anybody wants a patchbomb let me know.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Transparent Hugepage Support #22

[Ingo Molnar]

* Andrea Arcangeli <aarcange@redhat.com> wrote:

> If anybody wants a patchbomb let me know.

It would be nice and informative to have two diffstats in the announcement:

- an 'absolute' one that shows all the hugetlb changes relative to upstream 
  (or relative to -mm, whichever tree you use as a base),

- and [if possible] a 'delta' one that shows the diffstat to the previous
  version you've announced. [say in this current case the #21..#22 delta 
  diffstat] [this might not always be easy to provide, when the upstream base 
  changes.]

That way people can see the general direction and scope from the email, 
without having to fetch any of the trees.

	Ingo

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Transparent Hugepage Support #22

[Andrea Arcangeli]

On Fri, Apr 30, 2010 at 10:54:27AM +0200, Ingo Molnar wrote:
> It would be nice and informative to have two diffstats in the announcement:
> 
> - an 'absolute' one that shows all the hugetlb changes relative to upstream 
>   (or relative to -mm, whichever tree you use as a base),

That's easy, I will do next times. I'm based on mainline right now.

> - and [if possible] a 'delta' one that shows the diffstat to the previous
>   version you've announced. [say in this current case the #21..#22 delta 
>   diffstat] [this might not always be easy to provide, when the upstream base 
>   changes.]

I've revision control on the quilt patchset, I can send the diffstat
of the quilt patchset.

You can also monitor the changes by running:

git fetch
git diff origin/master

before "git checkout origin/master".

but as you said that will also show the new mainline changes mixed
with my changes.

> That way people can see the general direction and scope from the email, 
> without having to fetch any of the trees.

It's informative yes, but I hope that people really fetch the tree,
review the changes with "git log -p" or just blind test it and run
some benchmark. Many already did and that is the only way to get
feedback (positive or negative) and to be sure that we're going into
the right direction. The only feedback I got so far from people
testing the tree has been exceedingly positive which in addition to
the benchmarks I run myself, makes me more confident this is going in
the right direction and helping a wider scope of workloads. The
research I did initially on the prefault logic I think helped me
keeping things simpler and more efficient and I think the decision
that it was worth it do larger copy-page/clear-page only if we also
get something more than a prefault speedup in return (as those copies
slowdown the page faults and trashes the cache) is paying off.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>