linux-mm.kvack.org archive mirror
 help / color / mirror / Atom feed
* Re: [PATCH 2/2] thp: support split page table lock
@ 2013-09-09  2:34 Daniel J Blueman
  2013-09-09 15:43 ` Alex Thorlton
  0 siblings, 1 reply; 14+ messages in thread
From: Daniel J Blueman @ 2013-09-09  2:34 UTC (permalink / raw)
  To: Naoya Horiguchi
  Cc: linux-mm, Andrew Morton, Mel Gorman, Andi Kleen, Michal Hocko,
	KOSAKI Motohiro, Rik van Riel, Andrea Arcangeli, kirill.shutemov,
	Aneesh Kumar K.V, LKML, Alex Thorlton, Steffen Persvold

On Saturday, 7 September 2013 02:10:02 UTC+8, Naoya Horiguchi  wrote:
> Hi Alex,
>
> On Fri, Sep 06, 2013 at 11:04:23AM -0500, Alex Thorlton wrote:
> > On Thu, Sep 05, 2013 at 05:27:46PM -0400, Naoya Horiguchi wrote:
> > > Thp related code also uses per process mm->page_table_lock now.
> > > So making it fine-grained can provide better performance.
> > >
> > > This patch makes thp support split page table lock by using page->ptl
> > > of the pages storing "pmd_trans_huge" pmds.
> > >
> > > Some functions like pmd_trans_huge_lock() and
page_check_address_pmd()
> > > are expected by their caller to pass back the pointer of ptl, so this
> > > patch adds to those functions new arguments for that. Rather than
that,
> > > this patch gives only straightforward replacement.
> > >
> > > ChangeLog v3:
> > >  - fixed argument of huge_pmd_lockptr() in copy_huge_pmd()
> > >  - added missing declaration of ptl in do_huge_pmd_anonymous_page()
> >
> > I've applied these and tested them using the same tests program that I
> > used when I was working on the same issue, and I'm running into some
> > bugs.  Here's a stack trace:
>
> Thank you for helping testing. This bug is new to me.

With 3.11, this patch series and CONFIG_TRANSPARENT_HUGEPAGE_ALWAYS, I 
consistently hit the same failure when exiting one of my stress-testers 
[1] when using eg 24 cores.

Doesn't happen with 8 cores, so likely needs enough virtual memory to 
use multiple split locks. Otherwise, this is very promising work!

[1] http://quora.org/2013/fft3d.c
-- 
Daniel J Blueman
Principal Software Engineer, Numascale

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 14+ messages in thread
* [PATCH 0/2 v3] split page table lock for hugepage
@ 2013-09-05 21:27 Naoya Horiguchi
  2013-09-05 21:27 ` [PATCH 2/2] thp: support split page table lock Naoya Horiguchi
  0 siblings, 1 reply; 14+ messages in thread
From: Naoya Horiguchi @ 2013-09-05 21:27 UTC (permalink / raw)
  To: linux-mm
  Cc: Andrew Morton, Mel Gorman, Andi Kleen, Michal Hocko,
	KOSAKI Motohiro, Rik van Riel, Andrea Arcangeli, kirill.shutemov,
	Aneesh Kumar K.V, Alex Thorlton, linux-kernel

I revised the split ptl patchset with small fixes.
See also the previous post [1] for the motivation and the numbers.

Any comments and reviews are welcomed.

[1] http://thread.gmane.org/gmane.linux.kernel.mm/106292/

Thanks,
Naoya Horiguchi
---
Summary:

Naoya Horiguchi (2):
      hugetlbfs: support split page table lock
      thp: support split page table lock

 arch/powerpc/mm/pgtable_64.c |   8 +-
 arch/s390/mm/pgtable.c       |   4 +-
 arch/sparc/mm/tlb.c          |   4 +-
 fs/proc/task_mmu.c           |  17 +++--
 include/linux/huge_mm.h      |  11 +--
 include/linux/hugetlb.h      |  20 +++++
 include/linux/mm.h           |   3 +
 include/linux/mm_types.h     |   2 +
 mm/huge_memory.c             | 171 ++++++++++++++++++++++++++-----------------
 mm/hugetlb.c                 |  92 ++++++++++++++---------
 mm/memcontrol.c              |  14 ++--
 mm/memory.c                  |  15 ++--
 mm/mempolicy.c               |   5 +-
 mm/migrate.c                 |  12 +--
 mm/mprotect.c                |   5 +-
 mm/pgtable-generic.c         |  10 +--
 mm/rmap.c                    |  13 ++--
 17 files changed, 246 insertions(+), 160 deletions(-)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 14+ messages in thread
* [PATCH 0/2 v2] split page table lock for hugepage
@ 2013-08-30 17:18 Naoya Horiguchi
  2013-08-30 17:18 ` [PATCH 2/2] thp: support split page table lock Naoya Horiguchi
  0 siblings, 1 reply; 14+ messages in thread
From: Naoya Horiguchi @ 2013-08-30 17:18 UTC (permalink / raw)
  To: linux-mm
  Cc: Andrew Morton, Mel Gorman, Andi Kleen, Michal Hocko,
	KOSAKI Motohiro, Rik van Riel, Andrea Arcangeli, kirill.shutemov,
	Aneesh Kumar K.V, Alex Thorlton, linux-kernel

Hi,

I revised the split page table lock patch (v1 is [1]), and got some numbers
to confirm the performance improvement.

This patchset simply replaces all of locking/unlocking of mm->page_table_lock
in hugepage context with page->ptl when USE_SPLIT_PTLOCKS is true, which
breaks single mm wide locking into multiple small (pmd/pte sized) address
range locking, so we can clearly expect better performance when many threads
access to virtual memory of a process simultaneously.

Here is the result of my testing [2], where I measured the time (in seconds)
taken to execute a specific workload in various conditions. So the smaller
number means the better performance. The workload is like this:
  1) allocate N hugepages/thps and touch them once,
  2) create T threads with pthread_create(), and
  3) each thread accesses to the whole pages sequentially 10 times.

           |             hugetlb             |               thp               |
 N      T  |   v3.11-rc3    |    patched     |   v3.11-rc3    |    patched     |
100    100 |  0.13 (+-0.04) |  0.07 (+-0.01) |  0.10 (+-0.01) |  0.08 (+-0.03) |
100   3000 | 11.67 (+-0.47) |  6.54 (+-0.38) | 11.21 (+-0.28) |  6.44 (+-0.26) |
6000   100 |  2.87 (+-0.07) |  2.79 (+-0.06) |  3.21 (+-0.06) |  3.10 (+-0.06) |
6000  3000 | 18.76 (+-0.50) | 13.68 (+-0.35) | 19.44 (+-0.78) | 14.03 (+-0.43) |

  * Numbers are the averages (and stddev) of 20 testing respectively.

This result shows that for both of hugetlb/thp patched kernel provides better
results, so patches works fine. The performance gain is larger for larger T.
Interestingly, in more detailed analysis the improvement mostly comes from 2).
I got a little improvement for 3), but no visible improvement for 1).

[1] http://thread.gmane.org/gmane.linux.kernel.mm/100856/focus=100858
[2] https://github.com/Naoya-Horiguchi/test_split_page_table_lock_for_hugepage

Naoya Horiguchi (2):
      hugetlbfs: support split page table lock
      thp: support split page table lock

 arch/powerpc/mm/hugetlbpage.c |   6 +-
 arch/powerpc/mm/pgtable_64.c  |   8 +-
 arch/s390/mm/pgtable.c        |   4 +-
 arch/sparc/mm/tlb.c           |   4 +-
 arch/tile/mm/hugetlbpage.c    |   6 +-
 fs/proc/task_mmu.c            |  17 +++--
 include/linux/huge_mm.h       |  11 +--
 include/linux/hugetlb.h       |  20 +++++
 include/linux/mm.h            |   3 +
 mm/huge_memory.c              | 170 +++++++++++++++++++++++++-----------------
 mm/hugetlb.c                  |  92 ++++++++++++++---------
 mm/memcontrol.c               |  14 ++--
 mm/memory.c                   |  15 ++--
 mm/mempolicy.c                |   5 +-
 mm/migrate.c                  |  12 +--
 mm/mprotect.c                 |   5 +-
 mm/pgtable-generic.c          |  10 +--
 mm/rmap.c                     |  13 ++--
 18 files changed, 251 insertions(+), 164 deletions(-)

Thanks,
Naoya Horiguchi

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 14+ messages in thread
end of thread, other threads:[~2013-09-09 15:43 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-09-09  2:34 [PATCH 2/2] thp: support split page table lock Daniel J Blueman
2013-09-09 15:43 ` Alex Thorlton
  -- strict thread matches above, loose matches on Subject: below --
2013-09-05 21:27 [PATCH 0/2 v3] split page table lock for hugepage Naoya Horiguchi
2013-09-05 21:27 ` [PATCH 2/2] thp: support split page table lock Naoya Horiguchi
2013-09-06 10:48   ` Kirill A. Shutemov
2013-09-06 16:27     ` Naoya Horiguchi
2013-09-06 16:46       ` Kirill A. Shutemov
2013-09-06 17:15         ` Naoya Horiguchi
2013-09-08 16:56     ` Aneesh Kumar K.V
2013-09-06 16:04   ` Alex Thorlton
2013-09-06 18:01     ` Naoya Horiguchi
2013-08-30 17:18 [PATCH 0/2 v2] split page table lock for hugepage Naoya Horiguchi
2013-08-30 17:18 ` [PATCH 2/2] thp: support split page table lock Naoya Horiguchi
2013-09-02 10:53   ` Kirill A. Shutemov
2013-09-02 16:37     ` Naoya Horiguchi
2013-09-03 20:52   ` Naoya Horiguchi

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).