From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
To: Alex Thorlton <athorlton@sgi.com>
Cc: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>,
Ingo Molnar <mingo@redhat.com>,
Andrew Morton <akpm@linux-foundation.org>,
Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>,
"Eric W . Biederman" <ebiederm@xmission.com>,
"Paul E . McKenney" <paulmck@linux.vnet.ibm.com>,
Al Viro <viro@zeniv.linux.org.uk>,
Andi Kleen <ak@linux.intel.com>,
Andrea Arcangeli <aarcange@redhat.com>,
Dave Hansen <dave.hansen@intel.com>,
Dave Jones <davej@redhat.com>,
David Howells <dhowells@redhat.com>,
Frederic Weisbecker <fweisbec@gmail.com>,
Johannes Weiner <hannes@cmpxchg.org>,
Kees Cook <keescook@chromium.org>, Mel Gorman <mgorman@suse.de>,
Michael Kerrisk <mtk.manpages@gmail.com>,
Oleg Nesterov <oleg@redhat.com>,
Peter Zijlstra <peterz@infradead.org>,
Rik van Riel <riel@redhat.com>, Robin Holt <robinmholt@gmail.com>,
Sedat Dilek <sedat.dilek@gmail.com>,
Srikar Dronamraju <srikar@linux.vnet.ibm.com>,
Thomas Gleixner <tglx@linutronix.de>,
linux-kernel@vger.kernel.org, linux-mm@kvack.org
Subject: Re: [PATCHv2 0/9] split page table lock for PMD tables
Date: Thu, 26 Sep 2013 13:50:51 +0300 (EEST) [thread overview]
Message-ID: <20130926105052.0205AE0090@blue.fi.intel.com> (raw)
In-Reply-To: <20130924164443.GB2940@sgi.com>
Alex Thorlton wrote:
> > THP off:
> > --------
...
> > 36.540185552 seconds time elapsed ( +- 18.36% )
>
> I'm assuming this was THP off, no patchset, correct?
Yes. But THP off patched is *very* close to this, so I didn't post it separately.
> Here are my results from this test on 3.12-rc1:
...
> 1138.759708820 seconds time elapsed ( +- 0.47% )
>
> And the same test on 3.12-rc1 with your patchset:
>
> Performance counter stats for './runt -t -c 512 -b 512m' (5 runs):
...
> 1115.214191126 seconds time elapsed ( +- 0.18% )
>
> Looks like we're getting a mild performance increase here, but we still
> have a problem.
Let me guess: you have HUGETLBFS enabled in your config, right? ;)
HUGETLBFS hasn't converted to new locking and we disable split pmd lock if
HUGETLBFS is enabled.
I'm going to convert HUGETLBFS too, but it might take some time.
Without HUGETLBFS numbers looks pretty solid on my machine:
THP off, v3.12-rc2:
-------------------
Performance counter stats for './thp_memscale -c 80 -b 512m' (5 runs):
1037072.835207 task-clock # 57.426 CPUs utilized ( +- 3.59% )
95,093 context-switches # 0.092 K/sec ( +- 3.93% )
140 cpu-migrations # 0.000 K/sec ( +- 5.28% )
10,000,550 page-faults # 0.010 M/sec ( +- 0.00% )
2,455,210,400,261 cycles # 2.367 GHz ( +- 3.62% ) [83.33%]
2,429,281,882,056 stalled-cycles-frontend # 98.94% frontend cycles idle ( +- 3.67% ) [83.33%]
1,975,960,019,659 stalled-cycles-backend # 80.48% backend cycles idle ( +- 3.88% ) [66.68%]
46,503,296,013 instructions # 0.02 insns per cycle
# 52.24 stalled cycles per insn ( +- 3.21% ) [83.34%]
9,278,997,542 branches # 8.947 M/sec ( +- 4.00% ) [83.34%]
89,881,640 branch-misses # 0.97% of all branches ( +- 1.17% ) [83.33%]
18.059261877 seconds time elapsed ( +- 2.65% )
THP on, v3.12-rc2:
------------------
Performance counter stats for './thp_memscale -c 80 -b 512m' (5 runs):
3114745.395974 task-clock # 73.875 CPUs utilized ( +- 1.84% )
267,356 context-switches # 0.086 K/sec ( +- 1.84% )
99 cpu-migrations # 0.000 K/sec ( +- 1.40% )
58,313 page-faults # 0.019 K/sec ( +- 0.28% )
7,416,635,817,510 cycles # 2.381 GHz ( +- 1.83% ) [83.33%]
7,342,619,196,993 stalled-cycles-frontend # 99.00% frontend cycles idle ( +- 1.88% ) [83.33%]
6,267,671,641,967 stalled-cycles-backend # 84.51% backend cycles idle ( +- 2.03% ) [66.67%]
117,819,935,165 instructions # 0.02 insns per cycle
# 62.32 stalled cycles per insn ( +- 4.39% ) [83.34%]
28,899,314,777 branches # 9.278 M/sec ( +- 4.48% ) [83.34%]
71,787,032 branch-misses # 0.25% of all branches ( +- 1.03% ) [83.33%]
42.162306788 seconds time elapsed ( +- 1.73% )
THP off, patched, no HUGETLBFS:
-------------------------------
Performance counter stats for './thp_memscale -c 80 -b 512m' (5 runs):
943301.957892 task-clock # 56.256 CPUs utilized ( +- 3.01% )
86,218 context-switches # 0.091 K/sec ( +- 3.17% )
121 cpu-migrations # 0.000 K/sec ( +- 6.64% )
10,000,551 page-faults # 0.011 M/sec ( +- 0.00% )
2,230,462,457,654 cycles # 2.365 GHz ( +- 3.04% ) [83.32%]
2,204,616,385,805 stalled-cycles-frontend # 98.84% frontend cycles idle ( +- 3.09% ) [83.32%]
1,778,640,046,926 stalled-cycles-backend # 79.74% backend cycles idle ( +- 3.47% ) [66.69%]
45,995,472,617 instructions # 0.02 insns per cycle
# 47.93 stalled cycles per insn ( +- 2.51% ) [83.34%]
9,179,700,174 branches # 9.731 M/sec ( +- 3.04% ) [83.35%]
89,166,529 branch-misses # 0.97% of all branches ( +- 1.45% ) [83.33%]
16.768027318 seconds time elapsed ( +- 2.47% )
THP on, patched, no HUGETLBFS:
------------------------------
Performance counter stats for './thp_memscale -c 80 -b 512m' (5 runs):
458793.837905 task-clock # 54.632 CPUs utilized ( +- 0.79% )
41,831 context-switches # 0.091 K/sec ( +- 0.97% )
98 cpu-migrations # 0.000 K/sec ( +- 1.66% )
57,829 page-faults # 0.126 K/sec ( +- 0.62% )
1,077,543,336,716 cycles # 2.349 GHz ( +- 0.81% ) [83.33%]
1,067,403,802,964 stalled-cycles-frontend # 99.06% frontend cycles idle ( +- 0.87% ) [83.33%]
864,764,616,143 stalled-cycles-backend # 80.25% backend cycles idle ( +- 0.73% ) [66.68%]
16,129,177,440 instructions # 0.01 insns per cycle
# 66.18 stalled cycles per insn ( +- 7.94% ) [83.35%]
3,618,938,569 branches # 7.888 M/sec ( +- 8.46% ) [83.36%]
33,242,032 branch-misses # 0.92% of all branches ( +- 2.02% ) [83.32%]
8.397885779 seconds time elapsed ( +- 0.18% )
--
Kirill A. Shutemov
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2013-09-26 10:51 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-09-16 11:25 [PATCHv2 0/9] split page table lock for PMD tables Kirill A. Shutemov
2013-09-16 11:25 ` [PATCHv2 1/9] mm: rename USE_SPLIT_PTLOCKS to USE_SPLIT_PTE_PTLOCKS Kirill A. Shutemov
2013-09-16 11:25 ` [PATCHv2 2/9] mm: convert mm->nr_ptes to atomic_t Kirill A. Shutemov
2013-09-17 14:33 ` Kirill A. Shutemov
2013-09-16 11:25 ` [PATCHv2 3/9] mm: introduce api for split page table lock for PMD level Kirill A. Shutemov
2013-09-16 11:25 ` [PATCHv2 4/9] mm, thp: change pmd_trans_huge_lock() to return taken lock Kirill A. Shutemov
2013-09-16 11:25 ` [PATCHv2 5/9] mm, thp: move ptl taking inside page_check_address_pmd() Kirill A. Shutemov
2013-09-16 11:25 ` [PATCHv2 6/9] mm, thp: do not access mm->pmd_huge_pte directly Kirill A. Shutemov
2013-09-16 11:25 ` [PATCHv2 7/9] mm: convent the rest to new page table lock api Kirill A. Shutemov
2013-09-16 11:25 ` [PATCHv2 8/9] mm: implement split page table lock for PMD level Kirill A. Shutemov
2013-09-17 14:28 ` Kirill A. Shutemov
2013-09-16 11:25 ` [PATCHv2 9/9] x86, mm: enable " Kirill A. Shutemov
2013-09-16 11:44 ` [PATCHv2 0/9] split page table lock for PMD tables Peter Zijlstra
2013-09-16 12:11 ` Kirill A. Shutemov
2013-09-19 17:17 ` Alex Thorlton
2013-09-20 12:31 ` Kirill A. Shutemov
2013-09-24 16:44 ` Alex Thorlton
2013-09-26 10:50 ` Kirill A. Shutemov [this message]
2013-09-26 15:42 ` Kirill A. Shutemov
2013-09-27 0:04 ` Naoya Horiguchi
2013-09-26 21:19 ` Alex Thorlton
2013-09-26 21:38 ` Kirill A. Shutemov
2013-09-26 21:42 ` Kirill A. Shutemov
2013-09-26 21:44 ` Alex Thorlton
2013-09-26 21:43 ` Alex Thorlton
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20130926105052.0205AE0090@blue.fi.intel.com \
--to=kirill.shutemov@linux.intel.com \
--cc=aarcange@redhat.com \
--cc=ak@linux.intel.com \
--cc=akpm@linux-foundation.org \
--cc=athorlton@sgi.com \
--cc=dave.hansen@intel.com \
--cc=davej@redhat.com \
--cc=dhowells@redhat.com \
--cc=ebiederm@xmission.com \
--cc=fweisbec@gmail.com \
--cc=hannes@cmpxchg.org \
--cc=keescook@chromium.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mgorman@suse.de \
--cc=mingo@redhat.com \
--cc=mtk.manpages@gmail.com \
--cc=n-horiguchi@ah.jp.nec.com \
--cc=oleg@redhat.com \
--cc=paulmck@linux.vnet.ibm.com \
--cc=peterz@infradead.org \
--cc=riel@redhat.com \
--cc=robinmholt@gmail.com \
--cc=sedat.dilek@gmail.com \
--cc=srikar@linux.vnet.ibm.com \
--cc=tglx@linutronix.de \
--cc=viro@zeniv.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).