From: Madhavan Srinivasan <maddy@linux.vnet.ibm.com>
To: linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org,
linux-mm@kvack.org, linux-arch@vger.kernel.org, x86@kernel.org
Cc: benh@kernel.crashing.org, paulus@samba.org,
kirill.shutemov@linux.intel.com, rusty@rustcorp.com.au,
akpm@linux-foundation.org, riel@redhat.com, mgorman@suse.de,
ak@linux.intel.com, peterz@infradead.org, mingo@kernel.org,
Madhavan Srinivasan <maddy@linux.vnet.ibm.com>
Subject: [PATCH V2 2/2] mm: add FAULT_AROUND_ORDER Kconfig paramater for powerpc
Date: Fri, 4 Apr 2014 11:57:15 +0530 [thread overview]
Message-ID: <1396592835-24767-3-git-send-email-maddy@linux.vnet.ibm.com> (raw)
In-Reply-To: <1396592835-24767-1-git-send-email-maddy@linux.vnet.ibm.com>
Performance data for different FAULT_AROUND_ORDER values from 4 socket
Power7 system (128 Threads and 128GB memory) is below. perf stat with
repeat of 5 is used to get the stddev values. This patch create
FAULT_AROUND_ORDER Kconfig parameter and defaults it to 3 based on the
performance data.
FAULT_AROUND_ORDER Baseline 1 3 4 5 7
Linux build (make -j64)
minor-faults 7184385 5874015 4567289 4318518 4193815 4159193
times in seconds 61.433776136 60.865935292 59.245368038 60.630675011 60.56587624 59.828271924
stddev for time ( +- 1.18% ) ( +- 1.78% ) ( +- 0.44% ) ( +- 2.03% ) ( +- 1.66% ) ( +- 1.45% )
Linux rebuild (make -j64)
minor-faults 303018 226392 146170 132480 126878 126236
times in seconds 5.659819172 5.723996942 5.591238319 5.622533357 5.878811995 5.550133096
stddev for time ( +- 0.71% ) ( +- 0.78% ) ( +- 0.62% ) ( +- 0.45% ) ( +- 1.55% ) ( +- 0.29% )
Two synthetic tests: access every word in file in sequential/random order.
Marginal Performance gains seen for FAO value of 3 when compared to value
of 4.
Sequential access 16GiB file
FAULT_AROUND_ORDER Baseline 1 3 4 5 7
1 thread
minor-faults 262302 131192 32873 16486 8291 2351
times in seconds 53.071497352 52.945826882 52.931417302 52.928577184 52.859285439 53.116800539
stddev for time ( +- 0.01% ) ( +- 0.02% ) ( +- 0.02% ) ( +- 0.04% ) ( +- 0.04% ) ( +- 0.01% )
8 threads
minor-faults 2097314 1051046 263336 131715 66098 16653
times in seconds 54.385698561 54.603652339 54.771282004 54.488565674 54.496701531 54.962142189
stddev for time ( +- 0.05% ) ( +- 0.02% ) ( +- 0.37% ) ( +- 0.08% ) ( +- 0.07% ) ( +- 0.51% )
32 threads
minor-faults 8389267 4218595 1059961 531319 266463 67271
times in seconds 60.61715047 60.827964038 60.46412673 60.266045885 60.492398315 60.24531921
stddev for time ( +- 0.65% ) ( +- 0.21% ) ( +- 0.25% ) ( +- 0.29% ) ( +- 0.19% ) ( +- 0.35% )
64 threads
minor-faults 16777455 8485998 2178582 1092106 544302 137693
times in seconds 86.471334554 84.412415735 85.208303832 84.331473392 85.598793479 84.695469266
stddev for time ( +- 0.60% ) ( +- 1.47% ) ( +- 0.74% ) ( +- 1.55% ) ( +- 0.92% ) ( +- 1.16% )
128 threads
minor-faults 33555267 17734522 4710107 2380821 1182707 292077
times in seconds 117.535385569 114.291359037 112.593908276 113.081807611 114.358686588 114.491043011
stddev for time ( +- 2.24% ) ( +- 0.92% ) ( +- 0.36% ) ( +- 0.53% ) ( +- 0.70% ) ( +- 0.53% )
Random access 1GiB file
FAULT_AROUND_ORDER Baseline 1 3 4 5 7
1 thread
minor-faults 16503 8664 2149 1126 610 437
times in seconds 43.843573808 48.042069805 50.580779682 54.282884593 52.641739876 51.803302129
stddev for time ( +- 1.30% ) ( +- 2.25% ) ( +- 2.92% ) ( +- 1.44% ) ( +- 4.49% ) ( +- 3.78% )
8 threads
minor-faults 131201 70916 17760 8665 4250 1149
times in seconds 46.262626804 55.942851041 56.629191584 57.97044714 55.417557594 56.019709166
stddev for time ( +- 4.66% ) ( +- 1.52% ) ( +- 1.43% ) ( +- 1.61% ) ( +- 0.65% ) ( +- 1.27% )
32 threads
minor-faults 524959 265980 67282 33601 16930 4316
times in seconds 67.754175928 69.85012331 71.750338061 71.053074643 68.90728294 71.250103217
stddev for time ( +- 3.79% ) ( +- 0.77% ) ( +- 1.15% ) ( +- 1.08% ) ( +- 2.14% ) ( +- 1.17% )
64 threads
minor-faults 1048831 528829 133256 66700 33428 8776
times in seconds 96.674025305 93.109961822 87.441777715 91.986332028 88.686748472 93.101434306
stddev for time ( +- 2.85% ) ( +- 2.42% ) ( +- 0.42% ) ( +- 1.58% ) ( +- 1.29% ) ( +- 2.01% )
128 threads
minor-faults 2098043 1053224 266271 133702 66966 17276
times in seconds 156.525792044 152.117971403 147.523673243 148.560226602 148.596575663 149.389288429
stddev for time ( +- 2.39% ) ( +- 0.71% ) ( +- 0.72% ) ( +- 0.35% ) ( +- 0.41% ) ( +- 1.04% )
Worst case scenario: we touch one page every 16M to demonstrate overhead.
Touch only one page in page table in 16GiB file
FAULT_AROUND_ORDER Baseline 1 3 4 5 7
1 thread
minor-faults 1077 1064 1051 1048 1046 1045
times in seconds 0.00615347 0.008327379 0.019775282 0.034444003 0.05905971 0.220863339
stddev for time ( +- 3.12% ) ( +- 2.29% ) ( +- 1.68% ) ( +- 1.40% ) ( +- 1.68% ) ( +- 1.53% )
8 threads
minor-faults 8252 8239 8226 8223 8220 8224
times in seconds 0.04387392 0.059859294 0.113897648 0.199707764 0.361585762 1.343366843
stddev for time ( +- 3.66% ) ( +- 2.18% ) ( +- 1.44% ) ( +- 2.40% ) ( +- 2.08% ) ( +- 1.75% )
32 threads
minor-faults 32852 32841 32825 32826 32824 32828
times in seconds 0.191404544 0.21907773 0.433207123 0.72430447 1.334983196 4.97727449
stddev for time ( +- 5.08% ) ( +- 3.19% ) ( +- 4.45% ) ( +- 2.18% ) ( +- 2.75% ) ( +- 1.52% )
64 threads
minor-faults 65652 65642 65629 65622 65623 65634
times in seconds 0.402140429 0.510806718 0.854288645 1.412329805 2.556707704 8.711074863
stddev for time ( +- 2.90% ) ( +- 3.04% ) ( +- 1.83% ) ( +- 2.75% ) ( +- 2.49% ) ( +- 0.68% )
128 threads
minor-faults 131255 131239 131228 131228 131229 131243
times in seconds 0.817782148 1.124631348 2.023730928 3.184792382 5.331392072 17.309524609
stddev for time ( +- 3.35% ) ( +- 11.74% ) ( +- 4.55% ) ( +- 2.00% ) ( +- 1.31% ) ( +- 1.30% )
Signed-off-by: Madhavan Srinivasan <maddy@linux.vnet.ibm.com>
---
arch/powerpc/platforms/pseries/Kconfig | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/arch/powerpc/platforms/pseries/Kconfig b/arch/powerpc/platforms/pseries/Kconfig
index 2cb8b77..2246d9f 100644
--- a/arch/powerpc/platforms/pseries/Kconfig
+++ b/arch/powerpc/platforms/pseries/Kconfig
@@ -132,3 +132,8 @@ config DTL
which are accessible through a debugfs file.
Say N if you are unsure.
+
+config FAULT_AROUND_ORDER
+ int
+ default "3"
+
--
1.7.10.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2014-04-04 6:27 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-04-04 6:27 [PATCH V2 0/2] FAULT_AROUND_ORDER patchset performance data for powerpc Madhavan Srinivasan
2014-04-04 6:27 ` [PATCH V2 1/2] mm: move FAULT_AROUND_ORDER to arch/ Madhavan Srinivasan
2014-04-04 13:17 ` Kirill A. Shutemov
2014-04-09 1:14 ` Madhavan Srinivasan
2014-04-04 16:18 ` Dave Hansen
2014-04-04 17:50 ` David Miller
2014-04-09 1:44 ` Madhavan Srinivasan
2014-04-07 5:45 ` Benjamin Herrenschmidt
2014-04-09 1:32 ` Madhavan Srinivasan
2014-04-09 8:20 ` Peter Zijlstra
2014-04-09 15:48 ` Dave Hansen
2014-04-10 8:29 ` Madhavan Srinivasan
2014-04-09 15:46 ` Dave Hansen
2014-04-22 7:22 ` Rusty Russell
2014-04-04 6:27 ` Madhavan Srinivasan [this message]
2014-04-04 7:02 ` [PATCH V2 2/2] mm: add FAULT_AROUND_ORDER Kconfig paramater for powerpc Ingo Molnar
2014-04-04 7:10 ` Ingo Molnar
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1396592835-24767-3-git-send-email-maddy@linux.vnet.ibm.com \
--to=maddy@linux.vnet.ibm.com \
--cc=ak@linux.intel.com \
--cc=akpm@linux-foundation.org \
--cc=benh@kernel.crashing.org \
--cc=kirill.shutemov@linux.intel.com \
--cc=linux-arch@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=mgorman@suse.de \
--cc=mingo@kernel.org \
--cc=paulus@samba.org \
--cc=peterz@infradead.org \
--cc=riel@redhat.com \
--cc=rusty@rustcorp.com.au \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).