From: Aaron Lu <aaron.lu@intel.com>
To: Joonsoo Kim <iamjoonsoo.kim@lge.com>, Vlastimil Babka <vbabka@suse.cz>
Cc: linux-mm@kvack.org, Huang Ying <ying.huang@intel.com>,
Dave Hansen <dave.hansen@intel.com>,
Tim Chen <tim.c.chen@linux.intel.com>,
lkp@lists.01.org, Andrea Arcangeli <aarcange@redhat.com>,
David Rientjes <rientjes@google.com>
Subject: Re: hugepage compaction causes performance drop
Date: Mon, 23 Nov 2015 16:33:04 +0800 [thread overview]
Message-ID: <5652CF40.6040400@intel.com> (raw)
In-Reply-To: <20151123081601.GA29397@js1304-P5Q-DELUXE>
[-- Attachment #1: Type: text/plain, Size: 881 bytes --]
On 11/23/2015 04:16 PM, Joonsoo Kim wrote:
> Numbers looks fine to me. I guess this performance degradation is
> caused by COMPACT_CLUSTER_MAX change (from 32 to 256). THP allocation
> is async so should be aborted quickly. But, after isolating 256
> migratable pages, it can't be aborted and will finish 256 pages
> migration (at least, current implementation).
>
> Aaron, please test again with setting COMPACT_CLUSTER_MAX to 32
> (in swap.h)?
This is what I found in include/linux/swap.h:
#define SWAP_CLUSTER_MAX 32UL
#define COMPACT_CLUSTER_MAX SWAP_CLUSTER_MAX
Looks like it is already 32, or am I looking at the wrong place?
BTW, I'm using v4.3 for all these tests, and I just checked v4.4-rc2,
the above definition doesn't change.
>
> And, please attach always-always's vmstat numbers, too.
Sure, attached the vmstat tool output, taken every second.
Thanks,
Aaron
[-- Attachment #2: vmstat --]
[-- Type: text/plain, Size: 9298 bytes --]
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu----- -----timestamp-----
r b swpd free buff cache si so bi bo in cs us sy id wa st CST
6 1 0 25647504 580 626540 0 0 0 0 66 19 0 1 99 0 0 2015-11-20 02:19:37
1 0 0 25563796 580 638000 0 0 0 0 769 6085 0 1 99 0 0 2015-11-20 02:19:38
1 0 0 22010336 580 638168 0 0 0 0 1698 930 0 1 99 0 0 2015-11-20 02:19:39
1 0 0 18589868 580 638084 0 0 0 0 1198 793 0 1 99 0 0 2015-11-20 02:19:40
1 0 0 15173252 580 638104 0 0 0 0 1234 738 0 1 99 0 0 2015-11-20 02:19:41
1 0 0 11751756 580 638120 0 0 0 0 1224 679 0 1 99 0 0 2015-11-20 02:19:42
1 0 0 8322416 580 638156 0 0 0 0 1213 726 0 1 99 0 0 2015-11-20 02:19:43
1 0 0 4877336 580 638232 0 0 0 0 1171 726 0 1 99 0 0 2015-11-20 02:19:44
1 0 0 1437496 580 638300 0 0 0 0 1203 641 0 1 99 0 0 2015-11-20 02:19:45
1 0 460904 439088 284 631300 1020 465260 1020 465260 7392 6468 0 2 98 0 0 2015-11-20 02:19:46
3 1 1656704 371028 148 633716 2216 1203792 2216 1203792 253072 5293 0 4 95 1 0 2015-11-20 02:19:47
2 0 2989412 385264 140 631940 1772 1325968 1772 1325968 291189 4987 0 4 95 0 0 2015-11-20 02:19:48
1 0 4271348 396588 140 634024 604 1281156 604 1281156 114622 5095 0 2 97 0 0 2015-11-20 02:19:49
1 0 5590260 391532 140 634208 324 1318916 324 1318916 1550 5516 0 1 99 0 0 2015-11-20 02:19:50
3 0 6735960 373428 140 634744 20 1147804 20 1147804 106941 4821 0 2 98 0 0 2015-11-20 02:19:51
3 0 7933896 374244 140 636020 632 1197576 632 1197572 240440 4690 0 4 96 0 0 2015-11-20 02:19:52
3 0 9262464 366936 140 638332 128 1327512 128 1327516 291280 4277 0 4 96 0 0 2015-11-20 02:19:53
1 0 10465268 400632 140 637240 56 1204884 56 1204884 119208 4982 0 2 97 0 0 2015-11-20 02:19:54
1 0 11487212 401092 140 636896 24 1019904 24 1019904 1579 5249 0 1 99 0 0 2015-11-20 02:19:55
1 0 12398600 400644 140 637240 8 911396 8 911396 1434 4825 0 1 99 0 0 2015-11-20 02:19:56
1 1 13407712 396808 140 636480 108 1010376 108 1010376 1741 9335 0 1 98 0 0 2015-11-20 02:19:57
1 0 14212948 397452 140 637192 120 804160 120 804160 1414 4490 0 1 99 0 0 2015-11-20 02:19:58
1 0 14976844 399148 140 636696 0 763904 0 763904 1473 4379 0 1 99 0 0 2015-11-20 02:19:59
1 0 15765336 401612 140 636656 12 788508 12 788508 1387 4378 0 1 99 0 0 2015-11-20 02:20:00
1 0 16737876 403216 140 636468 80 975368 80 975364 1469 4950 0 1 99 0 0 2015-11-20 02:20:01
1 0 17532708 403472 140 637256 0 792056 0 792060 1375 4558 0 1 99 0 0 2015-11-20 02:20:02
1 0 18263000 402296 140 637784 784 733184 784 733184 15557 4555 0 1 98 0 0 2015-11-20 02:20:03
1 0 19246408 404008 140 639284 0 981040 0 981040 15169 4835 0 1 99 0 0 2015-11-20 02:20:04
1 0 19713820 407392 140 638924 0 467420 0 467420 15464 3788 0 1 99 0 0 2015-11-20 02:20:05
1 0 20326740 401112 140 639860 60 612936 60 612936 15072 4204 0 1 99 0 0 2015-11-20 02:20:06
1 0 21001060 402152 140 640008 0 676376 0 676376 15018 4148 0 1 99 0 0 2015-11-20 02:20:07
1 0 21563284 406060 140 639804 20 560188 20 560188 17919 8419 0 2 98 0 0 2015-11-20 02:20:08
1 0 22077856 403296 140 640604 0 514576 0 514576 15618 3734 0 1 99 0 0 2015-11-20 02:20:09
1 0 22578344 402016 140 640896 32 500516 32 500516 15288 3848 0 1 99 0 0 2015-11-20 02:20:10
1 0 23054368 401156 140 641000 0 476024 0 476024 15534 3896 0 1 99 0 0 2015-11-20 02:20:11
1 0 23678064 403060 140 640980 0 623700 0 623700 15184 4009 0 1 99 0 0 2015-11-20 02:20:12
1 0 24152136 424660 140 646608 7564 483848 7564 483848 3544 4709 0 1 98 0 0 2015-11-20 02:20:13
1 0 24631332 402948 140 646572 124 479232 124 479232 1475 4037 0 1 99 0 0 2015-11-20 02:20:14
1 0 25137188 399836 140 646496 0 505856 0 505856 1546 3745 0 1 99 0 0 2015-11-20 02:20:15
1 0 25809492 399544 140 639732 0 672304 0 672304 1500 4242 0 1 99 0 0 2015-11-20 02:20:16
1 0 26839604 397088 140 639816 100 1030144 100 1030144 1476 5131 0 1 99 0 0 2015-11-20 02:20:17
1 0 27873840 392212 140 640160 0 1034240 0 1034240 1387 5104 0 1 99 0 0 2015-11-20 02:20:18
1 0 28866508 423100 140 634052 40 992692 40 992692 43633 8766 0 2 98 0 0 2015-11-20 02:20:19
1 0 29953544 384884 140 634228 1020 1087856 1020 1087856 244003 2850 0 2 97 0 0 2015-11-20 02:20:20
1 0 30991516 388644 140 634544 928 1038824 928 1038824 104550 4616 0 2 98 0 0 2015-11-20 02:20:21
1 0 32099728 393432 140 634540 40 1108220 40 1108220 36817 5281 0 2 98 0 0 2015-11-20 02:20:22
1 0 33346816 398860 140 634820 864 1248384 864 1248384 203863 4811 0 3 96 0 0 2015-11-20 02:20:23
3 0 34256800 396392 92 636452 376 912708 376 912708 106741 4122 0 2 98 0 0 2015-11-20 02:20:24
3 0 35305064 360224 92 637684 756 1047104 756 1047104 215548 3407 0 4 96 0 0 2015-11-20 02:20:25
1 0 36096908 399740 92 636628 180 791124 180 791124 91507 4034 0 2 98 0 0 2015-11-20 02:20:26
3 0 37168508 388644 92 637652 596 1072448 596 1072444 33876 5317 0 2 98 0 0 2015-11-20 02:20:27
1 0 38356828 383224 92 635984 764 1189040 764 1189044 15618 5839 0 1 98 0 0 2015-11-20 02:20:28
1 0 39697584 383288 92 636848 8 1342204 8 1342204 1466 5839 0 1 99 0 0 2015-11-20 02:20:29
1 0 40936988 393532 92 636784 196 1239952 196 1239952 9621 10147 0 2 98 0 0 2015-11-20 02:20:30
1 0 42314612 393596 92 636908 0 1375824 0 1375824 1513 5957 0 1 99 0 0 2015-11-20 02:20:31
1 0 43648364 388076 92 637308 4 1333860 4 1333860 1403 5806 0 1 99 0 0 2015-11-20 02:20:32
1 0 44909932 395256 92 637168 0 1261472 0 1261472 1407 5562 0 1 99 0 0 2015-11-20 02:20:33
1 0 46161428 387000 92 637192 4 1253376 4 1253376 1341 5563 0 1 99 0 0 2015-11-20 02:20:34
1 0 47420772 389516 92 637100 0 1257472 0 1257472 1490 5671 0 1 99 0 0 2015-11-20 02:20:35
1 0 48688468 389928 92 637712 24 1267712 24 1267712 1406 5615 0 1 99 0 0 2015-11-20 02:20:36
1 1 50005344 385328 92 636720 60 1316908 60 1316908 1472 5744 0 1 99 0 0 2015-11-20 02:20:37
1 0 51232704 383852 92 636824 464 1230888 464 1230888 1564 5727 0 1 98 0 0 2015-11-20 02:20:38
1 0 52472728 383456 92 637680 4 1236964 4 1236964 1419 5537 0 1 99 0 0 2015-11-20 02:20:39
1 0 53671788 381408 92 637304 4 1201744 4 1201744 1368 5411 0 1 99 0 0 2015-11-20 02:20:40
2 0 54444956 401360 92 636396 952 771872 952 771872 55875 9172 0 2 97 0 0 2015-11-20 02:20:41
1 0 55317332 391568 92 637668 852 875836 852 875836 85727 4794 0 2 97 0 0 2015-11-20 02:20:42
1 0 56218404 409888 92 634484 764 900928 764 900928 89499 4867 0 2 97 0 0 2015-11-20 02:20:43
1 0 56989380 392256 92 633052 2016 773700 2016 773700 50196 5161 0 2 98 1 0 2015-11-20 02:20:44
3 0 57633636 378960 92 633920 944 642984 944 642984 32478 4148 0 2 98 0 0 2015-11-20 02:20:45
1 0 58693956 392100 92 633332 792 1061092 792 1061092 92713 4976 0 2 97 0 0 2015-11-20 02:20:46
1 0 59583820 407380 92 633848 912 890916 912 890916 98123 4765 0 2 98 0 0 2015-11-20 02:20:47
1 0 60493136 376532 92 633184 1128 912444 1128 912444 50279 5276 0 2 98 0 0 2015-11-20 02:20:48
1 0 61168796 398636 92 634220 920 674796 920 674796 40032 4843 0 2 98 0 0 2015-11-20 02:20:49
1 0 61952912 387680 92 633732 840 784944 840 784944 10903 4938 0 1 98 0 0 2015-11-20 02:20:50
1 0 63190048 387784 92 634132 564 1237880 564 1237880 1759 5928 0 1 98 0 0 2015-11-20 02:20:51
1 1 64455336 383276 92 633660 76 1265664 76 1265664 1541 5668 0 1 99 0 0 2015-11-20 02:20:52
1 0 65825828 386788 92 633696 84 1370948 84 1370948 2145 9935 0 1 98 0 0 2015-11-20 02:20:53
1 0 66505700 386764 92 634404 876 679936 876 679936 1574 4153 0 1 99 0 0 2015-11-20 02:20:54
1 0 67357744 384320 92 634760 8 852056 8 852056 1470 4443 0 1 99 0 0 2015-11-20 02:20:55
1 0 68500536 386720 92 634268 12 1142800 12 1142800 1367 5366 0 1 99 0 0 2015-11-20 02:20:56
1 0 69670512 385312 92 634572 0 1171456 0 1171456 1403 5325 0 1 99 0 0 2015-11-20 02:20:57
1 0 70771360 378756 92 634888 484 1099776 484 1099776 1459 5365 0 1 99 0 0 2015-11-20 02:20:58
1 0 71882880 384664 92 635176 0 1114192 0 1114192 1454 5193 0 1 99 0 0 2015-11-20 02:20:59
1 0 43315772 379880 92 634952 40 634800 40 634800 1382 3564 0 1 99 0 0 2015-11-20 02:21:00
0 0 33600 25734156 92 634884 4536 0 4536 0 2215 4864 0 1 98 1 0 2015-11-20 02:21:01
next prev parent reply other threads:[~2015-11-23 8:33 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-11-19 9:29 hugepage compaction causes performance drop Aaron Lu
2015-11-19 13:29 ` Vlastimil Babka
2015-11-20 8:55 ` Aaron Lu
2015-11-20 9:33 ` Aaron Lu
2015-11-20 10:06 ` Vlastimil Babka
2015-11-23 8:16 ` Joonsoo Kim
2015-11-23 8:33 ` Aaron Lu [this message]
2015-11-23 9:24 ` Joonsoo Kim
2015-11-24 3:40 ` Aaron Lu
2015-11-24 4:55 ` Joonsoo Kim
2015-11-24 7:27 ` Aaron Lu
2015-11-24 8:29 ` Joonsoo Kim
2015-11-25 12:44 ` Vlastimil Babka
2015-11-26 5:47 ` Aaron Lu
2015-11-24 2:45 ` Joonsoo Kim
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5652CF40.6040400@intel.com \
--to=aaron.lu@intel.com \
--cc=aarcange@redhat.com \
--cc=dave.hansen@intel.com \
--cc=iamjoonsoo.kim@lge.com \
--cc=linux-mm@kvack.org \
--cc=lkp@lists.01.org \
--cc=rientjes@google.com \
--cc=tim.c.chen@linux.intel.com \
--cc=vbabka@suse.cz \
--cc=ying.huang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).