From mboxrd@z Thu Jan 1 00:00:00 1970 Content-Type: multipart/mixed; boundary="===============9138453034477325740==" MIME-Version: 1.0 From: Aaron Lu To: lkp@lists.01.org Subject: Re: hugepage compaction causes performance drop Date: Mon, 23 Nov 2015 16:33:04 +0800 Message-ID: <5652CF40.6040400@intel.com> In-Reply-To: <20151123081601.GA29397@js1304-P5Q-DELUXE> List-Id: --===============9138453034477325740== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable On 11/23/2015 04:16 PM, Joonsoo Kim wrote: > Numbers looks fine to me. I guess this performance degradation is > caused by COMPACT_CLUSTER_MAX change (from 32 to 256). THP allocation > is async so should be aborted quickly. But, after isolating 256 > migratable pages, it can't be aborted and will finish 256 pages > migration (at least, current implementation). > = > Aaron, please test again with setting COMPACT_CLUSTER_MAX to 32 > (in swap.h)? This is what I found in include/linux/swap.h: #define SWAP_CLUSTER_MAX 32UL #define COMPACT_CLUSTER_MAX SWAP_CLUSTER_MAX Looks like it is already 32, or am I looking at the wrong place? BTW, I'm using v4.3 for all these tests, and I just checked v4.4-rc2, the above definition doesn't change. > = > And, please attach always-always's vmstat numbers, too. Sure, attached the vmstat tool output, taken every second. Thanks, Aaron --===============9138453034477325740== Content-Type: text/plain MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="vmstat.ksh" cHJvY3MgLS0tLS0tLS0tLS1tZW1vcnktLS0tLS0tLS0tIC0tLXN3YXAtLSAtLS0tLWlvLS0tLSAt c3lzdGVtLS0gLS0tLS0tY3B1LS0tLS0gLS0tLS10aW1lc3RhbXAtLS0tLQogciAgYiAgIHN3cGQg ICBmcmVlICAgYnVmZiAgY2FjaGUgICBzaSAgIHNvICAgIGJpICAgIGJvICAgaW4gICBjcyB1cyBz eSBpZCB3YSBzdCAgICAgICAgICAgICAgICAgQ1NUCiA2ICAxICAgICAgMCAyNTY0NzUwNCAgICA1 ODAgNjI2NTQwICAgIDAgICAgMCAgICAgMCAgICAgMCAgIDY2ICAgMTkgIDAgIDEgOTkgIDAgIDAg MjAxNS0xMS0yMCAwMjoxOTozNwogMSAgMCAgICAgIDAgMjU1NjM3OTYgICAgNTgwIDYzODAwMCAg ICAwICAgIDAgICAgIDAgICAgIDAgIDc2OSA2MDg1ICAwICAxIDk5ICAwICAwIDIwMTUtMTEtMjAg MDI6MTk6MzgKIDEgIDAgICAgICAwIDIyMDEwMzM2ICAgIDU4MCA2MzgxNjggICAgMCAgICAwICAg ICAwICAgICAwIDE2OTggIDkzMCAgMCAgMSA5OSAgMCAgMCAyMDE1LTExLTIwIDAyOjE5OjM5CiAx ICAwICAgICAgMCAxODU4OTg2OCAgICA1ODAgNjM4MDg0ICAgIDAgICAgMCAgICAgMCAgICAgMCAx MTk4ICA3OTMgIDAgIDEgOTkgIDAgIDAgMjAxNS0xMS0yMCAwMjoxOTo0MAogMSAgMCAgICAgIDAg MTUxNzMyNTIgICAgNTgwIDYzODEwNCAgICAwICAgIDAgICAgIDAgICAgIDAgMTIzNCAgNzM4ICAw ICAxIDk5ICAwICAwIDIwMTUtMTEtMjAgMDI6MTk6NDEKIDEgIDAgICAgICAwIDExNzUxNzU2ICAg IDU4MCA2MzgxMjAgICAgMCAgICAwICAgICAwICAgICAwIDEyMjQgIDY3OSAgMCAgMSA5OSAgMCAg MCAyMDE1LTExLTIwIDAyOjE5OjQyCiAxICAwICAgICAgMCA4MzIyNDE2ICAgIDU4MCA2MzgxNTYg ICAgMCAgICAwICAgICAwICAgICAwIDEyMTMgIDcyNiAgMCAgMSA5OSAgMCAgMCAyMDE1LTExLTIw IDAyOjE5OjQzCiAxICAwICAgICAgMCA0ODc3MzM2ICAgIDU4MCA2MzgyMzIgICAgMCAgICAwICAg ICAwICAgICAwIDExNzEgIDcyNiAgMCAgMSA5OSAgMCAgMCAyMDE1LTExLTIwIDAyOjE5OjQ0CiAx ICAwICAgICAgMCAxNDM3NDk2ICAgIDU4MCA2MzgzMDAgICAgMCAgICAwICAgICAwICAgICAwIDEy MDMgIDY0MSAgMCAgMSA5OSAgMCAgMCAyMDE1LTExLTIwIDAyOjE5OjQ1CiAxICAwIDQ2MDkwNCA0 MzkwODggICAgMjg0IDYzMTMwMCAxMDIwIDQ2NTI2MCAgMTAyMCA0NjUyNjAgNzM5MiA2NDY4ICAw ICAyIDk4ICAwICAwIDIwMTUtMTEtMjAgMDI6MTk6NDYKIDMgIDEgMTY1NjcwNCAzNzEwMjggICAg MTQ4IDYzMzcxNiAyMjE2IDEyMDM3OTIgIDIyMTYgMTIwMzc5MiAyNTMwNzIgNTI5MyAgMCAgNCA5 NSAgMSAgMCAyMDE1LTExLTIwIDAyOjE5OjQ3CiAyICAwIDI5ODk0MTIgMzg1MjY0ICAgIDE0MCA2 MzE5NDAgMTc3MiAxMzI1OTY4ICAxNzcyIDEzMjU5NjggMjkxMTg5IDQ5ODcgIDAgIDQgOTUgIDAg IDAgMjAxNS0xMS0yMCAwMjoxOTo0OAogMSAgMCA0MjcxMzQ4IDM5NjU4OCAgICAxNDAgNjM0MDI0 ICA2MDQgMTI4MTE1NiAgIDYwNCAxMjgxMTU2IDExNDYyMiA1MDk1ICAwICAyIDk3ICAwICAwIDIw MTUtMTEtMjAgMDI6MTk6NDkKIDEgIDAgNTU5MDI2MCAzOTE1MzIgICAgMTQwIDYzNDIwOCAgMzI0 IDEzMTg5MTYgICAzMjQgMTMxODkxNiAxNTUwIDU1MTYgIDAgIDEgOTkgIDAgIDAgMjAxNS0xMS0y MCAwMjoxOTo1MAogMyAgMCA2NzM1OTYwIDM3MzQyOCAgICAxNDAgNjM0NzQ0ICAgMjAgMTE0Nzgw NCAgICAyMCAxMTQ3ODA0IDEwNjk0MSA0ODIxICAwICAyIDk4ICAwICAwIDIwMTUtMTEtMjAgMDI6 MTk6NTEKIDMgIDAgNzkzMzg5NiAzNzQyNDQgICAgMTQwIDYzNjAyMCAgNjMyIDExOTc1NzYgICA2 MzIgMTE5NzU3MiAyNDA0NDAgNDY5MCAgMCAgNCA5NiAgMCAgMCAyMDE1LTExLTIwIDAyOjE5OjUy CiAzICAwIDkyNjI0NjQgMzY2OTM2ICAgIDE0MCA2MzgzMzIgIDEyOCAxMzI3NTEyICAgMTI4IDEz Mjc1MTYgMjkxMjgwIDQyNzcgIDAgIDQgOTYgIDAgIDAgMjAxNS0xMS0yMCAwMjoxOTo1MwogMSAg MCAxMDQ2NTI2OCA0MDA2MzIgICAgMTQwIDYzNzI0MCAgIDU2IDEyMDQ4ODQgICAgNTYgMTIwNDg4 NCAxMTkyMDggNDk4MiAgMCAgMiA5NyAgMCAgMCAyMDE1LTExLTIwIDAyOjE5OjU0CiAxICAwIDEx NDg3MjEyIDQwMTA5MiAgICAxNDAgNjM2ODk2ICAgMjQgMTAxOTkwNCAgICAyNCAxMDE5OTA0IDE1 NzkgNTI0OSAgMCAgMSA5OSAgMCAgMCAyMDE1LTExLTIwIDAyOjE5OjU1CiAxICAwIDEyMzk4NjAw IDQwMDY0NCAgICAxNDAgNjM3MjQwICAgIDggOTExMzk2ICAgICA4IDkxMTM5NiAxNDM0IDQ4MjUg IDAgIDEgOTkgIDAgIDAgMjAxNS0xMS0yMCAwMjoxOTo1NgogMSAgMSAxMzQwNzcxMiAzOTY4MDgg ICAgMTQwIDYzNjQ4MCAgMTA4IDEwMTAzNzYgICAxMDggMTAxMDM3NiAxNzQxIDkzMzUgIDAgIDEg OTggIDAgIDAgMjAxNS0xMS0yMCAwMjoxOTo1NwogMSAgMCAxNDIxMjk0OCAzOTc0NTIgICAgMTQw IDYzNzE5MiAgMTIwIDgwNDE2MCAgIDEyMCA4MDQxNjAgMTQxNCA0NDkwICAwICAxIDk5ICAwICAw IDIwMTUtMTEtMjAgMDI6MTk6NTgKIDEgIDAgMTQ5NzY4NDQgMzk5MTQ4ICAgIDE0MCA2MzY2OTYg ICAgMCA3NjM5MDQgICAgIDAgNzYzOTA0IDE0NzMgNDM3OSAgMCAgMSA5OSAgMCAgMCAyMDE1LTEx LTIwIDAyOjE5OjU5CiAxICAwIDE1NzY1MzM2IDQwMTYxMiAgICAxNDAgNjM2NjU2ICAgMTIgNzg4 NTA4ICAgIDEyIDc4ODUwOCAxMzg3IDQzNzggIDAgIDEgOTkgIDAgIDAgMjAxNS0xMS0yMCAwMjoy MDowMAogMSAgMCAxNjczNzg3NiA0MDMyMTYgICAgMTQwIDYzNjQ2OCAgIDgwIDk3NTM2OCAgICA4 MCA5NzUzNjQgMTQ2OSA0OTUwICAwICAxIDk5ICAwICAwIDIwMTUtMTEtMjAgMDI6MjA6MDEKIDEg IDAgMTc1MzI3MDggNDAzNDcyICAgIDE0MCA2MzcyNTYgICAgMCA3OTIwNTYgICAgIDAgNzkyMDYw IDEzNzUgNDU1OCAgMCAgMSA5OSAgMCAgMCAyMDE1LTExLTIwIDAyOjIwOjAyCiAxICAwIDE4MjYz MDAwIDQwMjI5NiAgICAxNDAgNjM3Nzg0ICA3ODQgNzMzMTg0ICAgNzg0IDczMzE4NCAxNTU1NyA0 NTU1ICAwICAxIDk4ICAwICAwIDIwMTUtMTEtMjAgMDI6MjA6MDMKIDEgIDAgMTkyNDY0MDggNDA0 MDA4ICAgIDE0MCA2MzkyODQgICAgMCA5ODEwNDAgICAgIDAgOTgxMDQwIDE1MTY5IDQ4MzUgIDAg IDEgOTkgIDAgIDAgMjAxNS0xMS0yMCAwMjoyMDowNAogMSAgMCAxOTcxMzgyMCA0MDczOTIgICAg MTQwIDYzODkyNCAgICAwIDQ2NzQyMCAgICAgMCA0Njc0MjAgMTU0NjQgMzc4OCAgMCAgMSA5OSAg MCAgMCAyMDE1LTExLTIwIDAyOjIwOjA1CiAxICAwIDIwMzI2NzQwIDQwMTExMiAgICAxNDAgNjM5 ODYwICAgNjAgNjEyOTM2ICAgIDYwIDYxMjkzNiAxNTA3MiA0MjA0ICAwICAxIDk5ICAwICAwIDIw MTUtMTEtMjAgMDI6MjA6MDYKIDEgIDAgMjEwMDEwNjAgNDAyMTUyICAgIDE0MCA2NDAwMDggICAg MCA2NzYzNzYgICAgIDAgNjc2Mzc2IDE1MDE4IDQxNDggIDAgIDEgOTkgIDAgIDAgMjAxNS0xMS0y MCAwMjoyMDowNwogMSAgMCAyMTU2MzI4NCA0MDYwNjAgICAgMTQwIDYzOTgwNCAgIDIwIDU2MDE4 OCAgICAyMCA1NjAxODggMTc5MTkgODQxOSAgMCAgMiA5OCAgMCAgMCAyMDE1LTExLTIwIDAyOjIw OjA4CiAxICAwIDIyMDc3ODU2IDQwMzI5NiAgICAxNDAgNjQwNjA0ICAgIDAgNTE0NTc2ICAgICAw IDUxNDU3NiAxNTYxOCAzNzM0ICAwICAxIDk5ICAwICAwIDIwMTUtMTEtMjAgMDI6MjA6MDkKIDEg IDAgMjI1NzgzNDQgNDAyMDE2ICAgIDE0MCA2NDA4OTYgICAzMiA1MDA1MTYgICAgMzIgNTAwNTE2 IDE1Mjg4IDM4NDggIDAgIDEgOTkgIDAgIDAgMjAxNS0xMS0yMCAwMjoyMDoxMAogMSAgMCAyMzA1 NDM2OCA0MDExNTYgICAgMTQwIDY0MTAwMCAgICAwIDQ3NjAyNCAgICAgMCA0NzYwMjQgMTU1MzQg Mzg5NiAgMCAgMSA5OSAgMCAgMCAyMDE1LTExLTIwIDAyOjIwOjExCiAxICAwIDIzNjc4MDY0IDQw MzA2MCAgICAxNDAgNjQwOTgwICAgIDAgNjIzNzAwICAgICAwIDYyMzcwMCAxNTE4NCA0MDA5ICAw ICAxIDk5ICAwICAwIDIwMTUtMTEtMjAgMDI6MjA6MTIKIDEgIDAgMjQxNTIxMzYgNDI0NjYwICAg IDE0MCA2NDY2MDggNzU2NCA0ODM4NDggIDc1NjQgNDgzODQ4IDM1NDQgNDcwOSAgMCAgMSA5OCAg MCAgMCAyMDE1LTExLTIwIDAyOjIwOjEzCiAxICAwIDI0NjMxMzMyIDQwMjk0OCAgICAxNDAgNjQ2 NTcyICAxMjQgNDc5MjMyICAgMTI0IDQ3OTIzMiAxNDc1IDQwMzcgIDAgIDEgOTkgIDAgIDAgMjAx NS0xMS0yMCAwMjoyMDoxNAogMSAgMCAyNTEzNzE4OCAzOTk4MzYgICAgMTQwIDY0NjQ5NiAgICAw IDUwNTg1NiAgICAgMCA1MDU4NTYgMTU0NiAzNzQ1ICAwICAxIDk5ICAwICAwIDIwMTUtMTEtMjAg MDI6MjA6MTUKIDEgIDAgMjU4MDk0OTIgMzk5NTQ0ICAgIDE0MCA2Mzk3MzIgICAgMCA2NzIzMDQg ICAgIDAgNjcyMzA0IDE1MDAgNDI0MiAgMCAgMSA5OSAgMCAgMCAyMDE1LTExLTIwIDAyOjIwOjE2 CiAxICAwIDI2ODM5NjA0IDM5NzA4OCAgICAxNDAgNjM5ODE2ICAxMDAgMTAzMDE0NCAgIDEwMCAx MDMwMTQ0IDE0NzYgNTEzMSAgMCAgMSA5OSAgMCAgMCAyMDE1LTExLTIwIDAyOjIwOjE3CiAxICAw IDI3ODczODQwIDM5MjIxMiAgICAxNDAgNjQwMTYwICAgIDAgMTAzNDI0MCAgICAgMCAxMDM0MjQw IDEzODcgNTEwNCAgMCAgMSA5OSAgMCAgMCAyMDE1LTExLTIwIDAyOjIwOjE4CiAxICAwIDI4ODY2 NTA4IDQyMzEwMCAgICAxNDAgNjM0MDUyICAgNDAgOTkyNjkyICAgIDQwIDk5MjY5MiA0MzYzMyA4 NzY2ICAwICAyIDk4ICAwICAwIDIwMTUtMTEtMjAgMDI6MjA6MTkKIDEgIDAgMjk5NTM1NDQgMzg0 ODg0ICAgIDE0MCA2MzQyMjggMTAyMCAxMDg3ODU2ICAxMDIwIDEwODc4NTYgMjQ0MDAzIDI4NTAg IDAgIDIgOTcgIDAgIDAgMjAxNS0xMS0yMCAwMjoyMDoyMAogMSAgMCAzMDk5MTUxNiAzODg2NDQg ICAgMTQwIDYzNDU0NCAgOTI4IDEwMzg4MjQgICA5MjggMTAzODgyNCAxMDQ1NTAgNDYxNiAgMCAg MiA5OCAgMCAgMCAyMDE1LTExLTIwIDAyOjIwOjIxCiAxICAwIDMyMDk5NzI4IDM5MzQzMiAgICAx NDAgNjM0NTQwICAgNDAgMTEwODIyMCAgICA0MCAxMTA4MjIwIDM2ODE3IDUyODEgIDAgIDIgOTgg IDAgIDAgMjAxNS0xMS0yMCAwMjoyMDoyMgogMSAgMCAzMzM0NjgxNiAzOTg4NjAgICAgMTQwIDYz NDgyMCAgODY0IDEyNDgzODQgICA4NjQgMTI0ODM4NCAyMDM4NjMgNDgxMSAgMCAgMyA5NiAgMCAg MCAyMDE1LTExLTIwIDAyOjIwOjIzCiAzICAwIDM0MjU2ODAwIDM5NjM5MiAgICAgOTIgNjM2NDUy ICAzNzYgOTEyNzA4ICAgMzc2IDkxMjcwOCAxMDY3NDEgNDEyMiAgMCAgMiA5OCAgMCAgMCAyMDE1 LTExLTIwIDAyOjIwOjI0CiAzICAwIDM1MzA1MDY0IDM2MDIyNCAgICAgOTIgNjM3Njg0ICA3NTYg MTA0NzEwNCAgIDc1NiAxMDQ3MTA0IDIxNTU0OCAzNDA3ICAwICA0IDk2ICAwICAwIDIwMTUtMTEt MjAgMDI6MjA6MjUKIDEgIDAgMzYwOTY5MDggMzk5NzQwICAgICA5MiA2MzY2MjggIDE4MCA3OTEx MjQgICAxODAgNzkxMTI0IDkxNTA3IDQwMzQgIDAgIDIgOTggIDAgIDAgMjAxNS0xMS0yMCAwMjoy MDoyNgogMyAgMCAzNzE2ODUwOCAzODg2NDQgICAgIDkyIDYzNzY1MiAgNTk2IDEwNzI0NDggICA1 OTYgMTA3MjQ0NCAzMzg3NiA1MzE3ICAwICAyIDk4ICAwICAwIDIwMTUtMTEtMjAgMDI6MjA6MjcK IDEgIDAgMzgzNTY4MjggMzgzMjI0ICAgICA5MiA2MzU5ODQgIDc2NCAxMTg5MDQwICAgNzY0IDEx ODkwNDQgMTU2MTggNTgzOSAgMCAgMSA5OCAgMCAgMCAyMDE1LTExLTIwIDAyOjIwOjI4CiAxICAw IDM5Njk3NTg0IDM4MzI4OCAgICAgOTIgNjM2ODQ4ICAgIDggMTM0MjIwNCAgICAgOCAxMzQyMjA0 IDE0NjYgNTgzOSAgMCAgMSA5OSAgMCAgMCAyMDE1LTExLTIwIDAyOjIwOjI5CiAxICAwIDQwOTM2 OTg4IDM5MzUzMiAgICAgOTIgNjM2Nzg0ICAxOTYgMTIzOTk1MiAgIDE5NiAxMjM5OTUyIDk2MjEg MTAxNDcgIDAgIDIgOTggIDAgIDAgMjAxNS0xMS0yMCAwMjoyMDozMAogMSAgMCA0MjMxNDYxMiAz OTM1OTYgICAgIDkyIDYzNjkwOCAgICAwIDEzNzU4MjQgICAgIDAgMTM3NTgyNCAxNTEzIDU5NTcg IDAgIDEgOTkgIDAgIDAgMjAxNS0xMS0yMCAwMjoyMDozMQogMSAgMCA0MzY0ODM2NCAzODgwNzYg ICAgIDkyIDYzNzMwOCAgICA0IDEzMzM4NjAgICAgIDQgMTMzMzg2MCAxNDAzIDU4MDYgIDAgIDEg OTkgIDAgIDAgMjAxNS0xMS0yMCAwMjoyMDozMgogMSAgMCA0NDkwOTkzMiAzOTUyNTYgICAgIDky IDYzNzE2OCAgICAwIDEyNjE0NzIgICAgIDAgMTI2MTQ3MiAxNDA3IDU1NjIgIDAgIDEgOTkgIDAg IDAgMjAxNS0xMS0yMCAwMjoyMDozMwogMSAgMCA0NjE2MTQyOCAzODcwMDAgICAgIDkyIDYzNzE5 MiAgICA0IDEyNTMzNzYgICAgIDQgMTI1MzM3NiAxMzQxIDU1NjMgIDAgIDEgOTkgIDAgIDAgMjAx NS0xMS0yMCAwMjoyMDozNAogMSAgMCA0NzQyMDc3MiAzODk1MTYgICAgIDkyIDYzNzEwMCAgICAw IDEyNTc0NzIgICAgIDAgMTI1NzQ3MiAxNDkwIDU2NzEgIDAgIDEgOTkgIDAgIDAgMjAxNS0xMS0y MCAwMjoyMDozNQogMSAgMCA0ODY4ODQ2OCAzODk5MjggICAgIDkyIDYzNzcxMiAgIDI0IDEyNjc3 MTIgICAgMjQgMTI2NzcxMiAxNDA2IDU2MTUgIDAgIDEgOTkgIDAgIDAgMjAxNS0xMS0yMCAwMjoy MDozNgogMSAgMSA1MDAwNTM0NCAzODUzMjggICAgIDkyIDYzNjcyMCAgIDYwIDEzMTY5MDggICAg NjAgMTMxNjkwOCAxNDcyIDU3NDQgIDAgIDEgOTkgIDAgIDAgMjAxNS0xMS0yMCAwMjoyMDozNwog MSAgMCA1MTIzMjcwNCAzODM4NTIgICAgIDkyIDYzNjgyNCAgNDY0IDEyMzA4ODggICA0NjQgMTIz MDg4OCAxNTY0IDU3MjcgIDAgIDEgOTggIDAgIDAgMjAxNS0xMS0yMCAwMjoyMDozOAogMSAgMCA1 MjQ3MjcyOCAzODM0NTYgICAgIDkyIDYzNzY4MCAgICA0IDEyMzY5NjQgICAgIDQgMTIzNjk2NCAx NDE5IDU1MzcgIDAgIDEgOTkgIDAgIDAgMjAxNS0xMS0yMCAwMjoyMDozOQogMSAgMCA1MzY3MTc4 OCAzODE0MDggICAgIDkyIDYzNzMwNCAgICA0IDEyMDE3NDQgICAgIDQgMTIwMTc0NCAxMzY4IDU0 MTEgIDAgIDEgOTkgIDAgIDAgMjAxNS0xMS0yMCAwMjoyMDo0MAogMiAgMCA1NDQ0NDk1NiA0MDEz NjAgICAgIDkyIDYzNjM5NiAgOTUyIDc3MTg3MiAgIDk1MiA3NzE4NzIgNTU4NzUgOTE3MiAgMCAg MiA5NyAgMCAgMCAyMDE1LTExLTIwIDAyOjIwOjQxCiAxICAwIDU1MzE3MzMyIDM5MTU2OCAgICAg OTIgNjM3NjY4ICA4NTIgODc1ODM2ICAgODUyIDg3NTgzNiA4NTcyNyA0Nzk0ICAwICAyIDk3ICAw ICAwIDIwMTUtMTEtMjAgMDI6MjA6NDIKIDEgIDAgNTYyMTg0MDQgNDA5ODg4ICAgICA5MiA2MzQ0 ODQgIDc2NCA5MDA5MjggICA3NjQgOTAwOTI4IDg5NDk5IDQ4NjcgIDAgIDIgOTcgIDAgIDAgMjAx NS0xMS0yMCAwMjoyMDo0MwogMSAgMCA1Njk4OTM4MCAzOTIyNTYgICAgIDkyIDYzMzA1MiAyMDE2 IDc3MzcwMCAgMjAxNiA3NzM3MDAgNTAxOTYgNTE2MSAgMCAgMiA5OCAgMSAgMCAyMDE1LTExLTIw IDAyOjIwOjQ0CiAzICAwIDU3NjMzNjM2IDM3ODk2MCAgICAgOTIgNjMzOTIwICA5NDQgNjQyOTg0 ICAgOTQ0IDY0Mjk4NCAzMjQ3OCA0MTQ4ICAwICAyIDk4ICAwICAwIDIwMTUtMTEtMjAgMDI6MjA6 NDUKIDEgIDAgNTg2OTM5NTYgMzkyMTAwICAgICA5MiA2MzMzMzIgIDc5MiAxMDYxMDkyICAgNzky IDEwNjEwOTIgOTI3MTMgNDk3NiAgMCAgMiA5NyAgMCAgMCAyMDE1LTExLTIwIDAyOjIwOjQ2CiAx ICAwIDU5NTgzODIwIDQwNzM4MCAgICAgOTIgNjMzODQ4ICA5MTIgODkwOTE2ICAgOTEyIDg5MDkx NiA5ODEyMyA0NzY1ICAwICAyIDk4ICAwICAwIDIwMTUtMTEtMjAgMDI6MjA6NDcKIDEgIDAgNjA0 OTMxMzYgMzc2NTMyICAgICA5MiA2MzMxODQgMTEyOCA5MTI0NDQgIDExMjggOTEyNDQ0IDUwMjc5 IDUyNzYgIDAgIDIgOTggIDAgIDAgMjAxNS0xMS0yMCAwMjoyMDo0OAogMSAgMCA2MTE2ODc5NiAz OTg2MzYgICAgIDkyIDYzNDIyMCAgOTIwIDY3NDc5NiAgIDkyMCA2NzQ3OTYgNDAwMzIgNDg0MyAg MCAgMiA5OCAgMCAgMCAyMDE1LTExLTIwIDAyOjIwOjQ5CiAxICAwIDYxOTUyOTEyIDM4NzY4MCAg ICAgOTIgNjMzNzMyICA4NDAgNzg0OTQ0ICAgODQwIDc4NDk0NCAxMDkwMyA0OTM4ICAwICAxIDk4 ICAwICAwIDIwMTUtMTEtMjAgMDI6MjA6NTAKIDEgIDAgNjMxOTAwNDggMzg3Nzg0ICAgICA5MiA2 MzQxMzIgIDU2NCAxMjM3ODgwICAgNTY0IDEyMzc4ODAgMTc1OSA1OTI4ICAwICAxIDk4ICAwICAw IDIwMTUtMTEtMjAgMDI6MjA6NTEKIDEgIDEgNjQ0NTUzMzYgMzgzMjc2ICAgICA5MiA2MzM2NjAg ICA3NiAxMjY1NjY0ICAgIDc2IDEyNjU2NjQgMTU0MSA1NjY4ICAwICAxIDk5ICAwICAwIDIwMTUt MTEtMjAgMDI6MjA6NTIKIDEgIDAgNjU4MjU4MjggMzg2Nzg4ICAgICA5MiA2MzM2OTYgICA4NCAx MzcwOTQ4ICAgIDg0IDEzNzA5NDggMjE0NSA5OTM1ICAwICAxIDk4ICAwICAwIDIwMTUtMTEtMjAg MDI6MjA6NTMKIDEgIDAgNjY1MDU3MDAgMzg2NzY0ICAgICA5MiA2MzQ0MDQgIDg3NiA2Nzk5MzYg ICA4NzYgNjc5OTM2IDE1NzQgNDE1MyAgMCAgMSA5OSAgMCAgMCAyMDE1LTExLTIwIDAyOjIwOjU0 CiAxICAwIDY3MzU3NzQ0IDM4NDMyMCAgICAgOTIgNjM0NzYwICAgIDggODUyMDU2ICAgICA4IDg1 MjA1NiAxNDcwIDQ0NDMgIDAgIDEgOTkgIDAgIDAgMjAxNS0xMS0yMCAwMjoyMDo1NQogMSAgMCA2 ODUwMDUzNiAzODY3MjAgICAgIDkyIDYzNDI2OCAgIDEyIDExNDI4MDAgICAgMTIgMTE0MjgwMCAx MzY3IDUzNjYgIDAgIDEgOTkgIDAgIDAgMjAxNS0xMS0yMCAwMjoyMDo1NgogMSAgMCA2OTY3MDUx MiAzODUzMTIgICAgIDkyIDYzNDU3MiAgICAwIDExNzE0NTYgICAgIDAgMTE3MTQ1NiAxNDAzIDUz MjUgIDAgIDEgOTkgIDAgIDAgMjAxNS0xMS0yMCAwMjoyMDo1NwogMSAgMCA3MDc3MTM2MCAzNzg3 NTYgICAgIDkyIDYzNDg4OCAgNDg0IDEwOTk3NzYgICA0ODQgMTA5OTc3NiAxNDU5IDUzNjUgIDAg IDEgOTkgIDAgIDAgMjAxNS0xMS0yMCAwMjoyMDo1OAogMSAgMCA3MTg4Mjg4MCAzODQ2NjQgICAg IDkyIDYzNTE3NiAgICAwIDExMTQxOTIgICAgIDAgMTExNDE5MiAxNDU0IDUxOTMgIDAgIDEgOTkg IDAgIDAgMjAxNS0xMS0yMCAwMjoyMDo1OQogMSAgMCA0MzMxNTc3MiAzNzk4ODAgICAgIDkyIDYz NDk1MiAgIDQwIDYzNDgwMCAgICA0MCA2MzQ4MDAgMTM4MiAzNTY0ICAwICAxIDk5ICAwICAwIDIw MTUtMTEtMjAgMDI6MjE6MDAKIDAgIDAgIDMzNjAwIDI1NzM0MTU2ICAgICA5MiA2MzQ4ODQgNDUz NiAgICAwICA0NTM2ICAgICAwIDIyMTUgNDg2NCAgMCAgMSA5OCAgMSAgMCAyMDE1LTExLTIwIDAy OjIxOjAxCg== --===============9138453034477325740==-- From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pa0-f46.google.com (mail-pa0-f46.google.com [209.85.220.46]) by kanga.kvack.org (Postfix) with ESMTP id DD0606B0038 for ; Mon, 23 Nov 2015 03:33:30 -0500 (EST) Received: by pacdm15 with SMTP id dm15so184540172pac.3 for ; Mon, 23 Nov 2015 00:33:30 -0800 (PST) Received: from mga14.intel.com (mga14.intel.com. [192.55.52.115]) by mx.google.com with ESMTP id lf12si17806430pab.22.2015.11.23.00.33.30 for ; Mon, 23 Nov 2015 00:33:30 -0800 (PST) Subject: Re: hugepage compaction causes performance drop References: <20151119092920.GA11806@aaronlu.sh.intel.com> <564DCEA6.3000802@suse.cz> <564EDFE5.5010709@intel.com> <564EE8FD.7090702@intel.com> <564EF0B6.10508@suse.cz> <20151123081601.GA29397@js1304-P5Q-DELUXE> From: Aaron Lu Message-ID: <5652CF40.6040400@intel.com> Date: Mon, 23 Nov 2015 16:33:04 +0800 MIME-Version: 1.0 In-Reply-To: <20151123081601.GA29397@js1304-P5Q-DELUXE> Content-Type: multipart/mixed; boundary="------------060301050006050000090309" Sender: owner-linux-mm@kvack.org List-ID: To: Joonsoo Kim , Vlastimil Babka Cc: linux-mm@kvack.org, Huang Ying , Dave Hansen , Tim Chen , lkp@lists.01.org, Andrea Arcangeli , David Rientjes This is a multi-part message in MIME format. --------------060301050006050000090309 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit On 11/23/2015 04:16 PM, Joonsoo Kim wrote: > Numbers looks fine to me. I guess this performance degradation is > caused by COMPACT_CLUSTER_MAX change (from 32 to 256). THP allocation > is async so should be aborted quickly. But, after isolating 256 > migratable pages, it can't be aborted and will finish 256 pages > migration (at least, current implementation). > > Aaron, please test again with setting COMPACT_CLUSTER_MAX to 32 > (in swap.h)? This is what I found in include/linux/swap.h: #define SWAP_CLUSTER_MAX 32UL #define COMPACT_CLUSTER_MAX SWAP_CLUSTER_MAX Looks like it is already 32, or am I looking at the wrong place? BTW, I'm using v4.3 for all these tests, and I just checked v4.4-rc2, the above definition doesn't change. > > And, please attach always-always's vmstat numbers, too. Sure, attached the vmstat tool output, taken every second. Thanks, Aaron --------------060301050006050000090309 Content-Type: text/plain; charset=UTF-8; name="vmstat" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="vmstat" cHJvY3MgLS0tLS0tLS0tLS1tZW1vcnktLS0tLS0tLS0tIC0tLXN3YXAtLSAtLS0tLWlvLS0t LSAtc3lzdGVtLS0gLS0tLS0tY3B1LS0tLS0gLS0tLS10aW1lc3RhbXAtLS0tLQogciAgYiAg IHN3cGQgICBmcmVlICAgYnVmZiAgY2FjaGUgICBzaSAgIHNvICAgIGJpICAgIGJvICAgaW4g ICBjcyB1cyBzeSBpZCB3YSBzdCAgICAgICAgICAgICAgICAgQ1NUCiA2ICAxICAgICAgMCAy NTY0NzUwNCAgICA1ODAgNjI2NTQwICAgIDAgICAgMCAgICAgMCAgICAgMCAgIDY2ICAgMTkg IDAgIDEgOTkgIDAgIDAgMjAxNS0xMS0yMCAwMjoxOTozNwogMSAgMCAgICAgIDAgMjU1NjM3 OTYgICAgNTgwIDYzODAwMCAgICAwICAgIDAgICAgIDAgICAgIDAgIDc2OSA2MDg1ICAwICAx IDk5ICAwICAwIDIwMTUtMTEtMjAgMDI6MTk6MzgKIDEgIDAgICAgICAwIDIyMDEwMzM2ICAg IDU4MCA2MzgxNjggICAgMCAgICAwICAgICAwICAgICAwIDE2OTggIDkzMCAgMCAgMSA5OSAg MCAgMCAyMDE1LTExLTIwIDAyOjE5OjM5CiAxICAwICAgICAgMCAxODU4OTg2OCAgICA1ODAg NjM4MDg0ICAgIDAgICAgMCAgICAgMCAgICAgMCAxMTk4ICA3OTMgIDAgIDEgOTkgIDAgIDAg MjAxNS0xMS0yMCAwMjoxOTo0MAogMSAgMCAgICAgIDAgMTUxNzMyNTIgICAgNTgwIDYzODEw NCAgICAwICAgIDAgICAgIDAgICAgIDAgMTIzNCAgNzM4ICAwICAxIDk5ICAwICAwIDIwMTUt MTEtMjAgMDI6MTk6NDEKIDEgIDAgICAgICAwIDExNzUxNzU2ICAgIDU4MCA2MzgxMjAgICAg MCAgICAwICAgICAwICAgICAwIDEyMjQgIDY3OSAgMCAgMSA5OSAgMCAgMCAyMDE1LTExLTIw IDAyOjE5OjQyCiAxICAwICAgICAgMCA4MzIyNDE2ICAgIDU4MCA2MzgxNTYgICAgMCAgICAw ICAgICAwICAgICAwIDEyMTMgIDcyNiAgMCAgMSA5OSAgMCAgMCAyMDE1LTExLTIwIDAyOjE5 OjQzCiAxICAwICAgICAgMCA0ODc3MzM2ICAgIDU4MCA2MzgyMzIgICAgMCAgICAwICAgICAw ICAgICAwIDExNzEgIDcyNiAgMCAgMSA5OSAgMCAgMCAyMDE1LTExLTIwIDAyOjE5OjQ0CiAx ICAwICAgICAgMCAxNDM3NDk2ICAgIDU4MCA2MzgzMDAgICAgMCAgICAwICAgICAwICAgICAw IDEyMDMgIDY0MSAgMCAgMSA5OSAgMCAgMCAyMDE1LTExLTIwIDAyOjE5OjQ1CiAxICAwIDQ2 MDkwNCA0MzkwODggICAgMjg0IDYzMTMwMCAxMDIwIDQ2NTI2MCAgMTAyMCA0NjUyNjAgNzM5 MiA2NDY4ICAwICAyIDk4ICAwICAwIDIwMTUtMTEtMjAgMDI6MTk6NDYKIDMgIDEgMTY1Njcw NCAzNzEwMjggICAgMTQ4IDYzMzcxNiAyMjE2IDEyMDM3OTIgIDIyMTYgMTIwMzc5MiAyNTMw NzIgNTI5MyAgMCAgNCA5NSAgMSAgMCAyMDE1LTExLTIwIDAyOjE5OjQ3CiAyICAwIDI5ODk0 MTIgMzg1MjY0ICAgIDE0MCA2MzE5NDAgMTc3MiAxMzI1OTY4ICAxNzcyIDEzMjU5NjggMjkx MTg5IDQ5ODcgIDAgIDQgOTUgIDAgIDAgMjAxNS0xMS0yMCAwMjoxOTo0OAogMSAgMCA0Mjcx MzQ4IDM5NjU4OCAgICAxNDAgNjM0MDI0ICA2MDQgMTI4MTE1NiAgIDYwNCAxMjgxMTU2IDEx NDYyMiA1MDk1ICAwICAyIDk3ICAwICAwIDIwMTUtMTEtMjAgMDI6MTk6NDkKIDEgIDAgNTU5 MDI2MCAzOTE1MzIgICAgMTQwIDYzNDIwOCAgMzI0IDEzMTg5MTYgICAzMjQgMTMxODkxNiAx NTUwIDU1MTYgIDAgIDEgOTkgIDAgIDAgMjAxNS0xMS0yMCAwMjoxOTo1MAogMyAgMCA2NzM1 OTYwIDM3MzQyOCAgICAxNDAgNjM0NzQ0ICAgMjAgMTE0NzgwNCAgICAyMCAxMTQ3ODA0IDEw Njk0MSA0ODIxICAwICAyIDk4ICAwICAwIDIwMTUtMTEtMjAgMDI6MTk6NTEKIDMgIDAgNzkz Mzg5NiAzNzQyNDQgICAgMTQwIDYzNjAyMCAgNjMyIDExOTc1NzYgICA2MzIgMTE5NzU3MiAy NDA0NDAgNDY5MCAgMCAgNCA5NiAgMCAgMCAyMDE1LTExLTIwIDAyOjE5OjUyCiAzICAwIDky NjI0NjQgMzY2OTM2ICAgIDE0MCA2MzgzMzIgIDEyOCAxMzI3NTEyICAgMTI4IDEzMjc1MTYg MjkxMjgwIDQyNzcgIDAgIDQgOTYgIDAgIDAgMjAxNS0xMS0yMCAwMjoxOTo1MwogMSAgMCAx MDQ2NTI2OCA0MDA2MzIgICAgMTQwIDYzNzI0MCAgIDU2IDEyMDQ4ODQgICAgNTYgMTIwNDg4 NCAxMTkyMDggNDk4MiAgMCAgMiA5NyAgMCAgMCAyMDE1LTExLTIwIDAyOjE5OjU0CiAxICAw IDExNDg3MjEyIDQwMTA5MiAgICAxNDAgNjM2ODk2ICAgMjQgMTAxOTkwNCAgICAyNCAxMDE5 OTA0IDE1NzkgNTI0OSAgMCAgMSA5OSAgMCAgMCAyMDE1LTExLTIwIDAyOjE5OjU1CiAxICAw IDEyMzk4NjAwIDQwMDY0NCAgICAxNDAgNjM3MjQwICAgIDggOTExMzk2ICAgICA4IDkxMTM5 NiAxNDM0IDQ4MjUgIDAgIDEgOTkgIDAgIDAgMjAxNS0xMS0yMCAwMjoxOTo1NgogMSAgMSAx MzQwNzcxMiAzOTY4MDggICAgMTQwIDYzNjQ4MCAgMTA4IDEwMTAzNzYgICAxMDggMTAxMDM3 NiAxNzQxIDkzMzUgIDAgIDEgOTggIDAgIDAgMjAxNS0xMS0yMCAwMjoxOTo1NwogMSAgMCAx NDIxMjk0OCAzOTc0NTIgICAgMTQwIDYzNzE5MiAgMTIwIDgwNDE2MCAgIDEyMCA4MDQxNjAg MTQxNCA0NDkwICAwICAxIDk5ICAwICAwIDIwMTUtMTEtMjAgMDI6MTk6NTgKIDEgIDAgMTQ5 NzY4NDQgMzk5MTQ4ICAgIDE0MCA2MzY2OTYgICAgMCA3NjM5MDQgICAgIDAgNzYzOTA0IDE0 NzMgNDM3OSAgMCAgMSA5OSAgMCAgMCAyMDE1LTExLTIwIDAyOjE5OjU5CiAxICAwIDE1NzY1 MzM2IDQwMTYxMiAgICAxNDAgNjM2NjU2ICAgMTIgNzg4NTA4ICAgIDEyIDc4ODUwOCAxMzg3 IDQzNzggIDAgIDEgOTkgIDAgIDAgMjAxNS0xMS0yMCAwMjoyMDowMAogMSAgMCAxNjczNzg3 NiA0MDMyMTYgICAgMTQwIDYzNjQ2OCAgIDgwIDk3NTM2OCAgICA4MCA5NzUzNjQgMTQ2OSA0 OTUwICAwICAxIDk5ICAwICAwIDIwMTUtMTEtMjAgMDI6MjA6MDEKIDEgIDAgMTc1MzI3MDgg NDAzNDcyICAgIDE0MCA2MzcyNTYgICAgMCA3OTIwNTYgICAgIDAgNzkyMDYwIDEzNzUgNDU1 OCAgMCAgMSA5OSAgMCAgMCAyMDE1LTExLTIwIDAyOjIwOjAyCiAxICAwIDE4MjYzMDAwIDQw MjI5NiAgICAxNDAgNjM3Nzg0ICA3ODQgNzMzMTg0ICAgNzg0IDczMzE4NCAxNTU1NyA0NTU1 ICAwICAxIDk4ICAwICAwIDIwMTUtMTEtMjAgMDI6MjA6MDMKIDEgIDAgMTkyNDY0MDggNDA0 MDA4ICAgIDE0MCA2MzkyODQgICAgMCA5ODEwNDAgICAgIDAgOTgxMDQwIDE1MTY5IDQ4MzUg IDAgIDEgOTkgIDAgIDAgMjAxNS0xMS0yMCAwMjoyMDowNAogMSAgMCAxOTcxMzgyMCA0MDcz OTIgICAgMTQwIDYzODkyNCAgICAwIDQ2NzQyMCAgICAgMCA0Njc0MjAgMTU0NjQgMzc4OCAg MCAgMSA5OSAgMCAgMCAyMDE1LTExLTIwIDAyOjIwOjA1CiAxICAwIDIwMzI2NzQwIDQwMTEx MiAgICAxNDAgNjM5ODYwICAgNjAgNjEyOTM2ICAgIDYwIDYxMjkzNiAxNTA3MiA0MjA0ICAw ICAxIDk5ICAwICAwIDIwMTUtMTEtMjAgMDI6MjA6MDYKIDEgIDAgMjEwMDEwNjAgNDAyMTUy ICAgIDE0MCA2NDAwMDggICAgMCA2NzYzNzYgICAgIDAgNjc2Mzc2IDE1MDE4IDQxNDggIDAg IDEgOTkgIDAgIDAgMjAxNS0xMS0yMCAwMjoyMDowNwogMSAgMCAyMTU2MzI4NCA0MDYwNjAg ICAgMTQwIDYzOTgwNCAgIDIwIDU2MDE4OCAgICAyMCA1NjAxODggMTc5MTkgODQxOSAgMCAg MiA5OCAgMCAgMCAyMDE1LTExLTIwIDAyOjIwOjA4CiAxICAwIDIyMDc3ODU2IDQwMzI5NiAg ICAxNDAgNjQwNjA0ICAgIDAgNTE0NTc2ICAgICAwIDUxNDU3NiAxNTYxOCAzNzM0ICAwICAx IDk5ICAwICAwIDIwMTUtMTEtMjAgMDI6MjA6MDkKIDEgIDAgMjI1NzgzNDQgNDAyMDE2ICAg IDE0MCA2NDA4OTYgICAzMiA1MDA1MTYgICAgMzIgNTAwNTE2IDE1Mjg4IDM4NDggIDAgIDEg OTkgIDAgIDAgMjAxNS0xMS0yMCAwMjoyMDoxMAogMSAgMCAyMzA1NDM2OCA0MDExNTYgICAg MTQwIDY0MTAwMCAgICAwIDQ3NjAyNCAgICAgMCA0NzYwMjQgMTU1MzQgMzg5NiAgMCAgMSA5 OSAgMCAgMCAyMDE1LTExLTIwIDAyOjIwOjExCiAxICAwIDIzNjc4MDY0IDQwMzA2MCAgICAx NDAgNjQwOTgwICAgIDAgNjIzNzAwICAgICAwIDYyMzcwMCAxNTE4NCA0MDA5ICAwICAxIDk5 ICAwICAwIDIwMTUtMTEtMjAgMDI6MjA6MTIKIDEgIDAgMjQxNTIxMzYgNDI0NjYwICAgIDE0 MCA2NDY2MDggNzU2NCA0ODM4NDggIDc1NjQgNDgzODQ4IDM1NDQgNDcwOSAgMCAgMSA5OCAg MCAgMCAyMDE1LTExLTIwIDAyOjIwOjEzCiAxICAwIDI0NjMxMzMyIDQwMjk0OCAgICAxNDAg NjQ2NTcyICAxMjQgNDc5MjMyICAgMTI0IDQ3OTIzMiAxNDc1IDQwMzcgIDAgIDEgOTkgIDAg IDAgMjAxNS0xMS0yMCAwMjoyMDoxNAogMSAgMCAyNTEzNzE4OCAzOTk4MzYgICAgMTQwIDY0 NjQ5NiAgICAwIDUwNTg1NiAgICAgMCA1MDU4NTYgMTU0NiAzNzQ1ICAwICAxIDk5ICAwICAw IDIwMTUtMTEtMjAgMDI6MjA6MTUKIDEgIDAgMjU4MDk0OTIgMzk5NTQ0ICAgIDE0MCA2Mzk3 MzIgICAgMCA2NzIzMDQgICAgIDAgNjcyMzA0IDE1MDAgNDI0MiAgMCAgMSA5OSAgMCAgMCAy MDE1LTExLTIwIDAyOjIwOjE2CiAxICAwIDI2ODM5NjA0IDM5NzA4OCAgICAxNDAgNjM5ODE2 ICAxMDAgMTAzMDE0NCAgIDEwMCAxMDMwMTQ0IDE0NzYgNTEzMSAgMCAgMSA5OSAgMCAgMCAy MDE1LTExLTIwIDAyOjIwOjE3CiAxICAwIDI3ODczODQwIDM5MjIxMiAgICAxNDAgNjQwMTYw ICAgIDAgMTAzNDI0MCAgICAgMCAxMDM0MjQwIDEzODcgNTEwNCAgMCAgMSA5OSAgMCAgMCAy MDE1LTExLTIwIDAyOjIwOjE4CiAxICAwIDI4ODY2NTA4IDQyMzEwMCAgICAxNDAgNjM0MDUy ICAgNDAgOTkyNjkyICAgIDQwIDk5MjY5MiA0MzYzMyA4NzY2ICAwICAyIDk4ICAwICAwIDIw MTUtMTEtMjAgMDI6MjA6MTkKIDEgIDAgMjk5NTM1NDQgMzg0ODg0ICAgIDE0MCA2MzQyMjgg MTAyMCAxMDg3ODU2ICAxMDIwIDEwODc4NTYgMjQ0MDAzIDI4NTAgIDAgIDIgOTcgIDAgIDAg MjAxNS0xMS0yMCAwMjoyMDoyMAogMSAgMCAzMDk5MTUxNiAzODg2NDQgICAgMTQwIDYzNDU0 NCAgOTI4IDEwMzg4MjQgICA5MjggMTAzODgyNCAxMDQ1NTAgNDYxNiAgMCAgMiA5OCAgMCAg MCAyMDE1LTExLTIwIDAyOjIwOjIxCiAxICAwIDMyMDk5NzI4IDM5MzQzMiAgICAxNDAgNjM0 NTQwICAgNDAgMTEwODIyMCAgICA0MCAxMTA4MjIwIDM2ODE3IDUyODEgIDAgIDIgOTggIDAg IDAgMjAxNS0xMS0yMCAwMjoyMDoyMgogMSAgMCAzMzM0NjgxNiAzOTg4NjAgICAgMTQwIDYz NDgyMCAgODY0IDEyNDgzODQgICA4NjQgMTI0ODM4NCAyMDM4NjMgNDgxMSAgMCAgMyA5NiAg MCAgMCAyMDE1LTExLTIwIDAyOjIwOjIzCiAzICAwIDM0MjU2ODAwIDM5NjM5MiAgICAgOTIg NjM2NDUyICAzNzYgOTEyNzA4ICAgMzc2IDkxMjcwOCAxMDY3NDEgNDEyMiAgMCAgMiA5OCAg MCAgMCAyMDE1LTExLTIwIDAyOjIwOjI0CiAzICAwIDM1MzA1MDY0IDM2MDIyNCAgICAgOTIg NjM3Njg0ICA3NTYgMTA0NzEwNCAgIDc1NiAxMDQ3MTA0IDIxNTU0OCAzNDA3ICAwICA0IDk2 ICAwICAwIDIwMTUtMTEtMjAgMDI6MjA6MjUKIDEgIDAgMzYwOTY5MDggMzk5NzQwICAgICA5 MiA2MzY2MjggIDE4MCA3OTExMjQgICAxODAgNzkxMTI0IDkxNTA3IDQwMzQgIDAgIDIgOTgg IDAgIDAgMjAxNS0xMS0yMCAwMjoyMDoyNgogMyAgMCAzNzE2ODUwOCAzODg2NDQgICAgIDky IDYzNzY1MiAgNTk2IDEwNzI0NDggICA1OTYgMTA3MjQ0NCAzMzg3NiA1MzE3ICAwICAyIDk4 ICAwICAwIDIwMTUtMTEtMjAgMDI6MjA6MjcKIDEgIDAgMzgzNTY4MjggMzgzMjI0ICAgICA5 MiA2MzU5ODQgIDc2NCAxMTg5MDQwICAgNzY0IDExODkwNDQgMTU2MTggNTgzOSAgMCAgMSA5 OCAgMCAgMCAyMDE1LTExLTIwIDAyOjIwOjI4CiAxICAwIDM5Njk3NTg0IDM4MzI4OCAgICAg OTIgNjM2ODQ4ICAgIDggMTM0MjIwNCAgICAgOCAxMzQyMjA0IDE0NjYgNTgzOSAgMCAgMSA5 OSAgMCAgMCAyMDE1LTExLTIwIDAyOjIwOjI5CiAxICAwIDQwOTM2OTg4IDM5MzUzMiAgICAg OTIgNjM2Nzg0ICAxOTYgMTIzOTk1MiAgIDE5NiAxMjM5OTUyIDk2MjEgMTAxNDcgIDAgIDIg OTggIDAgIDAgMjAxNS0xMS0yMCAwMjoyMDozMAogMSAgMCA0MjMxNDYxMiAzOTM1OTYgICAg IDkyIDYzNjkwOCAgICAwIDEzNzU4MjQgICAgIDAgMTM3NTgyNCAxNTEzIDU5NTcgIDAgIDEg OTkgIDAgIDAgMjAxNS0xMS0yMCAwMjoyMDozMQogMSAgMCA0MzY0ODM2NCAzODgwNzYgICAg IDkyIDYzNzMwOCAgICA0IDEzMzM4NjAgICAgIDQgMTMzMzg2MCAxNDAzIDU4MDYgIDAgIDEg OTkgIDAgIDAgMjAxNS0xMS0yMCAwMjoyMDozMgogMSAgMCA0NDkwOTkzMiAzOTUyNTYgICAg IDkyIDYzNzE2OCAgICAwIDEyNjE0NzIgICAgIDAgMTI2MTQ3MiAxNDA3IDU1NjIgIDAgIDEg OTkgIDAgIDAgMjAxNS0xMS0yMCAwMjoyMDozMwogMSAgMCA0NjE2MTQyOCAzODcwMDAgICAg IDkyIDYzNzE5MiAgICA0IDEyNTMzNzYgICAgIDQgMTI1MzM3NiAxMzQxIDU1NjMgIDAgIDEg OTkgIDAgIDAgMjAxNS0xMS0yMCAwMjoyMDozNAogMSAgMCA0NzQyMDc3MiAzODk1MTYgICAg IDkyIDYzNzEwMCAgICAwIDEyNTc0NzIgICAgIDAgMTI1NzQ3MiAxNDkwIDU2NzEgIDAgIDEg OTkgIDAgIDAgMjAxNS0xMS0yMCAwMjoyMDozNQogMSAgMCA0ODY4ODQ2OCAzODk5MjggICAg IDkyIDYzNzcxMiAgIDI0IDEyNjc3MTIgICAgMjQgMTI2NzcxMiAxNDA2IDU2MTUgIDAgIDEg OTkgIDAgIDAgMjAxNS0xMS0yMCAwMjoyMDozNgogMSAgMSA1MDAwNTM0NCAzODUzMjggICAg IDkyIDYzNjcyMCAgIDYwIDEzMTY5MDggICAgNjAgMTMxNjkwOCAxNDcyIDU3NDQgIDAgIDEg OTkgIDAgIDAgMjAxNS0xMS0yMCAwMjoyMDozNwogMSAgMCA1MTIzMjcwNCAzODM4NTIgICAg IDkyIDYzNjgyNCAgNDY0IDEyMzA4ODggICA0NjQgMTIzMDg4OCAxNTY0IDU3MjcgIDAgIDEg OTggIDAgIDAgMjAxNS0xMS0yMCAwMjoyMDozOAogMSAgMCA1MjQ3MjcyOCAzODM0NTYgICAg IDkyIDYzNzY4MCAgICA0IDEyMzY5NjQgICAgIDQgMTIzNjk2NCAxNDE5IDU1MzcgIDAgIDEg OTkgIDAgIDAgMjAxNS0xMS0yMCAwMjoyMDozOQogMSAgMCA1MzY3MTc4OCAzODE0MDggICAg IDkyIDYzNzMwNCAgICA0IDEyMDE3NDQgICAgIDQgMTIwMTc0NCAxMzY4IDU0MTEgIDAgIDEg OTkgIDAgIDAgMjAxNS0xMS0yMCAwMjoyMDo0MAogMiAgMCA1NDQ0NDk1NiA0MDEzNjAgICAg IDkyIDYzNjM5NiAgOTUyIDc3MTg3MiAgIDk1MiA3NzE4NzIgNTU4NzUgOTE3MiAgMCAgMiA5 NyAgMCAgMCAyMDE1LTExLTIwIDAyOjIwOjQxCiAxICAwIDU1MzE3MzMyIDM5MTU2OCAgICAg OTIgNjM3NjY4ICA4NTIgODc1ODM2ICAgODUyIDg3NTgzNiA4NTcyNyA0Nzk0ICAwICAyIDk3 ICAwICAwIDIwMTUtMTEtMjAgMDI6MjA6NDIKIDEgIDAgNTYyMTg0MDQgNDA5ODg4ICAgICA5 MiA2MzQ0ODQgIDc2NCA5MDA5MjggICA3NjQgOTAwOTI4IDg5NDk5IDQ4NjcgIDAgIDIgOTcg IDAgIDAgMjAxNS0xMS0yMCAwMjoyMDo0MwogMSAgMCA1Njk4OTM4MCAzOTIyNTYgICAgIDky IDYzMzA1MiAyMDE2IDc3MzcwMCAgMjAxNiA3NzM3MDAgNTAxOTYgNTE2MSAgMCAgMiA5OCAg MSAgMCAyMDE1LTExLTIwIDAyOjIwOjQ0CiAzICAwIDU3NjMzNjM2IDM3ODk2MCAgICAgOTIg NjMzOTIwICA5NDQgNjQyOTg0ICAgOTQ0IDY0Mjk4NCAzMjQ3OCA0MTQ4ICAwICAyIDk4ICAw ICAwIDIwMTUtMTEtMjAgMDI6MjA6NDUKIDEgIDAgNTg2OTM5NTYgMzkyMTAwICAgICA5MiA2 MzMzMzIgIDc5MiAxMDYxMDkyICAgNzkyIDEwNjEwOTIgOTI3MTMgNDk3NiAgMCAgMiA5NyAg MCAgMCAyMDE1LTExLTIwIDAyOjIwOjQ2CiAxICAwIDU5NTgzODIwIDQwNzM4MCAgICAgOTIg NjMzODQ4ICA5MTIgODkwOTE2ICAgOTEyIDg5MDkxNiA5ODEyMyA0NzY1ICAwICAyIDk4ICAw ICAwIDIwMTUtMTEtMjAgMDI6MjA6NDcKIDEgIDAgNjA0OTMxMzYgMzc2NTMyICAgICA5MiA2 MzMxODQgMTEyOCA5MTI0NDQgIDExMjggOTEyNDQ0IDUwMjc5IDUyNzYgIDAgIDIgOTggIDAg IDAgMjAxNS0xMS0yMCAwMjoyMDo0OAogMSAgMCA2MTE2ODc5NiAzOTg2MzYgICAgIDkyIDYz NDIyMCAgOTIwIDY3NDc5NiAgIDkyMCA2NzQ3OTYgNDAwMzIgNDg0MyAgMCAgMiA5OCAgMCAg MCAyMDE1LTExLTIwIDAyOjIwOjQ5CiAxICAwIDYxOTUyOTEyIDM4NzY4MCAgICAgOTIgNjMz NzMyICA4NDAgNzg0OTQ0ICAgODQwIDc4NDk0NCAxMDkwMyA0OTM4ICAwICAxIDk4ICAwICAw IDIwMTUtMTEtMjAgMDI6MjA6NTAKIDEgIDAgNjMxOTAwNDggMzg3Nzg0ICAgICA5MiA2MzQx MzIgIDU2NCAxMjM3ODgwICAgNTY0IDEyMzc4ODAgMTc1OSA1OTI4ICAwICAxIDk4ICAwICAw IDIwMTUtMTEtMjAgMDI6MjA6NTEKIDEgIDEgNjQ0NTUzMzYgMzgzMjc2ICAgICA5MiA2MzM2 NjAgICA3NiAxMjY1NjY0ICAgIDc2IDEyNjU2NjQgMTU0MSA1NjY4ICAwICAxIDk5ICAwICAw IDIwMTUtMTEtMjAgMDI6MjA6NTIKIDEgIDAgNjU4MjU4MjggMzg2Nzg4ICAgICA5MiA2MzM2 OTYgICA4NCAxMzcwOTQ4ICAgIDg0IDEzNzA5NDggMjE0NSA5OTM1ICAwICAxIDk4ICAwICAw IDIwMTUtMTEtMjAgMDI6MjA6NTMKIDEgIDAgNjY1MDU3MDAgMzg2NzY0ICAgICA5MiA2MzQ0 MDQgIDg3NiA2Nzk5MzYgICA4NzYgNjc5OTM2IDE1NzQgNDE1MyAgMCAgMSA5OSAgMCAgMCAy MDE1LTExLTIwIDAyOjIwOjU0CiAxICAwIDY3MzU3NzQ0IDM4NDMyMCAgICAgOTIgNjM0NzYw ICAgIDggODUyMDU2ICAgICA4IDg1MjA1NiAxNDcwIDQ0NDMgIDAgIDEgOTkgIDAgIDAgMjAx NS0xMS0yMCAwMjoyMDo1NQogMSAgMCA2ODUwMDUzNiAzODY3MjAgICAgIDkyIDYzNDI2OCAg IDEyIDExNDI4MDAgICAgMTIgMTE0MjgwMCAxMzY3IDUzNjYgIDAgIDEgOTkgIDAgIDAgMjAx NS0xMS0yMCAwMjoyMDo1NgogMSAgMCA2OTY3MDUxMiAzODUzMTIgICAgIDkyIDYzNDU3MiAg ICAwIDExNzE0NTYgICAgIDAgMTE3MTQ1NiAxNDAzIDUzMjUgIDAgIDEgOTkgIDAgIDAgMjAx NS0xMS0yMCAwMjoyMDo1NwogMSAgMCA3MDc3MTM2MCAzNzg3NTYgICAgIDkyIDYzNDg4OCAg NDg0IDEwOTk3NzYgICA0ODQgMTA5OTc3NiAxNDU5IDUzNjUgIDAgIDEgOTkgIDAgIDAgMjAx NS0xMS0yMCAwMjoyMDo1OAogMSAgMCA3MTg4Mjg4MCAzODQ2NjQgICAgIDkyIDYzNTE3NiAg ICAwIDExMTQxOTIgICAgIDAgMTExNDE5MiAxNDU0IDUxOTMgIDAgIDEgOTkgIDAgIDAgMjAx NS0xMS0yMCAwMjoyMDo1OQogMSAgMCA0MzMxNTc3MiAzNzk4ODAgICAgIDkyIDYzNDk1MiAg IDQwIDYzNDgwMCAgICA0MCA2MzQ4MDAgMTM4MiAzNTY0ICAwICAxIDk5ICAwICAwIDIwMTUt MTEtMjAgMDI6MjE6MDAKIDAgIDAgIDMzNjAwIDI1NzM0MTU2ICAgICA5MiA2MzQ4ODQgNDUz NiAgICAwICA0NTM2ICAgICAwIDIyMTUgNDg2NCAgMCAgMSA5OCAgMSAgMCAyMDE1LTExLTIw IDAyOjIxOjAxCg== --------------060301050006050000090309-- -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org