From: Andrea Arcangeli <aarcange@redhat.com>
To: Dave Hansen <dave@linux.vnet.ibm.com>
Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org,
Michael J Wolf <mjwolf@us.ibm.com>
Subject: Re: [RFC][PATCH 0/6] more detailed per-process transparent hugepage statistics
Date: Tue, 1 Feb 2011 16:38:57 +0100 [thread overview]
Message-ID: <20110201153857.GA18740@random.random> (raw)
In-Reply-To: <20110201003357.D6F0BE0D@kernel>
On Mon, Jan 31, 2011 at 04:33:57PM -0800, Dave Hansen wrote:
> I'm working on some more reports that transparent huge pages and
> KSM do not play nicely together. Basically, whenever THP's are
> present along with KSM, there is a lot of attrition over time,
> and we do not see much overall progress keeping THP's around:
>
> http://sr71.net/~dave/ibm/038_System_Anonymous_Pages.png
>
> (That's Karl Rister's graph, thanks Karl!)
Well if the pages_sharing/pages_shared count goes up, this is a
feature not a bug.... You need to print that too in the chart to show
this is not ok.
KSM will slowdown performance also during copy-on-writes when
pages_sharing goes up, not only because of creating non-linearity
inside 2m chunks (which makes mandatory to use ptes and not hugepmd,
it's not an inefficiency of some sort that can be optimized away
unfortunately). We sure could change KSM to merge 2M pages instead of
4k pages, but then the memory-density would decrease of several order
of magnitudes making the KSM scan almost useless (ok, with guest
heavily using THP that may change, but all pagecache is still 4k... so
for now it'd be next to useless).
I'm in the process of adding a no-ksm option to qemu-kvm command line
so you can selectively choose which VM runs with KSM or not (otherwise
you can switch ksm off globally to be sure not to degrade
performance).
> However, I realized that we do not currently have a nice way to find
> out where individual THP's might be on the system. We have an
> overall count, but no way of telling which processes or VMAs they
> might be in.
>
> I started to implement this in the /proc/$pid/smaps code, but
> quickly realized that the lib/pagewalk.c code unconditionally
> splits THPs up. This set reworks that code a bit and, in the
> end, gives you a per-map count of the numbers of huge pages.
> It also makes it possible for page walks to _not_ split THPs.
That's something in the TODO list indeed thanks a lot for working on
this (I think we discussed this earlier too).
I would prefer to close the issues that you just previously reported,
sometime with mmap_sem and issues like that, before adding more
features though but I don't want to defer things either so it's up to
you.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom policy in Canada: sign http://dissolvethecrtc.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2011-02-01 15:39 UTC|newest]
Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-02-01 0:33 [RFC][PATCH 0/6] more detailed per-process transparent hugepage statistics Dave Hansen
2011-02-01 0:33 ` [RFC][PATCH 1/6] count transparent hugepage splits Dave Hansen
2011-02-01 9:58 ` Johannes Weiner
2011-02-03 21:22 ` David Rientjes
2011-02-04 21:18 ` Andrea Arcangeli
2011-02-04 21:28 ` Dave Hansen
2011-02-01 0:33 ` [RFC][PATCH 2/6] pagewalk: only split huge pages when necessary Dave Hansen
2011-02-01 10:04 ` Johannes Weiner
2011-02-01 15:03 ` Dave Hansen
2011-02-03 21:22 ` David Rientjes
2011-02-03 21:33 ` Dave Hansen
2011-02-03 21:46 ` David Rientjes
2011-02-04 17:19 ` Dave Hansen
2011-02-04 21:10 ` Andrea Arcangeli
2011-02-01 0:34 ` [RFC][PATCH 3/6] break out smaps_pte_entry() from smaps_pte_range() Dave Hansen
2011-02-01 10:08 ` Johannes Weiner
2011-02-03 21:22 ` David Rientjes
2011-02-03 21:40 ` Dave Hansen
2011-02-01 0:34 ` [RFC][PATCH 4/6] pass pte size argument in to smaps_pte_entry() Dave Hansen
2011-02-01 10:09 ` Johannes Weiner
2011-02-03 21:22 ` David Rientjes
2011-02-01 0:34 ` [RFC][PATCH 5/6] teach smaps_pte_range() about THP pmds Dave Hansen
2011-02-01 10:11 ` Johannes Weiner
2011-02-01 15:02 ` Dave Hansen
2011-02-01 16:09 ` Andrea Arcangeli
2011-02-03 21:22 ` David Rientjes
2011-02-03 21:34 ` Dave Hansen
2011-02-01 0:34 ` [RFC][PATCH 6/6] have smaps show transparent huge pages Dave Hansen
2011-02-01 10:12 ` Johannes Weiner
2011-02-03 21:22 ` David Rientjes
2011-02-01 15:38 ` Andrea Arcangeli [this message]
2011-02-01 17:15 ` [RFC][PATCH 0/6] more detailed per-process transparent hugepage statistics Dave Hansen
2011-02-01 20:39 ` Andrea Arcangeli
2011-02-01 20:56 ` Dave Hansen
2011-02-02 0:07 ` Andrea Arcangeli
2011-02-08 17:54 ` Dave Hansen
2011-02-08 18:17 ` Andrea Arcangeli
2011-02-03 21:54 ` David Rientjes
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20110201153857.GA18740@random.random \
--to=aarcange@redhat.com \
--cc=dave@linux.vnet.ibm.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mjwolf@us.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).