From: Andrew Morton <akpm@google.com>
To: Michel Lespinasse <walken@google.com>
Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org,
KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>,
Dave Hansen <dave@linux.vnet.ibm.com>,
Andrea Arcangeli <aarcange@redhat.com>,
Rik van Riel <riel@redhat.com>,
Johannes Weiner <jweiner@redhat.com>,
KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>,
Hugh Dickins <hughd@google.com>,
Peter Zijlstra <a.p.zijlstra@chello.nl>,
Michael Wolf <mjwolf@us.ibm.com>,
Andrew Morton <akpm@linux-foundation.org>
Subject: Re: [PATCH 0/8] idle page tracking / working set estimation
Date: Thu, 22 Sep 2011 16:13:25 -0700 [thread overview]
Message-ID: <20110922161325.f94f9c9e.akpm@google.com> (raw)
In-Reply-To: <1316230753-8693-1-git-send-email-walken@google.com>
On Fri, 16 Sep 2011 20:39:05 -0700
Michel Lespinasse <walken@google.com> wrote:
> Please comment on the following patches (which are against the v3.0 kernel).
> We are using these to collect memory utilization statistics for each cgroup
> accross many machines, and optimize job placement accordingly.
Please consider updating /proc/kpageflags with the three new page
flags. If "yes": update. If "no": explain/justify.
Which prompts the obvious: the whole feature could have been mostly
implemented in userspace, using kpageflags. Some additional kernel
support would presumably be needed, but I'm not sure how much.
If you haven't already done so, please sketch down what that
infrastructure would look like and have a think about which approach is
preferable?
What bugs me a bit about the proposal is its cgroups-centricity. The
question "how much memory is my application really using" comes up
again and again. It predates cgroups. One way to answer that question
is to force a massive amount of swapout on the entire machine, then let
the system recover and take a look at your app's RSS two minutes later.
This is very lame.
It's a legitimate requirement, and the kstaled infrastructure puts a
lot of things in place to answer it well. But as far as I can tell it
doesn't quite get over the line. Then again, maybe it _does_ get
there: put the application into a memcg all of its own, just for
instrumentation purposes and then use kstaled to monitor it?
<later> OK, I'm surprised to discover that kstaled is doing a physical
scan and not a virtual one. I assume it works, but I don't know why.
But it makes the above requirement harder, methinks.
How does all this code get along with hugepages, btw?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2011-09-22 23:13 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-09-17 3:39 [PATCH 0/8] idle page tracking / working set estimation Michel Lespinasse
2011-09-17 3:39 ` [PATCH 1/8] page_referenced: replace vm_flags parameter with struct pr_info Michel Lespinasse
2011-09-17 3:44 ` Joe Perches
2011-09-17 4:51 ` Michel Lespinasse
2011-09-20 19:05 ` Rik van Riel
2011-09-21 2:51 ` Michel Lespinasse
2011-09-17 3:39 ` [PATCH 2/8] kstaled: documentation and config option Michel Lespinasse
2011-09-20 21:23 ` Rik van Riel
2011-09-23 19:27 ` Rik van Riel
2011-09-17 3:39 ` [PATCH 3/8] kstaled: page_referenced_kstaled() and supporting infrastructure Michel Lespinasse
2011-09-20 19:36 ` Peter Zijlstra
2011-09-17 3:39 ` [PATCH 4/8] kstaled: minimalistic implementation Michel Lespinasse
2011-09-22 23:14 ` Andrew Morton
2011-09-23 8:37 ` Michel Lespinasse
2011-09-17 3:39 ` [PATCH 5/8] kstaled: skip non-RAM regions Michel Lespinasse
2011-09-17 3:39 ` [PATCH 6/8] kstaled: rate limit pages scanned per second Michel Lespinasse
2011-09-22 23:15 ` Andrew Morton
2011-09-23 10:18 ` Michel Lespinasse
2011-09-17 3:39 ` [PATCH 7/8] kstaled: add histogram sampling functionality Michel Lespinasse
2011-09-22 23:15 ` Andrew Morton
2011-09-23 10:26 ` Michel Lespinasse
2011-09-17 3:39 ` [PATCH 8/8] kstaled: add incrementally updating stale page count Michel Lespinasse
2011-09-22 23:13 ` Andrew Morton [this message]
2011-09-23 1:23 ` [PATCH 0/8] idle page tracking / working set estimation Michel Lespinasse
2011-09-27 10:03 ` Balbir Singh
2011-09-27 10:14 ` Michel Lespinasse
2011-09-27 16:50 ` Balbir Singh
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20110922161325.f94f9c9e.akpm@google.com \
--to=akpm@google.com \
--cc=a.p.zijlstra@chello.nl \
--cc=aarcange@redhat.com \
--cc=akpm@linux-foundation.org \
--cc=dave@linux.vnet.ibm.com \
--cc=hughd@google.com \
--cc=jweiner@redhat.com \
--cc=kamezawa.hiroyu@jp.fujitsu.com \
--cc=kosaki.motohiro@jp.fujitsu.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mjwolf@us.ibm.com \
--cc=riel@redhat.com \
--cc=walken@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).