* Re: Ping? RE: [GIT PULL] mm/vfs/fs:cleancache for 2.6.37 merge window
2010-10-30 19:06 ` Andrew Morton
@ 2010-10-30 20:49 ` Jeremy Fitzhardinge
2010-10-31 2:19 ` Dan Magenheimer
2011-01-19 16:42 ` Dan Magenheimer
2 siblings, 0 replies; 9+ messages in thread
From: Jeremy Fitzhardinge @ 2010-10-30 20:49 UTC (permalink / raw)
To: Andrew Morton
Cc: Dan Magenheimer, torvalds, linux-kernel, Christoph Hellwig,
Chris Mason, Nitin Gupta
On 10/30/2010 12:06 PM, Andrew Morton wrote:
> On Wed, 27 Oct 2010 11:37:47 -0700 (PDT) Dan Magenheimer <dan.magenheimer@oracle.com> wrote:
>
>> Ping? I hope you are still considering this. If not or if
>> there are any questions I can answer, please let me know.
> What's happened here is that the patchset has gone through its
> iterations and a few people have commented and then after a while,
> nobody had anything to say about the code so nobody said anything more.
>
> But silence doesn't mean acceptance - it just means that nobody had
> anything to say.
>
> I think I looked at the earlier iterations, tried to understand the
> point behind it all, made a few code suggestions and eventually tuned
> out. At that time (and hence at this time) I just cannot explain to
> myself why we would want to merge this code.
>
> All new code is a cost/benefit decision. The costs are pretty well
> known: larger codebase, more code for us and our "customers" to
> maintain and support, etc. That the code pokes around in vfs and
> various filesystems does increase those costs a little.
>
> But the extent of the benefits to our users aren't obvious to me. The
> code is still xen-specific, I believe? If so, that immediately reduces
> the benefit side by a large amount simply because of the reduced
> audience.
>
> We did spend some time trying to get this wired up to zram so that the
> feature would be potentially useful to *all* users, thereby setting the
> usefulness multiplier back to 1.0. But I don't recall that anything
> came of this?
Nitin was definitely working on this and made some constructive comments
as a result, but I don't know if there's any completed/usable code or not.
> I also don't know how useful the code is to its intended
> micro-audience: xen users!
The benefit is that it allows memory to be much more fluidly assigned
between domains as needed, rather than having to statically allocate big
chunks of memory. The result is that its possible to provision domains
with much smaller amounts of memory while still reducing the number of
real pagefault IOs. Dan's numbers are certainly very interesting (Dan,
perhaps you can repost those results).
However, I don't think it has been widely deployed yet, since most users
are using upstream/distro kernels.
> So can we please revisit all this from the top level? Jeremy, your
> input would be valuable.
OK, I'll need to get myself up to speed on the issues again. Will you
be about in Boston next week?
> Christoph, I recall that you had technical
> objections - can you please repeat those?
I think (and I don't want to misrepresent or minimize Christoph's
concerns here) the most acute one were the need for a one-line addition
to each filesystem which wants to participate.
> It's the best I can do to kick this along, sorry.
Thanks,
J
^ permalink raw reply [flat|nested] 9+ messages in thread* RE: Ping? RE: [GIT PULL] mm/vfs/fs:cleancache for 2.6.37 merge window
2010-10-30 19:06 ` Andrew Morton
2010-10-30 20:49 ` Jeremy Fitzhardinge
@ 2010-10-31 2:19 ` Dan Magenheimer
2011-01-19 16:42 ` Dan Magenheimer
2 siblings, 0 replies; 9+ messages in thread
From: Dan Magenheimer @ 2010-10-31 2:19 UTC (permalink / raw)
To: Andrew Morton
Cc: torvalds, linux-kernel, Christoph Hellwig, Jeremy Fitzhardinge,
Nitin Gupta, Chris Mason
> From: Andrew Morton [mailto:akpm@linux-foundation.org]
> So can we please revisit all this from the top level?
> Jeremy, your input would be valuable.
Hi Andrew --
Thanks for your reply! Between preparing for LPC and some
upcoming personal time off, I may not be able to reply in
a timely way to some future discussion on this thread, so
I will try to respond now but still encourage others to
respond. I would also be happy to talk f2f at LPC. I see as
I type this that Jeremy has already replied and will try
to incorporate information re his comments.
Andrew, I think you raise four interesting questions...
I hope it's OK if I paraphrase rather than quote directly?
1) Is something Xen-specific [though no, it's not] worth
the cost of this addition to the code base?
2) Even if (1) is yes, is this going to be used by a
significant percentage of Xen users?
3) Is cleancache beneficial to NON-Xen users?
4) Ignoring the "user-base" questions, are there technical
objections to and issues with cleancache that would
stop it from being merged?
So, I hope you have the time to read my long-winded reply:
1) By using the term, "micro-audience" to refer to the xen
user base, I think you are grossly minimizing their
number. Since this is a technical list, we can leave
it up to industry analysts to argue that point, but I
think it is at least fair to point out that there are
literally dozens of merges accepted in just this window
that had a much larger code impact than cleancache
but have a much smaller user base than Xen.
While it is reasonable to argue that most of those other
merges don't touch core mm/vfs code, please realize that
the cleancache impact to mm/vfs totals less than 50 lines,
(written by Chris Mason, not me) and these patched lines
have been essentially static since about 2.6.18, and
in all of the versions I've posted. Most of the
cleancache version churn was designing a clean layer
so that those hooks are more broadly useful plus
my inexperience in Linux coding style and the process
for posting patches to lkml. (The layering, plus
some sysfs info and documentation contributes nearly
all of the additional lines of the patch.)
2) Will a lot of Xen users use this? That's sort of a
chicken and egg thing. I have talked to many real
Xen users who would love to use cleancache (actually
Transcendent Memory of which cleancache is the
key part), but they get scared when I tell them it
requires patches that are not upstream yet. Distros
(including Oracle's) are similarly skittish.
At Linux and Xen conferences[1], I've shown a nice performance
improvement on some workloads and negligible worst case
loss. The on-list controversies over cleancache have rarely
involved any performance questions/issues, but I can
revisit the data if it makes a difference on the
decision to merge.
3) GregKH was ready to apply Nitin's zram (actually called
zcache) patch until I pointed out that it was dependent
on cleancache which wasn't yet merged. See:
http://lkml.org/lkml/2010/7/22/491. Due to an internal
API change at v5 (to use exportfs to support fs's where
an inode_t doesn't uniquely represent a file -- with
input and guidance from Andreas Dilger),
zcache needs a few changes and Nitin appears otherwise
occupied right now. If Nitin doesn't get `round to it
and doesn't object, and this is the only barrier to merging
cleancache, I'll be happy to make those changes myself.
I'm separately working on some similar in-kernel compression
ideas, plus the "page-accessible memory" ideas I proposed
for LSF10/MM where, ahem, certain future idiosyncratic fast
solid-state-ish memory technologies are a good match for
cleancache. The core hooks are highly similar to what was
used for Geiger (google Geiger ASPLOS 2006) and I've heard
from several university students that are interested in
researching other ideas built on top of cleancache.
Oh, and at LinuxCon, Anthony Liguori told me he thought
there were at least parts of it that KVM can use.
So, no, this isn't a xen-only thing, nor a one-time
thing. Cleancache is a generic mechanism for grabbing
data from clean pages when they are reclaimed, cacheing
the data in non-kernel-directly-addressable memory, and
avoiding kernel disk reads into file cache when the pages
can be found in cleancache. I just happen to get paid to
work on Xen, so that's where this story started.
4) The on-list lkml patch review process was very helpful
in helping to clean up the cleancache patchset. The
biggest technical hole -- filesystems for which an
inode_t can't uniquely identify a file -- is fixed.
Other technical questions/feedback are summarized
in the commit comments and in the FAQ included with
the patch including, I believe, everything both
on-list and f2f from hch.
A LOT of people have provided review, useful feedback
and questions, and I've tried to be very diligent in
replying to all reviewers. If I've missed any that
would lead anyone to disagree with merging cleancache,
I hope they will re-raise them prior to the next
merge window.
Hope that helps... and I hope I am not sounding defensive.
Thanks again for offering to revisit it.
Thanks,
Dan
[1] For a quick performance summary, see slides 37-39 of http://oss.oracle.com/projects/tmem/dist/documentation/presentations/TranscendentMemoryXenSummit2010.pdf
^ permalink raw reply [flat|nested] 9+ messages in thread
* RE: Ping? RE: [GIT PULL] mm/vfs/fs:cleancache for 2.6.37 merge window
2010-10-30 19:06 ` Andrew Morton
2010-10-30 20:49 ` Jeremy Fitzhardinge
2010-10-31 2:19 ` Dan Magenheimer
@ 2011-01-19 16:42 ` Dan Magenheimer
2011-02-17 20:14 ` PING? cleancache for 2.6.39 window? Dan Magenheimer
2 siblings, 1 reply; 9+ messages in thread
From: Dan Magenheimer @ 2011-01-19 16:42 UTC (permalink / raw)
To: Andrew Morton
Cc: torvalds, linux-kernel, Christoph Hellwig, Jeremy Fitzhardinge
> On Wed, 27 Oct 2010 11:37:47 -0700 (PDT) Dan Magenheimer
> <dan.magenheimer@oracle.com> wrote:
>
> > Ping? I hope you are still considering this. If not or if
> > there are any questions I can answer, please let me know.
>
> What's happened here is that the patchset has gone through its
> iterations and a few people have commented and then after a while,
> nobody had anything to say about the code so nobody said anything more.
>
> But silence doesn't mean acceptance - it just means that nobody had
> anything to say.
>
> I think I looked at the earlier iterations, tried to understand the
> point behind it all, made a few code suggestions and eventually tuned
> out. At that time (and hence at this time) I just cannot explain to
> myself why we would want to merge this code.
>
> All new code is a cost/benefit decision. The costs are pretty well
> known: larger codebase, more code for us and our "customers" to
> maintain and support, etc. That the code pokes around in vfs and
> various filesystems does increase those costs a little.
>
> But the extent of the benefits to our users aren't obvious to me. The
> coe is still xen-specific, I believe? If so, that immediately reduces
> the benefit side by a large amount simply because of the reduced
> audience.
>
> We did spend some time trying to get this wired up to zram so that the
> feature would be potentially useful to *all* users, thereby setting the
> usefulness multiplier back to 1.0. But I don't recall that anything
> came of this?
>
> I also don't know how useful the code is to its intended
> micro-audience: xen users!
>
> So can we please revisit all this from the top level? Jeremy, your
> input would be valuable. Christoph, I recall that you had technical
> objections - can you please repeat those?
>
> It's the best I can do to kick this along, sorry.
Hi Andrew (and Linus) --
Time to re-open this conversation (for 2.6.39 merge window)?
Assuming GregKH approves kztmem as a staging driver, it should
now set "the usefulness multiplier back to 1.0". Kztmem
is a superset of Nitin's zcache and zram but more dynamic
and is completely independent of Xen and virtualization.
See kztmem overview: https://lkml.org/lkml/2011/1/18/170
And I believe Christoph's technical objections have all been
resolved. See longer version of previous reply here:
https://lkml.org/lkml/2010/10/30/226
So please reconsider cleancache!
Thanks,
Dan
P.S. Christoph, apologies, I see I didn't have you on the dist list
for the kztmem patch.
^ permalink raw reply [flat|nested] 9+ messages in thread
* PING? cleancache for 2.6.39 window?
2011-01-19 16:42 ` Dan Magenheimer
@ 2011-02-17 20:14 ` Dan Magenheimer
0 siblings, 0 replies; 9+ messages in thread
From: Dan Magenheimer @ 2011-02-17 20:14 UTC (permalink / raw)
To: Andrew Morton, torvalds; +Cc: linux-kernel
> From: Dan Magenheimer
> Sent: Wednesday, January 19, 2011 9:42 AM
> To: Andrew Morton
> Cc: torvalds@linux-foundation.org; linux-kernel@vger.kernel.org;
> Christoph Hellwig; Jeremy Fitzhardinge
> Subject: RE: Ping? RE: [GIT PULL] mm/vfs/fs:cleancache for 2.6.37 merge
> window
>
> > On Wed, 27 Oct 2010 11:37:47 -0700 (PDT) Dan Magenheimer
> > <dan.magenheimer@oracle.com> wrote:
> >
> > > Ping? I hope you are still considering this. If not or if
> > > there are any questions I can answer, please let me know.
> >
> > What's happened here is that the patchset has gone through its
> > iterations and a few people have commented and then after a while,
> > nobody had anything to say about the code so nobody said anything
> more.
> >
> > But silence doesn't mean acceptance - it just means that nobody had
> > anything to say.
> >
> > I think I looked at the earlier iterations, tried to understand the
> > point behind it all, made a few code suggestions and eventually tuned
> > out. At that time (and hence at this time) I just cannot explain to
> > myself why we would want to merge this code.
> >
> > All new code is a cost/benefit decision. The costs are pretty well
> > known: larger codebase, more code for us and our "customers" to
> > maintain and support, etc. That the code pokes around in vfs and
> > various filesystems does increase those costs a little.
> >
> > But the extent of the benefits to our users aren't obvious to me.
> The
> > coe is still xen-specific, I believe? If so, that immediately
> reduces
> > the benefit side by a large amount simply because of the reduced
> > audience.
> >
> > We did spend some time trying to get this wired up to zram so that
> the
> > feature would be potentially useful to *all* users, thereby setting
> the
> > usefulness multiplier back to 1.0. But I don't recall that anything
> > came of this?
> >
> > I also don't know how useful the code is to its intended
> > micro-audience: xen users!
> >
> > So can we please revisit all this from the top level? Jeremy, your
> > input would be valuable. Christoph, I recall that you had technical
> > objections - can you please repeat those?
> >
> > It's the best I can do to kick this along, sorry.
>
> Hi Andrew (and Linus) --
>
> Time to re-open this conversation (for 2.6.39 merge window)?
>
> Assuming GregKH approves kztmem as a staging driver, it should
> now set "the usefulness multiplier back to 1.0". Kztmem
> is a superset of Nitin's zcache and zram but more dynamic
> and is completely independent of Xen and virtualization.
>
> See kztmem overview: https://lkml.org/lkml/2011/1/18/170
>
> And I believe Christoph's technical objections have all been
> resolved. See longer version of previous reply here:
> https://lkml.org/lkml/2010/10/30/226
>
> So please reconsider cleancache!
Hi Andrew (and Linus) --
As you may have seen, gregkh has taken zcache into drivers/staging
for merging when the 2.6.39 window opens. And zcache (which
works entirely in-kernel, no virtualization required) depends on
cleancache.
Cleancache and zcache are both in linux-next and, via linux-next,
in mmotm. Last October, Linus said that he preferred cleancache
to merge via you (Andrew), not pulled from my git tree. So:
1) Is cleancache finally now acceptable for merging in the
upcoming 2.6.39 window? and
2) If so, is there anything else I need to do to ensure the
merging of cleancache happens during the open window or will
it all happen automagically (from my point of view) through
your (Andrew's) normal open-window processes?
Sorry to be excessively persistent, but I don't want to miss
the 2.6.39 window due to my newbie-ness. And I plan to be on
vacation for some time during March and would also like to ensure
I don't miss something *I* need to do before or during the
window.
Thanks,
Dan Magenheimer
^ permalink raw reply [flat|nested] 9+ messages in thread