Re: KSM: numa awareness sysfs knob

[Nai Xia <nai.xia@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 3130 bytes --]

On Thursday 01 December 2011 18:16:40 Petr Holasek wrote:
> On Wed, 30 Nov 2011, Andrew Morton wrote:
> 
> > Date: Wed, 30 Nov 2011 15:47:19 -0800
> > From: Andrew Morton <akpm@linux-foundation.org>
> > To: Petr Holasek <pholasek@redhat.com>
> > Cc: Hugh Dickins <hughd@google.com>, Andrea Arcangeli
> >  <aarcange@redhat.com>, linux-kernel@vger.kernel.org, linux-mm@kvack.org,
> >  Anton Arapov <anton@redhat.com>
> > Subject: Re: [PATCH] [RFC] KSM: numa awareness sysfs knob
> > 
> > On Wed, 30 Nov 2011 11:37:26 +0100
> > Petr Holasek <pholasek@redhat.com> wrote:
> > 
> > > Introduce a new sysfs knob /sys/kernel/mm/ksm/max_node_dist, whose
> > > value will be used as the limitation for node distance of merged pages.
> > > 
> > 
> > The changelog doesn't really describe why you think Linux needs this
> > feature?  What's the reasoning?  Use cases?  What value does it provide?
> 
> Typical use-case could be a lot of KVM guests on NUMA machine and cpus from
> more distant nodes would have significant increase of access latency to the
> merged ksm page. I chose sysfs knob for higher scalability.

Seems this consideration for NUMA is sound. 

> 
> > 
> > > index b392e49..b882140 100644
> > > --- a/Documentation/vm/ksm.txt
> > > +++ b/Documentation/vm/ksm.txt
> > > @@ -58,6 +58,10 @@ sleep_millisecs  - how many milliseconds ksmd should sleep before next scan
> > >                     e.g. "echo 20 > /sys/kernel/mm/ksm/sleep_millisecs"
> > >                     Default: 20 (chosen for demonstration purposes)
> > >  
> > > +max_node_dist    - maximum node distance between two pages which could be
> > > +                   merged.
> > > +                   Default: 255 (without any limitations)
> > 
> > And this doesn't explain to our users why they might want to alter it,
> > and what effects they would see from doing so.  Maybe that's obvious to
> > them...
> 
> Now I can't figure out more extensive description of this feature, but we
> could explain it deeply, of course.

However, if we don't know what the number fed into this knob really means, 
seems nobody would think of using this knob...

Then why not make this NUMA feature automatically adjusted by some algorithm
instread of dropping it to userland?

BTW, the algrothim you already include in this patch seems unstable itself:

Suppose we have three duplicated pages in order: Page_a, Page_b, Page_c with 
distance(Page_a, Page_b) == distance(Page_b, Page_c) == 3, 
but distance(Page_a, Page_c) == 6 and if max_node_dist == 3, 
a stable algorithm should result in Page_a and Page_c being merged to Page_b,
independent of the order these pages get scanned. 

But with your patch, if ksmd goes Page_b --> Page_c --> Page_a, will it 
result in Page_b being merged to Page_c but Page_a not merged since its 
distance to Page_c is 6?

It may easy to further deduce that maybe a worst case(or even many cases?) 
for your patch will get many many could-be-merged pages not merged simply 
because of the sequence they are scanned.

The problem you plan to solve maybe worthwhile, but it may also be much more
complex than you expected ;-)
  

BR,

Nai

[-- Attachment #2: Type: text/html, Size: 14833 bytes --]