All of lore.kernel.org
 help / color / mirror / Atom feed
From: Andrea Righi <arighi-vWjgImWzx8FBDgjK7y7TUQ@public.gmane.org>
To: Vivek Goyal <vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Cc: Daisuke Nishimura
	<nishimura-YQH0OdQVrdy45+QrQBaojngSJqDPrsil@public.gmane.org>,
	linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	Trond Myklebust
	<trond.myklebust-41N18TsMXrtuMpJDpNschA@public.gmane.org>,
	linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org,
	Suleiman Souhlal
	<suleiman-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>,
	Andrew Morton
	<akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org>,
	containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org,
	Balbir Singh
	<balbir-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
Subject: Re: [PATCH -mmotm 4/4] memcg: dirty pages instrumentation
Date: Thu, 4 Mar 2010 22:51:01 +0100	[thread overview]
Message-ID: <20100304215101.GB4787@linux> (raw)
In-Reply-To: <20100304194144.GE18786-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>

On Thu, Mar 04, 2010 at 02:41:44PM -0500, Vivek Goyal wrote:
> On Thu, Mar 04, 2010 at 11:40:15AM +0100, Andrea Righi wrote:
> 
> [..]
> > diff --git a/mm/page-writeback.c b/mm/page-writeback.c
> > index 5a0f8f3..c5d14ea 100644
> > --- a/mm/page-writeback.c
> > +++ b/mm/page-writeback.c
> > @@ -137,13 +137,16 @@ static struct prop_descriptor vm_dirties;
> >   */
> >  static int calc_period_shift(void)
> >  {
> > +	struct dirty_param dirty_param;
> >  	unsigned long dirty_total;
> >  
> > -	if (vm_dirty_bytes)
> > -		dirty_total = vm_dirty_bytes / PAGE_SIZE;
> > +	get_dirty_param(&dirty_param);
> > +
> > +	if (dirty_param.dirty_bytes)
> > +		dirty_total = dirty_param.dirty_bytes / PAGE_SIZE;
> >  	else
> > -		dirty_total = (vm_dirty_ratio * determine_dirtyable_memory()) /
> > -				100;
> > +		dirty_total = (dirty_param.dirty_ratio *
> > +				determine_dirtyable_memory()) / 100;
> >  	return 2 + ilog2(dirty_total - 1);
> >  }
> >  
> 
> Hmm.., I have been staring at this for some time and I think something is
> wrong. I don't fully understand the way floating proportions are working
> but this function seems to be calculating the period over which we need
> to measuer the proportions. (vm_completion proportion and vm_dirties
> proportions).
> 
> And we this period (shift), when admin updates dirty_ratio or dirty_bytes
> etc. In that case we recalculate the global dirty limit and take log2 and
> use that as period over which we monitor and calculate proportions.
> 
> If yes, then it should be global and not per cgroup (because all our 
> accouting of bdi completion is global and not per cgroup).
> 
> PeterZ, can tell us more about it. I am just raising the flag here to be
> sure.
> 
> Thanks
> Vivek

Hi Vivek,

I tend to agree, we must use global dirty values here.

BTW, update_completion_period() is called from dirty_* handlers, so it's
totally unrelated to use the current memcg. That's the memcg where the
admin is running, so probably it's the root memcg almost all the time,
but it's wrong in principle. In conclusion this patch shouldn't touch
calc_period_shift().

Thanks,
-Andrea

WARNING: multiple messages have this Message-ID (diff)
From: Andrea Righi <arighi@develer.com>
To: Vivek Goyal <vgoyal@redhat.com>
Cc: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>,
	Balbir Singh <balbir@linux.vnet.ibm.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Trond Myklebust <trond.myklebust@fys.uio.no>,
	Suleiman Souhlal <suleiman@google.com>,
	Greg Thelen <gthelen@google.com>,
	Daisuke Nishimura <nishimura@mxp.nes.nec.co.jp>,
	"Kirill A. Shutemov" <kirill@shutemov.name>,
	Andrew Morton <akpm@linux-foundation.org>,
	containers@lists.linux-foundation.org,
	linux-kernel@vger.kernel.org, linux-mm@kvack.org
Subject: Re: [PATCH -mmotm 4/4] memcg: dirty pages instrumentation
Date: Thu, 4 Mar 2010 22:51:01 +0100	[thread overview]
Message-ID: <20100304215101.GB4787@linux> (raw)
In-Reply-To: <20100304194144.GE18786@redhat.com>

On Thu, Mar 04, 2010 at 02:41:44PM -0500, Vivek Goyal wrote:
> On Thu, Mar 04, 2010 at 11:40:15AM +0100, Andrea Righi wrote:
> 
> [..]
> > diff --git a/mm/page-writeback.c b/mm/page-writeback.c
> > index 5a0f8f3..c5d14ea 100644
> > --- a/mm/page-writeback.c
> > +++ b/mm/page-writeback.c
> > @@ -137,13 +137,16 @@ static struct prop_descriptor vm_dirties;
> >   */
> >  static int calc_period_shift(void)
> >  {
> > +	struct dirty_param dirty_param;
> >  	unsigned long dirty_total;
> >  
> > -	if (vm_dirty_bytes)
> > -		dirty_total = vm_dirty_bytes / PAGE_SIZE;
> > +	get_dirty_param(&dirty_param);
> > +
> > +	if (dirty_param.dirty_bytes)
> > +		dirty_total = dirty_param.dirty_bytes / PAGE_SIZE;
> >  	else
> > -		dirty_total = (vm_dirty_ratio * determine_dirtyable_memory()) /
> > -				100;
> > +		dirty_total = (dirty_param.dirty_ratio *
> > +				determine_dirtyable_memory()) / 100;
> >  	return 2 + ilog2(dirty_total - 1);
> >  }
> >  
> 
> Hmm.., I have been staring at this for some time and I think something is
> wrong. I don't fully understand the way floating proportions are working
> but this function seems to be calculating the period over which we need
> to measuer the proportions. (vm_completion proportion and vm_dirties
> proportions).
> 
> And we this period (shift), when admin updates dirty_ratio or dirty_bytes
> etc. In that case we recalculate the global dirty limit and take log2 and
> use that as period over which we monitor and calculate proportions.
> 
> If yes, then it should be global and not per cgroup (because all our 
> accouting of bdi completion is global and not per cgroup).
> 
> PeterZ, can tell us more about it. I am just raising the flag here to be
> sure.
> 
> Thanks
> Vivek

Hi Vivek,

I tend to agree, we must use global dirty values here.

BTW, update_completion_period() is called from dirty_* handlers, so it's
totally unrelated to use the current memcg. That's the memcg where the
admin is running, so probably it's the root memcg almost all the time,
but it's wrong in principle. In conclusion this patch shouldn't touch
calc_period_shift().

Thanks,
-Andrea

WARNING: multiple messages have this Message-ID (diff)
From: Andrea Righi <arighi@develer.com>
To: Vivek Goyal <vgoyal@redhat.com>
Cc: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>,
	Balbir Singh <balbir@linux.vnet.ibm.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Trond Myklebust <trond.myklebust@fys.uio.no>,
	Suleiman Souhlal <suleiman@google.com>,
	Greg Thelen <gthelen@google.com>,
	Daisuke Nishimura <nishimura@mxp.nes.nec.co.jp>,
	"Kirill A. Shutemov" <kirill@shutemov.name>,
	Andrew Morton <akpm@linux-foundation.org>,
	containers@lists.linux-foundation.org,
	linux-kernel@vger.kernel.org, linux-mm@kvack.org
Subject: Re: [PATCH -mmotm 4/4] memcg: dirty pages instrumentation
Date: Thu, 4 Mar 2010 22:51:01 +0100	[thread overview]
Message-ID: <20100304215101.GB4787@linux> (raw)
In-Reply-To: <20100304194144.GE18786@redhat.com>

On Thu, Mar 04, 2010 at 02:41:44PM -0500, Vivek Goyal wrote:
> On Thu, Mar 04, 2010 at 11:40:15AM +0100, Andrea Righi wrote:
> 
> [..]
> > diff --git a/mm/page-writeback.c b/mm/page-writeback.c
> > index 5a0f8f3..c5d14ea 100644
> > --- a/mm/page-writeback.c
> > +++ b/mm/page-writeback.c
> > @@ -137,13 +137,16 @@ static struct prop_descriptor vm_dirties;
> >   */
> >  static int calc_period_shift(void)
> >  {
> > +	struct dirty_param dirty_param;
> >  	unsigned long dirty_total;
> >  
> > -	if (vm_dirty_bytes)
> > -		dirty_total = vm_dirty_bytes / PAGE_SIZE;
> > +	get_dirty_param(&dirty_param);
> > +
> > +	if (dirty_param.dirty_bytes)
> > +		dirty_total = dirty_param.dirty_bytes / PAGE_SIZE;
> >  	else
> > -		dirty_total = (vm_dirty_ratio * determine_dirtyable_memory()) /
> > -				100;
> > +		dirty_total = (dirty_param.dirty_ratio *
> > +				determine_dirtyable_memory()) / 100;
> >  	return 2 + ilog2(dirty_total - 1);
> >  }
> >  
> 
> Hmm.., I have been staring at this for some time and I think something is
> wrong. I don't fully understand the way floating proportions are working
> but this function seems to be calculating the period over which we need
> to measuer the proportions. (vm_completion proportion and vm_dirties
> proportions).
> 
> And we this period (shift), when admin updates dirty_ratio or dirty_bytes
> etc. In that case we recalculate the global dirty limit and take log2 and
> use that as period over which we monitor and calculate proportions.
> 
> If yes, then it should be global and not per cgroup (because all our 
> accouting of bdi completion is global and not per cgroup).
> 
> PeterZ, can tell us more about it. I am just raising the flag here to be
> sure.
> 
> Thanks
> Vivek

Hi Vivek,

I tend to agree, we must use global dirty values here.

BTW, update_completion_period() is called from dirty_* handlers, so it's
totally unrelated to use the current memcg. That's the memcg where the
admin is running, so probably it's the root memcg almost all the time,
but it's wrong in principle. In conclusion this patch shouldn't touch
calc_period_shift().

Thanks,
-Andrea

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  parent reply	other threads:[~2010-03-04 21:51 UTC|newest]

Thread overview: 68+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2010-03-04 10:40 [PATCH -mmotm 0/4] memcg: per cgroup dirty limit (v4) Andrea Righi
2010-03-04 10:40 ` Andrea Righi
2010-03-04 10:40 ` [PATCH -mmotm 1/4] memcg: dirty memory documentation Andrea Righi
2010-03-04 10:40   ` Andrea Righi
2010-03-04 10:40 ` [PATCH -mmotm 2/4] page_cgroup: introduce file cache flags Andrea Righi
2010-03-04 10:40   ` Andrea Righi
     [not found]   ` <1267699215-4101-3-git-send-email-arighi-vWjgImWzx8FBDgjK7y7TUQ@public.gmane.org>
2010-03-05  6:32     ` Balbir Singh
2010-03-05  6:32   ` Balbir Singh
2010-03-05  6:32     ` Balbir Singh
2010-03-05 22:35     ` Andrea Righi
2010-03-05 22:35       ` Andrea Righi
     [not found]     ` <20100305063249.GH3073-SINUvgVNF2CyUtPGxGje5AC/G2K4zDHf@public.gmane.org>
2010-03-05 22:35       ` Andrea Righi
     [not found] ` <1267699215-4101-1-git-send-email-arighi-vWjgImWzx8FBDgjK7y7TUQ@public.gmane.org>
2010-03-04 10:40   ` [PATCH -mmotm 1/4] memcg: dirty memory documentation Andrea Righi
2010-03-04 10:40   ` [PATCH -mmotm 2/4] page_cgroup: introduce file cache flags Andrea Righi
2010-03-04 10:40   ` [PATCH -mmotm 3/4] memcg: dirty pages accounting and limiting infrastructure Andrea Righi
2010-03-04 10:40   ` [PATCH -mmotm 4/4] memcg: dirty pages instrumentation Andrea Righi
2010-03-04 17:11   ` [PATCH -mmotm 0/4] memcg: per cgroup dirty limit (v4) Balbir Singh
2010-03-04 10:40 ` [PATCH -mmotm 3/4] memcg: dirty pages accounting and limiting infrastructure Andrea Righi
2010-03-04 10:40   ` Andrea Righi
     [not found]   ` <1267699215-4101-4-git-send-email-arighi-vWjgImWzx8FBDgjK7y7TUQ@public.gmane.org>
2010-03-04 11:54     ` Kirill A. Shutemov
2010-03-05  1:12     ` Daisuke Nishimura
2010-03-04 11:54   ` Kirill A. Shutemov
2010-03-04 11:54     ` Kirill A. Shutemov
2010-03-05  1:12   ` Daisuke Nishimura
2010-03-05  1:12     ` Daisuke Nishimura
     [not found]     ` <20100305101234.909001e8.nishimura-YQH0OdQVrdy45+QrQBaojngSJqDPrsil@public.gmane.org>
2010-03-05  1:58       ` KAMEZAWA Hiroyuki
2010-03-05 22:14       ` Andrea Righi
2010-03-05  1:58     ` KAMEZAWA Hiroyuki
2010-03-05  1:58       ` KAMEZAWA Hiroyuki
     [not found]       ` <20100305105855.9b53176c.kamezawa.hiroyu-+CUm20s59erQFUHtdCDX3A@public.gmane.org>
2010-03-05  7:01         ` Balbir Singh
2010-03-05  7:01           ` Balbir Singh
2010-03-05  7:01           ` Balbir Singh
2010-03-05 22:14         ` Andrea Righi
2010-03-05 22:14       ` Andrea Righi
2010-03-05 22:14         ` Andrea Righi
2010-03-05 22:14     ` Andrea Righi
2010-03-05 22:14       ` Andrea Righi
2010-03-04 10:40 ` [PATCH -mmotm 4/4] memcg: dirty pages instrumentation Andrea Righi
2010-03-04 10:40   ` Andrea Righi
     [not found]   ` <1267699215-4101-5-git-send-email-arighi-vWjgImWzx8FBDgjK7y7TUQ@public.gmane.org>
2010-03-04 16:18     ` Vivek Goyal
2010-03-04 19:41     ` Vivek Goyal
2010-03-04 19:41       ` Vivek Goyal
2010-03-04 19:41       ` Vivek Goyal
     [not found]       ` <20100304194144.GE18786-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2010-03-04 21:51         ` Andrea Righi [this message]
2010-03-04 21:51           ` Andrea Righi
2010-03-04 21:51           ` Andrea Righi
2010-03-05  6:38     ` Balbir Singh
2010-03-04 16:18   ` Vivek Goyal
2010-03-04 16:18     ` Vivek Goyal
2010-03-04 16:28     ` Andrea Righi
2010-03-04 16:28       ` Andrea Righi
     [not found]     ` <20100304161828.GC18786-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2010-03-04 16:28       ` Andrea Righi
2010-03-05  6:38   ` Balbir Singh
2010-03-05  6:38     ` Balbir Singh
2010-03-05 22:55     ` Andrea Righi
2010-03-05 22:55       ` Andrea Righi
     [not found]     ` <20100305063843.GI3073-SINUvgVNF2CyUtPGxGje5AC/G2K4zDHf@public.gmane.org>
2010-03-05 22:55       ` Andrea Righi
2010-03-04 17:11 ` [PATCH -mmotm 0/4] memcg: per cgroup dirty limit (v4) Balbir Singh
2010-03-04 17:11   ` Balbir Singh
2010-03-04 21:37   ` Andrea Righi
2010-03-04 21:37     ` Andrea Righi
     [not found]   ` <20100304171143.GG3073-SINUvgVNF2CyUtPGxGje5AC/G2K4zDHf@public.gmane.org>
2010-03-04 21:37     ` Andrea Righi
  -- strict thread matches above, loose matches on Subject: below --
2010-03-07 20:57 [PATCH -mmotm 0/4] memcg: per cgroup dirty limit (v5) Andrea Righi
     [not found] ` <1267995474-9117-1-git-send-email-arighi-vWjgImWzx8FBDgjK7y7TUQ@public.gmane.org>
2010-03-07 20:57   ` [PATCH -mmotm 4/4] memcg: dirty pages instrumentation Andrea Righi
2010-03-07 20:57 ` Andrea Righi
2010-03-07 20:57   ` Andrea Righi
2010-03-08  2:31   ` KAMEZAWA Hiroyuki
2010-03-08  2:31     ` KAMEZAWA Hiroyuki
     [not found]   ` <1267995474-9117-5-git-send-email-arighi-vWjgImWzx8FBDgjK7y7TUQ@public.gmane.org>
2010-03-08  2:31     ` KAMEZAWA Hiroyuki

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20100304215101.GB4787@linux \
    --to=arighi-vwjgimwzx8fbdgjk7y7tuq@public.gmane.org \
    --cc=akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org \
    --cc=balbir-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org \
    --cc=containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org \
    --cc=linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org \
    --cc=nishimura-YQH0OdQVrdy45+QrQBaojngSJqDPrsil@public.gmane.org \
    --cc=suleiman-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org \
    --cc=trond.myklebust-41N18TsMXrtuMpJDpNschA@public.gmane.org \
    --cc=vgoyal-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.