From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail6.bemta8.messagelabs.com (mail6.bemta8.messagelabs.com [216.82.243.55]) by kanga.kvack.org (Postfix) with ESMTP id BACCE6B0011 for ; Thu, 19 May 2011 11:52:40 -0400 (EDT) Received: from hpaq5.eem.corp.google.com (hpaq5.eem.corp.google.com [172.25.149.5]) by smtp-out.google.com with ESMTP id p4JFqOV7013269 for ; Thu, 19 May 2011 08:52:24 -0700 Received: from qwe5 (qwe5.prod.google.com [10.241.194.5]) by hpaq5.eem.corp.google.com with ESMTP id p4JFprrp005546 (version=TLSv1/SSLv3 cipher=RC4-SHA bits=128 verify=NOT) for ; Thu, 19 May 2011 08:52:23 -0700 Received: by qwe5 with SMTP id 5so1499510qwe.9 for ; Thu, 19 May 2011 08:52:23 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <20110519152206.1dac20af.nishimura@mxp.nes.nec.co.jp> References: <1305766511-11469-1-git-send-email-yinghan@google.com> <1305766511-11469-2-git-send-email-yinghan@google.com> <20110519152206.1dac20af.nishimura@mxp.nes.nec.co.jp> Date: Thu, 19 May 2011 08:23:47 -0700 Message-ID: Subject: Re: [PATCH V2 2/2] memcg: add memory.numastat api for numa statistics From: Ying Han Content-Type: multipart/alternative; boundary=20cf302efce277c12c04a3a297fe Sender: owner-linux-mm@kvack.org List-ID: To: Daisuke Nishimura Cc: KOSAKI Motohiro , Minchan Kim , Balbir Singh , Tejun Heo , Pavel Emelyanov , KAMEZAWA Hiroyuki , Andrew Morton , Li Zefan , Mel Gorman , Christoph Lameter , Johannes Weiner , Rik van Riel , Hugh Dickins , Michal Hocko , Dave Hansen , Zhu Yanhai , linux-mm@kvack.org --20cf302efce277c12c04a3a297fe Content-Type: text/plain; charset=ISO-8859-1 On Wed, May 18, 2011 at 11:22 PM, Daisuke Nishimura < nishimura@mxp.nes.nec.co.jp> wrote: > On Wed, 18 May 2011 17:55:11 -0700 > Ying Han wrote: > > > The new API exports numa_maps per-memcg basis. This is a piece of useful > > information where it exports per-memcg page distribution across real numa > > nodes. > > > > One of the usecase is evaluating application performance by combining > this > > information w/ the cpu allocation to the application. > > > > The output of the memory.numastat tries to follow w/ simiar format of > numa_maps > > like: > > > > total= N0= N1= ... > > file= N0= N1= ... > > anon= N0= N1= ... > > > > $ cat /dev/cgroup/memory/memory.numa_stat > > total=317674 N0=101850 N1=72552 N2=30120 N3=113142 > > file=288219 N0=98046 N1=59220 N2=23578 N3=107375 > > anon=25699 N0=3804 N1=10124 N2=6540 N3=5231 > > > > Note: I noticed is not equal to the sum of the rest of > counters. > > I might need to change the way get that counter, comments are welcomed. > > > Isn't it just because (mem_cgroup_local_usage()) includes > pages > which are not on any LRU, while other counters doesn't ? > Yes, i noticed that also and I am preparing for the next post :) > > > change v2..v1: > > 1. add also the file and anon pages on per-node distribution. > > > > Signed-off-by: Ying Han > > --- > > mm/memcontrol.c | 109 > +++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > 1 files changed, 109 insertions(+), 0 deletions(-) > > > > diff --git a/mm/memcontrol.c b/mm/memcontrol.c > > index da183dc..cffc3a6 100644 > > --- a/mm/memcontrol.c > > +++ b/mm/memcontrol.c > > @@ -1162,6 +1162,62 @@ unsigned long mem_cgroup_zone_nr_lru_pages(struct > mem_cgroup *memcg, > > return MEM_CGROUP_ZSTAT(mz, lru); > > } > > > > +unsigned long mem_cgroup_node_nr_file_pages(struct mem_cgroup *memcg, > int nid) > > +{ > > + unsigned long ret; > > + > > + ret = mem_cgroup_get_zonestat_node(memcg, nid, LRU_INACTIVE_FILE) + > > + mem_cgroup_get_zonestat_node(memcg, nid, LRU_ACTIVE_FILE); > > + > > + return ret; > > +} > > + > > +unsigned long mem_cgroup_node_nr_anon_pages(struct mem_cgroup *memcg, > int nid) > > +{ > > + unsigned long ret; > > + > > + ret = mem_cgroup_get_zonestat_node(memcg, nid, LRU_INACTIVE_ANON) + > > + mem_cgroup_get_zonestat_node(memcg, nid, LRU_ACTIVE_ANON); > > + > > + return ret; > > +} > > + > > +unsigned long mem_cgroup_node_nr_lru_pages(struct mem_cgroup *memcg, > > + int nid, bool file) > > +{ > > + if (file) > > + return mem_cgroup_node_nr_file_pages(memcg, nid); > > + else > > + return mem_cgroup_node_nr_anon_pages(memcg, nid); > > +} > > + > > +unsigned long mem_cgroup_nr_lru_pages(struct mem_cgroup *memcg, bool > file) > > +{ > > + u64 total = 0; > > + int nid; > > + > > + for_each_node_state(nid, N_HIGH_MEMORY) > > + total += mem_cgroup_node_nr_lru_pages(memcg, nid, file); > > + > > + return total; > > +} > > + > Can these functions defined as "static" ? > > > > +unsigned long mem_cgroup_node_nr_pages(struct mem_cgroup *memcg, int > nid) > > +{ > > + int zid; > > + struct mem_cgroup_per_zone *mz; > > + enum lru_list l; > > + u64 total = 0; > > + > > + for (zid = 0; zid < MAX_NR_ZONES; zid++) { > > + mz = mem_cgroup_zoneinfo(memcg, nid, zid); > > + for_each_lru(l) > > + total += MEM_CGROUP_ZSTAT(mz, l); > > + } > > + > > + return total; > > +} > > + > ditto. > And I think this function can be implemented by using > mem_cgroup_get_zonestat_node(). > > for_each_lru(l) > total += mem_cgroup_get_zonestat_node(memcg, nid, l); > > As KAMEZAWA-san posted a fix already, mem_cgroup_get_zonestat_node() must > be fixed first. > > > Yes, I can include that in the patchset. Thanks --Ying > Thanks, > Daisuke Nishimura. > > > struct zone_reclaim_stat *mem_cgroup_get_reclaim_stat(struct mem_cgroup > *memcg, > > struct zone *zone) > > { > > @@ -4048,6 +4104,41 @@ mem_cgroup_get_total_stat(struct mem_cgroup *mem, > struct mcs_total_stat *s) > > mem_cgroup_get_local_stat(iter, s); > > } > > > > +static int mem_control_numa_stat_show(struct seq_file *m, void *arg) > > +{ > > + int nid; > > + unsigned long total_nr, file_nr, anon_nr; > > + unsigned long node_nr; > > + struct cgroup *cont = m->private; > > + struct mem_cgroup *mem_cont = mem_cgroup_from_cont(cont); > > + > > + total_nr = mem_cgroup_local_usage(mem_cont); > > + seq_printf(m, "total=%lu", total_nr); > > + for_each_node_state(nid, N_HIGH_MEMORY) { > > + node_nr = mem_cgroup_node_nr_pages(mem_cont, nid); > > + seq_printf(m, " N%d=%lu", nid, node_nr); > > + } > > + seq_putc(m, '\n'); > > + > > + file_nr = mem_cgroup_nr_lru_pages(mem_cont, 1); > > + seq_printf(m, "file=%lu", file_nr); > > + for_each_node_state(nid, N_HIGH_MEMORY) { > > + node_nr = mem_cgroup_node_nr_lru_pages(mem_cont, nid, 1); > > + seq_printf(m, " N%d=%lu", nid, node_nr); > > + } > > + seq_putc(m, '\n'); > > + > > + anon_nr = mem_cgroup_nr_lru_pages(mem_cont, 0); > > + seq_printf(m, "anon=%lu", anon_nr); > > + for_each_node_state(nid, N_HIGH_MEMORY) { > > + node_nr = mem_cgroup_node_nr_lru_pages(mem_cont, nid, 0); > > + seq_printf(m, " N%d=%lu", nid, node_nr); > > + } > > + seq_putc(m, '\n'); > > + > > + return 0; > > +} > > + > > static int mem_control_stat_show(struct cgroup *cont, struct cftype > *cft, > > struct cgroup_map_cb *cb) > > { > > @@ -4481,6 +4572,20 @@ static int mem_cgroup_oom_control_write(struct > cgroup *cgrp, > > return 0; > > } > > > > +static const struct file_operations > mem_control_numa_stat_file_operations = { > > + .read = seq_read, > > + .llseek = seq_lseek, > > + .release = single_release, > > +}; > > + > > +static int mem_control_numa_stat_open(struct inode *unused, struct file > *file) > > +{ > > + struct cgroup *cont = file->f_dentry->d_parent->d_fsdata; > > + > > + file->f_op = &mem_control_numa_stat_file_operations; > > + return single_open(file, mem_control_numa_stat_show, cont); > > +} > > + > > static struct cftype mem_cgroup_files[] = { > > { > > .name = "usage_in_bytes", > > @@ -4544,6 +4649,10 @@ static struct cftype mem_cgroup_files[] = { > > .unregister_event = mem_cgroup_oom_unregister_event, > > .private = MEMFILE_PRIVATE(_OOM_TYPE, OOM_CONTROL), > > }, > > + { > > + .name = "numa_stat", > > + .open = mem_control_numa_stat_open, > > + }, > > }; > > > > #ifdef CONFIG_CGROUP_MEM_RES_CTLR_SWAP > > -- > > 1.7.3.1 > > > > -- > To unsubscribe, send a message with 'unsubscribe linux-mm' in > the body to majordomo@kvack.org. For more info on Linux MM, > see: http://www.linux-mm.org/ . > Fight unfair telecom internet charges in Canada: sign > http://stopthemeter.ca/ > Don't email: email@kvack.org > --20cf302efce277c12c04a3a297fe Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

On Wed, May 18, 2011 at 11:22 PM, Daisuk= e Nishimura <nishimura@mxp.nes.nec.co.jp> wrote:
On Wed, 18 May 2011 17:55:11 -0700
Ying Han <yinghan@google.com&g= t; wrote:

> The new API exports numa_maps per-memcg basis. This is a piece of usef= ul
> information where it exports per-memcg page distribution across real n= uma
> nodes.
>
> One of the usecase is evaluating application performance by combining = this
> information w/ the cpu allocation to the application.
>
> The output of the memory.numastat tries to follow w/ simiar format of = numa_maps
> like:
>
> total=3D<total pages> N0=3D<node 0 pages> N1=3D<node 1 = pages> ...
> file=3D<total file pages> N0=3D<node 0 pages> N1=3D<nod= e 1 pages> ...
> anon=3D<total anon pages> N0=3D<node 0 pages> N1=3D<nod= e 1 pages> ...
>
> $ cat /dev/cgroup/memory/memory.numa_stat
> total=3D317674 N0=3D101850 N1=3D72552 N2=3D30120 N3=3D113142
> file=3D288219 N0=3D98046 N1=3D59220 N2=3D23578 N3=3D107375
> anon=3D25699 N0=3D3804 N1=3D10124 N2=3D6540 N3=3D5231
>
> Note: I noticed <total pages> is not equal to the sum of the res= t of counters.
> I might need to change the way get that counter, comments are welcomed= .
>
Isn't it just because <total pages>(mem_cgroup_local_usage(= )) includes pages
which are not on any LRU, while other counters doesn't ?

Yes, i noticed that also and I am preparing for the n= ext post :)=A0

> change v2..v1:
> 1. add also the file and anon pages on per-node distribution.
>
> Signed-off-by: Ying Han <ying= han@google.com>
> ---
> =A0mm/memcontrol.c | =A0109 ++++++++++++++++++++++++++++++++++++++++++= +++++++++++++
> =A01 files changed, 109 insertions(+), 0 deletions(-)
>
> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> index da183dc..cffc3a6 100644
> --- a/mm/memcontrol.c
> +++ b/mm/memcontrol.c
> @@ -1162,6 +1162,62 @@ unsigned long mem_cgroup_zone_nr_lru_pages(stru= ct mem_cgroup *memcg,
> =A0 =A0 =A0 return MEM_CGROUP_ZSTAT(mz, lru);
> =A0}
>
> +unsigned long mem_cgroup_node_nr_file_pages(struct mem_cgroup *memcg,= int nid)
> +{
> + =A0 =A0 unsigned long ret;
> +
> + =A0 =A0 ret =3D mem_cgroup_get_zonestat_node(memcg, nid, LRU_INACTIV= E_FILE) +
> + =A0 =A0 =A0 =A0 =A0 =A0 mem_cgroup_get_zonestat_node(memcg, nid, LRU= _ACTIVE_FILE);
> +
> + =A0 =A0 return ret;
> +}
> +
> +unsigned long mem_cgroup_node_nr_anon_pages(struct mem_cgroup *memcg,= int nid)
> +{
> + =A0 =A0 unsigned long ret;
> +
> + =A0 =A0 ret =3D mem_cgroup_get_zonestat_node(memcg, nid, LRU_INACTIV= E_ANON) +
> + =A0 =A0 =A0 =A0 =A0 =A0 mem_cgroup_get_zonestat_node(memcg, nid, LRU= _ACTIVE_ANON);
> +
> + =A0 =A0 return ret;
> +}
> +
> +unsigned long mem_cgroup_node_nr_lru_pages(struct mem_cgroup *memcg,<= br> > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 int nid, bool file)
> +{
> + =A0 =A0 if (file)
> + =A0 =A0 =A0 =A0 =A0 =A0 return mem_cgroup_node_nr_file_pages(memcg, = nid);
> + =A0 =A0 else
> + =A0 =A0 =A0 =A0 =A0 =A0 return mem_cgroup_node_nr_anon_pages(memcg, = nid);
> +}
> +
> +unsigned long mem_cgroup_nr_lru_pages(struct mem_cgroup *memcg, bool = file)
> +{
> + =A0 =A0 u64 total =3D 0;
> + =A0 =A0 int nid;
> +
> + =A0 =A0 for_each_node_state(nid, N_HIGH_MEMORY)
> + =A0 =A0 =A0 =A0 =A0 =A0 total +=3D mem_cgroup_node_nr_lru_pages(memc= g, nid, file);
> +
> + =A0 =A0 return total;
> +}
> +
Can these functions defined as "static" ?

=A0

> +unsigned long mem_cgroup_node_nr_pages(struct mem_cgroup *memcg, int = nid)
> +{
> + =A0 =A0 int zid;
> + =A0 =A0 struct mem_cgroup_per_zone *mz;
> + =A0 =A0 enum lru_list l;
> + =A0 =A0 u64 total =3D 0;
> +
> + =A0 =A0 for (zid =3D 0; zid < MAX_NR_ZONES; zid++) {
> + =A0 =A0 =A0 =A0 =A0 =A0 mz =3D mem_cgroup_zoneinfo(memcg, nid, zid);=
> + =A0 =A0 =A0 =A0 =A0 =A0 for_each_lru(l)
> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 total +=3D MEM_CGROUP_ZSTAT(= mz, l);
> + =A0 =A0 }
> +
> + =A0 =A0 return total;
> +}
> +
ditto.
And I think this function can be implemented by using mem_cgroup_get_zonest= at_node().

=A0 =A0 =A0 =A0for_each_lru(l)
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0total +=3D mem_cgroup_get_zonestat_node(mem= cg, nid, l);

As KAMEZAWA-san posted a fix already, mem_cgroup_get_zonestat_node() must b= e fixed first.


Yes, I can include that in the patchset.
Thanks

--Ying
=A0
Thanks,
Daisuke Nishimura.

> =A0struct zone_reclaim_stat *mem_cgroup_get_reclaim_stat(struct mem_cg= roup *memcg,
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 struct zone *zone)
> =A0{
> @@ -4048,6 +4104,41 @@ mem_cgroup_get_total_stat(struct mem_cgroup *me= m, struct mcs_total_stat *s)
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 mem_cgroup_get_local_stat(iter, s);
> =A0}
>
> +static int mem_control_numa_stat_show(struct seq_file *m, void *arg)<= br> > +{
> + =A0 =A0 int nid;
> + =A0 =A0 unsigned long total_nr, file_nr, anon_nr;
> + =A0 =A0 unsigned long node_nr;
> + =A0 =A0 struct cgroup *cont =3D m->private;
> + =A0 =A0 struct mem_cgroup *mem_cont =3D mem_cgroup_from_cont(cont);<= br> > +
> + =A0 =A0 total_nr =3D mem_cgroup_local_usage(mem_cont);
> + =A0 =A0 seq_printf(m, "total=3D%lu", total_nr);
> + =A0 =A0 for_each_node_state(nid, N_HIGH_MEMORY) {
> + =A0 =A0 =A0 =A0 =A0 =A0 node_nr =3D mem_cgroup_node_nr_pages(mem_con= t, nid);
> + =A0 =A0 =A0 =A0 =A0 =A0 seq_printf(m, " N%d=3D%lu", nid, n= ode_nr);
> + =A0 =A0 }
> + =A0 =A0 seq_putc(m, '\n');
> +
> + =A0 =A0 file_nr =3D mem_cgroup_nr_lru_pages(mem_cont, 1);
> + =A0 =A0 seq_printf(m, "file=3D%lu", file_nr);
> + =A0 =A0 for_each_node_state(nid, N_HIGH_MEMORY) {
> + =A0 =A0 =A0 =A0 =A0 =A0 node_nr =3D mem_cgroup_node_nr_lru_pages(mem= _cont, nid, 1);
> + =A0 =A0 =A0 =A0 =A0 =A0 seq_printf(m, " N%d=3D%lu", nid, n= ode_nr);
> + =A0 =A0 }
> + =A0 =A0 seq_putc(m, '\n');
> +
> + =A0 =A0 anon_nr =3D mem_cgroup_nr_lru_pages(mem_cont, 0);
> + =A0 =A0 seq_printf(m, "anon=3D%lu", anon_nr);
> + =A0 =A0 for_each_node_state(nid, N_HIGH_MEMORY) {
> + =A0 =A0 =A0 =A0 =A0 =A0 node_nr =3D mem_cgroup_node_nr_lru_pages(mem= _cont, nid, 0);
> + =A0 =A0 =A0 =A0 =A0 =A0 seq_printf(m, " N%d=3D%lu", nid, n= ode_nr);
> + =A0 =A0 }
> + =A0 =A0 seq_putc(m, '\n');
> +
> + =A0 =A0 return 0;
> +}
> +
> =A0static int mem_control_stat_show(struct cgroup *cont, struct cftype= *cft,
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0struct = cgroup_map_cb *cb)
> =A0{
> @@ -4481,6 +4572,20 @@ static int mem_cgroup_oom_control_write(struct = cgroup *cgrp,
> =A0 =A0 =A0 return 0;
> =A0}
>
> +static const struct file_operations mem_control_numa_stat_file_operat= ions =3D {
> + =A0 =A0 .read =3D seq_read,
> + =A0 =A0 .llseek =3D seq_lseek,
> + =A0 =A0 .release =3D single_release,
> +};
> +
> +static int mem_control_numa_stat_open(struct inode *unused, struct fi= le *file)
> +{
> + =A0 =A0 struct cgroup *cont =3D file->f_dentry->d_parent->d= _fsdata;
> +
> + =A0 =A0 file->f_op =3D &mem_control_numa_stat_file_operations= ;
> + =A0 =A0 return single_open(file, mem_control_numa_stat_show, cont);<= br> > +}
> +
> =A0static struct cftype mem_cgroup_files[] =3D {
> =A0 =A0 =A0 {
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 .name =3D "usage_in_bytes",
> @@ -4544,6 +4649,10 @@ static struct cftype mem_cgroup_files[] =3D { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 .unregister_event =3D mem_cgroup_oom_unreg= ister_event,
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 .private =3D MEMFILE_PRIVATE(_OOM_TYPE, OO= M_CONTROL),
> =A0 =A0 =A0 },
> + =A0 =A0 {
> + =A0 =A0 =A0 =A0 =A0 =A0 .name =3D "numa_stat",
> + =A0 =A0 =A0 =A0 =A0 =A0 .open =3D mem_control_numa_stat_open,
> + =A0 =A0 },
> =A0};
>
> =A0#ifdef CONFIG_CGROUP_MEM_RES_CTLR_SWAP
> --
> 1.7.3.1
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.= =A0For more info on Linux MM,
see: http://www.linu= x-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=3Dmailto:"dont@kvack.org"> email@kva= ck.org </a>

--20cf302efce277c12c04a3a297fe-- -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/ Don't email: email@kvack.org