Re: [PATCH] Add file based RSS accounting for memory resource controller (v2)

[Balbir Singh <balbir@linux.vnet.ibm.com>]

* KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> [2009-04-17 14:17:26]:

> On Fri, 17 Apr 2009 10:26:23 +0530
> Balbir Singh <balbir@linux.vnet.ibm.com> wrote:
> 
> > * KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> [2009-04-17 12:49:51]:
> > 
> > > On Fri, 17 Apr 2009 09:15:39 +0530
> > > Balbir Singh <balbir@linux.vnet.ibm.com> wrote:
> > > 
> > > > * KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> [2009-04-17 11:03:50]:
> > > > 
> > > > > On Fri, 17 Apr 2009 07:10:42 +0530
> > > > > Balbir Singh <balbir@linux.vnet.ibm.com> wrote:
> > > > > 
> > > > > > * KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> [2009-04-17 09:14:59]:
> > > > > > 
> > > > > > > On Thu, 16 Apr 2009 17:33:16 +0530
> > > > > > > Balbir Singh <balbir@linux.vnet.ibm.com> wrote:
> > > > > > > 
> > > > > > > > * KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> [2009-04-16 17:15:35]:
> > > > > > > > 
> > > > > > > > > 
> > > > > > > > > > Sorry, some troubles found. Ignore above Ack. 3points now.
> > > > > > > > > > 
> > > > > > > > > > 1. get_cpu should be after (*)
> > > > > > > > > > ==mem_cgroup_update_mapped_file_stat()
> > > > > > > > > > +	int cpu = get_cpu();
> > > > > > > > > > +
> > > > > > > > > > +	if (!page_is_file_cache(page))
> > > > > > > > > > +		return;
> > > > > > > > > > +
> > > > > > > > > > +	if (unlikely(!mm))
> > > > > > > > > > +		mm = &init_mm;
> > > > > > > > > > +
> > > > > > > > > > +	mem = try_get_mem_cgroup_from_mm(mm);
> > > > > > > > > > +	if (!mem)
> > > > > > > > > > +		return;
> > > > > > > > > > + ----------------------------------------(*)
> > > > > > > > > > +	stat = &mem->stat;
> > > > > > > > > > +	cpustat = &stat->cpustat[cpu];
> > > > > > > > > > +
> > > > > > > > > > +	__mem_cgroup_stat_add_safe(cpustat, MEM_CGROUP_STAT_MAPPED_FILE, val);
> > > > > > > > > > +	put_cpu();
> > > > > > > > > > +}
> > > > > > > > > > ==
> > > > > > > > 
> > > > > > > > Yes or I should have a goto
> > > > > > > > 
> > > > > > > > > > 
> > > > > > > > > > 2. In above, "mem" shouldn't be got from "mm"....please get "mem" from page_cgroup.
> > > > > > > > > > (Because it's file cache, pc->mem_cgroup is not NULL always.)
> > > > > > > > 
> > > > > > > > Hmmm.. not sure I understand this part. Are you suggesting that mm can
> > > > > > > > be NULL?
> > > > > > > No.
> > > > > > > 
> > > > > > > > I added the check for !mm as a safety check. Since this
> > > > > > > > routine is only called from rmap context, mm is not NULL, hence mem
> > > > > > > > should not be NULL. Did you find a race between mm->owner assignment
> > > > > > > > and lookup via mm->owner?
> > > > > > > > 
> > > > > > > No.
> > > > > > > 
> > > > > > > page_cgroup->mem_cgroup != try_get_mem_cgroup_from_mm(mm);  in many many cases.
> > > > > > > 
> > > > > > > For example, libc and /bin/*** is tend to be loaded into default cgroup at boot but
> > > > > > > used by many cgroups. But mapcount of page caches for /bin/*** is 0 if not running.
> > > > > > > 
> > > > > > > Then, File_Mapped can be greater than Cached easily if you use mm->owner.
> > > > > > > 
> > > > > > > I can't estimate RSS in *my* cgroup if File_Mapped includes pages which is under 
> > > > > > > other cgroups. It's meaningless.
> > > > > > > Especially, when Cached==0 but File_Mapped > 0, I think "oh, the kernel leaks somehing..hmm..."
> > > > > > > 
> > > > > > > By useing page_cgroup->mem_cgroup, we can avoid above mess.
> > > > > > 
> > > > > > Yes, I see your point. I wanted mapped_file to show up in the cgroup
> > > > > > that mapped the page. But this works for me as well, but that means
> > > > > > we'll nest the page cgroup lock under the PTE lock.
> > > > > 
> > > > > Don't worry. we do that nest at ANON's uncharge(), already.
> > > > > 
> > > > > About cost:
> > > > > 
> > > > > IIUC, the number of "mapcount 0->1/1->0" of file caches are much smaller than
> > > > > that of o Anon. And there will be not very much cache pingpong.
> > > > > 
> > > > > If you use PCG_MAPPED flag in page_cgroup (as my patch), you can use
> > > > > not-atomic version of set/clear when update is only under lock_page_cgroup().
> > > > > If you find better way, plz use it. But we can't avoid some kind of atomic ops
> > > > > for correct accounting, I think.
> > > > >
> > > > 
> > > > Can you sign off on your patch, so that I can take it with your
> > > > signed-off-by. I will also make some minor changes, get_cpu() is not
> > > > needed, since we are in preempt disable context. 
> > > > 
> > > Hmm, 
> > > Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
> > > 
> > > But some more clean up is necesarry.
> > > 
> > > === This part ==
> > > +	lock_page_cgroup(pc);
> > > +	mem = pc->mem_cgroup;
> > > +	if (mem) {
> > > +		cpu = get_cpu();
> > > +		stat = &mem->stat;
> > > +		cpustat = &stat->cpustat[cpu];
> > > +		if (map)
> > > 
> > > === Should be ==
> > > +	lock_page_cgroup(pc);
> > > 	if (!PageCgroupUsed(pc)) {
> > > 		unlock_page_cgroup(pc);
> > > 		return;
> > > 	}
> > 
> > Do we need this? If the page is mapped, pc should be used right?
> > 
> 
> About file cache, it'd definitely charged at add-to-radix-tree
> regardless of being mapped or not.
>

Yes, what I meant was that before being mapped, the page should be
charged by the memory controller.
 
> *But* we still have following code.
> ==
> 820 static int __mem_cgroup_try_charge(struct mm_struct *mm,
>  821                         gfp_t gfp_mask, struct mem_cgroup **memcg,
>  822    
>  834         /*
>  835          * We always charge the cgroup the mm_struct belongs to.
>  836          * The mm_struct's mem_cgroup changes on task migration if the
>  837          * thread group leader migrates. It's possible that mm is not
>  838          * set, if so charge the init_mm (happens for pagecache usage).
>  839          */
>  840         mem = *memcg;
>  841         if (likely(!mem)) {
>  842                 mem = try_get_mem_cgroup_from_mm(mm);
>  843                 *memcg = mem;
>  844         } else {
>  845                 css_get(&mem->css);
>  846         }
>  847         if (unlikely(!mem))
>  848                 return 0;
> ==
> 
> So, for _now_, we should use this style of checking page_cgroup is used or not.
> Until we fix/confirm try_charge() does.
>

Hmm... I think we need to fix this loop hole, if not mem, we should
look at charging the root cgroup. I suspect !mem cases should be 0,
I'll keep that as a TODO. 

-- 
	Balbir

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>