Re: [ATTEND][LSF/VM TOPIC] deterministic cgroup charging using file path

[Balbir Singh <balbir@linux.vnet.ibm.com>]

* KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> [2010-06-28 11:03:27]:

> On Fri, 25 Jun 2010 13:43:45 -0700
> Greg Thelen <gthelen@google.com> wrote:
> 
> > For the upcoming Linux VM summit, I am interesting in discussing the
> > following proposal.
> > 
> > Problem: When tasks from multiple cgroups share files the charging can be
> > non-deterministic.  This requires that all such cgroups have unnecessarily high
> > limits.  It would be nice if the charging was deterministic, using the file's
> > path to determine which cgroup to charge.  This would benefit charging of
> > commonly used files (eg: libc) as well as large databases shared by only a few
> > tasks.
> > 
> > Example: assume two tasks (T1 and T2), each in a separate cgroup.  Each task
> > wants to access a large (1GB) database file.  To catch memory leaks a tight
> > memory limit on each task's cgroup is set.  However, the large database file
> > presents a problem.  If the file has not been cached, then the first task to
> > access the file is charged, thereby requiring that task's cgroup to have a limit
> > large enough to include the database file.  If the order of access is unknown
> > (due to process restart, etc), then all cgroups accessing the file need to have
> > a limit large enough to include the database.  This is wasteful because the
> > database won't be charged to both T1 and T2.  It would be useful to introduce
> > determinism by declaring that a particular cgroup is charged for a particular
> > set of files.
> > 
> > /dev/cgroup/cg1/cg11  # T1: want memory.limit = 30MB
> > /dev/cgroup/cg1/cg12  # T2: want memory.limit = 100MB
> > /dev/cgroup/cg1       # want memory.limit = 1GB + 30MB + 100MB
> > 
> > I have implemented a prototype that allows a file system hierarchy be charge a
> > particular cgroup using a new bind mount option:
> > + mount -t cgroup none /cgroup -o memory
> > + mount --bind /tmp/db /tmp/db -o cgroup=/dev/cgroup/cg1
> > 
> > Any accesses to files within /tmp/db are charged to /dev/cgroup/cg1.  Access to
> > other files behave normally - they charge the cgroup of the current task.
> > 
> 
> Interesting, but I want to use madvice() etc..for this kind of jobs, rather than
> deep hooks into the kernel.
> 
> madvise(addr, size, MEMORY_RECHAEGE_THIS_PAGES_TO_ME);
> 
> Then, you can write a command as:
> 
>   file_recharge [path name] [cgroup]
>   - this commands move a file cache to specified cgroup.
> 
> A daemon program which uses this command + inotify will give us much
> flexible controls on file cache on memcg. Do you have some requirements
> that this move-charge shouldn't be done in lazy manner ?
> 
> Status:
> We have codes for move-charge, inotify but have no code for new madvise.

I have not see the approach yet, but ideally one would want to avoid
changing the application, otherwise we are going to get very tightly
bound in the API issues.

I want to understand why do we need bind mounts? I think this needs
more discussion.

-- 
	Three Cheers,
	Balbir

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>