* [RFC PATCH v2 05/17] cgroup: implement cgroup v2 thread support
From: Waiman Long @ 2017-05-15 13:34 UTC (permalink / raw)
To: Tejun Heo, Li Zefan, Johannes Weiner, Peter Zijlstra, Ingo Molnar
Cc: cgroups, linux-kernel, linux-doc, linux-mm, kernel-team, pjt,
luto, efault, longman
In-Reply-To: <1494855256-12558-1-git-send-email-longman@redhat.com>
From: Tejun Heo <tj@kernel.org>
This patch implements cgroup v2 thread support. The goal of the
thread mode is supporting hierarchical accounting and control at
thread granularity while staying inside the resource domain model
which allows coordination across different resource controllers and
handling of anonymous resource consumptions.
Once thread mode is enabled on a cgroup, the threads of the processes
which are in its subtree can be placed inside the subtree without
being restricted by process granularity or no-internal-process
constraint. Note that the threads aren't allowed to escape to a
different threaded subtree. To be used inside a threaded subtree, a
controller should explicitly support threaded mode and be able to
handle internal competition in the way which is appropriate for the
resource.
The root of a threaded subtree, where thread mode is enabled in the
first place, is called the thread root and serves as the resource
domain for the whole subtree. This is the last cgroup where
non-threaded controllers are operational and where all the
domain-level resource consumptions in the subtree are accounted. This
allows threaded controllers to operate at thread granularity when
requested while staying inside the scope of system-level resource
distribution.
Internally, in a threaded subtree, each css_set has its ->proc_cset
pointing to a matching css_set which belongs to the thread root. This
ensures that thread root level cgroup_subsys_state for all threaded
controllers are readily accessible for domain-level operations.
This patch enables threaded mode for the pids and perf_events
controllers. Neither has to worry about domain-level resource
consumptions and it's enough to simply set the flag.
For more details on the interface and behavior of the thread mode,
please refer to the section 2-2-2 in Documentation/cgroup-v2.txt added
by this patch. Note that the documentation update is not complete as
the rest of the documentation needs to be updated accordingly.
Rolling those updates into this patch can be confusing so that will be
separate patches.
Signed-off-by: Tejun Heo <tj@kernel.org>
---
Documentation/cgroup-v2.txt | 75 +++++++++++++-
include/linux/cgroup-defs.h | 16 +++
kernel/cgroup/cgroup.c | 240 +++++++++++++++++++++++++++++++++++++++++++-
kernel/cgroup/pids.c | 1 +
kernel/events/core.c | 1 +
5 files changed, 326 insertions(+), 7 deletions(-)
diff --git a/Documentation/cgroup-v2.txt b/Documentation/cgroup-v2.txt
index dc5e2dc..1c6f5a9 100644
--- a/Documentation/cgroup-v2.txt
+++ b/Documentation/cgroup-v2.txt
@@ -16,7 +16,9 @@ CONTENTS
1-2. What is cgroup?
2. Basic Operations
2-1. Mounting
- 2-2. Organizing Processes
+ 2-2. Organizing Processes and Threads
+ 2-2-1. Processes
+ 2-2-2. Threads
2-3. [Un]populated Notification
2-4. Controlling Controllers
2-4-1. Enabling and Disabling
@@ -150,7 +152,9 @@ and experimenting easier, the kernel parameter cgroup_no_v1= allows
disabling controllers in v1 and make them always available in v2.
-2-2. Organizing Processes
+2-2. Organizing Processes and Threads
+
+2-2-1. Processes
Initially, only the root cgroup exists to which all processes belong.
A child cgroup can be created by creating a sub-directory.
@@ -201,6 +205,73 @@ is removed subsequently, " (deleted)" is appended to the path.
0::/test-cgroup/test-cgroup-nested (deleted)
+2-2-2. Threads
+
+cgroup v2 supports thread granularity for a subset of controllers to
+support use cases requiring hierarchical resource distribution across
+the threads of a group of processes. By default, all threads of a
+process belong to the same cgroup, which also serves as the resource
+domain to host resource consumptions which are not specific to a
+process or thread. The thread mode allows threads to be spread across
+a subtree while still maintaining the common resource domain for them.
+
+Enabling thread mode on a subtree makes it threaded. The root of a
+threaded subtree is called thread root and serves as the resource
+domain for the entire subtree. In a threaded subtree, threads of a
+process can be put in different cgroups and are not subject to the no
+internal process constraint - threaded controllers can be enabled on
+non-leaf cgroups whether they have threads in them or not.
+
+To enable the thread mode, the following conditions must be met.
+
+- The thread root doesn't have any child cgroups.
+
+- The thread root doesn't have any controllers enabled.
+
+Thread mode can be enabled by writing "enable" to "cgroup.threads"
+file.
+
+ # echo enable > cgroup.threads
+
+Inside a threaded subtree, "cgroup.threads" can be read and contains
+the list of the thread IDs of all threads in the cgroup. Except that
+the operations are per-thread instead of per-process, "cgroup.threads"
+has the same format and behaves the same way as "cgroup.procs".
+
+The thread root serves as the resource domain for the whole subtree,
+and, while the threads can be scattered across the subtree, all the
+processes are considered to be in the thread root. "cgroup.procs" in
+a thread root contains the PIDs of all processes in the subtree and is
+not readable in the subtree proper. However, "cgroup.procs" can be
+written to from anywhere in the subtree to migrate all threads of the
+matching process to the cgroup.
+
+Only threaded controllers can be enabled in a threaded subtree. When
+a threaded controller is enabled inside a threaded subtree, it only
+accounts for and controls resource consumptions associated with the
+threads in the cgroup and its descendants. All consumptions which
+aren't tied to a specific thread belong to the thread root.
+
+Because a threaded subtree is exempt from no internal process
+constraint, a threaded controller must be able to handle competition
+between threads in a non-leaf cgroup and its child cgroups. Each
+threaded controller defines how such competitions are handled.
+
+To disable the thread mode, the following conditions must be met.
+
+- The cgroup is a thread root. Thread mode can't be disabled
+ partially in the subtree.
+
+- The thread root doesn't have any child cgroups.
+
+- The thread root doesn't have any controllers enabled.
+
+Thread mode can be disabled by writing "disable" to "cgroup.threads"
+file.
+
+ # echo disable > cgroup.threads
+
+
2-3. [Un]populated Notification
Each non-root cgroup has a "cgroup.events" file which contains
diff --git a/include/linux/cgroup-defs.h b/include/linux/cgroup-defs.h
index 3f3cfdd..e8d0cfc 100644
--- a/include/linux/cgroup-defs.h
+++ b/include/linux/cgroup-defs.h
@@ -230,6 +230,10 @@ struct css_set {
struct cgroup *mg_dst_cgrp;
struct css_set *mg_dst_cset;
+ /* used while updating ->proc_cset to enable/disable threaded mode */
+ struct list_head pcset_preload_node;
+ struct css_set *pcset_preload;
+
/* dead and being drained, ignore for migration */
bool dead;
@@ -501,6 +505,18 @@ struct cgroup_subsys {
bool implicit_on_dfl:1;
/*
+ * If %true, the controller, supports threaded mode on the default
+ * hierarchy. In a threaded subtree, both process granularity and
+ * no-internal-process constraint are ignored and a threaded
+ * controllers should be able to handle that.
+ *
+ * Note that as an implicit controller is automatically enabled on
+ * all cgroups on the default hierarchy, it should also be
+ * threaded. implicit && !threaded is not supported.
+ */
+ bool threaded:1;
+
+ /*
* If %false, this subsystem is properly hierarchical -
* configuration, resource accounting and restriction on a parent
* cgroup cover those of its children. If %true, hierarchy support
diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c
index 7efb5da..d7bab5e 100644
--- a/kernel/cgroup/cgroup.c
+++ b/kernel/cgroup/cgroup.c
@@ -162,6 +162,9 @@ struct cgroup_subsys *cgroup_subsys[] = {
/* some controllers are implicitly enabled on the default hierarchy */
static u16 cgrp_dfl_implicit_ss_mask;
+/* some controllers can be threaded on the default hierarchy */
+static u16 cgrp_dfl_threaded_ss_mask;
+
/* The list of hierarchy roots */
LIST_HEAD(cgroup_roots);
static int cgroup_root_count;
@@ -2916,11 +2919,18 @@ static ssize_t cgroup_subtree_control_write(struct kernfs_open_file *of,
goto out_unlock;
}
+ /* can't enable !threaded controllers on a threaded cgroup */
+ if (cgrp->proc_cgrp && (enable & ~cgrp_dfl_threaded_ss_mask)) {
+ ret = -EBUSY;
+ goto out_unlock;
+ }
+
/*
- * Except for the root, subtree_control must be zero for a cgroup
- * with tasks so that child cgroups don't compete against tasks.
+ * Except for root and threaded cgroups, subtree_control must be
+ * zero for a cgroup with tasks so that child cgroups don't compete
+ * against tasks.
*/
- if (enable && cgroup_parent(cgrp)) {
+ if (enable && cgroup_parent(cgrp) && !cgrp->proc_cgrp) {
struct cgrp_cset_link *link;
/*
@@ -2961,6 +2971,124 @@ static ssize_t cgroup_subtree_control_write(struct kernfs_open_file *of,
return ret ?: nbytes;
}
+static int cgroup_enable_threaded(struct cgroup *cgrp)
+{
+ LIST_HEAD(csets);
+ struct cgrp_cset_link *link;
+ struct css_set *cset, *cset_next;
+ int ret;
+
+ lockdep_assert_held(&cgroup_mutex);
+
+ /* noop if already threaded */
+ if (cgrp->proc_cgrp)
+ return 0;
+
+ /* allow only if there are neither children or enabled controllers */
+ if (css_has_online_children(&cgrp->self) || cgrp->subtree_control)
+ return -EBUSY;
+
+ /* find all csets which need ->proc_cset updated */
+ spin_lock_irq(&css_set_lock);
+ list_for_each_entry(link, &cgrp->cset_links, cset_link) {
+ cset = link->cset;
+ if (css_set_populated(cset)) {
+ WARN_ON_ONCE(css_set_threaded(cset));
+ WARN_ON_ONCE(cset->pcset_preload);
+
+ list_add_tail(&cset->pcset_preload_node, &csets);
+ get_css_set(cset);
+ }
+ }
+ spin_unlock_irq(&css_set_lock);
+
+ /* find the proc_csets to associate */
+ list_for_each_entry(cset, &csets, pcset_preload_node) {
+ struct css_set *pcset = find_css_set(cset, cgrp, true);
+
+ WARN_ON_ONCE(cset == pcset);
+ if (!pcset) {
+ ret = -ENOMEM;
+ goto err_put_csets;
+ }
+ cset->pcset_preload = pcset;
+ }
+
+ /* install ->proc_cset */
+ spin_lock_irq(&css_set_lock);
+ list_for_each_entry_safe(cset, cset_next, &csets, pcset_preload_node) {
+ rcu_assign_pointer(cset->proc_cset, cset->pcset_preload);
+ list_add_tail(&cset->threaded_csets_node,
+ &cset->pcset_preload->threaded_csets);
+
+ cset->pcset_preload = NULL;
+ list_del(&cset->pcset_preload_node);
+ put_css_set_locked(cset);
+ }
+ spin_unlock_irq(&css_set_lock);
+
+ /* mark it threaded */
+ cgrp->proc_cgrp = cgrp;
+
+ return 0;
+
+err_put_csets:
+ spin_lock_irq(&css_set_lock);
+ list_for_each_entry_safe(cset, cset_next, &csets, pcset_preload_node) {
+ if (cset->pcset_preload) {
+ put_css_set_locked(cset->pcset_preload);
+ cset->pcset_preload = NULL;
+ }
+ list_del(&cset->pcset_preload_node);
+ put_css_set_locked(cset);
+ }
+ spin_unlock_irq(&css_set_lock);
+ return ret;
+}
+
+static int cgroup_disable_threaded(struct cgroup *cgrp)
+{
+ struct cgrp_cset_link *link;
+
+ lockdep_assert_held(&cgroup_mutex);
+
+ /* noop if already !threaded */
+ if (!cgrp->proc_cgrp)
+ return 0;
+
+ /* partial disable isn't supported */
+ if (cgrp->proc_cgrp != cgrp)
+ return -EBUSY;
+
+ /* allow only if there are neither children or enabled controllers */
+ if (css_has_online_children(&cgrp->self) || cgrp->subtree_control)
+ return -EBUSY;
+
+ /* walk all csets and reset ->proc_cset */
+ spin_lock_irq(&css_set_lock);
+ list_for_each_entry(link, &cgrp->cset_links, cset_link) {
+ struct css_set *cset = link->cset;
+
+ if (css_set_threaded(cset)) {
+ struct css_set *pcset = proc_css_set(cset);
+
+ WARN_ON_ONCE(pcset->dfl_cgrp != cgrp);
+ rcu_assign_pointer(cset->proc_cset, cset);
+ list_del(&cset->threaded_csets_node);
+
+ /*
+ * @pcset is never @cset and safe to put during
+ * iteration.
+ */
+ put_css_set_locked(pcset);
+ }
+ }
+ cgrp->proc_cgrp = NULL;
+ spin_unlock_irq(&css_set_lock);
+
+ return 0;
+}
+
static int cgroup_events_show(struct seq_file *seq, void *v)
{
seq_printf(seq, "populated %d\n",
@@ -3845,12 +3973,12 @@ static void *cgroup_procs_next(struct seq_file *s, void *v, loff_t *pos)
return css_task_iter_next(it);
}
-static void *cgroup_procs_start(struct seq_file *s, loff_t *pos)
+static void *__cgroup_procs_start(struct seq_file *s, loff_t *pos,
+ unsigned int iter_flags)
{
struct kernfs_open_file *of = s->private;
struct cgroup *cgrp = seq_css(s)->cgroup;
struct css_task_iter *it = of->priv;
- unsigned iter_flags = CSS_TASK_ITER_PROCS | CSS_TASK_ITER_THREADED;
/*
* When a seq_file is seeked, it's always traversed sequentially
@@ -3873,6 +4001,23 @@ static void *cgroup_procs_start(struct seq_file *s, loff_t *pos)
return cgroup_procs_next(s, NULL, NULL);
}
+static void *cgroup_procs_start(struct seq_file *s, loff_t *pos)
+{
+ struct cgroup *cgrp = seq_css(s)->cgroup;
+
+ /*
+ * All processes of a threaded subtree are in the top threaded
+ * cgroup. Only threads can be distributed across the subtree.
+ * Reject reads on cgroup.procs in the subtree proper. They're
+ * always empty anyway.
+ */
+ if (cgrp->proc_cgrp && cgrp->proc_cgrp != cgrp)
+ return ERR_PTR(-EINVAL);
+
+ return __cgroup_procs_start(s, pos, CSS_TASK_ITER_PROCS |
+ CSS_TASK_ITER_THREADED);
+}
+
static int cgroup_procs_show(struct seq_file *s, void *v)
{
seq_printf(s, "%d\n", task_pid_vnr(v));
@@ -3927,6 +4072,76 @@ static ssize_t cgroup_procs_write(struct kernfs_open_file *of,
return ret ?: nbytes;
}
+static void *cgroup_threads_start(struct seq_file *s, loff_t *pos)
+{
+ struct cgroup *cgrp = seq_css(s)->cgroup;
+
+ if (!cgrp->proc_cgrp)
+ return ERR_PTR(-EINVAL);
+
+ return __cgroup_procs_start(s, pos, 0);
+}
+
+static ssize_t cgroup_threads_write(struct kernfs_open_file *of,
+ char *buf, size_t nbytes, loff_t off)
+{
+ struct super_block *sb = of->file->f_path.dentry->d_sb;
+ struct cgroup *cgrp, *common_ancestor;
+ struct task_struct *task;
+ ssize_t ret;
+
+ buf = strstrip(buf);
+
+ cgrp = cgroup_kn_lock_live(of->kn, false);
+ if (!cgrp)
+ return -ENODEV;
+
+ /* cgroup.procs determines delegation, require permission on it too */
+ ret = cgroup_procs_write_permission(cgrp, sb);
+ if (ret)
+ goto out_unlock;
+
+ /* enable or disable? */
+ if (!strcmp(buf, "enable")) {
+ ret = cgroup_enable_threaded(cgrp);
+ goto out_unlock;
+ } else if (!strcmp(buf, "disable")) {
+ ret = cgroup_disable_threaded(cgrp);
+ goto out_unlock;
+ }
+
+ /* thread migration */
+ ret = -EINVAL;
+ if (!cgrp->proc_cgrp)
+ goto out_unlock;
+
+ task = cgroup_procs_write_start(buf, false);
+ ret = PTR_ERR_OR_ZERO(task);
+ if (ret)
+ goto out_unlock;
+
+ common_ancestor = cgroup_migrate_common_ancestor(task, cgrp);
+
+ /* can't migrate across disjoint threaded subtrees */
+ ret = -EACCES;
+ if (common_ancestor->proc_cgrp != cgrp->proc_cgrp)
+ goto out_finish;
+
+ /* and follow the cgroup.procs delegation rule */
+ ret = cgroup_procs_write_permission(common_ancestor, sb);
+ if (ret)
+ goto out_finish;
+
+ ret = cgroup_attach_task(cgrp, task, false);
+
+out_finish:
+ cgroup_procs_write_finish();
+out_unlock:
+ cgroup_kn_unlock(of->kn);
+
+ return ret ?: nbytes;
+}
+
/* cgroup core interface files for the default hierarchy */
static struct cftype cgroup_base_files[] = {
{
@@ -3939,6 +4154,14 @@ static ssize_t cgroup_procs_write(struct kernfs_open_file *of,
.write = cgroup_procs_write,
},
{
+ .name = "cgroup.threads",
+ .release = cgroup_procs_release,
+ .seq_start = cgroup_threads_start,
+ .seq_next = cgroup_procs_next,
+ .seq_show = cgroup_procs_show,
+ .write = cgroup_threads_write,
+ },
+ {
.name = "cgroup.controllers",
.seq_show = cgroup_controllers_show,
},
@@ -4252,6 +4475,7 @@ static struct cgroup *cgroup_create(struct cgroup *parent)
cgrp->self.parent = &parent->self;
cgrp->root = root;
cgrp->level = level;
+ cgrp->proc_cgrp = parent->proc_cgrp;
for (tcgrp = cgrp; tcgrp; tcgrp = cgroup_parent(tcgrp))
cgrp->ancestor_ids[tcgrp->level] = tcgrp->id;
@@ -4694,11 +4918,17 @@ int __init cgroup_init(void)
cgrp_dfl_root.subsys_mask |= 1 << ss->id;
+ /* implicit controllers must be threaded too */
+ WARN_ON(ss->implicit_on_dfl && !ss->threaded);
+
if (ss->implicit_on_dfl)
cgrp_dfl_implicit_ss_mask |= 1 << ss->id;
else if (!ss->dfl_cftypes)
cgrp_dfl_inhibit_ss_mask |= 1 << ss->id;
+ if (ss->threaded)
+ cgrp_dfl_threaded_ss_mask |= 1 << ss->id;
+
if (ss->dfl_cftypes == ss->legacy_cftypes) {
WARN_ON(cgroup_add_cftypes(ss, ss->dfl_cftypes));
} else {
diff --git a/kernel/cgroup/pids.c b/kernel/cgroup/pids.c
index 2237201..9829c67 100644
--- a/kernel/cgroup/pids.c
+++ b/kernel/cgroup/pids.c
@@ -345,4 +345,5 @@ struct cgroup_subsys pids_cgrp_subsys = {
.free = pids_free,
.legacy_cftypes = pids_files,
.dfl_cftypes = pids_files,
+ .threaded = true,
};
diff --git a/kernel/events/core.c b/kernel/events/core.c
index 13f5b94..6ba1d06 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -11150,5 +11150,6 @@ struct cgroup_subsys perf_event_cgrp_subsys = {
* controller is not mounted on a legacy hierarchy.
*/
.implicit_on_dfl = true,
+ .threaded = true,
};
#endif /* CONFIG_CGROUP_PERF */
--
1.8.3.1
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [RFC PATCH v2 06/17] cgroup: Fix reference counting bug in cgroup_procs_write()
From: Waiman Long @ 2017-05-15 13:34 UTC (permalink / raw)
To: Tejun Heo, Li Zefan, Johannes Weiner, Peter Zijlstra, Ingo Molnar
Cc: cgroups, linux-kernel, linux-doc, linux-mm, kernel-team, pjt,
luto, efault, longman
In-Reply-To: <1494855256-12558-1-git-send-email-longman@redhat.com>
The cgroup_procs_write_start() took a reference to the task structure
which was not properly released within cgroup_procs_write() and so
on. So a put_task_struct() call is added to cgroup_procs_write_finish()
to match the get_task_struct() in cgroup_procs_write_start() to fix
this reference counting error.
Signed-off-by: Waiman Long <longman@redhat.com>
---
kernel/cgroup/cgroup-internal.h | 2 +-
kernel/cgroup/cgroup-v1.c | 2 +-
kernel/cgroup/cgroup.c | 10 ++++++----
3 files changed, 8 insertions(+), 6 deletions(-)
diff --git a/kernel/cgroup/cgroup-internal.h b/kernel/cgroup/cgroup-internal.h
index f0a0dba..2c8e3a9 100644
--- a/kernel/cgroup/cgroup-internal.h
+++ b/kernel/cgroup/cgroup-internal.h
@@ -182,7 +182,7 @@ int cgroup_attach_task(struct cgroup *dst_cgrp, struct task_struct *leader,
bool threadgroup);
struct task_struct *cgroup_procs_write_start(char *buf, bool threadgroup)
__acquires(&cgroup_threadgroup_rwsem);
-void cgroup_procs_write_finish(void)
+void cgroup_procs_write_finish(struct task_struct *task)
__releases(&cgroup_threadgroup_rwsem);
void cgroup_lock_and_drain_offline(struct cgroup *cgrp);
diff --git a/kernel/cgroup/cgroup-v1.c b/kernel/cgroup/cgroup-v1.c
index c212856..1e101b9 100644
--- a/kernel/cgroup/cgroup-v1.c
+++ b/kernel/cgroup/cgroup-v1.c
@@ -549,7 +549,7 @@ static ssize_t __cgroup1_procs_write(struct kernfs_open_file *of,
ret = cgroup_attach_task(cgrp, task, threadgroup);
out_finish:
- cgroup_procs_write_finish();
+ cgroup_procs_write_finish(task);
out_unlock:
cgroup_kn_unlock(of->kn);
diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c
index d7bab5e..f14deca 100644
--- a/kernel/cgroup/cgroup.c
+++ b/kernel/cgroup/cgroup.c
@@ -2492,12 +2492,15 @@ struct task_struct *cgroup_procs_write_start(char *buf, bool threadgroup)
return tsk;
}
-void cgroup_procs_write_finish(void)
+void cgroup_procs_write_finish(struct task_struct *task)
__releases(&cgroup_threadgroup_rwsem)
{
struct cgroup_subsys *ss;
int ssid;
+ /* release reference from cgroup_procs_write_start() */
+ put_task_struct(task);
+
percpu_up_write(&cgroup_threadgroup_rwsem);
for_each_subsys(ss, ssid)
if (ss->post_attach)
@@ -3300,7 +3303,6 @@ static int cgroup_addrm_files(struct cgroup_subsys_state *css,
static int cgroup_apply_cftypes(struct cftype *cfts, bool is_add)
{
- LIST_HEAD(pending);
struct cgroup_subsys *ss = cfts[0].ss;
struct cgroup *root = &ss->root->cgrp;
struct cgroup_subsys_state *css;
@@ -4065,7 +4067,7 @@ static ssize_t cgroup_procs_write(struct kernfs_open_file *of,
ret = cgroup_attach_task(cgrp, task, true);
out_finish:
- cgroup_procs_write_finish();
+ cgroup_procs_write_finish(task);
out_unlock:
cgroup_kn_unlock(of->kn);
@@ -4135,7 +4137,7 @@ static ssize_t cgroup_threads_write(struct kernfs_open_file *of,
ret = cgroup_attach_task(cgrp, task, false);
out_finish:
- cgroup_procs_write_finish();
+ cgroup_procs_write_finish(task);
out_unlock:
cgroup_kn_unlock(of->kn);
--
1.8.3.1
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [RFC PATCH v2 07/17] cgroup: Prevent kill_css() from being called more than once
From: Waiman Long @ 2017-05-15 13:34 UTC (permalink / raw)
To: Tejun Heo, Li Zefan, Johannes Weiner, Peter Zijlstra, Ingo Molnar
Cc: cgroups, linux-kernel, linux-doc, linux-mm, kernel-team, pjt,
luto, efault, longman
In-Reply-To: <1494855256-12558-1-git-send-email-longman@redhat.com>
The kill_css() function may be called more than once under the condition
that the css was killed but not physically removed yet followed by the
removal of the cgroup that is hosting the css. This patch prevents any
harmm from being done when that happens.
Signed-off-by: Waiman Long <longman@redhat.com>
---
include/linux/cgroup-defs.h | 1 +
kernel/cgroup/cgroup.c | 5 +++++
2 files changed, 6 insertions(+)
diff --git a/include/linux/cgroup-defs.h b/include/linux/cgroup-defs.h
index e8d0cfc..b123afc 100644
--- a/include/linux/cgroup-defs.h
+++ b/include/linux/cgroup-defs.h
@@ -48,6 +48,7 @@ enum {
CSS_ONLINE = (1 << 1), /* between ->css_online() and ->css_offline() */
CSS_RELEASED = (1 << 2), /* refcnt reached zero, released */
CSS_VISIBLE = (1 << 3), /* css is visible to userland */
+ CSS_DYING = (1 << 4), /* css is dying */
};
/* bits in struct cgroup flags field */
diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c
index f14deca..7b085d5 100644
--- a/kernel/cgroup/cgroup.c
+++ b/kernel/cgroup/cgroup.c
@@ -4630,6 +4630,11 @@ static void kill_css(struct cgroup_subsys_state *css)
{
lockdep_assert_held(&cgroup_mutex);
+ if (css->flags & CSS_DYING)
+ return;
+
+ css->flags |= CSS_DYING;
+
/*
* This must happen before css is disassociated with its cgroup.
* See seq_css() for details.
--
1.8.3.1
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [RFC PATCH v2 08/17] cgroup: Move debug cgroup to its own file
From: Waiman Long @ 2017-05-15 13:34 UTC (permalink / raw)
To: Tejun Heo, Li Zefan, Johannes Weiner, Peter Zijlstra, Ingo Molnar
Cc: cgroups, linux-kernel, linux-doc, linux-mm, kernel-team, pjt,
luto, efault, longman
In-Reply-To: <1494855256-12558-1-git-send-email-longman@redhat.com>
The debug cgroup currently resides within cgroup-v1.c and is enabled
only for v1 cgroup. To enable the debug cgroup also for v2, it
makes sense to put the code into its own file as it will no longer
be v1 specific. The only change in this patch is the expansion of
cgroup_task_count() within the debug_taskcount_read() function.
Signed-off-by: Waiman Long <longman@redhat.com>
---
kernel/cgroup/Makefile | 1 +
kernel/cgroup/cgroup-v1.c | 147 -----------------------------------------
kernel/cgroup/debug.c | 165 ++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 166 insertions(+), 147 deletions(-)
create mode 100644 kernel/cgroup/debug.c
diff --git a/kernel/cgroup/Makefile b/kernel/cgroup/Makefile
index 387348a..ce693cc 100644
--- a/kernel/cgroup/Makefile
+++ b/kernel/cgroup/Makefile
@@ -4,3 +4,4 @@ obj-$(CONFIG_CGROUP_FREEZER) += freezer.o
obj-$(CONFIG_CGROUP_PIDS) += pids.o
obj-$(CONFIG_CGROUP_RDMA) += rdma.o
obj-$(CONFIG_CPUSETS) += cpuset.o
+obj-$(CONFIG_CGROUP_DEBUG) += debug.o
diff --git a/kernel/cgroup/cgroup-v1.c b/kernel/cgroup/cgroup-v1.c
index 1e101b9..7ad6b17 100644
--- a/kernel/cgroup/cgroup-v1.c
+++ b/kernel/cgroup/cgroup-v1.c
@@ -1311,150 +1311,3 @@ static int __init cgroup_no_v1(char *str)
return 1;
}
__setup("cgroup_no_v1=", cgroup_no_v1);
-
-
-#ifdef CONFIG_CGROUP_DEBUG
-static struct cgroup_subsys_state *
-debug_css_alloc(struct cgroup_subsys_state *parent_css)
-{
- struct cgroup_subsys_state *css = kzalloc(sizeof(*css), GFP_KERNEL);
-
- if (!css)
- return ERR_PTR(-ENOMEM);
-
- return css;
-}
-
-static void debug_css_free(struct cgroup_subsys_state *css)
-{
- kfree(css);
-}
-
-static u64 debug_taskcount_read(struct cgroup_subsys_state *css,
- struct cftype *cft)
-{
- return cgroup_task_count(css->cgroup);
-}
-
-static u64 current_css_set_read(struct cgroup_subsys_state *css,
- struct cftype *cft)
-{
- return (u64)(unsigned long)current->cgroups;
-}
-
-static u64 current_css_set_refcount_read(struct cgroup_subsys_state *css,
- struct cftype *cft)
-{
- u64 count;
-
- rcu_read_lock();
- count = refcount_read(&task_css_set(current)->refcount);
- rcu_read_unlock();
- return count;
-}
-
-static int current_css_set_cg_links_read(struct seq_file *seq, void *v)
-{
- struct cgrp_cset_link *link;
- struct css_set *cset;
- char *name_buf;
-
- name_buf = kmalloc(NAME_MAX + 1, GFP_KERNEL);
- if (!name_buf)
- return -ENOMEM;
-
- spin_lock_irq(&css_set_lock);
- rcu_read_lock();
- cset = rcu_dereference(current->cgroups);
- list_for_each_entry(link, &cset->cgrp_links, cgrp_link) {
- struct cgroup *c = link->cgrp;
-
- cgroup_name(c, name_buf, NAME_MAX + 1);
- seq_printf(seq, "Root %d group %s\n",
- c->root->hierarchy_id, name_buf);
- }
- rcu_read_unlock();
- spin_unlock_irq(&css_set_lock);
- kfree(name_buf);
- return 0;
-}
-
-#define MAX_TASKS_SHOWN_PER_CSS 25
-static int cgroup_css_links_read(struct seq_file *seq, void *v)
-{
- struct cgroup_subsys_state *css = seq_css(seq);
- struct cgrp_cset_link *link;
-
- spin_lock_irq(&css_set_lock);
- list_for_each_entry(link, &css->cgroup->cset_links, cset_link) {
- struct css_set *cset = link->cset;
- struct task_struct *task;
- int count = 0;
-
- seq_printf(seq, "css_set %pK\n", cset);
-
- list_for_each_entry(task, &cset->tasks, cg_list) {
- if (count++ > MAX_TASKS_SHOWN_PER_CSS)
- goto overflow;
- seq_printf(seq, " task %d\n", task_pid_vnr(task));
- }
-
- list_for_each_entry(task, &cset->mg_tasks, cg_list) {
- if (count++ > MAX_TASKS_SHOWN_PER_CSS)
- goto overflow;
- seq_printf(seq, " task %d\n", task_pid_vnr(task));
- }
- continue;
- overflow:
- seq_puts(seq, " ...\n");
- }
- spin_unlock_irq(&css_set_lock);
- return 0;
-}
-
-static u64 releasable_read(struct cgroup_subsys_state *css, struct cftype *cft)
-{
- return (!cgroup_is_populated(css->cgroup) &&
- !css_has_online_children(&css->cgroup->self));
-}
-
-static struct cftype debug_files[] = {
- {
- .name = "taskcount",
- .read_u64 = debug_taskcount_read,
- },
-
- {
- .name = "current_css_set",
- .read_u64 = current_css_set_read,
- },
-
- {
- .name = "current_css_set_refcount",
- .read_u64 = current_css_set_refcount_read,
- },
-
- {
- .name = "current_css_set_cg_links",
- .seq_show = current_css_set_cg_links_read,
- },
-
- {
- .name = "cgroup_css_links",
- .seq_show = cgroup_css_links_read,
- },
-
- {
- .name = "releasable",
- .read_u64 = releasable_read,
- },
-
- { } /* terminate */
-};
-
-struct cgroup_subsys debug_cgrp_subsys = {
- .css_alloc = debug_css_alloc,
- .css_free = debug_css_free,
- .legacy_cftypes = debug_files,
-};
-#endif /* CONFIG_CGROUP_DEBUG */
diff --git a/kernel/cgroup/debug.c b/kernel/cgroup/debug.c
new file mode 100644
index 0000000..56e60a2
--- /dev/null
+++ b/kernel/cgroup/debug.c
@@ -0,0 +1,165 @@
+#include <linux/ctype.h>
+#include <linux/mm.h>
+#include <linux/slab.h>
+
+#include "cgroup-internal.h"
+
+static struct cgroup_subsys_state *
+debug_css_alloc(struct cgroup_subsys_state *parent_css)
+{
+ struct cgroup_subsys_state *css = kzalloc(sizeof(*css), GFP_KERNEL);
+
+ if (!css)
+ return ERR_PTR(-ENOMEM);
+
+ return css;
+}
+
+static void debug_css_free(struct cgroup_subsys_state *css)
+{
+ kfree(css);
+}
+
+/*
+ * debug_taskcount_read - return the number of tasks in a cgroup.
+ * @cgrp: the cgroup in question
+ *
+ * Return the number of tasks in the cgroup. The returned number can be
+ * higher than the actual number of tasks due to css_set references from
+ * namespace roots and temporary usages.
+ */
+static u64 debug_taskcount_read(struct cgroup_subsys_state *css,
+ struct cftype *cft)
+{
+ struct cgroup *cgrp = css->cgroup;
+ u64 count = 0;
+ struct cgrp_cset_link *link;
+
+ spin_lock_irq(&css_set_lock);
+ list_for_each_entry(link, &cgrp->cset_links, cset_link)
+ count += refcount_read(&link->cset->refcount);
+ spin_unlock_irq(&css_set_lock);
+ return count;
+}
+
+static u64 current_css_set_read(struct cgroup_subsys_state *css,
+ struct cftype *cft)
+{
+ return (u64)(unsigned long)current->cgroups;
+}
+
+static u64 current_css_set_refcount_read(struct cgroup_subsys_state *css,
+ struct cftype *cft)
+{
+ u64 count;
+
+ rcu_read_lock();
+ count = refcount_read(&task_css_set(current)->refcount);
+ rcu_read_unlock();
+ return count;
+}
+
+static int current_css_set_cg_links_read(struct seq_file *seq, void *v)
+{
+ struct cgrp_cset_link *link;
+ struct css_set *cset;
+ char *name_buf;
+
+ name_buf = kmalloc(NAME_MAX + 1, GFP_KERNEL);
+ if (!name_buf)
+ return -ENOMEM;
+
+ spin_lock_irq(&css_set_lock);
+ rcu_read_lock();
+ cset = rcu_dereference(current->cgroups);
+ list_for_each_entry(link, &cset->cgrp_links, cgrp_link) {
+ struct cgroup *c = link->cgrp;
+
+ cgroup_name(c, name_buf, NAME_MAX + 1);
+ seq_printf(seq, "Root %d group %s\n",
+ c->root->hierarchy_id, name_buf);
+ }
+ rcu_read_unlock();
+ spin_unlock_irq(&css_set_lock);
+ kfree(name_buf);
+ return 0;
+}
+
+#define MAX_TASKS_SHOWN_PER_CSS 25
+static int cgroup_css_links_read(struct seq_file *seq, void *v)
+{
+ struct cgroup_subsys_state *css = seq_css(seq);
+ struct cgrp_cset_link *link;
+
+ spin_lock_irq(&css_set_lock);
+ list_for_each_entry(link, &css->cgroup->cset_links, cset_link) {
+ struct css_set *cset = link->cset;
+ struct task_struct *task;
+ int count = 0;
+
+ seq_printf(seq, "css_set %pK\n", cset);
+
+ list_for_each_entry(task, &cset->tasks, cg_list) {
+ if (count++ > MAX_TASKS_SHOWN_PER_CSS)
+ goto overflow;
+ seq_printf(seq, " task %d\n", task_pid_vnr(task));
+ }
+
+ list_for_each_entry(task, &cset->mg_tasks, cg_list) {
+ if (count++ > MAX_TASKS_SHOWN_PER_CSS)
+ goto overflow;
+ seq_printf(seq, " task %d\n", task_pid_vnr(task));
+ }
+ continue;
+ overflow:
+ seq_puts(seq, " ...\n");
+ }
+ spin_unlock_irq(&css_set_lock);
+ return 0;
+}
+
+static u64 releasable_read(struct cgroup_subsys_state *css, struct cftype *cft)
+{
+ return (!cgroup_is_populated(css->cgroup) &&
+ !css_has_online_children(&css->cgroup->self));
+}
+
+static struct cftype debug_files[] = {
+ {
+ .name = "taskcount",
+ .read_u64 = debug_taskcount_read,
+ },
+
+ {
+ .name = "current_css_set",
+ .read_u64 = current_css_set_read,
+ },
+
+ {
+ .name = "current_css_set_refcount",
+ .read_u64 = current_css_set_refcount_read,
+ },
+
+ {
+ .name = "current_css_set_cg_links",
+ .seq_show = current_css_set_cg_links_read,
+ },
+
+ {
+ .name = "cgroup_css_links",
+ .seq_show = cgroup_css_links_read,
+ },
+
+ {
+ .name = "releasable",
+ .read_u64 = releasable_read,
+ },
+
+ { } /* terminate */
+};
+
+struct cgroup_subsys debug_cgrp_subsys = {
+ .css_alloc = debug_css_alloc,
+ .css_free = debug_css_free,
+ .legacy_cftypes = debug_files,
+};
--
1.8.3.1
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [RFC PATCH v2 09/17] cgroup: Keep accurate count of tasks in each css_set
From: Waiman Long @ 2017-05-15 13:34 UTC (permalink / raw)
To: Tejun Heo, Li Zefan, Johannes Weiner, Peter Zijlstra, Ingo Molnar
Cc: cgroups, linux-kernel, linux-doc, linux-mm, kernel-team, pjt,
luto, efault, longman
In-Reply-To: <1494855256-12558-1-git-send-email-longman@redhat.com>
The reference count in the css_set data structure was used as a
proxy of the number of tasks attached to that css_set. However, that
count is actually not an accurate measure especially with thread mode
support. So a new variable task_count is added to the css_set to keep
track of the actual task count. This new variable is protected by
the css_set_lock. Functions that require the actual task count are
updated to use the new variable.
Signed-off-by: Waiman Long <longman@redhat.com>
---
include/linux/cgroup-defs.h | 3 +++
kernel/cgroup/cgroup-v1.c | 6 +-----
kernel/cgroup/cgroup.c | 5 +++++
kernel/cgroup/debug.c | 6 +-----
4 files changed, 10 insertions(+), 10 deletions(-)
diff --git a/include/linux/cgroup-defs.h b/include/linux/cgroup-defs.h
index b123afc..104be73 100644
--- a/include/linux/cgroup-defs.h
+++ b/include/linux/cgroup-defs.h
@@ -163,6 +163,9 @@ struct css_set {
/* reference count */
refcount_t refcount;
+ /* internal task count, protected by css_set_lock */
+ int task_count;
+
/*
* If not threaded, the following points to self. If threaded, to
* a cset which belongs to the top cgroup of the threaded subtree.
diff --git a/kernel/cgroup/cgroup-v1.c b/kernel/cgroup/cgroup-v1.c
index 7ad6b17..302b3b8 100644
--- a/kernel/cgroup/cgroup-v1.c
+++ b/kernel/cgroup/cgroup-v1.c
@@ -334,10 +334,6 @@ static struct cgroup_pidlist *cgroup_pidlist_find_create(struct cgroup *cgrp,
/**
* cgroup_task_count - count the number of tasks in a cgroup.
* @cgrp: the cgroup in question
- *
- * Return the number of tasks in the cgroup. The returned number can be
- * higher than the actual number of tasks due to css_set references from
- * namespace roots and temporary usages.
*/
static int cgroup_task_count(const struct cgroup *cgrp)
{
@@ -346,7 +342,7 @@ static int cgroup_task_count(const struct cgroup *cgrp)
spin_lock_irq(&css_set_lock);
list_for_each_entry(link, &cgrp->cset_links, cset_link)
- count += refcount_read(&link->cset->refcount);
+ count += link->cset->task_count;
spin_unlock_irq(&css_set_lock);
return count;
}
diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c
index 7b085d5..7e3ddfb 100644
--- a/kernel/cgroup/cgroup.c
+++ b/kernel/cgroup/cgroup.c
@@ -1676,6 +1676,7 @@ static void cgroup_enable_task_cg_lists(void)
css_set_update_populated(cset, true);
list_add_tail(&p->cg_list, &cset->tasks);
get_css_set(cset);
+ cset->task_count++;
}
spin_unlock(&p->sighand->siglock);
} while_each_thread(g, p);
@@ -2159,8 +2160,10 @@ static int cgroup_migrate_execute(struct cgroup_mgctx *mgctx)
struct css_set *to_cset = cset->mg_dst_cset;
get_css_set(to_cset);
+ to_cset->task_count++;
css_set_move_task(task, from_cset, to_cset, true);
put_css_set_locked(from_cset);
+ from_cset->task_count--;
}
}
spin_unlock_irq(&css_set_lock);
@@ -5160,6 +5163,7 @@ void cgroup_post_fork(struct task_struct *child)
cset = task_css_set(current);
if (list_empty(&child->cg_list)) {
get_css_set(cset);
+ cset->task_count++;
css_set_move_task(child, NULL, cset, false);
}
spin_unlock_irq(&css_set_lock);
@@ -5209,6 +5213,7 @@ void cgroup_exit(struct task_struct *tsk)
if (!list_empty(&tsk->cg_list)) {
spin_lock_irq(&css_set_lock);
css_set_move_task(tsk, cset, NULL, false);
+ cset->task_count--;
spin_unlock_irq(&css_set_lock);
} else {
get_css_set(cset);
diff --git a/kernel/cgroup/debug.c b/kernel/cgroup/debug.c
index 56e60a2..ada53e6 100644
--- a/kernel/cgroup/debug.c
+++ b/kernel/cgroup/debug.c
@@ -23,10 +23,6 @@ static void debug_css_free(struct cgroup_subsys_state *css)
/*
* debug_taskcount_read - return the number of tasks in a cgroup.
* @cgrp: the cgroup in question
- *
- * Return the number of tasks in the cgroup. The returned number can be
- * higher than the actual number of tasks due to css_set references from
- * namespace roots and temporary usages.
*/
static u64 debug_taskcount_read(struct cgroup_subsys_state *css,
struct cftype *cft)
@@ -37,7 +33,7 @@ static u64 debug_taskcount_read(struct cgroup_subsys_state *css,
spin_lock_irq(&css_set_lock);
list_for_each_entry(link, &cgrp->cset_links, cset_link)
- count += refcount_read(&link->cset->refcount);
+ count += link->cset->task_count;
spin_unlock_irq(&css_set_lock);
return count;
}
--
1.8.3.1
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [RFC PATCH v2 10/17] cgroup: Make debug cgroup support v2 and thread mode
From: Waiman Long @ 2017-05-15 13:34 UTC (permalink / raw)
To: Tejun Heo, Li Zefan, Johannes Weiner, Peter Zijlstra, Ingo Molnar
Cc: cgroups, linux-kernel, linux-doc, linux-mm, kernel-team, pjt,
luto, efault, longman
In-Reply-To: <1494855256-12558-1-git-send-email-longman@redhat.com>
Besides supporting cgroup v2 and thread mode, the following changes
are also made:
1) current_* cgroup files now resides only at the root as we don't
need duplicated files of the same function all over the cgroup
hierarchy.
2) The cgroup_css_links_read() function is modified to report
the number of tasks that are skipped because of overflow.
3) The relationship between proc_cset and threaded_csets are displayed.
4) The number of extra unaccounted references are displayed.
5) The status of being a thread root or threaded cgroup is displayed.
6) The current_css_set_read() function now prints out the addresses of
the css'es associated with the current css_set.
7) A new cgroup_subsys_states file is added to display the css objects
associated with a cgroup.
8) A new cgroup_masks file is added to display the various controller
bit masks in the cgroup.
Signed-off-by: Waiman Long <longman@redhat.com>
---
kernel/cgroup/debug.c | 196 +++++++++++++++++++++++++++++++++++++++++++++-----
1 file changed, 179 insertions(+), 17 deletions(-)
diff --git a/kernel/cgroup/debug.c b/kernel/cgroup/debug.c
index ada53e6..3121811 100644
--- a/kernel/cgroup/debug.c
+++ b/kernel/cgroup/debug.c
@@ -38,10 +38,37 @@ static u64 debug_taskcount_read(struct cgroup_subsys_state *css,
return count;
}
-static u64 current_css_set_read(struct cgroup_subsys_state *css,
- struct cftype *cft)
+static int current_css_set_read(struct seq_file *seq, void *v)
{
- return (u64)(unsigned long)current->cgroups;
+ struct css_set *cset;
+ struct cgroup_subsys *ss;
+ struct cgroup_subsys_state *css;
+ int i, refcnt;
+
+ mutex_lock(&cgroup_mutex);
+ spin_lock_irq(&css_set_lock);
+ rcu_read_lock();
+ cset = rcu_dereference(current->cgroups);
+ refcnt = refcount_read(&cset->refcount);
+ seq_printf(seq, "css_set %pK %d", cset, refcnt);
+ if (refcnt > cset->task_count)
+ seq_printf(seq, " +%d", refcnt - cset->task_count);
+ seq_puts(seq, "\n");
+
+ /*
+ * Print the css'es stored in the current css_set.
+ */
+ for_each_subsys(ss, i) {
+ css = cset->subsys[ss->id];
+ if (!css)
+ continue;
+ seq_printf(seq, "%2d: %-4s\t- %lx[%d]\n", ss->id, ss->name,
+ (unsigned long)css, css->id);
+ }
+ rcu_read_unlock();
+ spin_unlock_irq(&css_set_lock);
+ mutex_unlock(&cgroup_mutex);
+ return 0;
}
static u64 current_css_set_refcount_read(struct cgroup_subsys_state *css,
@@ -86,31 +113,151 @@ static int cgroup_css_links_read(struct seq_file *seq, void *v)
{
struct cgroup_subsys_state *css = seq_css(seq);
struct cgrp_cset_link *link;
+ int dead_cnt = 0, extra_refs = 0, threaded_csets = 0;
spin_lock_irq(&css_set_lock);
+ if (css->cgroup->proc_cgrp)
+ seq_puts(seq, (css->cgroup->proc_cgrp == css->cgroup)
+ ? "[thread root]\n" : "[threaded]\n");
+
list_for_each_entry(link, &css->cgroup->cset_links, cset_link) {
struct css_set *cset = link->cset;
struct task_struct *task;
int count = 0;
+ int refcnt = refcount_read(&cset->refcount);
+
+ /*
+ * Print out the proc_cset and threaded_cset relationship
+ * and highlight difference between refcount and task_count.
+ */
+ seq_printf(seq, "css_set %pK", cset);
+ if (rcu_dereference_protected(cset->proc_cset, 1) != cset) {
+ threaded_csets++;
+ seq_printf(seq, "=>%pK", cset->proc_cset);
+ }
+ if (!list_empty(&cset->threaded_csets)) {
+ struct css_set *tcset;
+ int idx = 0;
- seq_printf(seq, "css_set %pK\n", cset);
+ list_for_each_entry(tcset, &cset->threaded_csets,
+ threaded_csets_node) {
+ seq_puts(seq, idx ? "," : "<=");
+ seq_printf(seq, "%pK", tcset);
+ idx++;
+ }
+ } else {
+ seq_printf(seq, " %d", refcnt);
+ if (refcnt - cset->task_count > 0) {
+ int extra = refcnt - cset->task_count;
+
+ seq_printf(seq, " +%d", extra);
+ /*
+ * Take out the one additional reference in
+ * init_css_set.
+ */
+ if (cset == &init_css_set)
+ extra--;
+ extra_refs += extra;
+ }
+ }
+ seq_puts(seq, "\n");
list_for_each_entry(task, &cset->tasks, cg_list) {
- if (count++ > MAX_TASKS_SHOWN_PER_CSS)
- goto overflow;
- seq_printf(seq, " task %d\n", task_pid_vnr(task));
+ if (count++ <= MAX_TASKS_SHOWN_PER_CSS)
+ seq_printf(seq, " task %d\n",
+ task_pid_vnr(task));
}
list_for_each_entry(task, &cset->mg_tasks, cg_list) {
- if (count++ > MAX_TASKS_SHOWN_PER_CSS)
- goto overflow;
- seq_printf(seq, " task %d\n", task_pid_vnr(task));
+ if (count++ <= MAX_TASKS_SHOWN_PER_CSS)
+ seq_printf(seq, " task %d\n",
+ task_pid_vnr(task));
}
- continue;
- overflow:
- seq_puts(seq, " ...\n");
+ /* show # of overflowed tasks */
+ if (count > MAX_TASKS_SHOWN_PER_CSS)
+ seq_printf(seq, " ... (%d)\n",
+ count - MAX_TASKS_SHOWN_PER_CSS);
+
+ if (cset->dead) {
+ seq_puts(seq, " [dead]\n");
+ dead_cnt++;
+ }
+
+ WARN_ON(count != cset->task_count);
}
spin_unlock_irq(&css_set_lock);
+
+ if (!dead_cnt && !extra_refs && !threaded_csets)
+ return 0;
+
+ seq_puts(seq, "\n");
+ if (threaded_csets)
+ seq_printf(seq, "threaded css_sets = %d\n", threaded_csets);
+ if (extra_refs)
+ seq_printf(seq, "extra references = %d\n", extra_refs);
+ if (dead_cnt)
+ seq_printf(seq, "dead css_sets = %d\n", dead_cnt);
+
+ return 0;
+}
+
+static int cgroup_subsys_states_read(struct seq_file *seq, void *v)
+{
+ struct cgroup *cgrp = seq_css(seq)->cgroup;
+ struct cgroup_subsys *ss;
+ struct cgroup_subsys_state *css;
+ char pbuf[16];
+ int i;
+
+ mutex_lock(&cgroup_mutex);
+ for_each_subsys(ss, i) {
+ css = rcu_dereference_check(cgrp->subsys[ss->id], true);
+ if (!css)
+ continue;
+ pbuf[0] = '\0';
+
+ /* Show the parent CSS if applicable*/
+ if (css->parent)
+ snprintf(pbuf, sizeof(pbuf) - 1, " P=%d",
+ css->parent->id);
+ seq_printf(seq, "%2d: %-4s\t- %lx[%d] %d%s\n", ss->id, ss->name,
+ (unsigned long)css, css->id,
+ atomic_read(&css->online_cnt), pbuf);
+ }
+ mutex_unlock(&cgroup_mutex);
+ return 0;
+}
+
+static int cgroup_masks_read(struct seq_file *seq, void *v)
+{
+ struct cgroup *cgrp = seq_css(seq)->cgroup;
+ struct cgroup_subsys *ss;
+ int i, j;
+ struct {
+ u16 *mask;
+ char *name;
+ } mask_list[] = {
+ { &cgrp->subtree_control, "subtree_control" },
+ { &cgrp->subtree_ss_mask, "subtree_ss_mask" },
+ };
+
+ mutex_lock(&cgroup_mutex);
+ for (i = 0; i < ARRAY_SIZE(mask_list); i++) {
+ u16 mask = *mask_list[i].mask;
+ bool first = true;
+
+ seq_printf(seq, "%-15s: ", mask_list[i].name);
+ for_each_subsys(ss, j) {
+ if (!(mask & (1 << ss->id)))
+ continue;
+ if (!first)
+ seq_puts(seq, ", ");
+ seq_puts(seq, ss->name);
+ first = false;
+ }
+ seq_putc(seq, '\n');
+ }
+ mutex_unlock(&cgroup_mutex);
return 0;
}
@@ -128,17 +275,20 @@ static u64 releasable_read(struct cgroup_subsys_state *css, struct cftype *cft)
{
.name = "current_css_set",
- .read_u64 = current_css_set_read,
+ .seq_show = current_css_set_read,
+ .flags = CFTYPE_ONLY_ON_ROOT,
},
{
.name = "current_css_set_refcount",
.read_u64 = current_css_set_refcount_read,
+ .flags = CFTYPE_ONLY_ON_ROOT,
},
{
.name = "current_css_set_cg_links",
.seq_show = current_css_set_cg_links_read,
+ .flags = CFTYPE_ONLY_ON_ROOT,
},
{
@@ -147,6 +297,16 @@ static u64 releasable_read(struct cgroup_subsys_state *css, struct cftype *cft)
},
{
+ .name = "cgroup_subsys_states",
+ .seq_show = cgroup_subsys_states_read,
+ },
+
+ {
+ .name = "cgroup_masks",
+ .seq_show = cgroup_masks_read,
+ },
+
+ {
.name = "releasable",
.read_u64 = releasable_read,
},
@@ -155,7 +315,9 @@ static u64 releasable_read(struct cgroup_subsys_state *css, struct cftype *cft)
};
struct cgroup_subsys debug_cgrp_subsys = {
- .css_alloc = debug_css_alloc,
- .css_free = debug_css_free,
- .legacy_cftypes = debug_files,
+ .css_alloc = debug_css_alloc,
+ .css_free = debug_css_free,
+ .legacy_cftypes = debug_files,
+ .dfl_cftypes = debug_files,
+ .threaded = true,
};
--
1.8.3.1
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [RFC PATCH v2 11/17] cgroup: Implement new thread mode semantics
From: Waiman Long @ 2017-05-15 13:34 UTC (permalink / raw)
To: Tejun Heo, Li Zefan, Johannes Weiner, Peter Zijlstra, Ingo Molnar
Cc: cgroups, linux-kernel, linux-doc, linux-mm, kernel-team, pjt,
luto, efault, longman
In-Reply-To: <1494855256-12558-1-git-send-email-longman@redhat.com>
The current thread mode semantics aren't sufficient to fully support
threaded controllers like cpu. The main problem is that when thread
mode is enabled at root (mainly for performance reason), all the
non-threaded controllers cannot be supported at all.
To alleviate this problem, the roles of thread root and threaded
cgroups are now further separated. Now thread mode can only be enabled
on a non-root leaf cgroup whose parent will then become the thread
root. All the descendants of a threaded cgroup will still need to be
threaded. All the non-threaded resource will be accounted for in the
thread root. Unlike the previous thread mode, however, a thread root
can have non-threaded children where system resources like memory
can be further split down the hierarchy.
Now we could have something like
R -- A -- B
\
T1 -- T2
where R is the thread root, A and B are non-threaded cgroups, T1 and
T2 are threaded cgroups. The cgroups R, T1, T2 form a threaded subtree
where all the non-threaded resources are accounted for in R. The no
internal process constraint does not apply in the threaded subtree.
Non-threaded controllers need to properly handle the competition
between internal processes and child cgroups at the thread root.
This model will be flexible enough to support the need of the threaded
controllers.
Signed-off-by: Waiman Long <longman@redhat.com>
---
Documentation/cgroup-v2.txt | 51 +++++++----
kernel/cgroup/cgroup-internal.h | 10 +++
kernel/cgroup/cgroup.c | 186 +++++++++++++++++++++++++++++++++++-----
3 files changed, 209 insertions(+), 38 deletions(-)
diff --git a/Documentation/cgroup-v2.txt b/Documentation/cgroup-v2.txt
index 1c6f5a9..3ae7e9c 100644
--- a/Documentation/cgroup-v2.txt
+++ b/Documentation/cgroup-v2.txt
@@ -222,21 +222,32 @@ process can be put in different cgroups and are not subject to the no
internal process constraint - threaded controllers can be enabled on
non-leaf cgroups whether they have threads in them or not.
-To enable the thread mode, the following conditions must be met.
+To enable the thread mode on a cgroup, the following conditions must
+be met.
-- The thread root doesn't have any child cgroups.
+- The cgroup doesn't have any child cgroups.
-- The thread root doesn't have any controllers enabled.
+- The cgroup doesn't have any non-threaded controllers enabled.
+
+- The cgroup doesn't have any processes attached to it.
Thread mode can be enabled by writing "enable" to "cgroup.threads"
file.
# echo enable > cgroup.threads
-Inside a threaded subtree, "cgroup.threads" can be read and contains
-the list of the thread IDs of all threads in the cgroup. Except that
-the operations are per-thread instead of per-process, "cgroup.threads"
-has the same format and behaves the same way as "cgroup.procs".
+The parent of the threaded cgroup will become the thread root, if
+it hasn't been a thread root yet. In other word, thread mode cannot
+be enabled on the root cgroup as it doesn't have a parent cgroup. A
+thread root can have child cgroups and controllers enabled before
+becoming one.
+
+A threaded subtree includes the thread root and all the threaded child
+cgroups as well as their descendants which are all threaded cgroups.
+"cgroup.threads" can be read and contains the list of the thread
+IDs of all threads in the cgroup. Except that the operations are
+per-thread instead of per-process, "cgroup.threads" has the same
+format and behaves the same way as "cgroup.procs".
The thread root serves as the resource domain for the whole subtree,
and, while the threads can be scattered across the subtree, all the
@@ -246,25 +257,30 @@ not readable in the subtree proper. However, "cgroup.procs" can be
written to from anywhere in the subtree to migrate all threads of the
matching process to the cgroup.
-Only threaded controllers can be enabled in a threaded subtree. When
-a threaded controller is enabled inside a threaded subtree, it only
-accounts for and controls resource consumptions associated with the
-threads in the cgroup and its descendants. All consumptions which
-aren't tied to a specific thread belong to the thread root.
+Only threaded controllers can be enabled in a non-root threaded cgroup.
+When a threaded controller is enabled inside a threaded subtree,
+it only accounts for and controls resource consumptions associated
+with the threads in the cgroup and its descendants. All consumptions
+which aren't tied to a specific thread belong to the thread root.
Because a threaded subtree is exempt from no internal process
constraint, a threaded controller must be able to handle competition
between threads in a non-leaf cgroup and its child cgroups. Each
threaded controller defines how such competitions are handled.
+A new child cgroup created under a thread root will not be threaded.
+Thread mode has to be explicitly enabled on each of the thread root's
+children. Descendants of a threaded cgroup, however, will always be
+threaded and that mode cannot be disabled.
+
To disable the thread mode, the following conditions must be met.
-- The cgroup is a thread root. Thread mode can't be disabled
- partially in the subtree.
+- The cgroup is a child of a thread root. Thread mode can't be
+ disabled partially further down the hierarchy.
-- The thread root doesn't have any child cgroups.
+- The cgroup doesn't have any child cgroups.
-- The thread root doesn't have any controllers enabled.
+- The cgroup doesn't have any threads attached to it.
Thread mode can be disabled by writing "disable" to "cgroup.threads"
file.
@@ -366,6 +382,9 @@ with any other cgroups and requires special treatment from most
controllers. How resource consumption in the root cgroup is governed
is up to each controller.
+The threaded cgroups and the thread roots are also exempt from this
+restriction.
+
Note that the restriction doesn't get in the way if there is no
enabled controller in the cgroup's "cgroup.subtree_control". This is
important as otherwise it wouldn't be possible to create children of a
diff --git a/kernel/cgroup/cgroup-internal.h b/kernel/cgroup/cgroup-internal.h
index 2c8e3a9..15abaa0 100644
--- a/kernel/cgroup/cgroup-internal.h
+++ b/kernel/cgroup/cgroup-internal.h
@@ -124,6 +124,16 @@ static inline bool notify_on_release(const struct cgroup *cgrp)
return test_bit(CGRP_NOTIFY_ON_RELEASE, &cgrp->flags);
}
+static inline bool cgroup_is_threaded(const struct cgroup *cgrp)
+{
+ return cgrp->proc_cgrp && (cgrp->proc_cgrp != cgrp);
+}
+
+static inline bool cgroup_is_thread_root(const struct cgroup *cgrp)
+{
+ return cgrp->proc_cgrp == cgrp;
+}
+
void put_css_set_locked(struct css_set *cset);
static inline void put_css_set(struct css_set *cset)
diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c
index 7e3ddfb..11cb091 100644
--- a/kernel/cgroup/cgroup.c
+++ b/kernel/cgroup/cgroup.c
@@ -334,8 +334,13 @@ static u16 cgroup_control(struct cgroup *cgrp)
struct cgroup *parent = cgroup_parent(cgrp);
u16 root_ss_mask = cgrp->root->subsys_mask;
- if (parent)
- return parent->subtree_control;
+ if (parent) {
+ u16 ss_mask = parent->subtree_control;
+
+ if (cgroup_is_threaded(cgrp))
+ ss_mask &= cgrp_dfl_threaded_ss_mask;
+ return ss_mask;
+ }
if (cgroup_on_dfl(cgrp))
root_ss_mask &= ~(cgrp_dfl_inhibit_ss_mask |
@@ -348,8 +353,13 @@ static u16 cgroup_ss_mask(struct cgroup *cgrp)
{
struct cgroup *parent = cgroup_parent(cgrp);
- if (parent)
- return parent->subtree_ss_mask;
+ if (parent) {
+ u16 ss_mask = parent->subtree_ss_mask;
+
+ if (cgroup_is_threaded(cgrp))
+ ss_mask &= cgrp_dfl_threaded_ss_mask;
+ return ss_mask;
+ }
return cgrp->root->subsys_mask;
}
@@ -598,6 +608,24 @@ static bool css_set_threaded(struct css_set *cset)
}
/**
+ * threaded_children_count - returns # of threaded children
+ * @cgrp: cgroup to be tested
+ *
+ * cgroup_mutex must be held by the caller.
+ */
+static int threaded_children_count(struct cgroup *cgrp)
+{
+ struct cgroup *child;
+ int count = 0;
+
+ lockdep_assert_held(&cgroup_mutex);
+ cgroup_for_each_live_child(child, cgrp)
+ if (cgroup_is_threaded(child))
+ count++;
+ return count;
+}
+
+/**
* cgroup_update_populated - updated populated count of a cgroup
* @cgrp: the target cgroup
* @populated: inc or dec populated count
@@ -2926,15 +2954,15 @@ static ssize_t cgroup_subtree_control_write(struct kernfs_open_file *of,
}
/* can't enable !threaded controllers on a threaded cgroup */
- if (cgrp->proc_cgrp && (enable & ~cgrp_dfl_threaded_ss_mask)) {
+ if (cgroup_is_threaded(cgrp) && (enable & ~cgrp_dfl_threaded_ss_mask)) {
ret = -EBUSY;
goto out_unlock;
}
/*
- * Except for root and threaded cgroups, subtree_control must be
- * zero for a cgroup with tasks so that child cgroups don't compete
- * against tasks.
+ * Except for root, thread roots and threaded cgroups, subtree_control
+ * must be zero for a cgroup with tasks so that child cgroups don't
+ * compete against tasks.
*/
if (enable && cgroup_parent(cgrp) && !cgrp->proc_cgrp) {
struct cgrp_cset_link *link;
@@ -2982,22 +3010,48 @@ static int cgroup_enable_threaded(struct cgroup *cgrp)
LIST_HEAD(csets);
struct cgrp_cset_link *link;
struct css_set *cset, *cset_next;
+ struct cgroup *child;
int ret;
+ u16 ss_mask;
lockdep_assert_held(&cgroup_mutex);
/* noop if already threaded */
- if (cgrp->proc_cgrp)
+ if (cgroup_is_threaded(cgrp))
return 0;
- /* allow only if there are neither children or enabled controllers */
- if (css_has_online_children(&cgrp->self) || cgrp->subtree_control)
+ /*
+ * Allow only if it is not the root and there are:
+ * 1) no children,
+ * 2) no non-threaded controllers are enabled, and
+ * 3) no attached tasks.
+ *
+ * With no attached tasks, it is assumed that no css_sets will be
+ * linked to the current cgroup. This may not be true if some dead
+ * css_sets linger around due to task_struct leakage, for example.
+ */
+ if (css_has_online_children(&cgrp->self) ||
+ (cgroup_control(cgrp) & ~cgrp_dfl_threaded_ss_mask) ||
+ !cgroup_parent(cgrp) || cgroup_is_populated(cgrp))
return -EBUSY;
- /* find all csets which need ->proc_cset updated */
+ /* make the parent cgroup a thread root */
+ child = cgrp;
+ cgrp = cgroup_parent(child);
+
+ /* noop for parent if parent has already been threaded */
+ if (cgrp->proc_cgrp)
+ goto setup_child;
+
+ /*
+ * For the parent cgroup, we need to find all csets which need
+ * ->proc_cset updated
+ */
spin_lock_irq(&css_set_lock);
list_for_each_entry(link, &cgrp->cset_links, cset_link) {
cset = link->cset;
+ if (cset->dead)
+ continue;
if (css_set_populated(cset)) {
WARN_ON_ONCE(css_set_threaded(cset));
WARN_ON_ONCE(cset->pcset_preload);
@@ -3036,7 +3090,34 @@ static int cgroup_enable_threaded(struct cgroup *cgrp)
/* mark it threaded */
cgrp->proc_cgrp = cgrp;
- return 0;
+setup_child:
+ ss_mask = cgroup_ss_mask(child);
+ /*
+ * If some non-threaded controllers are enabled, they have to be
+ * disabled.
+ */
+ if (ss_mask & ~cgrp_dfl_threaded_ss_mask) {
+ cgroup_save_control(child);
+ child->proc_cgrp = cgrp;
+ ret = cgroup_apply_control(child);
+ cgroup_finalize_control(child, ret);
+ kernfs_activate(child->kn);
+
+ /*
+ * If an error happen (it shouldn't), the thread mode
+ * enablement fails, but the parent will remain as thread
+ * root. That shouldn't be a problem as a thread root
+ * without threaded children is not much different from
+ * a non-threaded cgroup.
+ */
+ WARN_ON_ONCE(ret);
+ if (ret)
+ child->proc_cgrp = NULL;
+ } else {
+ child->proc_cgrp = cgrp;
+ ret = 0;
+ }
+ return ret;
err_put_csets:
spin_lock_irq(&css_set_lock);
@@ -3055,26 +3136,71 @@ static int cgroup_enable_threaded(struct cgroup *cgrp)
static int cgroup_disable_threaded(struct cgroup *cgrp)
{
struct cgrp_cset_link *link;
+ struct cgroup *parent = cgroup_parent(cgrp);
lockdep_assert_held(&cgroup_mutex);
- /* noop if already !threaded */
- if (!cgrp->proc_cgrp)
- return 0;
-
/* partial disable isn't supported */
- if (cgrp->proc_cgrp != cgrp)
+ if (cgrp->proc_cgrp != parent)
return -EBUSY;
- /* allow only if there are neither children or enabled controllers */
- if (css_has_online_children(&cgrp->self) || cgrp->subtree_control)
+ /* noop if not a threaded cgroup */
+ if (!cgroup_is_threaded(cgrp))
+ return 0;
+
+ /*
+ * Allow only if there are
+ * 1) no children, and
+ * 2) no attached tasks.
+ *
+ * With no attached tasks, it is assumed that no css_sets will be
+ * linked to the current cgroup. This may not be true if some dead
+ * css_sets linger around due to task_struct leakage, for example.
+ */
+ if (css_has_online_children(&cgrp->self) || cgroup_is_populated(cgrp))
return -EBUSY;
- /* walk all csets and reset ->proc_cset */
+ /*
+ * If the cgroup has some non-threaded controllers enabled at the
+ * subtree_control level of the parent, we need to re-enabled those
+ * controllers.
+ */
+ cgrp->proc_cgrp = NULL;
+ if (cgroup_ss_mask(cgrp) & ~cgrp_dfl_threaded_ss_mask) {
+ int ret;
+
+ cgrp->proc_cgrp = parent;
+ cgroup_save_control(cgrp);
+ cgrp->proc_cgrp = NULL;
+ ret = cgroup_apply_control(cgrp);
+ cgroup_finalize_control(cgrp, ret);
+ kernfs_activate(cgrp->kn);
+
+ /*
+ * If an error happen, we abandon update to the thread root
+ * and return the erorr.
+ */
+ if (ret)
+ return ret;
+ }
+
+ /*
+ * Check remaining threaded children count to see if the threaded
+ * csets of the parent need to be removed and ->proc_cset reset.
+ */
spin_lock_irq(&css_set_lock);
+
+ if (threaded_children_count(parent))
+ goto out_unlock; /* still have threaded children left */
+
+ cgrp = parent;
list_for_each_entry(link, &cgrp->cset_links, cset_link) {
struct css_set *cset = link->cset;
+ /* skip dead css_set */
+ if (cset->dead)
+ continue;
+
if (css_set_threaded(cset)) {
struct css_set *pcset = proc_css_set(cset);
@@ -3090,6 +3216,7 @@ static int cgroup_disable_threaded(struct cgroup *cgrp)
}
}
cgrp->proc_cgrp = NULL;
+out_unlock:
spin_unlock_irq(&css_set_lock);
return 0;
@@ -4480,7 +4607,16 @@ static struct cgroup *cgroup_create(struct cgroup *parent)
cgrp->self.parent = &parent->self;
cgrp->root = root;
cgrp->level = level;
- cgrp->proc_cgrp = parent->proc_cgrp;
+
+ /*
+ * A child cgroup created directly under a thread root will not
+ * be threaded. Thread mode has to be explicitly enabled for it.
+ * The child cgroup will be threaded if its parent is threaded.
+ */
+ if (cgroup_is_thread_root(parent))
+ cgrp->proc_cgrp = NULL;
+ else
+ cgrp->proc_cgrp = parent->proc_cgrp;
for (tcgrp = cgrp; tcgrp; tcgrp = cgroup_parent(tcgrp))
cgrp->ancestor_ids[tcgrp->level] = tcgrp->id;
@@ -4712,6 +4848,12 @@ static int cgroup_destroy_locked(struct cgroup *cgrp)
return -EBUSY;
/*
+ * Do an implicit thread mode disable if on default hierarchy.
+ */
+ if (cgroup_on_dfl(cgrp))
+ cgroup_disable_threaded(cgrp);
+
+ /*
* Mark @cgrp and the associated csets dead. The former prevents
* further task migration and child creation by disabling
* cgroup_lock_live_group(). The latter makes the csets ignored by
--
1.8.3.1
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [RFC PATCH v2 12/17] cgroup: Remove cgroup v2 no internal process constraint
From: Waiman Long @ 2017-05-15 13:34 UTC (permalink / raw)
To: Tejun Heo, Li Zefan, Johannes Weiner, Peter Zijlstra, Ingo Molnar
Cc: cgroups, linux-kernel, linux-doc, linux-mm, kernel-team, pjt,
luto, efault, longman
In-Reply-To: <1494855256-12558-1-git-send-email-longman@redhat.com>
The rationale behind the cgroup v2 no internal process constraint is
to avoid resouorce competition between internal processes and child
cgroups. However, not all controllers have problem with internal
process competiton. Enforcing this rule may lead to unnatural process
hierarchy and unneeded levels for those controllers.
This patch removes the no internal process contraint by enabling those
controllers that don't like internal process competition to have a
separate set of control knobs just for internal processes in a cgroup.
A new control file "cgroup.resource_control" is added. Enabling a
controller with a "+" prefix will create a separate set of control
knobs for that controller in the special "cgroup.resource_domain"
sub-directory for all the internal processes. The existing control
knobs in the cgroup will then be used to manage resource distribution
between internal processes as a group and other child cgroups.
Signed-off-by: Waiman Long <longman@redhat.com>
---
Documentation/cgroup-v2.txt | 76 ++++++-----
include/linux/cgroup-defs.h | 15 +++
kernel/cgroup/cgroup-internal.h | 1 -
kernel/cgroup/cgroup-v1.c | 3 -
kernel/cgroup/cgroup.c | 275 ++++++++++++++++++++++++++++------------
kernel/cgroup/debug.c | 7 +-
6 files changed, 260 insertions(+), 117 deletions(-)
diff --git a/Documentation/cgroup-v2.txt b/Documentation/cgroup-v2.txt
index 3ae7e9c..0f41282 100644
--- a/Documentation/cgroup-v2.txt
+++ b/Documentation/cgroup-v2.txt
@@ -23,7 +23,7 @@ CONTENTS
2-4. Controlling Controllers
2-4-1. Enabling and Disabling
2-4-2. Top-down Constraint
- 2-4-3. No Internal Process Constraint
+ 2-4-3. Managing Internal Process Competition
2-5. Delegation
2-5-1. Model of Delegation
2-5-2. Delegation Containment
@@ -218,9 +218,7 @@ a subtree while still maintaining the common resource domain for them.
Enabling thread mode on a subtree makes it threaded. The root of a
threaded subtree is called thread root and serves as the resource
domain for the entire subtree. In a threaded subtree, threads of a
-process can be put in different cgroups and are not subject to the no
-internal process constraint - threaded controllers can be enabled on
-non-leaf cgroups whether they have threads in them or not.
+process can be put in different cgroups.
To enable the thread mode on a cgroup, the following conditions must
be met.
@@ -263,11 +261,6 @@ it only accounts for and controls resource consumptions associated
with the threads in the cgroup and its descendants. All consumptions
which aren't tied to a specific thread belong to the thread root.
-Because a threaded subtree is exempt from no internal process
-constraint, a threaded controller must be able to handle competition
-between threads in a non-leaf cgroup and its child cgroups. Each
-threaded controller defines how such competitions are handled.
-
A new child cgroup created under a thread root will not be threaded.
Thread mode has to be explicitly enabled on each of the thread root's
children. Descendants of a threaded cgroup, however, will always be
@@ -364,35 +357,38 @@ the parent has the controller enabled and a controller can't be
disabled if one or more children have it enabled.
-2-4-3. No Internal Process Constraint
+2-4-3. Managing Internal Process Competition
-Non-root cgroups can only distribute resources to their children when
-they don't have any processes of their own. In other words, only
-cgroups which don't contain any processes can have controllers enabled
-in their "cgroup.subtree_control" files.
+There are resources managed by some controllers that don't work well
+if the internal processes in a non-leaf cgroup have to compete against
+the resource requirement of the other child cgroups. Other controllers
+work perfectly fine with internal process competition.
-This guarantees that, when a controller is looking at the part of the
-hierarchy which has it enabled, processes are always only on the
-leaves. This rules out situations where child cgroups compete against
-internal processes of the parent.
+Internal processes are allowed in a non-leaf cgroup. Controllers
+that don't like internal process competition can use
+the "cgroup.resource_control" file to create a special
+"cgroup.resource_domain" child cgroup that hold the control knobs
+for all the internal processes in the cgroup.
-The root cgroup is exempt from this restriction. Root contains
-processes and anonymous resource consumption which can't be associated
-with any other cgroups and requires special treatment from most
-controllers. How resource consumption in the root cgroup is governed
-is up to each controller.
+ # echo "+memory -pids" > cgroup.resource_control
-The threaded cgroups and the thread roots are also exempt from this
-restriction.
+Here, the control files for the memory controller are activated in the
+"cgroup.resource_domain" directory while that of the pids controller
+are removed. All the internal processes in the cgroup will use the
+memory control files in the "cgroup.resource_domain" directory to
+manage their memory. The memory control files in the cgroup itself
+can then be used to manage resource distribution between internal
+processes as a group and other child cgroups.
-Note that the restriction doesn't get in the way if there is no
-enabled controller in the cgroup's "cgroup.subtree_control". This is
-important as otherwise it wouldn't be possible to create children of a
-populated cgroup. To control resource distribution of a cgroup, the
-cgroup must create children and transfer all its processes to the
-children before enabling controllers in its "cgroup.subtree_control"
-file.
+Only controllers that are enabled in the "cgroup.controllers" file
+can be enabled in the "cgroup.resource_control" file. Once enabled,
+the parent cgroup cannot take away the controller until it has been
+disabled in the "cgroup.resource_control" file.
+The directory name "cgroup.resource_domain" is reserved. It cannot
+be created or deleted directly and no child cgroups can be created
+underneath it. All the "cgroup." control files are missing and so
+the users cannot move process into it.
2-5. Delegation
@@ -730,6 +726,22 @@ All cgroup core files are prefixed with "cgroup."
the last one is effective. When multiple enable and disable
operations are specified, either all succeed or all fail.
+ cgroup.resource_control
+
+ A read-write space separated values file which exists on all
+ cgroups. Starts out empty.
+
+ When read, it shows space separated list of the controllers
+ which are enabled to have separate control files in the
+ "cgroup.resource_domain" directory for internal processes.
+
+ Space separated list of controllers prefixed with '+' or '-'
+ can be written to enable or disable controllers. A controller
+ name prefixed with '+' enables the controller and '-'
+ disables. If a controller appears more than once on the list,
+ the last one is effective. When multiple enable and disable
+ operations are specified, either all succeed or all fail.
+
cgroup.events
A read-only flat-keyed file which exists on non-root cgroups.
diff --git a/include/linux/cgroup-defs.h b/include/linux/cgroup-defs.h
index 104be73..67ab326 100644
--- a/include/linux/cgroup-defs.h
+++ b/include/linux/cgroup-defs.h
@@ -61,6 +61,9 @@ enum {
* specified at mount time and thus is implemented here.
*/
CGRP_CPUSET_CLONE_CHILDREN,
+
+ /* Special child resource domain cgroup */
+ CGRP_RESOURCE_DOMAIN,
};
/* cgroup_root->flags */
@@ -293,11 +296,23 @@ struct cgroup {
u16 old_subtree_control;
u16 old_subtree_ss_mask;
+ /*
+ * The bitmask of subsystems that have separate sets of control
+ * knobs in a special child resource cgroup to control internal
+ * processes within the current cgroup so that they won't compete
+ * directly with other regular child cgroups. This is for the
+ * default hierarchy only.
+ */
+ u16 resource_control;
+
/* Private pointers for each registered subsystem */
struct cgroup_subsys_state __rcu *subsys[CGROUP_SUBSYS_COUNT];
struct cgroup_root *root;
+ /* Pointer to the special resource child cgroup */
+ struct cgroup *resource_domain;
+
/*
* List of cgrp_cset_links pointing at css_sets with tasks in this
* cgroup. Protected by css_set_lock.
diff --git a/kernel/cgroup/cgroup-internal.h b/kernel/cgroup/cgroup-internal.h
index 15abaa0..fc877e0 100644
--- a/kernel/cgroup/cgroup-internal.h
+++ b/kernel/cgroup/cgroup-internal.h
@@ -180,7 +180,6 @@ struct dentry *cgroup_do_mount(struct file_system_type *fs_type, int flags,
struct cgroup_root *root, unsigned long magic,
struct cgroup_namespace *ns);
-bool cgroup_may_migrate_to(struct cgroup *dst_cgrp);
void cgroup_migrate_finish(struct cgroup_mgctx *mgctx);
void cgroup_migrate_add_src(struct css_set *src_cset, struct cgroup *dst_cgrp,
struct cgroup_mgctx *mgctx);
diff --git a/kernel/cgroup/cgroup-v1.c b/kernel/cgroup/cgroup-v1.c
index 302b3b8..ef578b6 100644
--- a/kernel/cgroup/cgroup-v1.c
+++ b/kernel/cgroup/cgroup-v1.c
@@ -99,9 +99,6 @@ int cgroup_transfer_tasks(struct cgroup *to, struct cgroup *from)
if (cgroup_on_dfl(to))
return -EINVAL;
- if (!cgroup_may_migrate_to(to))
- return -EBUSY;
-
mutex_lock(&cgroup_mutex);
percpu_down_write(&cgroup_threadgroup_rwsem);
diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c
index 11cb091..c3be7e2 100644
--- a/kernel/cgroup/cgroup.c
+++ b/kernel/cgroup/cgroup.c
@@ -63,6 +63,12 @@
MAX_CFTYPE_NAME + 2)
/*
+ * Reserved cgroup name for the special resource domain child cgroup of
+ * the default hierarchy.
+ */
+#define CGROUP_RESOURCE_DOMAIN "cgroup.resource_domain"
+
+/*
* cgroup_mutex is the master lock. Any modification to cgroup or its
* hierarchy must be performed while holding it.
*
@@ -337,6 +343,9 @@ static u16 cgroup_control(struct cgroup *cgrp)
if (parent) {
u16 ss_mask = parent->subtree_control;
+ if (test_bit(CGRP_RESOURCE_DOMAIN, &cgrp->flags))
+ return parent->resource_control;
+
if (cgroup_is_threaded(cgrp))
ss_mask &= cgrp_dfl_threaded_ss_mask;
return ss_mask;
@@ -356,6 +365,9 @@ static u16 cgroup_ss_mask(struct cgroup *cgrp)
if (parent) {
u16 ss_mask = parent->subtree_ss_mask;
+ if (test_bit(CGRP_RESOURCE_DOMAIN, &cgrp->flags))
+ return parent->resource_control;
+
if (cgroup_is_threaded(cgrp))
ss_mask &= cgrp_dfl_threaded_ss_mask;
return ss_mask;
@@ -413,6 +425,11 @@ static struct cgroup_subsys_state *cgroup_e_css(struct cgroup *cgrp,
return NULL;
}
+ if (cgrp->resource_control & (1 << ss->id)) {
+ WARN_ON(!cgrp->resource_domain);
+ if (cgrp->resource_domain)
+ return cgroup_css(cgrp->resource_domain, ss);
+ }
return cgroup_css(cgrp, ss);
}
@@ -435,8 +452,10 @@ struct cgroup_subsys_state *cgroup_get_e_css(struct cgroup *cgrp,
rcu_read_lock();
do {
- css = cgroup_css(cgrp, ss);
-
+ if (cgrp->resource_control & (1 << ss->id))
+ css = cgroup_css(cgrp->resource_domain, ss);
+ else
+ css = cgroup_css(cgrp, ss);
if (css && css_tryget_online(css))
goto out_unlock;
cgrp = cgroup_parent(cgrp);
@@ -2234,20 +2253,6 @@ static int cgroup_migrate_execute(struct cgroup_mgctx *mgctx)
}
/**
- * cgroup_may_migrate_to - verify whether a cgroup can be migration destination
- * @dst_cgrp: destination cgroup to test
- *
- * On the default hierarchy, except for the root, subtree_control must be
- * zero for migration destination cgroups with tasks so that child cgroups
- * don't compete against tasks.
- */
-bool cgroup_may_migrate_to(struct cgroup *dst_cgrp)
-{
- return !cgroup_on_dfl(dst_cgrp) || !cgroup_parent(dst_cgrp) ||
- !dst_cgrp->subtree_control;
-}
-
-/**
* cgroup_migrate_finish - cleanup after attach
* @mgctx: migration context
*
@@ -2449,9 +2454,6 @@ int cgroup_attach_task(struct cgroup *dst_cgrp, struct task_struct *leader,
struct task_struct *task;
int ret;
- if (!cgroup_may_migrate_to(dst_cgrp))
- return -EBUSY;
-
/* look up all src csets */
spin_lock_irq(&css_set_lock);
rcu_read_lock();
@@ -2572,6 +2574,15 @@ static int cgroup_subtree_control_show(struct seq_file *seq, void *v)
return 0;
}
+/* show controlllers that have resource control knobs in resource_domain */
+static int cgroup_resource_control_show(struct seq_file *seq, void *v)
+{
+ struct cgroup *cgrp = seq_css(seq)->cgroup;
+
+ cgroup_print_ss_mask(seq, cgrp->resource_control);
+ return 0;
+}
+
/**
* cgroup_update_dfl_csses - update css assoc of a subtree in default hierarchy
* @cgrp: root of the subtree to update csses for
@@ -2921,33 +2932,30 @@ static ssize_t cgroup_subtree_control_write(struct kernfs_open_file *of,
if (!cgrp)
return -ENODEV;
- for_each_subsys(ss, ssid) {
- if (enable & (1 << ssid)) {
- if (cgrp->subtree_control & (1 << ssid)) {
- enable &= ~(1 << ssid);
- continue;
- }
-
- if (!(cgroup_control(cgrp) & (1 << ssid))) {
- ret = -ENOENT;
- goto out_unlock;
- }
- } else if (disable & (1 << ssid)) {
- if (!(cgrp->subtree_control & (1 << ssid))) {
- disable &= ~(1 << ssid);
- continue;
- }
-
- /* a child has it enabled? */
- cgroup_for_each_live_child(child, cgrp) {
- if (child->subtree_control & (1 << ssid)) {
- ret = -EBUSY;
- goto out_unlock;
- }
- }
+ /*
+ * We cannot disable controllers that are enabled in a child
+ * cgroup.
+ */
+ if (disable) {
+ u16 child_enable = cgrp->resource_control;
+
+ cgroup_for_each_live_child(child, cgrp)
+ child_enable |= child->subtree_control|
+ child->resource_control;
+ if (disable & child_enable) {
+ ret = -EBUSY;
+ goto out_unlock;
}
}
+ if (enable & ~cgroup_control(cgrp)) {
+ ret = -ENOENT;
+ goto out_unlock;
+ }
+
+ enable &= ~cgrp->subtree_control;
+ disable &= cgrp->subtree_control;
+
if (!enable && !disable) {
ret = 0;
goto out_unlock;
@@ -2959,45 +2967,116 @@ static ssize_t cgroup_subtree_control_write(struct kernfs_open_file *of,
goto out_unlock;
}
+ /* save and update control masks and prepare csses */
+ cgroup_save_control(cgrp);
+
+ cgrp->subtree_control |= enable;
+ cgrp->subtree_control &= ~disable;
+
+ ret = cgroup_apply_control(cgrp);
+
+ cgroup_finalize_control(cgrp, ret);
+
+ kernfs_activate(cgrp->kn);
+ ret = 0;
+out_unlock:
+ cgroup_kn_unlock(of->kn);
+ return ret ?: nbytes;
+}
+
+/*
+ * Change the list of resource domain controllers for a cgroup in the
+ * default hierarchy
+ */
+static ssize_t cgroup_resource_control_write(struct kernfs_open_file *of,
+ char *buf, size_t nbytes,
+ loff_t off)
+{
+ u16 enable = 0, disable = 0;
+ struct cgroup *cgrp;
+ struct cgroup_subsys *ss;
+ char *tok;
+ int ssid, ret;
+
/*
- * Except for root, thread roots and threaded cgroups, subtree_control
- * must be zero for a cgroup with tasks so that child cgroups don't
- * compete against tasks.
+ * Parse input - space separated list of subsystem names prefixed
+ * with either + or -.
*/
- if (enable && cgroup_parent(cgrp) && !cgrp->proc_cgrp) {
- struct cgrp_cset_link *link;
-
- /*
- * Because namespaces pin csets too, @cgrp->cset_links
- * might not be empty even when @cgrp is empty. Walk and
- * verify each cset.
- */
- spin_lock_irq(&css_set_lock);
+ buf = strstrip(buf);
+ while ((tok = strsep(&buf, " "))) {
+ if (tok[0] == '\0')
+ continue;
+ do_each_subsys_mask(ss, ssid, ~cgrp_dfl_inhibit_ss_mask) {
+ if (!cgroup_ssid_enabled(ssid) ||
+ strcmp(tok + 1, ss->name))
+ continue;
- ret = 0;
- list_for_each_entry(link, &cgrp->cset_links, cset_link) {
- if (css_set_populated(link->cset)) {
- ret = -EBUSY;
- break;
+ if (*tok == '+') {
+ enable |= 1 << ssid;
+ disable &= ~(1 << ssid);
+ } else if (*tok == '-') {
+ disable |= 1 << ssid;
+ enable &= ~(1 << ssid);
+ } else {
+ return -EINVAL;
}
- }
+ break;
+ } while_each_subsys_mask();
+ if (ssid == CGROUP_SUBSYS_COUNT)
+ return -EINVAL;
+ }
- spin_unlock_irq(&css_set_lock);
+ cgrp = cgroup_kn_lock_live(of->kn, true);
+ if (!cgrp)
+ return -ENODEV;
- if (ret)
- goto out_unlock;
+ /*
+ * All the enabled or disabled controllers must have been enabled
+ * in the current cgroup.
+ */
+ if ((cgroup_control(cgrp) & (enable|disable)) != (enable|disable)) {
+ ret = -ENOENT;
+ goto out_unlock;
}
+ /*
+ * Clear bits that are currently enabled and disabled in
+ * resource_control.
+ */
+ enable &= ~cgrp->resource_control;
+ disable &= cgrp->resource_control;
+
+ if (!enable && !disable) {
+ ret = 0;
+ goto out_unlock;
+ }
+
+ /*
+ * Create a new child resource domain cgroup if necessary.
+ */
+ if (!cgrp->resource_domain && enable)
+ cgroup_mkdir(cgrp->kn, NULL, 0755);
+
+ cgrp->resource_control &= ~disable;
+ cgrp->resource_control |= enable;
+
/* save and update control masks and prepare csses */
cgroup_save_control(cgrp);
- cgrp->subtree_control |= enable;
- cgrp->subtree_control &= ~disable;
-
ret = cgroup_apply_control(cgrp);
cgroup_finalize_control(cgrp, ret);
+ /*
+ * Destroy the child resource domain cgroup if no controllers are
+ * enabled in the resource_control.
+ */
+ if (!cgrp->resource_control) {
+ struct cgroup *rdomain = cgrp->resource_domain;
+
+ cgrp->resource_domain = NULL;
+ cgroup_destroy_locked(rdomain);
+ }
kernfs_activate(cgrp->kn);
ret = 0;
out_unlock:
@@ -4303,6 +4382,11 @@ static ssize_t cgroup_threads_write(struct kernfs_open_file *of,
.write = cgroup_subtree_control_write,
},
{
+ .name = "cgroup.resource_control",
+ .seq_show = cgroup_resource_control_show,
+ .write = cgroup_resource_control_write,
+ },
+ {
.name = "cgroup.events",
.flags = CFTYPE_NOT_ON_ROOT,
.file_offset = offsetof(struct cgroup, events_file),
@@ -4661,25 +4745,49 @@ static struct cgroup *cgroup_create(struct cgroup *parent)
return ERR_PTR(ret);
}
+/*
+ * The name parameter will be NULL if called internally for creating the
+ * special resource domain cgroup. In this case, the cgroup_mutex will be
+ * held and there is no need to acquire or release it.
+ */
int cgroup_mkdir(struct kernfs_node *parent_kn, const char *name, umode_t mode)
{
struct cgroup *parent, *cgrp;
struct kernfs_node *kn;
+ bool create_rd = (name == NULL);
int ret;
- /* do not accept '\n' to prevent making /proc/<pid>/cgroup unparsable */
- if (strchr(name, '\n'))
- return -EINVAL;
+ /*
+ * Do not accept '\n' to prevent making /proc/<pid>/cgroup unparsable.
+ * The reserved resource domain directory name cannot be used. A
+ * sub-directory cannot be created under a resource domain directory.
+ */
+ if (create_rd) {
+ lockdep_assert_held(&cgroup_mutex);
+ name = CGROUP_RESOURCE_DOMAIN;
+ parent = parent_kn->priv;
+ } else {
+ if (strchr(name, '\n') || !strcmp(name, CGROUP_RESOURCE_DOMAIN))
+ return -EINVAL;
- parent = cgroup_kn_lock_live(parent_kn, false);
- if (!parent)
- return -ENODEV;
+ parent = cgroup_kn_lock_live(parent_kn, false);
+ if (!parent)
+ return -ENODEV;
+ if (test_bit(CGRP_RESOURCE_DOMAIN, &parent->flags)) {
+ ret = -EINVAL;
+ goto out_unlock;
+ }
+ }
cgrp = cgroup_create(parent);
if (IS_ERR(cgrp)) {
ret = PTR_ERR(cgrp);
goto out_unlock;
}
+ if (create_rd) {
+ parent->resource_domain = cgrp;
+ set_bit(CGRP_RESOURCE_DOMAIN, &cgrp->flags);
+ }
/* create the directory */
kn = kernfs_create_dir(parent->kn, name, mode, cgrp);
@@ -4699,9 +4807,11 @@ int cgroup_mkdir(struct kernfs_node *parent_kn, const char *name, umode_t mode)
if (ret)
goto out_destroy;
- ret = css_populate_dir(&cgrp->self);
- if (ret)
- goto out_destroy;
+ if (!create_rd) {
+ ret = css_populate_dir(&cgrp->self);
+ if (ret)
+ goto out_destroy;
+ }
ret = cgroup_apply_control_enable(cgrp);
if (ret)
@@ -4718,7 +4828,8 @@ int cgroup_mkdir(struct kernfs_node *parent_kn, const char *name, umode_t mode)
out_destroy:
cgroup_destroy_locked(cgrp);
out_unlock:
- cgroup_kn_unlock(parent_kn);
+ if (!create_rd)
+ cgroup_kn_unlock(parent_kn);
return ret;
}
@@ -4893,7 +5004,15 @@ int cgroup_rmdir(struct kernfs_node *kn)
if (!cgrp)
return 0;
- ret = cgroup_destroy_locked(cgrp);
+ /*
+ * A resource domain cgroup cannot be removed directly by users.
+ * It can only be done indirectly by writing to the "cgroup.resource"
+ * control file.
+ */
+ if (test_bit(CGRP_RESOURCE_DOMAIN, &cgrp->flags))
+ ret = -EINVAL;
+ else
+ ret = cgroup_destroy_locked(cgrp);
if (!ret)
trace_cgroup_rmdir(cgrp);
diff --git a/kernel/cgroup/debug.c b/kernel/cgroup/debug.c
index 3121811..b565951 100644
--- a/kernel/cgroup/debug.c
+++ b/kernel/cgroup/debug.c
@@ -237,8 +237,9 @@ static int cgroup_masks_read(struct seq_file *seq, void *v)
u16 *mask;
char *name;
} mask_list[] = {
- { &cgrp->subtree_control, "subtree_control" },
- { &cgrp->subtree_ss_mask, "subtree_ss_mask" },
+ { &cgrp->subtree_control, "subtree_control" },
+ { &cgrp->subtree_ss_mask, "subtree_ss_mask" },
+ { &cgrp->resource_control, "resource_control" },
};
mutex_lock(&cgroup_mutex);
@@ -246,7 +247,7 @@ static int cgroup_masks_read(struct seq_file *seq, void *v)
u16 mask = *mask_list[i].mask;
bool first = true;
- seq_printf(seq, "%-15s: ", mask_list[i].name);
+ seq_printf(seq, "%-16s: ", mask_list[i].name);
for_each_subsys(ss, j) {
if (!(mask & (1 << ss->id)))
continue;
--
1.8.3.1
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [RFC PATCH v2 13/17] cgroup: Allow fine-grained controllers control in cgroup v2
From: Waiman Long @ 2017-05-15 13:34 UTC (permalink / raw)
To: Tejun Heo, Li Zefan, Johannes Weiner, Peter Zijlstra, Ingo Molnar
Cc: cgroups, linux-kernel, linux-doc, linux-mm, kernel-team, pjt,
luto, efault, longman
In-Reply-To: <1494855256-12558-1-git-send-email-longman@redhat.com>
For cgroup v1, different controllers can be binded to different cgroup
hierarchies optimized for their own use cases. That is not currently
the case for cgroup v2 where combining all these controllers into
the same hierarchy will probably require more levels than is needed
by each individual controller.
By not enabling a controller in a cgroup and its descendants, we can
effectively trim the hierarchy as seen by a controller from the leafs
up. However, there is currently no way to compress the hierarchy in
the intermediate levels.
This patch implements a fine-grained mechanism to allow a controller to
skip some intermediate levels in a hierarchy and effectively flatten
the hierarchy as seen by that controller.
Controllers can now be directly enabled or disabled in a cgroup
by writing to the "cgroup.controllers" file. The special prefix
'#' with the controller name is used to set that controller in
pass-through mode. In that mode, the controller is disabled for that
cgroup but it allows its children to have that controller enabled or
in pass-through mode again.
With this change, each controller can now have a unique view of their
virtual process hierarchy that can be quite different from other
controllers. We now have the freedom and flexibility to create the
right hierarchy for each controller to suit their own needs without
performance loss when compared with cgroup v1.
Signed-off-by: Waiman Long <longman@redhat.com>
---
Documentation/cgroup-v2.txt | 125 ++++++++++++++++++---
include/linux/cgroup-defs.h | 11 ++
kernel/cgroup/cgroup.c | 263 ++++++++++++++++++++++++++++++++++++++------
kernel/cgroup/debug.c | 8 +-
4 files changed, 359 insertions(+), 48 deletions(-)
diff --git a/Documentation/cgroup-v2.txt b/Documentation/cgroup-v2.txt
index 0f41282..bb27491 100644
--- a/Documentation/cgroup-v2.txt
+++ b/Documentation/cgroup-v2.txt
@@ -308,25 +308,28 @@ both cgroups.
2-4-1. Enabling and Disabling
Each cgroup has a "cgroup.controllers" file which lists all
-controllers available for the cgroup to enable.
+controllers available for the cgroup to enable for its children.
# cat cgroup.controllers
cpu io memory
-No controller is enabled by default. Controllers can be enabled and
-disabled by writing to the "cgroup.subtree_control" file.
+No controller is enabled by default. Controllers can be
+enabled and disabled on the child cgroups by writing to the
+"cgroup.subtree_control" file. A '+' prefix enables the controller,
+and a '-' prefix disables it.
# echo "+cpu +memory -io" > cgroup.subtree_control
-Only controllers which are listed in "cgroup.controllers" can be
-enabled. When multiple operations are specified as above, either they
-all succeed or fail. If multiple operations on the same controller
-are specified, the last one is effective.
+Only controllers which are listed in "cgroup.controllers" can
+be enabled in the "cgroup.subtree_control" file. When multiple
+operations are specified as above, either they all succeed or fail.
+If multiple operations on the same controller are specified, the last
+one is effective.
Enabling a controller in a cgroup indicates that the distribution of
the target resource across its immediate children will be controlled.
-Consider the following sub-hierarchy. The enabled controllers are
-listed in parentheses.
+Consider the following sub-hierarchy. The enabled controllers in the
+"cgroup.subtree_control" file are listed in parentheses.
A(cpu,memory) - B(memory) - C()
\ D()
@@ -336,6 +339,17 @@ of CPU cycles and memory to its children, in this case, B. As B has
"memory" enabled but not "CPU", C and D will compete freely on CPU
cycles but their division of memory available to B will be controlled.
+By not enabling a controller in a cgroup and its descendants, we can
+effectively trim the hierarchy as seen by a controller from the leafs
+up. From the perspective of the cpu controller, the hierarchy is:
+
+ A - B|C|D
+
+From the perspective of the memory controller, the hierarchy becomes:
+
+ A - B - C
+ \ D
+
As a controller regulates the distribution of the target resource to
the cgroup's children, enabling it creates the controller's interface
files in the child cgroups. In the above example, enabling "cpu" on B
@@ -343,7 +357,81 @@ would create the "cpu." prefixed controller interface files in C and
D. Likewise, disabling "memory" from B would remove the "memory."
prefixed controller interface files from C and D. This means that the
controller interface files - anything which doesn't start with
-"cgroup." are owned by the parent rather than the cgroup itself.
+"cgroup." can be considered to be owned by the parent under this
+control scheme.
+
+Enabling controllers via the "cgroup.subtree_control" file is
+relatively coarse-grained. Fine-grained control of the controllers in
+a non-root cgroup can be done by writing to its "cgroup.controllers"
+file directly. A '+' prefix enables an controller as long as that
+controller is also enabled on its parent. Similarly, the '-' prefix
+disables a controller as long that controller isn't enabled in its
+parent's subtree_control file.
+
+The special prefix '#' is used to mark a controller in pass-through
+mode. In this mode, the controller is disabled in the cgroup
+effectively collapsing it with its parent from the perspective of
+that controller. However, it allows its child cgroups to enable the
+controller or have it in pass-through mode again. For example,
+
+ + # # # +
+ A - B - C - D - E
+ \ F
+ +
+In this case, the effective hiearchy is:
+
+ A|B|C|D - E
+ \ F
+
+Under this control scheme, the interface files can be considered to be
+owned by the cgroup itself. The use of the special '#' prefix allows
+the users to trim away layers in the middle of the hierarchy, thus
+flattening the tree from the perspective of that particular controller.
+As a result, different controllers can have quite different views of
+their virtual process hierarchy that can best fit their own needs.
+
+In the diagram below, the controller name in the parenthesis represents
+controller enabled by writing to the "cgroup.controllers" file.
+
+ A(cpu,memory) - B(cpu,#memory) - C()
+ \ D(memory)
+
+From the memory controller's perspective, the hierarchy looks like:
+
+ A|B|C - D
+
+For the CPU controller, the hierarchy is:
+
+ A - B|C|D
+
+Both control schemes can be used together with some limitations
+as shown in the following table about the interaction between
+subtree_control file of the parent of a cgroup and its controllers
+file.
+
+ ++: enable a controller in parent's subtree_control
+ --: disable a controller in parent's subtree_control
+ +: enable a controller in controllers
+ -: disable a controller in controllers
+ #: skip a controller in controllers
+
+ Old State New Desired State Result
+ --------- ----------------- ------
+ ++ + Ignored
+ ++ # Accepted
+ ++ - Rejected*
+ -- + or # Accepted
+ -- - Ignored
+ + ++ Accepted
+ + -- Rejected
+ - ++ or -- Accepted
+ # ++ or -- Rejected
+
+In the special case that the cgroup is in both '++' & '#' states
+('++' followed by '#'), the '-' prefix can be used to turn off the
+'#' leading to an effective '++' state. A cgroup in '+' or '#' state
+cannot be changed back to '-' or switched to each other as long as
+its children have that controller in a non-'-' state.
2-4-2. Top-down Constraint
@@ -353,8 +441,8 @@ a resource only if the resource has been distributed to it from the
parent. This means that all non-root "cgroup.subtree_control" files
can only contain controllers which are enabled in the parent's
"cgroup.subtree_control" file. A controller can be enabled only if
-the parent has the controller enabled and a controller can't be
-disabled if one or more children have it enabled.
+the parent has the controller enabled ('+' or '#') and a controller
+can't be disabled if one or more children have it enabled.
2-4-3. Managing Internal Process Competition
@@ -704,11 +792,18 @@ All cgroup core files are prefixed with "cgroup."
cgroup.controllers
- A read-only space separated values file which exists on all
+ A read-write space separated values file which exists on all
cgroups.
- It shows space separated list of all controllers available to
- the cgroup. The controllers are not ordered.
+ When read, it shows space separated list of all controllers
+ available to the cgroup. The controllers are not ordered.
+
+ Space separated list of controllers prefixed with '+', '-' or
+ '#' can be written to enable, disable or set the controllers
+ in pass-through mode. If a controller appears more than once
+ on the list, the last one is effective. When multiple enable
+ and disable operations are specified, either all succeed or
+ all fail.
cgroup.subtree_control
diff --git a/include/linux/cgroup-defs.h b/include/linux/cgroup-defs.h
index 67ab326..5d30182 100644
--- a/include/linux/cgroup-defs.h
+++ b/include/linux/cgroup-defs.h
@@ -305,6 +305,17 @@ struct cgroup {
*/
u16 resource_control;
+ /*
+ * The bitmasks of subsystems enabled and in pass-through mode in
+ * the current cgroup. The parent's subtree_ss_mask has priority.
+ * A bit set in subtree_ss_mask will suppress the setting of the
+ * corresponding bit in enable_ss_mask and passthru_ss_mask.
+ */
+ u16 enable_ss_mask;
+ u16 passthru_ss_mask;
+ u16 old_enable_ss_mask;
+ u16 old_passthru_ss_mask;
+
/* Private pointers for each registered subsystem */
struct cgroup_subsys_state __rcu *subsys[CGROUP_SUBSYS_COUNT];
diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c
index c3be7e2..6e77ebe 100644
--- a/kernel/cgroup/cgroup.c
+++ b/kernel/cgroup/cgroup.c
@@ -341,7 +341,7 @@ static u16 cgroup_control(struct cgroup *cgrp)
u16 root_ss_mask = cgrp->root->subsys_mask;
if (parent) {
- u16 ss_mask = parent->subtree_control;
+ u16 ss_mask = parent->subtree_control|cgrp->enable_ss_mask;
if (test_bit(CGRP_RESOURCE_DOMAIN, &cgrp->flags))
return parent->resource_control;
@@ -363,7 +363,7 @@ static u16 cgroup_ss_mask(struct cgroup *cgrp)
struct cgroup *parent = cgroup_parent(cgrp);
if (parent) {
- u16 ss_mask = parent->subtree_ss_mask;
+ u16 ss_mask = parent->subtree_ss_mask|cgrp->enable_ss_mask;
if (test_bit(CGRP_RESOURCE_DOMAIN, &cgrp->flags))
return parent->resource_control;
@@ -2540,15 +2540,18 @@ void cgroup_procs_write_finish(struct task_struct *task)
ss->post_attach();
}
-static void cgroup_print_ss_mask(struct seq_file *seq, u16 ss_mask)
+static void cgroup_print_ss_mask(struct seq_file *seq, u16 ss_mask,
+ u16 passthru_mask)
{
struct cgroup_subsys *ss;
bool printed = false;
int ssid;
- do_each_subsys_mask(ss, ssid, ss_mask) {
+ do_each_subsys_mask(ss, ssid, ss_mask|passthru_mask) {
if (printed)
seq_putc(seq, ' ');
+ if (passthru_mask & (1 << ssid))
+ seq_putc(seq, '#');
seq_printf(seq, "%s", ss->name);
printed = true;
} while_each_subsys_mask();
@@ -2561,7 +2564,7 @@ static int cgroup_controllers_show(struct seq_file *seq, void *v)
{
struct cgroup *cgrp = seq_css(seq)->cgroup;
- cgroup_print_ss_mask(seq, cgroup_control(cgrp));
+ cgroup_print_ss_mask(seq, cgroup_control(cgrp), cgrp->passthru_ss_mask);
return 0;
}
@@ -2570,7 +2573,7 @@ static int cgroup_subtree_control_show(struct seq_file *seq, void *v)
{
struct cgroup *cgrp = seq_css(seq)->cgroup;
- cgroup_print_ss_mask(seq, cgrp->subtree_control);
+ cgroup_print_ss_mask(seq, cgrp->subtree_control, 0);
return 0;
}
@@ -2579,7 +2582,7 @@ static int cgroup_resource_control_show(struct seq_file *seq, void *v)
{
struct cgroup *cgrp = seq_css(seq)->cgroup;
- cgroup_print_ss_mask(seq, cgrp->resource_control);
+ cgroup_print_ss_mask(seq, cgrp->resource_control, 0);
return 0;
}
@@ -2692,6 +2695,8 @@ static void cgroup_save_control(struct cgroup *cgrp)
cgroup_for_each_live_descendant_pre(dsct, d_css, cgrp) {
dsct->old_subtree_control = dsct->subtree_control;
dsct->old_subtree_ss_mask = dsct->subtree_ss_mask;
+ dsct->old_enable_ss_mask = dsct->enable_ss_mask;
+ dsct->old_passthru_ss_mask = dsct->passthru_ss_mask;
}
}
@@ -2709,10 +2714,11 @@ static void cgroup_propagate_control(struct cgroup *cgrp)
struct cgroup_subsys_state *d_css;
cgroup_for_each_live_descendant_pre(dsct, d_css, cgrp) {
- dsct->subtree_control &= cgroup_control(dsct);
+ dsct->subtree_control &= cgroup_control(dsct)|
+ dsct->passthru_ss_mask;
dsct->subtree_ss_mask =
cgroup_calc_subtree_ss_mask(dsct->subtree_control,
- cgroup_ss_mask(dsct));
+ cgroup_ss_mask(dsct)|dsct->passthru_ss_mask);
}
}
@@ -2731,6 +2737,8 @@ static void cgroup_restore_control(struct cgroup *cgrp)
cgroup_for_each_live_descendant_post(dsct, d_css, cgrp) {
dsct->subtree_control = dsct->old_subtree_control;
dsct->subtree_ss_mask = dsct->old_subtree_ss_mask;
+ dsct->enable_ss_mask = dsct->old_enable_ss_mask;
+ dsct->passthru_ss_mask = dsct->old_passthru_ss_mask;
}
}
@@ -2772,7 +2780,8 @@ static int cgroup_apply_control_enable(struct cgroup *cgrp)
WARN_ON_ONCE(css && percpu_ref_is_dying(&css->refcnt));
- if (!(cgroup_ss_mask(dsct) & (1 << ss->id)))
+ if (!(cgroup_ss_mask(dsct) & (1 << ss->id)) ||
+ (dsct->passthru_ss_mask & (1 << ss->id)))
continue;
if (!css) {
@@ -2822,7 +2831,8 @@ static void cgroup_apply_control_disable(struct cgroup *cgrp)
continue;
if (css->parent &&
- !(cgroup_ss_mask(dsct) & (1 << ss->id))) {
+ (!(cgroup_ss_mask(dsct) & (1 << ss->id)) ||
+ (dsct->passthru_ss_mask & (1 << ss->id)))) {
kill_css(css);
} else if (!css_visible(css)) {
css_clear_dir(css);
@@ -2895,7 +2905,8 @@ static ssize_t cgroup_subtree_control_write(struct kernfs_open_file *of,
loff_t off)
{
u16 enable = 0, disable = 0;
- struct cgroup *cgrp, *child;
+ u16 child_enable, child_passthru = 0;
+ struct cgroup *cgrp, *child, *grandchild;
struct cgroup_subsys *ss;
char *tok;
int ssid, ret;
@@ -2933,22 +2944,36 @@ static ssize_t cgroup_subtree_control_write(struct kernfs_open_file *of,
return -ENODEV;
/*
- * We cannot disable controllers that are enabled in a child
- * cgroup.
+ * Because a controller can be enabled on a grandchild if it is
+ * enabled in subtree_control, we need to look at all the children
+ * and grandchildren for what are enabled.
*/
- if (disable) {
- u16 child_enable = cgrp->resource_control;
+ child_enable = cgrp->resource_control;
+ cgroup_for_each_live_child(child, cgrp) {
+ child_enable |= child->subtree_control|
+ child->resource_control|
+ child->enable_ss_mask;
+ child_passthru |= child->passthru_ss_mask;
+
+ cgroup_for_each_live_child(grandchild, child)
+ child_enable |= grandchild->subtree_control|
+ grandchild->resource_control|
+ grandchild->enable_ss_mask|
+ grandchild->passthru_ss_mask;
+ }
- cgroup_for_each_live_child(child, cgrp)
- child_enable |= child->subtree_control|
- child->resource_control;
- if (disable & child_enable) {
- ret = -EBUSY;
- goto out_unlock;
- }
+ /*
+ * We cannot disable controllers that are enabled or in pass-through
+ * mode in a child or grandchild cgroup. We also cannot enable
+ * controllers that are in pass-through mode in a child cgroup.
+ */
+ if ((disable & (child_enable|child_passthru)) ||
+ (enable & child_passthru)) {
+ ret = -EBUSY;
+ goto out_unlock;
}
- if (enable & ~cgroup_control(cgrp)) {
+ if (enable & ~(cgroup_control(cgrp)|cgrp->passthru_ss_mask)) {
ret = -ENOENT;
goto out_unlock;
}
@@ -2963,7 +2988,7 @@ static ssize_t cgroup_subtree_control_write(struct kernfs_open_file *of,
/* can't enable !threaded controllers on a threaded cgroup */
if (cgroup_is_threaded(cgrp) && (enable & ~cgrp_dfl_threaded_ss_mask)) {
- ret = -EBUSY;
+ ret = -EINVAL;
goto out_unlock;
}
@@ -2973,6 +2998,164 @@ static ssize_t cgroup_subtree_control_write(struct kernfs_open_file *of,
cgrp->subtree_control |= enable;
cgrp->subtree_control &= ~disable;
+ /*
+ * Clear the child's enable_ss_mask for those bits that are enabled
+ * in subtree_control.
+ */
+ if (child_enable & enable) {
+ cgroup_for_each_live_child(child, cgrp)
+ child->enable_ss_mask &= ~enable;
+ }
+
+ ret = cgroup_apply_control(cgrp);
+
+ cgroup_finalize_control(cgrp, ret);
+
+ kernfs_activate(cgrp->kn);
+ ret = 0;
+out_unlock:
+ cgroup_kn_unlock(of->kn);
+ return ret ?: nbytes;
+}
+
+/*
+ * Change the enabled and pass-through controllers for a cgroup in the
+ * default hierarchy
+ */
+static ssize_t cgroup_controllers_write(struct kernfs_open_file *of,
+ char *buf, size_t nbytes,
+ loff_t off)
+{
+ u16 enable = 0, disable = 0, passthru = 0;
+ u16 child_enable, parent_subtree;
+ struct cgroup *cgrp, *child, *parent;
+ struct cgroup_subsys *ss;
+ char *tok;
+ int ssid, ret;
+
+ /*
+ * Parse input - space separated list of subsystem names prefixed
+ * with either +, - or #.
+ */
+ buf = strstrip(buf);
+ while ((tok = strsep(&buf, " "))) {
+ if (tok[0] == '\0')
+ continue;
+ do_each_subsys_mask(ss, ssid, ~cgrp_dfl_inhibit_ss_mask) {
+ if (!cgroup_ssid_enabled(ssid) ||
+ strcmp(tok + 1, ss->name))
+ continue;
+
+ if (*tok == '+') {
+ enable |= 1 << ssid;
+ disable &= ~(1 << ssid);
+ passthru &= ~(1 << ssid);
+ } else if (*tok == '-') {
+ disable |= 1 << ssid;
+ enable &= ~(1 << ssid);
+ passthru &= ~(1 << ssid);
+ } else if (*tok == '#') {
+ passthru |= 1 << ssid;
+ enable &= ~(1 << ssid);
+ disable &= ~(1 << ssid);
+ } else {
+ return -EINVAL;
+ }
+ break;
+ } while_each_subsys_mask();
+ if (ssid == CGROUP_SUBSYS_COUNT)
+ return -EINVAL;
+ }
+
+ cgrp = cgroup_kn_lock_live(of->kn, true);
+ if (!cgrp)
+ return -ENODEV;
+
+ /*
+ * Write to root cgroup's controllers file is not allowed.
+ */
+ parent = cgroup_parent(cgrp);
+ if (!parent) {
+ ret = -EINVAL;
+ goto out_unlock;
+ }
+
+ /*
+ * We only looks at parent's subtree_control that are not in
+ * passthru_ss_mask.
+ */
+ parent_subtree = parent->subtree_control & ~cgrp->passthru_ss_mask;
+
+ /*
+ * Reject disable bits that are in parent's subtree_control except
+ * when they are also in passthru_ss_mask.
+ */
+ if (disable & parent_subtree) {
+ ret = -EINVAL;
+ goto out_unlock;
+ }
+
+ child_enable = cgrp->resource_control|cgrp->subtree_control;
+ cgroup_for_each_live_child(child, cgrp)
+ child_enable |= child->subtree_control|child->resource_control|
+ child->enable_ss_mask|child->passthru_ss_mask;
+
+ /*
+ * Mask off bits that have been set as well as enable bits set
+ * in parent's subtree_control, but not in passthru_ss_mask.
+ */
+ passthru &= ~cgrp->passthru_ss_mask;
+ enable &= ~(cgrp->enable_ss_mask|parent_subtree);
+
+ /*
+ * We cannot enable, disable or pass-through controllers that
+ * are enabled in children's passthru_ss_mask, enable_ss_mask,
+ * resource_control or subtree_control as well as its own
+ * resource_control and subtree_control.
+ */
+ if ((disable|passthru|enable) & child_enable) {
+ ret = -EBUSY;
+ goto out_unlock;
+ }
+
+ /*
+ * We also cannot enable or pass through controllers that are not
+ * enabled in its parent's passthru_ss_mask or controllers.
+ */
+ if (((enable|passthru) & (parent->passthru_ss_mask|
+ cgroup_control(parent)))
+ != (enable|passthru)) {
+ ret = -ENOENT;
+ goto out_unlock;
+ }
+
+ disable &= cgrp->enable_ss_mask|cgrp->passthru_ss_mask;
+ if (!enable && !disable && !passthru) {
+ ret = 0;
+ goto out_unlock;
+ }
+
+ /*
+ * Can't enable or pass through !threaded controllers on a
+ * threaded cgroup
+ */
+ if (cgroup_is_threaded(cgrp) &&
+ ((enable|passthru) & ~cgrp_dfl_threaded_ss_mask)) {
+ ret = -EINVAL;
+ goto out_unlock;
+ }
+
+ /* Save and update control masks and prepare csses */
+ cgroup_save_control(cgrp);
+
+ cgrp->passthru_ss_mask |= passthru;
+ cgrp->passthru_ss_mask &= ~(disable|enable);
+
+ /* Mask off enable bits set in parent's subtree_control */
+ enable &= ~parent->subtree_control;
+ cgrp->enable_ss_mask |= enable;
+ cgrp->enable_ss_mask &= ~(disable|passthru);
+
ret = cgroup_apply_control(cgrp);
cgroup_finalize_control(cgrp, ret);
@@ -3102,7 +3285,8 @@ static int cgroup_enable_threaded(struct cgroup *cgrp)
/*
* Allow only if it is not the root and there are:
* 1) no children,
- * 2) no non-threaded controllers are enabled, and
+ * 2) no non-threaded controllers are enabled or in pass-through
+ * mode, and
* 3) no attached tasks.
*
* With no attached tasks, it is assumed that no css_sets will be
@@ -3110,7 +3294,8 @@ static int cgroup_enable_threaded(struct cgroup *cgrp)
* css_sets linger around due to task_struct leakage, for example.
*/
if (css_has_online_children(&cgrp->self) ||
- (cgroup_control(cgrp) & ~cgrp_dfl_threaded_ss_mask) ||
+ ((cgroup_control(cgrp)|cgrp->passthru_ss_mask)
+ & ~cgrp_dfl_threaded_ss_mask) ||
!cgroup_parent(cgrp) || cgroup_is_populated(cgrp))
return -EBUSY;
@@ -4375,6 +4560,7 @@ static ssize_t cgroup_threads_write(struct kernfs_open_file *of,
{
.name = "cgroup.controllers",
.seq_show = cgroup_controllers_show,
+ .write = cgroup_controllers_write,
},
{
.name = "cgroup.subtree_control",
@@ -4526,7 +4712,8 @@ static void css_release(struct percpu_ref *ref)
}
static void init_and_link_css(struct cgroup_subsys_state *css,
- struct cgroup_subsys *ss, struct cgroup *cgrp)
+ struct cgroup_subsys *ss, struct cgroup *cgrp,
+ struct cgroup_subsys_state *parent_css)
{
lockdep_assert_held(&cgroup_mutex);
@@ -4542,7 +4729,7 @@ static void init_and_link_css(struct cgroup_subsys_state *css,
atomic_set(&css->online_cnt, 0);
if (cgroup_parent(cgrp)) {
- css->parent = cgroup_css(cgroup_parent(cgrp), ss);
+ css->parent = parent_css;
css_get(css->parent);
}
@@ -4605,19 +4792,31 @@ static struct cgroup_subsys_state *css_create(struct cgroup *cgrp,
struct cgroup_subsys *ss)
{
struct cgroup *parent = cgroup_parent(cgrp);
- struct cgroup_subsys_state *parent_css = cgroup_css(parent, ss);
+ struct cgroup_subsys_state *parent_css;
struct cgroup_subsys_state *css;
int err;
lockdep_assert_held(&cgroup_mutex);
+ /*
+ * Need to skip over ancestor cgroups with skip flag set.
+ */
+ while (parent && (parent->passthru_ss_mask & (1 << ss->id)))
+ parent = cgroup_parent(parent);
+
+ if (!parent) {
+ WARN_ON_ONCE(1);
+ return ERR_PTR(-EINVAL);
+ }
+ parent_css = cgroup_css(parent, ss);
+
css = ss->css_alloc(parent_css);
if (!css)
css = ERR_PTR(-ENOMEM);
if (IS_ERR(css))
return css;
- init_and_link_css(css, ss, cgrp);
+ init_and_link_css(css, ss, cgrp, parent_css);
err = percpu_ref_init(&css->refcnt, css_release, 0, GFP_KERNEL);
if (err)
@@ -5044,7 +5243,7 @@ static void __init cgroup_init_subsys(struct cgroup_subsys *ss, bool early)
css = ss->css_alloc(cgroup_css(&cgrp_dfl_root.cgrp, ss));
/* We don't handle early failures gracefully */
BUG_ON(IS_ERR(css));
- init_and_link_css(css, ss, &cgrp_dfl_root.cgrp);
+ init_and_link_css(css, ss, &cgrp_dfl_root.cgrp, NULL);
/*
* Root csses are never destroyed and we can't initialize
diff --git a/kernel/cgroup/debug.c b/kernel/cgroup/debug.c
index b565951..a2dbf77 100644
--- a/kernel/cgroup/debug.c
+++ b/kernel/cgroup/debug.c
@@ -212,8 +212,12 @@ static int cgroup_subsys_states_read(struct seq_file *seq, void *v)
mutex_lock(&cgroup_mutex);
for_each_subsys(ss, i) {
css = rcu_dereference_check(cgrp->subsys[ss->id], true);
- if (!css)
+ if (!css) {
+ if (cgrp->passthru_ss_mask & (1 << ss->id))
+ seq_printf(seq, "%2d: %-4s\t- [Pass-through]\n",
+ ss->id, ss->name);
continue;
+ }
pbuf[0] = '\0';
/* Show the parent CSS if applicable*/
@@ -240,6 +244,8 @@ static int cgroup_masks_read(struct seq_file *seq, void *v)
{ &cgrp->subtree_control, "subtree_control" },
{ &cgrp->subtree_ss_mask, "subtree_ss_mask" },
{ &cgrp->resource_control, "resource_control" },
+ { &cgrp->enable_ss_mask, "enable_ss_mask" },
+ { &cgrp->passthru_ss_mask, "passthru_ss_mask" },
};
mutex_lock(&cgroup_mutex);
--
1.8.3.1
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [RFC PATCH v2 14/17] cgroup: Enable printing of v2 controllers' cgroup hierarchy
From: Waiman Long @ 2017-05-15 13:34 UTC (permalink / raw)
To: Tejun Heo, Li Zefan, Johannes Weiner, Peter Zijlstra, Ingo Molnar
Cc: cgroups, linux-kernel, linux-doc, linux-mm, kernel-team, pjt,
luto, efault, longman
In-Reply-To: <1494855256-12558-1-git-send-email-longman@redhat.com>
This patch add a new debug control file on the cgroup v2 root directory
to print out the cgroup hierarchy for each of the v2 controllers.
Signed-off-by: Waiman Long <longman@redhat.com>
---
kernel/cgroup/debug.c | 141 ++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 141 insertions(+)
diff --git a/kernel/cgroup/debug.c b/kernel/cgroup/debug.c
index a2dbf77..3adb26a 100644
--- a/kernel/cgroup/debug.c
+++ b/kernel/cgroup/debug.c
@@ -268,6 +268,141 @@ static int cgroup_masks_read(struct seq_file *seq, void *v)
return 0;
}
+/*
+ * Print out all the child cgroup names that doesn't have a css for the
+ * corresponding cgroup_subsys. If a child cgroup has a css, put that into
+ * the given cglist to be processed in the next iteration.
+ */
+#define CGLIST_MAX 16
+static void print_hierarchy(struct seq_file *seq,
+ struct cgroup *cgrp,
+ struct cgroup_subsys *ss,
+ struct cgroup_subsys_state *css,
+ struct cgroup **cglist,
+ int *cgcnt)
+{
+ struct cgroup *child;
+ struct cgroup_subsys_state *child_css;
+ char cgname[64];
+
+ cgname[sizeof(cgname) - 1] = '\0';
+ /*
+ * Iterate all live children of the given cgroup.
+ */
+ list_for_each_entry(child, &cgrp->self.children, self.sibling) {
+ if (cgroup_is_dead(child))
+ continue;
+
+ child_css = rcu_dereference_check(child->subsys[ss->id], true);
+ if (child_css) {
+ WARN_ON(child_css->parent != css);
+ if (*cgcnt < CGLIST_MAX) {
+ cglist[*cgcnt] = child;
+ (*cgcnt)++;
+ }
+ continue;
+ }
+
+ /*
+ * Skip resource domain cgroup
+ */
+ if (test_bit(CGRP_RESOURCE_DOMAIN, &child->flags))
+ continue;
+
+ cgroup_name(child, cgname, sizeof(cgname)-1);
+ seq_putc(seq, ',');
+ seq_puts(seq, cgname);
+ print_hierarchy(seq, child, ss, css, cglist, cgcnt);
+ }
+}
+
+/*
+ * Print the hierachies with respect to each controller for the default
+ * hierarchy.
+ *
+ * Each child level is printed on a separate line. Set of cgroups that
+ * have the same css will be grouped together and separated by comma.
+ * Process in those cgroups will be in the same node (css) in the
+ * controller's hierarchy. There is an exception that for resource
+ * domain cgroup, the processes associated with its parent and its
+ * affiliates will be mapped to the css of that resource domain cgroup
+ * instead.
+ *
+ * If there are more than CGLIST_MAX sets of cgroups in each level,
+ * the extra ones will be skipped.
+ */
+static int controller_hierachies_read(struct seq_file *seq, void *v)
+{
+ struct cgroup *root = seq_css(seq)->cgroup;
+ struct cgroup_subsys *ss;
+ struct cgroup_subsys_state *css;
+ struct cgroup *cgrp;
+ struct cgroup *cglist[CGLIST_MAX];
+ struct cgroup *cg2list[CGLIST_MAX];
+ int i, idx, cgnum, cg2num;
+ char cgname[64];
+
+ cgname[sizeof(cgname) - 1] = '\0';
+ mutex_lock(&cgroup_mutex);
+ for_each_subsys(ss, i) {
+ if (!(root->root->subsys_mask & (1 << ss->id)))
+ continue;
+ seq_puts(seq, ss->name);
+ seq_puts(seq, ":\n");
+
+ cgnum = 1;
+ cg2num = 0;
+ cglist[0] = root;
+ idx = 0;
+ while (cgnum) {
+ if (idx)
+ seq_putc(seq, ' ');
+ cgrp = cglist[idx];
+ if (test_bit(CGRP_RESOURCE_DOMAIN, &cgrp->flags)) {
+ struct cgroup *parent;
+
+ parent = container_of(cgrp->self.parent,
+ struct cgroup, self);
+ cgroup_name(parent, cgname, sizeof(cgname)-1);
+ seq_printf(seq, "%s.rd", cgname);
+ } else {
+ cgroup_name(cgrp, cgname, sizeof(cgname)-1);
+ seq_puts(seq, cgname);
+ }
+ css = rcu_dereference_check(cgrp->subsys[ss->id], true);
+ WARN_ON(!css);
+
+ if (cgrp == root)
+ seq_printf(seq, "[%d]", css->id);
+ else
+ seq_printf(seq, "[%d:P=%d]", css->id,
+ css->parent->id);
+
+ /*
+ * List all the cgroups that use the current
+ * css.
+ */
+ print_hierarchy(seq, cgrp, ss, css, cg2list, &cg2num);
+
+ if (++idx < cgnum)
+ continue;
+
+ /*
+ * Move cg2list to cglist.
+ */
+ cgnum = cg2num;
+ idx = cg2num = 0;
+ if (cgnum)
+ memcpy(cglist, cg2list,
+ cgnum * sizeof(cglist[0]));
+ seq_putc(seq, '\n');
+ }
+ seq_putc(seq, '\n');
+ }
+ mutex_unlock(&cgroup_mutex);
+ return 0;
+}
+
static u64 releasable_read(struct cgroup_subsys_state *css, struct cftype *cft)
{
return (!cgroup_is_populated(css->cgroup) &&
@@ -314,6 +449,12 @@ static u64 releasable_read(struct cgroup_subsys_state *css, struct cftype *cft)
},
{
+ .name = "controller_hierachies",
+ .seq_show = controller_hierachies_read,
+ .flags = CFTYPE_ONLY_ON_ROOT|__CFTYPE_ONLY_ON_DFL,
+ },
+
+ {
.name = "releasable",
.read_u64 = releasable_read,
},
--
1.8.3.1
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [RFC PATCH v2 15/17] sched: Misc preps for cgroup unified hierarchy interface
From: Waiman Long @ 2017-05-15 13:34 UTC (permalink / raw)
To: Tejun Heo, Li Zefan, Johannes Weiner, Peter Zijlstra, Ingo Molnar
Cc: cgroups, linux-kernel, linux-doc, linux-mm, kernel-team, pjt,
luto, efault, longman
In-Reply-To: <1494855256-12558-1-git-send-email-longman@redhat.com>
From: Tejun Heo <tj@kernel.org>
Make the following changes in preparation for the cpu controller
interface implementation for the unified hierarchy. This patch
doesn't cause any functional differences.
* s/cpu_stats_show()/cpu_cfs_stats_show()/
* s/cpu_files/cpu_legacy_files/
* Separate out cpuacct_stats_read() from cpuacct_stats_show(). While
at it, make the @val array u64 for consistency.
Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Li Zefan <lizefan@huawei.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
---
kernel/sched/core.c | 8 ++++----
kernel/sched/cpuacct.c | 29 ++++++++++++++++++-----------
2 files changed, 22 insertions(+), 15 deletions(-)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index c888bd3..be2527b 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -7230,7 +7230,7 @@ static int __cfs_schedulable(struct task_group *tg, u64 period, u64 quota)
return ret;
}
-static int cpu_stats_show(struct seq_file *sf, void *v)
+static int cpu_cfs_stats_show(struct seq_file *sf, void *v)
{
struct task_group *tg = css_tg(seq_css(sf));
struct cfs_bandwidth *cfs_b = &tg->cfs_bandwidth;
@@ -7270,7 +7270,7 @@ static u64 cpu_rt_period_read_uint(struct cgroup_subsys_state *css,
}
#endif /* CONFIG_RT_GROUP_SCHED */
-static struct cftype cpu_files[] = {
+static struct cftype cpu_legacy_files[] = {
#ifdef CONFIG_FAIR_GROUP_SCHED
{
.name = "shares",
@@ -7291,7 +7291,7 @@ static u64 cpu_rt_period_read_uint(struct cgroup_subsys_state *css,
},
{
.name = "stat",
- .seq_show = cpu_stats_show,
+ .seq_show = cpu_cfs_stats_show,
},
#endif
#ifdef CONFIG_RT_GROUP_SCHED
@@ -7317,7 +7317,7 @@ struct cgroup_subsys cpu_cgrp_subsys = {
.fork = cpu_cgroup_fork,
.can_attach = cpu_cgroup_can_attach,
.attach = cpu_cgroup_attach,
- .legacy_cftypes = cpu_files,
+ .legacy_cftypes = cpu_legacy_files,
.early_init = true,
};
diff --git a/kernel/sched/cpuacct.c b/kernel/sched/cpuacct.c
index f95ab29..6151c23 100644
--- a/kernel/sched/cpuacct.c
+++ b/kernel/sched/cpuacct.c
@@ -276,26 +276,33 @@ static int cpuacct_all_seq_show(struct seq_file *m, void *V)
return 0;
}
-static int cpuacct_stats_show(struct seq_file *sf, void *v)
+static void cpuacct_stats_read(struct cpuacct *ca,
+ u64 (*val)[CPUACCT_STAT_NSTATS])
{
- struct cpuacct *ca = css_ca(seq_css(sf));
- s64 val[CPUACCT_STAT_NSTATS];
int cpu;
- int stat;
- memset(val, 0, sizeof(val));
+ memset(val, 0, sizeof(*val));
+
for_each_possible_cpu(cpu) {
u64 *cpustat = per_cpu_ptr(ca->cpustat, cpu)->cpustat;
- val[CPUACCT_STAT_USER] += cpustat[CPUTIME_USER];
- val[CPUACCT_STAT_USER] += cpustat[CPUTIME_NICE];
- val[CPUACCT_STAT_SYSTEM] += cpustat[CPUTIME_SYSTEM];
- val[CPUACCT_STAT_SYSTEM] += cpustat[CPUTIME_IRQ];
- val[CPUACCT_STAT_SYSTEM] += cpustat[CPUTIME_SOFTIRQ];
+ (*val)[CPUACCT_STAT_USER] += cpustat[CPUTIME_USER];
+ (*val)[CPUACCT_STAT_USER] += cpustat[CPUTIME_NICE];
+ (*val)[CPUACCT_STAT_SYSTEM] += cpustat[CPUTIME_SYSTEM];
+ (*val)[CPUACCT_STAT_SYSTEM] += cpustat[CPUTIME_IRQ];
+ (*val)[CPUACCT_STAT_SYSTEM] += cpustat[CPUTIME_SOFTIRQ];
}
+}
+
+static int cpuacct_stats_show(struct seq_file *sf, void *v)
+{
+ u64 val[CPUACCT_STAT_NSTATS];
+ int stat;
+
+ cpuacct_stats_read(css_ca(seq_css(sf)), &val);
for (stat = 0; stat < CPUACCT_STAT_NSTATS; stat++) {
- seq_printf(sf, "%s %lld\n",
+ seq_printf(sf, "%s %llu\n",
cpuacct_stat_desc[stat],
(long long)nsec_to_clock_t(val[stat]));
}
--
1.8.3.1
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [RFC PATCH v2 16/17] sched: Implement interface for cgroup unified hierarchy
From: Waiman Long @ 2017-05-15 13:34 UTC (permalink / raw)
To: Tejun Heo, Li Zefan, Johannes Weiner, Peter Zijlstra, Ingo Molnar
Cc: cgroups, linux-kernel, linux-doc, linux-mm, kernel-team, pjt,
luto, efault, longman
In-Reply-To: <1494855256-12558-1-git-send-email-longman@redhat.com>
From: Tejun Heo <tj@kernel.org>
While the cpu controller doesn't have any functional problems, there
are a couple interface issues which can be addressed in the v2
interface.
* cpuacct being a separate controller. This separation is artificial
and rather pointless as demonstrated by most use cases co-mounting
the two controllers. It also forces certain information to be
accounted twice.
* Use of different time units. Writable control knobs use
microseconds, some stat fields use nanoseconds while other cpuacct
stat fields use centiseconds.
* Control knobs which can't be used in the root cgroup still show up
in the root.
* Control knob names and semantics aren't consistent with other
controllers.
This patchset implements cpu controller's interface on the unified
hierarchy which adheres to the controller file conventions described in
Documentation/cgroup-v2.txt. Overall, the following changes are made.
* cpuacct is implictly enabled and disabled by cpu and its information
is reported through "cpu.stat" which now uses microseconds for all
time durations. All time duration fields now have "_usec" appended
to them for clarity. While this doesn't solve the double accounting
immediately, once majority of users switch to v2, cpu can directly
account and report the relevant stats and cpuacct can be disabled on
the unified hierarchy.
Note that cpuacct.usage_percpu is currently not included in
"cpu.stat". If this information is actually called for, it can be
added later.
* "cpu.shares" is replaced with "cpu.weight" and operates on the
standard scale defined by CGROUP_WEIGHT_MIN/DFL/MAX (1, 100, 10000).
The weight is scaled to scheduler weight so that 100 maps to 1024
and the ratio relationship is preserved - if weight is W and its
scaled value is S, W / 100 == S / 1024. While the mapped range is a
bit smaller than the original scheduler weight range, the dead zones
on both sides are relatively small and covers wider range than the
nice value mappings. This file doesn't make sense in the root
cgroup and isn't create on root.
* "cpu.cfs_quota_us" and "cpu.cfs_period_us" are replaced by "cpu.max"
which contains both quota and period.
* "cpu.rt_runtime_us" and "cpu.rt_period_us" are replaced by
"cpu.rt.max" which contains both runtime and period.
v2: cpu_stats_show() was incorrectly using CONFIG_FAIR_GROUP_SCHED for
CFS bandwidth stats and also using raw division for u64. Use
CONFIG_CFS_BANDWIDTH and do_div() instead.
The semantics of "cpu.rt.max" is not fully decided yet. Dropped
for now.
Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Li Zefan <lizefan@huawei.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
---
kernel/sched/core.c | 141 +++++++++++++++++++++++++++++++++++++++++++++++++
kernel/sched/cpuacct.c | 25 +++++++++
kernel/sched/cpuacct.h | 5 ++
3 files changed, 171 insertions(+)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index be2527b..b041081 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -7309,6 +7309,139 @@ static u64 cpu_rt_period_read_uint(struct cgroup_subsys_state *css,
{ } /* Terminate */
};
+static int cpu_stats_show(struct seq_file *sf, void *v)
+{
+ cpuacct_cpu_stats_show(sf);
+
+#ifdef CONFIG_CFS_BANDWIDTH
+ {
+ struct task_group *tg = css_tg(seq_css(sf));
+ struct cfs_bandwidth *cfs_b = &tg->cfs_bandwidth;
+ u64 throttled_usec;
+
+ throttled_usec = cfs_b->throttled_time;
+ do_div(throttled_usec, NSEC_PER_USEC);
+
+ seq_printf(sf, "nr_periods %d\n"
+ "nr_throttled %d\n"
+ "throttled_usec %llu\n",
+ cfs_b->nr_periods, cfs_b->nr_throttled,
+ throttled_usec);
+ }
+#endif
+ return 0;
+}
+
+#ifdef CONFIG_FAIR_GROUP_SCHED
+static u64 cpu_weight_read_u64(struct cgroup_subsys_state *css,
+ struct cftype *cft)
+{
+ struct task_group *tg = css_tg(css);
+ u64 weight = scale_load_down(tg->shares);
+
+ return DIV_ROUND_CLOSEST_ULL(weight * CGROUP_WEIGHT_DFL, 1024);
+}
+
+static int cpu_weight_write_u64(struct cgroup_subsys_state *css,
+ struct cftype *cftype, u64 weight)
+{
+ /*
+ * cgroup weight knobs should use the common MIN, DFL and MAX
+ * values which are 1, 100 and 10000 respectively. While it loses
+ * a bit of range on both ends, it maps pretty well onto the shares
+ * value used by scheduler and the round-trip conversions preserve
+ * the original value over the entire range.
+ */
+ if (weight < CGROUP_WEIGHT_MIN || weight > CGROUP_WEIGHT_MAX)
+ return -ERANGE;
+
+ weight = DIV_ROUND_CLOSEST_ULL(weight * 1024, CGROUP_WEIGHT_DFL);
+
+ return sched_group_set_shares(css_tg(css), scale_load(weight));
+}
+#endif
+
+static void __maybe_unused cpu_period_quota_print(struct seq_file *sf,
+ long period, long quota)
+{
+ if (quota < 0)
+ seq_puts(sf, "max");
+ else
+ seq_printf(sf, "%ld", quota);
+
+ seq_printf(sf, " %ld\n", period);
+}
+
+/* caller should put the current value in *@periodp before calling */
+static int __maybe_unused cpu_period_quota_parse(char *buf,
+ u64 *periodp, u64 *quotap)
+{
+ char tok[21]; /* U64_MAX */
+
+ if (!sscanf(buf, "%s %llu", tok, periodp))
+ return -EINVAL;
+
+ *periodp *= NSEC_PER_USEC;
+
+ if (sscanf(tok, "%llu", quotap))
+ *quotap *= NSEC_PER_USEC;
+ else if (!strcmp(tok, "max"))
+ *quotap = RUNTIME_INF;
+ else
+ return -EINVAL;
+
+ return 0;
+}
+
+#ifdef CONFIG_CFS_BANDWIDTH
+static int cpu_max_show(struct seq_file *sf, void *v)
+{
+ struct task_group *tg = css_tg(seq_css(sf));
+
+ cpu_period_quota_print(sf, tg_get_cfs_period(tg), tg_get_cfs_quota(tg));
+ return 0;
+}
+
+static ssize_t cpu_max_write(struct kernfs_open_file *of,
+ char *buf, size_t nbytes, loff_t off)
+{
+ struct task_group *tg = css_tg(of_css(of));
+ u64 period = tg_get_cfs_period(tg);
+ u64 quota;
+ int ret;
+
+ ret = cpu_period_quota_parse(buf, &period, "a);
+ if (!ret)
+ ret = tg_set_cfs_bandwidth(tg, period, quota);
+ return ret ?: nbytes;
+}
+#endif
+
+static struct cftype cpu_files[] = {
+ {
+ .name = "stat",
+ .flags = CFTYPE_NOT_ON_ROOT,
+ .seq_show = cpu_stats_show,
+ },
+#ifdef CONFIG_FAIR_GROUP_SCHED
+ {
+ .name = "weight",
+ .flags = CFTYPE_NOT_ON_ROOT,
+ .read_u64 = cpu_weight_read_u64,
+ .write_u64 = cpu_weight_write_u64,
+ },
+#endif
+#ifdef CONFIG_CFS_BANDWIDTH
+ {
+ .name = "max",
+ .flags = CFTYPE_NOT_ON_ROOT,
+ .seq_show = cpu_max_show,
+ .write = cpu_max_write,
+ },
+#endif
+ { } /* terminate */
+};
+
struct cgroup_subsys cpu_cgrp_subsys = {
.css_alloc = cpu_cgroup_css_alloc,
.css_online = cpu_cgroup_css_online,
@@ -7318,7 +7451,15 @@ struct cgroup_subsys cpu_cgrp_subsys = {
.can_attach = cpu_cgroup_can_attach,
.attach = cpu_cgroup_attach,
.legacy_cftypes = cpu_legacy_files,
+ .dfl_cftypes = cpu_files,
.early_init = true,
+#ifdef CONFIG_CGROUP_CPUACCT
+ /*
+ * cpuacct is enabled together with cpu on the unified hierarchy
+ * and its stats are reported through "cpu.stat".
+ */
+ .depends_on = 1 << cpuacct_cgrp_id,
+#endif
};
#endif /* CONFIG_CGROUP_SCHED */
diff --git a/kernel/sched/cpuacct.c b/kernel/sched/cpuacct.c
index 6151c23..fc1cf13 100644
--- a/kernel/sched/cpuacct.c
+++ b/kernel/sched/cpuacct.c
@@ -347,6 +347,31 @@ static int cpuacct_stats_show(struct seq_file *sf, void *v)
{ } /* terminate */
};
+/* used to print cpuacct stats in cpu.stat on the unified hierarchy */
+void cpuacct_cpu_stats_show(struct seq_file *sf)
+{
+ struct cgroup_subsys_state *css;
+ u64 usage, val[CPUACCT_STAT_NSTATS];
+
+ css = cgroup_get_e_css(seq_css(sf)->cgroup, &cpuacct_cgrp_subsys);
+
+ usage = cpuusage_read(css, seq_cft(sf));
+ cpuacct_stats_read(css_ca(css), &val);
+
+ val[CPUACCT_STAT_USER] *= TICK_NSEC;
+ val[CPUACCT_STAT_SYSTEM] *= TICK_NSEC;
+ do_div(usage, NSEC_PER_USEC);
+ do_div(val[CPUACCT_STAT_USER], NSEC_PER_USEC);
+ do_div(val[CPUACCT_STAT_SYSTEM], NSEC_PER_USEC);
+
+ seq_printf(sf, "usage_usec %llu\n"
+ "user_usec %llu\n"
+ "system_usec %llu\n",
+ usage, val[CPUACCT_STAT_USER], val[CPUACCT_STAT_SYSTEM]);
+
+ css_put(css);
+}
+
/*
* charge this task's execution time to its accounting group.
*
diff --git a/kernel/sched/cpuacct.h b/kernel/sched/cpuacct.h
index ba72807..ddf7af4 100644
--- a/kernel/sched/cpuacct.h
+++ b/kernel/sched/cpuacct.h
@@ -2,6 +2,7 @@
extern void cpuacct_charge(struct task_struct *tsk, u64 cputime);
extern void cpuacct_account_field(struct task_struct *tsk, int index, u64 val);
+extern void cpuacct_cpu_stats_show(struct seq_file *sf);
#else
@@ -14,4 +15,8 @@ static inline void cpuacct_charge(struct task_struct *tsk, u64 cputime)
{
}
+static inline void cpuacct_cpu_stats_show(struct seq_file *sf)
+{
+}
+
#endif
--
1.8.3.1
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [RFC PATCH v2 17/17] sched: Make cpu/cpuacct threaded controllers
From: Waiman Long @ 2017-05-15 13:34 UTC (permalink / raw)
To: Tejun Heo, Li Zefan, Johannes Weiner, Peter Zijlstra, Ingo Molnar
Cc: cgroups, linux-kernel, linux-doc, linux-mm, kernel-team, pjt,
luto, efault, longman
In-Reply-To: <1494855256-12558-1-git-send-email-longman@redhat.com>
Make cpu and cpuacct cgroup controllers usable within a threaded cgroup.
Signed-off-by: Waiman Long <longman@redhat.com>
---
kernel/sched/core.c | 1 +
kernel/sched/cpuacct.c | 1 +
2 files changed, 2 insertions(+)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index b041081..479f69e 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -7453,6 +7453,7 @@ struct cgroup_subsys cpu_cgrp_subsys = {
.legacy_cftypes = cpu_legacy_files,
.dfl_cftypes = cpu_files,
.early_init = true,
+ .threaded = true,
#ifdef CONFIG_CGROUP_CPUACCT
/*
* cpuacct is enabled together with cpu on the unified hierarchy
diff --git a/kernel/sched/cpuacct.c b/kernel/sched/cpuacct.c
index fc1cf13..853d18a 100644
--- a/kernel/sched/cpuacct.c
+++ b/kernel/sched/cpuacct.c
@@ -414,4 +414,5 @@ struct cgroup_subsys cpuacct_cgrp_subsys = {
.css_free = cpuacct_css_free,
.legacy_cftypes = files,
.early_init = true,
+ .threaded = true,
};
--
1.8.3.1
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* Re: [PATCHv5, REBASED 8/9] x86: Enable 5-level paging support
From: Kirill A. Shutemov @ 2017-05-15 14:11 UTC (permalink / raw)
To: Juergen Gross
Cc: x86, Thomas Gleixner, Ingo Molnar, H. Peter Anvin, Andi Kleen,
Dave Hansen, Andy Lutomirski, Dan Williams, linux-mm,
linux-kernel
In-Reply-To: <9af22de7-89f3-576a-f933-c4e593924091@suse.com>
On Mon, May 15, 2017 at 02:31:00PM +0200, Juergen Gross wrote:
> > diff --git a/arch/x86/xen/Kconfig b/arch/x86/xen/Kconfig
> > index 027987638e98..12205e6dfa59 100644
> > --- a/arch/x86/xen/Kconfig
> > +++ b/arch/x86/xen/Kconfig
> > @@ -5,6 +5,7 @@
> > config XEN
> > bool "Xen guest support"
> > depends on PARAVIRT
> > + depends on !X86_5LEVEL
>
> I'd rather put this under "config XEN_PV".
Makes sense.
----------------------8<----------------------------
^ permalink raw reply
* Re: [PATCHv5, REBASED 8/9] x86: Enable 5-level paging support
From: Juergen Gross @ 2017-05-15 14:13 UTC (permalink / raw)
To: Kirill A. Shutemov
Cc: x86, Thomas Gleixner, Ingo Molnar, H. Peter Anvin, Andi Kleen,
Dave Hansen, Andy Lutomirski, Dan Williams, linux-mm,
linux-kernel
In-Reply-To: <20170515141118.wh45ham64unjk5y2@black.fi.intel.com>
On 15/05/17 16:11, Kirill A. Shutemov wrote:
> On Mon, May 15, 2017 at 02:31:00PM +0200, Juergen Gross wrote:
>>> diff --git a/arch/x86/xen/Kconfig b/arch/x86/xen/Kconfig
>>> index 027987638e98..12205e6dfa59 100644
>>> --- a/arch/x86/xen/Kconfig
>>> +++ b/arch/x86/xen/Kconfig
>>> @@ -5,6 +5,7 @@
>>> config XEN
>>> bool "Xen guest support"
>>> depends on PARAVIRT
>>> + depends on !X86_5LEVEL
>>
>> I'd rather put this under "config XEN_PV".
>
> Makes sense.
>
> ----------------------8<----------------------------
>
> From 422a980c748a5b84a013258eb7c00d61edc34492 Mon Sep 17 00:00:00 2001
> From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
> Date: Sat, 5 Nov 2016 03:24:03 +0300
> Subject: [PATCHv6 8/9] x86: Enable 5-level paging support
>
> Most of things are in place and we can enable support of 5-level paging.
>
> The patch makes XEN_PV dependent on !X86_5LEVEL. XEN_PV is not ready to
> work with 5-level paging.
>
> Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Xen part:
Reviewed-by: Juergen Gross <jgross@suse.com>
Juergen
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCHv5, REBASED 9/9] x86/mm: Allow to have userspace mappings above 47-bits
From: kbuild test robot @ 2017-05-15 14:49 UTC (permalink / raw)
To: Kirill A. Shutemov
Cc: kbuild-all, x86, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
Andi Kleen, Dave Hansen, Andy Lutomirski, Dan Williams, linux-mm,
linux-kernel, linux-api
In-Reply-To: <20170515121218.27610-10-kirill.shutemov@linux.intel.com>
[-- Attachment #1: Type: text/plain, Size: 5255 bytes --]
Hi Kirill,
[auto build test ERROR on linus/master]
[also build test ERROR on v4.12-rc1 next-20170515]
[cannot apply to tip/x86/core xen-tip/linux-next]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Kirill-A-Shutemov/x86-5-level-paging-enabling-for-v4-12-Part-4/20170515-202736
config: i386-defconfig (attached as .config)
compiler: gcc-6 (Debian 6.2.0-3) 6.2.0 20160901
reproduce:
# save the attached .config to linux build tree
make ARCH=i386
All error/warnings (new ones prefixed by >>):
In file included from include/linux/cache.h:4:0,
from include/linux/printk.h:8,
from include/linux/kernel.h:13,
from mm/mmap.c:11:
mm/mmap.c: In function 'arch_get_unmapped_area_topdown':
arch/x86/include/asm/processor.h:878:50: error: 'TASK_SIZE_LOW' undeclared (first use in this function)
#define TASK_UNMAPPED_BASE __TASK_UNMAPPED_BASE(TASK_SIZE_LOW)
^
include/uapi/linux/kernel.h:10:41: note: in definition of macro '__ALIGN_KERNEL_MASK'
#define __ALIGN_KERNEL_MASK(x, mask) (((x) + (mask)) & ~(mask))
^
include/linux/kernel.h:49:22: note: in expansion of macro '__ALIGN_KERNEL'
#define ALIGN(x, a) __ALIGN_KERNEL((x), (a))
^~~~~~~~~~~~~~
include/linux/mm.h:132:26: note: in expansion of macro 'ALIGN'
#define PAGE_ALIGN(addr) ALIGN(addr, PAGE_SIZE)
^~~~~
arch/x86/include/asm/processor.h:877:42: note: in expansion of macro 'PAGE_ALIGN'
#define __TASK_UNMAPPED_BASE(task_size) (PAGE_ALIGN(task_size / 3))
^~~~~~~~~~
arch/x86/include/asm/processor.h:878:29: note: in expansion of macro '__TASK_UNMAPPED_BASE'
#define TASK_UNMAPPED_BASE __TASK_UNMAPPED_BASE(TASK_SIZE_LOW)
^~~~~~~~~~~~~~~~~~~~
>> mm/mmap.c:2043:20: note: in expansion of macro 'TASK_UNMAPPED_BASE'
info.low_limit = TASK_UNMAPPED_BASE;
^~~~~~~~~~~~~~~~~~
arch/x86/include/asm/processor.h:878:50: note: each undeclared identifier is reported only once for each function it appears in
#define TASK_UNMAPPED_BASE __TASK_UNMAPPED_BASE(TASK_SIZE_LOW)
^
include/uapi/linux/kernel.h:10:41: note: in definition of macro '__ALIGN_KERNEL_MASK'
#define __ALIGN_KERNEL_MASK(x, mask) (((x) + (mask)) & ~(mask))
^
include/linux/kernel.h:49:22: note: in expansion of macro '__ALIGN_KERNEL'
#define ALIGN(x, a) __ALIGN_KERNEL((x), (a))
^~~~~~~~~~~~~~
include/linux/mm.h:132:26: note: in expansion of macro 'ALIGN'
#define PAGE_ALIGN(addr) ALIGN(addr, PAGE_SIZE)
^~~~~
arch/x86/include/asm/processor.h:877:42: note: in expansion of macro 'PAGE_ALIGN'
#define __TASK_UNMAPPED_BASE(task_size) (PAGE_ALIGN(task_size / 3))
^~~~~~~~~~
arch/x86/include/asm/processor.h:878:29: note: in expansion of macro '__TASK_UNMAPPED_BASE'
#define TASK_UNMAPPED_BASE __TASK_UNMAPPED_BASE(TASK_SIZE_LOW)
^~~~~~~~~~~~~~~~~~~~
>> mm/mmap.c:2043:20: note: in expansion of macro 'TASK_UNMAPPED_BASE'
info.low_limit = TASK_UNMAPPED_BASE;
^~~~~~~~~~~~~~~~~~
--
In file included from include/linux/elf.h:4:0,
from include/linux/module.h:15,
from fs/binfmt_elf.c:12:
fs/binfmt_elf.c: In function 'load_elf_binary':
>> arch/x86/include/asm/elf.h:253:27: error: 'TASK_SIZE_LOW' undeclared (first use in this function)
#define ELF_ET_DYN_BASE (TASK_SIZE_LOW / 3 * 2)
^
>> fs/binfmt_elf.c:937:16: note: in expansion of macro 'ELF_ET_DYN_BASE'
load_bias = ELF_ET_DYN_BASE - vaddr;
^~~~~~~~~~~~~~~
arch/x86/include/asm/elf.h:253:27: note: each undeclared identifier is reported only once for each function it appears in
#define ELF_ET_DYN_BASE (TASK_SIZE_LOW / 3 * 2)
^
>> fs/binfmt_elf.c:937:16: note: in expansion of macro 'ELF_ET_DYN_BASE'
load_bias = ELF_ET_DYN_BASE - vaddr;
^~~~~~~~~~~~~~~
vim +/TASK_SIZE_LOW +253 arch/x86/include/asm/elf.h
247
248 /* This is the location that an ET_DYN program is loaded if exec'ed. Typical
249 use of this is to invoke "./ld.so someprog" to test out a new version of
250 the loader. We need to make sure that it is out of the way of the program
251 that it will "exec", and that there is sufficient room for the brk. */
252
> 253 #define ELF_ET_DYN_BASE (TASK_SIZE_LOW / 3 * 2)
254
255 /* This yields a mask that user programs can use to figure out what
256 instruction set this CPU supports. This could be done in user space,
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 26192 bytes --]
^ permalink raw reply
* Re: [patch 15/18] mm/vmscan: Adjust system_state checks
From: Steven Rostedt @ 2017-05-15 14:54 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, Peter Zijlstra, Ingo Molnar, Mark Rutland, Andrew Morton,
Johannes Weiner, Mel Gorman, Michal Hocko, Vlastimil Babka,
linux-mm
In-Reply-To: <20170514183613.675463242@linutronix.de>
On Sun, 14 May 2017 20:27:31 +0200
Thomas Gleixner <tglx@linutronix.de> wrote:
> To enable smp_processor_id() and might_sleep() debug checks earlier, it's
> required to add system states between SYSTEM_BOOTING and SYSTEM_RUNNING.
>
> Adjust the system_state check in kswapd_run() to handle the extra states.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
-- Steve
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Johannes Weiner <hannes@cmpxchg.org>
> Cc: Mel Gorman <mgorman@techsingularity.net>
> Cc: Michal Hocko <mhocko@suse.com>
> Cc: Vlastimil Babka <vbabka@suse.cz>
> Cc: linux-mm@kvack.org
> ---
> mm/vmscan.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -3643,7 +3643,7 @@ int kswapd_run(int nid)
> pgdat->kswapd = kthread_run(kswapd, pgdat, "kswapd%d", nid);
> if (IS_ERR(pgdat->kswapd)) {
> /* failure at boot is fatal */
> - BUG_ON(system_state == SYSTEM_BOOTING);
> + BUG_ON(system_state < SYSTEM_RUNNING);
> pr_err("Failed to start kswapd on node %d\n", nid);
> ret = PTR_ERR(pgdat->kswapd);
> pgdat->kswapd = NULL;
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [RFC 0/4] RFC - Coherent Device Memory (Not for inclusion)
From: Christoph Lameter @ 2017-05-15 15:53 UTC (permalink / raw)
To: Michal Hocko
Cc: Benjamin Herrenschmidt, Balbir Singh, linux-mm, akpm, khandual,
aneesh.kumar, paulmck, srikar, haren, jglisse, mgorman, arbab,
vbabka
In-Reply-To: <20170515125530.GH6056@dhcp22.suse.cz>
On Mon, 15 May 2017, Michal Hocko wrote:
> With the proposed solution, they would need to set up mempolicy/cpuset
> so I must be missing something here...
>
> > Of course, the special case of the HPC user trying to milk the last
> > cycle out of the system is probably going to do what you suggest. But
> > most users won't.
Its going to be the HPC users who will be trying to take advantage of it
anyways. I doubt that enterprise class users will even be buying the
accellerators. If it goes that way (after a couple of years) we hopefully
have matured things a bit and have experience how to configure the special
NUMA nodes in the system to behave properly with an accellerator.
I think the simplest way is to just go ahead create the NUMA node
approach and see how much can be covered with the existing NUMA features.
Then work from there to simplify and enhance.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH 1/1] ksm: fix use after free with merge_across_nodes = 0
From: Andrey Ryabinin @ 2017-05-15 16:14 UTC (permalink / raw)
To: Andrea Arcangeli, Andrew Morton, linux-mm
Cc: Evgheni Dereveanchin, Petr Holasek, Hugh Dickins,
Arjan van de Ven, Davidlohr Bueso, Gavin Guo, Jay Vosburgh,
Mel Gorman
In-Reply-To: <20170512193805.8807-2-aarcange@redhat.com>
On 05/12/2017 10:38 PM, Andrea Arcangeli wrote:
> If merge_across_nodes was manually set to 0 (not the default value) by
> the admin or a tuned profile on NUMA systems triggering cross-NODE
> page migrations, a stable_node use after free could materialize.
>
> If the chain is collapsed stable_node would point to the old chain
> that was already freed. stable_node_dup would be the stable_node dup
> now converted to a regular stable_node and indexed in the rbtree in
> replacement of the freed stable_node chain (not anymore a dup).
>
> This special case where the chain is collapsed in the NUMA replacement
> path, is now detected by setting stable_node to NULL by the
> chain_prune callee if it decides to collapse the chain. This tells the
> NUMA replacement code that even if stable_node and stable_node_dup are
> different, this is not a chain if stable_node is NULL, as the
> stable_node_dup was converted to a regular stable_node and the chain
> was collapsed.
>
> It is generally safer for the callee to force the caller stable_node
> to NULL the moment it become stale so any other mistake like this
> would result in an instant Oops easier to debug than an use after free.
>
> Otherwise the replace logic would act like if stable_node was a valid
> chain, when in fact it was freed. Notably
> stable_node_chain_add_dup(page_node, stable_node) would run on a
> stable stable_node.
>
> Andrey Ryabinin found the source of the use after free in
> chain_prune().
>
> Reported-by: Andrey Ryabinin <aryabinin@virtuozzo.com>
> Reported-by: Evgheni Dereveanchin <ederevea@redhat.com>
> Signed-off-by: Andrea Arcangeli <aarcange@redhat.com>
> ---
Works for me,
Tested-by: Andrey Ryabinin <aryabinin@virtuozzo.com>
Bellow is reproducer which causes crash in ksm in several minutes without this fix.
$ cat ksm_test.c
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <sys/mman.h>
#include <stdlib.h>
#include <unistd.h>
#include <stdio.h>
#include <numaif.h>
#include <sys/types.h>
#include <sys/wait.h>
#define NR_NODES 4
#define MAP_SIZE 4096
#define NR_THREADS 1024
pid_t pids[NR_THREADS];
int merge_and_migrate(void)
{
void *p;
unsigned long rnd;
unsigned long old_node, new_node;
pid_t p_pid, pid;
int j;
p = mmap(NULL, MAP_SIZE, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);
if (p == MAP_FAILED)
perror("mmap"), exit(1);
memset(p, 0xff, MAP_SIZE);
if (madvise(p, MAP_SIZE, MADV_MERGEABLE))
perror("madvise"), exit(1);
while (1) {
sleep(0);
rnd = rand() % 2;
switch (rnd) {
case 0: {
rnd = rand() % 128;
memset(p, rnd, MAP_SIZE);
break;
}
case 1: {
j = rand()%NR_NODES;
old_node = 1 << j;
new_node = 1<<((j+1)%NR_NODES);
migrate_pages(0, NR_NODES, &old_node, &new_node);
break;
}
}
}
return 0;
}
int main(void)
{
int i,ret,j;
pid_t pid;
int wstatus;
unsigned long old_node, new_node;
for (i = 0; i < NR_THREADS; i++) {
pid = fork();
if (pid < 0) {
perror("fork");
return 1;
}
if (pid) {
pids[i] = pid;
continue;
} else
merge_and_migrate();
}
while (1) {
pid = waitpid(-1, &wstatus, WNOHANG);
if (pid < 0) {
perror("waitpid failed");
return 1;
}
if (pid) {
for (i = 0; i< NR_THREADS; i++) {
if (pids[i] == pid) {
pid = fork();
if (pid < 0) {
perror("fork in while");
return 1;
}
if (pid) {
pids[i] = pid;
break;
} else
merge_and_migrate();
}
}
continue; /*while(1)*/
}
i = rand()%NR_THREADS;
kill(pids[i], SIGKILL);
}
return 0;
}
$ cat run_ksm.sh
#!/bin/bash
gcc -lnuma -O2 ksm_test.c -o ksm_test
echo 1 > /sys/kernel/mm/ksm/run
echo 0 > /sys/kernel/mm/ksm/merge_across_nodes
echo 2 > /sys/kernel/mm/ksm/max_page_sharing
echo 0 > /sys/kernel/mm/ksm/stable_node_chains_prune_millisecs
./ksm_test
$ ./run_ksm.sh
[ 203.251200] ==================================================================
[ 203.251679] BUG: KASAN: use-after-free in stable_tree_search+0x1450/0x16f0
[ 203.252229] Read of size 4 at addr ffff880037e9d938 by task ksmd/170
[ 203.252800]
[ 203.252957] CPU: 2 PID: 170 Comm: ksmd Not tainted 4.12.0-rc1-next-20170515+ #639
[ 203.253627] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.10.2-0-g5f4c7b1-prebuilt.qemu-project.org 04/01/2014
[ 203.254670] Call Trace:
[ 203.254907] dump_stack+0x67/0x98
[ 203.255222] print_address_description+0x7c/0x290
[ 203.255652] ? stable_tree_search+0x1450/0x16f0
[ 203.256073] kasan_report+0x26e/0x350
[ 203.256418] __asan_report_load4_noabort+0x19/0x20
[ 203.256852] stable_tree_search+0x1450/0x16f0
[ 203.257262] ? __stable_node_chain+0x8a0/0x8a0
[ 203.257668] ? follow_page_mask+0x5f9/0xd80
[ 203.258060] ksm_scan_thread+0xb47/0x2790
[ 203.258438] ? stable_tree_search+0x16f0/0x16f0
[ 203.258858] ? __schedule+0x904/0x1ad0
[ 203.259214] ? clkdev_alloc+0xd0/0xd0
[ 203.259553] ? wake_atomic_t_function+0x2a0/0x2a0
[ 203.259985] ? trace_hardirqs_on+0xd/0x10
[ 203.260361] kthread+0x2d6/0x3d0
[ 203.260658] ? stable_tree_search+0x16f0/0x16f0
[ 203.261073] ? kthread_create_on_node+0xb0/0xb0
[ 203.261485] ret_from_fork+0x2e/0x40
[ 203.261819]
[ 203.261936] Allocated by task 170:
[ 203.262251] save_stack_trace+0x1b/0x20
[ 203.262601] kasan_kmalloc+0xee/0x180
[ 203.262938] kasan_slab_alloc+0x12/0x20
[ 203.263290] kmem_cache_alloc+0x129/0x2d0
[ 203.263654] alloc_stable_node_chain+0x29/0x310
[ 203.264072] ksm_scan_thread+0x2048/0x2790
[ 203.264444] kthread+0x2d6/0x3d0
[ 203.264744] ret_from_fork+0x2e/0x40
[ 203.265075]
[ 203.265220] Freed by task 170:
[ 203.265503] save_stack_trace+0x1b/0x20
[ 203.265852] kasan_slab_free+0xad/0x180
[ 203.266208] kmem_cache_free+0xc7/0x300
[ 203.266558] __stable_node_chain+0x68a/0x8a0
[ 203.266948] stable_tree_search+0x18e/0x16f0
[ 203.267339] ksm_scan_thread+0xb47/0x2790
[ 203.267655] kthread+0x2d6/0x3d0
[ 203.267910] ret_from_fork+0x2e/0x40
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [xfstests PATCH v2 2/3] ext4: allow ext4 to use $SCRATCH_LOGDEV
From: Darrick J. Wong @ 2017-05-15 17:42 UTC (permalink / raw)
To: Jeff Layton
Cc: linux-fsdevel, linux-kernel, linux-btrfs, linux-ext4, linux-cifs,
linux-nfs, linux-mm, jfs-discussion, linux-xfs, cluster-devel,
linux-f2fs-devel, v9fs-developer, linux-nilfs, linux-block,
fstests, dhowells, akpm, hch, ross.zwisler, mawilcox, jack, viro,
corbet, neilb, clm, tytso, axboe, josef, hubcap, rpeterso,
bo.li.liu
In-Reply-To: <20170509161245.29908-3-jlayton@redhat.com>
On Tue, May 09, 2017 at 12:12:44PM -0400, Jeff Layton wrote:
> The writeback error handling test requires that you put the journal on a
> separate device. This allows us to use dmerror to simulate data
> writeback failure, without affecting the journal.
>
> xfs already has infrastructure for this (a'la $SCRATCH_LOGDEV), so wire
> up the ext4 code so that it can do the same thing when _scratch_mkfs is
> called.
>
> Signed-off-by: Jeff Layton <jlayton@redhat.com>
Looks ok,
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
--D
> ---
> common/rc | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/common/rc b/common/rc
> index 257b1903359d..8b815d9c8c33 100644
> --- a/common/rc
> +++ b/common/rc
> @@ -675,6 +675,9 @@ _scratch_mkfs_ext4()
> local tmp=`mktemp`
> local mkfs_status
>
> + [ "$USE_EXTERNAL" = yes -a ! -z "$SCRATCH_LOGDEV" ] && \
> + $mkfs_cmd -O journal_dev $SCRATCH_LOGDEV && \
> + mkfs_cmd="$mkfs_cmd -J device=$SCRATCH_LOGDEV"
>
> _scratch_do_mkfs "$mkfs_cmd" "$mkfs_filter" $* 2>$tmp.mkfserr 1>$tmp.mkfsstd
> mkfs_status=$?
> --
> 2.9.3
>
> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v4 15/27] fs: retrofit old error reporting API onto new infrastructure
From: Jeff Layton @ 2017-05-15 17:58 UTC (permalink / raw)
To: Jan Kara
Cc: linux-fsdevel, linux-kernel, linux-btrfs, linux-ext4, linux-cifs,
linux-nfs, linux-mm, jfs-discussion, linux-xfs, cluster-devel,
linux-f2fs-devel, v9fs-developer, linux-nilfs, linux-block,
dhowells, akpm, hch, ross.zwisler, mawilcox, jack, viro, corbet,
neilb, clm, tytso, axboe, josef, hubcap, rpeterso, bo.li.liu
In-Reply-To: <20170515104246.GC16182@quack2.suse.cz>
On Mon, 2017-05-15 at 12:42 +0200, Jan Kara wrote:
> On Tue 09-05-17 11:49:18, Jeff Layton wrote:
> > Now that we have a better way to store and report errors that occur
> > during writeback, we need to convert the existing codebase to use it. We
> > could just adapt all of the filesystem code and related infrastructure
> > to the new API, but that's a lot of churn.
> >
> > When it comes to setting errors in the mapping, filemap_set_wb_error is
> > a drop-in replacement for mapping_set_error. Turn that function into a
> > simple wrapper around the new one.
> >
> > Because we want to ensure that writeback errors are always reported at
> > fsync time, inject filemap_report_wb_error calls much closer to the
> > syscall boundary, in call_fsync.
> >
> > For fsync calls (and things like the nfsd equivalent), we either return
> > the error that the fsync operation returns, or the one returned by
> > filemap_report_wb_error. In both cases, we advance the file->f_wb_err to
> > the latest value. This allows us to provide new fsync semantics that
> > will return errors that may have occurred previously and been viewed
> > via other file descriptors.
> >
> > The final piece of the puzzle is what to do about filemap_check_errors
> > calls that are being called directly or via filemap_* functions. Here,
> > we must take a little "creative license".
> >
> > Since we now handle advancing the file->f_wb_err value at the generic
> > filesystem layer, we no longer need those callers to clear errors out
> > of the mapping or advance an errseq_t.
> >
> > A lot of the existing codebase relies on being getting an error back
> > from those functions when there is a writeback problem, so we do still
> > want to have them report writeback errors somehow.
> >
> > When reporting writeback errors, we will always report errors that have
> > occurred since a particular point in time. With the old writeback error
> > reporting, the time we used was "since it was last tested/cleared" which
> > is entirely arbitrary and potentially racy. Now, we can at least report
> > the latest error that has occurred since an arbitrary point in time
> > (represented as a sampled errseq_t value).
> >
> > In the case where we don't have a struct file to work with, this patch
> > just has the wrappers sample the current mapping->wb_err value, and use
> > that as an arbitrary point from which to check for errors.
>
> I think this is really dangerous and we shouldn't do this. You are quite
> likely to lose IO errors in such calls because you will ignore all errors
> that happened during previous background writeback or even for IO that
> managed to complete before we called filemap_fdatawait(). Maybe we need to
> keep the original set-clear-bit IO error reporting for these cases, until
> we can convert them to fdatawait_range_since()?
>
Yes that is a danger.
In fact, late last week I was finally able to make my xfstest work with
btrfs, and started hitting oopses with it because of the way the error
reporting changed. I rolled up a patch to fix that (and it simplifies
the code some, IMO), but other callers of filemap_fdatawait may have
similar problems here.
I'll pick up working on this again in a more piecemeal way. I had
originally looked at doing that, but there are some problematic places
(e.g. buffer.c), where the code is shared by a bunch of different
filesystems that makes it difficult.
We may end up needing some sort of FS_WB_ERRSEQ flag to do this
correctly, at least until the transition to errseq_t is done.
> > That's probably not "correct" in all cases, particularly in the case of
> > something like filemap_fdatawait, but I'm not sure it's any worse than
> > what we already have, and this gives us a basis from which to work.
> >
> > A lot of those callers will likely want to change to a model where they
> > sample the errseq_t much earlier (perhaps when starting a transaction),
> > store it in an appropriate place and then use that value later when
> > checking to see if an error occurred.
> >
> > That will almost certainly take some involvement from other subsystem
> > maintainers. I'm quite open to adding new API functions to help enable
> > this if that would be helpful, but I don't really want to do that until
> > I better understand what's needed.
> >
> > Signed-off-by: Jeff Layton <jlayton@redhat.com>
>
> ...
>
> > diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
> > index 5f7317875a67..7ce13281925f 100644
> > --- a/fs/f2fs/file.c
> > +++ b/fs/f2fs/file.c
> > @@ -187,6 +187,7 @@ static int f2fs_do_sync_file(struct file *file, loff_t start, loff_t end,
> > .nr_to_write = LONG_MAX,
> > .for_reclaim = 0,
> > };
> > + errseq_t since = READ_ONCE(file->f_wb_err);
> >
> > if (unlikely(f2fs_readonly(inode->i_sb)))
> > return 0;
> > @@ -265,6 +266,8 @@ static int f2fs_do_sync_file(struct file *file, loff_t start, loff_t end,
> > }
> >
> > ret = wait_on_node_pages_writeback(sbi, ino);
> > + if (ret == 0)
> > + ret = filemap_check_wb_error(NODE_MAPPING(sbi), since);
> > if (ret)
> > goto out;
>
> So this conversion looks wrong and actually points to a larger issue with
> the scheme. The problem is there are two mappings that come into play here
> - file_inode(file)->i_mapping which is the data mapping and
> NODE_MAPPING(sbi) which is the metadata mapping (and this is not a problem
> specific to f2fs. For example ext2 also uses this scheme where block
> devices' mapping is the metadata mapping). And we need to merge error
> information from these two mappings so for the stamping scheme to work,
> we'd need two stamps stored in struct file. One for data mapping and one
> for metadata mapping. Or maybe there's some more clever scheme but for now
> I don't see one...
>
Got it -- since there are two mappings, there are two errseq_t's and
you'd need a since cursor for each. I don't really like having to add a
second 32-bit word to struct file, but I also don't see a real
alternative right offhand. May be able to stash it in file->private_data
for some of these filesystems.
In any case, I'll ponder how to do this in a more piecemeal way.
Thanks for all of the review so far!
--
Jeff Layton <jlayton@redhat.com>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v5 14/32] efi: Add an EFI table address match function
From: Borislav Petkov @ 2017-05-15 18:09 UTC (permalink / raw)
To: Tom Lendacky
Cc: linux-arch, linux-efi, kvm, linux-doc, x86, kexec, linux-kernel,
kasan-dev, linux-mm, iommu, Rik van Riel,
Radim Krčmář, Toshimitsu Kani, Arnd Bergmann,
Jonathan Corbet, Matt Fleming, Michael S. Tsirkin, Joerg Roedel,
Konrad Rzeszutek Wilk, Paolo Bonzini, Larry Woodman,
Brijesh Singh, Ingo Molnar, Andy Lutomirski, H. Peter Anvin,
Andrey Ryabinin, Alexander Potapenko, Dave Young, Thomas Gleixner,
Dmitry Vyukov
In-Reply-To: <20170418211848.10190.65062.stgit@tlendack-t1.amdoffice.net>
On Tue, Apr 18, 2017 at 04:18:48PM -0500, Tom Lendacky wrote:
> Add a function that will determine if a supplied physical address matches
> the address of an EFI table.
>
> Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
> ---
> drivers/firmware/efi/efi.c | 33 +++++++++++++++++++++++++++++++++
> include/linux/efi.h | 7 +++++++
> 2 files changed, 40 insertions(+)
>
> diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c
> index b372aad..8f606a3 100644
> --- a/drivers/firmware/efi/efi.c
> +++ b/drivers/firmware/efi/efi.c
> @@ -55,6 +55,25 @@ struct efi __read_mostly efi = {
> };
> EXPORT_SYMBOL(efi);
>
> +static unsigned long *efi_tables[] = {
> + &efi.mps,
> + &efi.acpi,
> + &efi.acpi20,
> + &efi.smbios,
> + &efi.smbios3,
> + &efi.sal_systab,
> + &efi.boot_info,
> + &efi.hcdp,
> + &efi.uga,
> + &efi.uv_systab,
> + &efi.fw_vendor,
> + &efi.runtime,
> + &efi.config_table,
> + &efi.esrt,
> + &efi.properties_table,
> + &efi.mem_attr_table,
> +};
> +
> static bool disable_runtime;
> static int __init setup_noefi(char *arg)
> {
> @@ -854,6 +873,20 @@ int efi_status_to_err(efi_status_t status)
> return err;
> }
>
> +bool efi_table_address_match(unsigned long phys_addr)
efi_is_table_address() reads easier/better in the code.
--
Regards/Gruss,
Boris.
Good mailing practices for 400: avoid top-posting and trim the reply.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [v3 0/9] parallelized "struct page" zeroing
From: Pasha Tatashin @ 2017-05-15 18:12 UTC (permalink / raw)
To: Michal Hocko
Cc: linux-kernel, sparclinux, linux-mm, linuxppc-dev, linux-s390,
borntraeger, heiko.carstens, davem
In-Reply-To: <20170509181234.GA4397@dhcp22.suse.cz>
Hi Michal,
After looking at your suggested memblock_virt_alloc_core() change again,
I decided to keep what I have. I do not want to inline
memblock_virt_alloc_internal(), because it is not a performance critical
path, and by inlining it we will unnecessarily increase the text size on
all platforms.
Also, because it will be very hard to make sure that no platform
regresses by making memset() default in _memblock_virt_alloc_core() (as
I already showed last week at least sun4v SPARC64 will require special
changes in order for this to work), I decided to make it available only
for "deferred struct page init" case. As, what is already in the patch.
I am working on testing to make sure we do not need to double zero in
the two cases that you found: sparsemem, and mem hotplug. Please let me
know if you have any more comments, or if I can send new patches out
when they are ready.
Thank you,
Pasha
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [v3 9/9] s390: teach platforms not to zero struct pages memory
From: Pasha Tatashin @ 2017-05-15 18:24 UTC (permalink / raw)
To: Heiko Carstens
Cc: linux-kernel, sparclinux, linux-mm, linuxppc-dev, linux-s390,
borntraeger, davem
In-Reply-To: <20170508113624.GA4876@osiris>
Hi Heiko,
Thank you for looking at this patch. I am worried to make the proposed
change, because, as I understand in this case we allocate memory not for
"struct page"s but for table that hold them. So, we will change the
behavior from the current one, where this table is allocated zeroed, but
now it won't be zeroed.
Pasha
>
> If you add the hunk below then this is
>
> Acked-by: Heiko Carstens <heiko.carstens@de.ibm.com>
>
> diff --git a/arch/s390/mm/vmem.c b/arch/s390/mm/vmem.c
> index ffe9ba1aec8b..bf88a8b9c24d 100644
> --- a/arch/s390/mm/vmem.c
> +++ b/arch/s390/mm/vmem.c
> @@ -272,7 +272,7 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node)
> if (pte_none(*pt_dir)) {
> void *new_page;
>
> - new_page = vmemmap_alloc_block(PAGE_SIZE, node, true);
> + new_page = vmemmap_alloc_block(PAGE_SIZE, node, VMEMMAP_ZERO);
> if (!new_page)
> goto out;
> pte_val(*pt_dir) = __pa(new_page) | pgt_prot;
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v5 17/32] x86/mm: Add support to access boot related data in the clear
From: Borislav Petkov @ 2017-05-15 18:35 UTC (permalink / raw)
To: Tom Lendacky
Cc: linux-arch, linux-efi, kvm, linux-doc, x86, kexec, linux-kernel,
kasan-dev, linux-mm, iommu, Rik van Riel,
Radim Krčmář, Toshimitsu Kani, Arnd Bergmann,
Jonathan Corbet, Matt Fleming, Michael S. Tsirkin, Joerg Roedel,
Konrad Rzeszutek Wilk, Paolo Bonzini, Larry Woodman,
Brijesh Singh, Ingo Molnar, Andy Lutomirski, H. Peter Anvin,
Andrey Ryabinin, Alexander Potapenko, Dave Young, Thomas Gleixner,
Dmitry Vyukov
In-Reply-To: <20170418211921.10190.1537.stgit@tlendack-t1.amdoffice.net>
On Tue, Apr 18, 2017 at 04:19:21PM -0500, Tom Lendacky wrote:
> Boot data (such as EFI related data) is not encrypted when the system is
> booted because UEFI/BIOS does not run with SME active. In order to access
> this data properly it needs to be mapped decrypted.
>
> The early_memremap() support is updated to provide an arch specific
"Update early_memremap() to provide... "
> routine to modify the pagetable protection attributes before they are
> applied to the new mapping. This is used to remove the encryption mask
> for boot related data.
>
> The memremap() support is updated to provide an arch specific routine
Ditto. Passive tone always reads harder than an active tone,
"doer"-sentence.
> to determine if RAM remapping is allowed. RAM remapping will cause an
> encrypted mapping to be generated. By preventing RAM remapping,
> ioremap_cache() will be used instead, which will provide a decrypted
> mapping of the boot related data.
>
> Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
> ---
> arch/x86/include/asm/io.h | 4 +
> arch/x86/mm/ioremap.c | 182 +++++++++++++++++++++++++++++++++++++++++++++
> include/linux/io.h | 2
> kernel/memremap.c | 20 ++++-
> mm/early_ioremap.c | 18 ++++
> 5 files changed, 219 insertions(+), 7 deletions(-)
>
> diff --git a/arch/x86/include/asm/io.h b/arch/x86/include/asm/io.h
> index 7afb0e2..75f2858 100644
> --- a/arch/x86/include/asm/io.h
> +++ b/arch/x86/include/asm/io.h
> @@ -381,4 +381,8 @@ extern int __must_check arch_phys_wc_add(unsigned long base,
> #define arch_io_reserve_memtype_wc arch_io_reserve_memtype_wc
> #endif
>
> +extern bool arch_memremap_do_ram_remap(resource_size_t offset, size_t size,
> + unsigned long flags);
> +#define arch_memremap_do_ram_remap arch_memremap_do_ram_remap
> +
> #endif /* _ASM_X86_IO_H */
> diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
> index 9bfcb1f..bce0604 100644
> --- a/arch/x86/mm/ioremap.c
> +++ b/arch/x86/mm/ioremap.c
> @@ -13,6 +13,7 @@
> #include <linux/slab.h>
> #include <linux/vmalloc.h>
> #include <linux/mmiotrace.h>
> +#include <linux/efi.h>
>
> #include <asm/cacheflush.h>
> #include <asm/e820/api.h>
> @@ -21,6 +22,7 @@
> #include <asm/tlbflush.h>
> #include <asm/pgalloc.h>
> #include <asm/pat.h>
> +#include <asm/setup.h>
>
> #include "physaddr.h"
>
> @@ -419,6 +421,186 @@ void unxlate_dev_mem_ptr(phys_addr_t phys, void *addr)
> iounmap((void __iomem *)((unsigned long)addr & PAGE_MASK));
> }
>
> +/*
> + * Examine the physical address to determine if it is an area of memory
> + * that should be mapped decrypted. If the memory is not part of the
> + * kernel usable area it was accessed and created decrypted, so these
> + * areas should be mapped decrypted.
> + */
> +static bool memremap_should_map_decrypted(resource_size_t phys_addr,
> + unsigned long size)
> +{
> + /* Check if the address is outside kernel usable area */
> + switch (e820__get_entry_type(phys_addr, phys_addr + size - 1)) {
> + case E820_TYPE_RESERVED:
> + case E820_TYPE_ACPI:
> + case E820_TYPE_NVS:
> + case E820_TYPE_UNUSABLE:
> + return true;
> + default:
> + break;
> + }
> +
> + return false;
> +}
> +
> +/*
> + * Examine the physical address to determine if it is EFI data. Check
> + * it against the boot params structure and EFI tables and memory types.
> + */
> +static bool memremap_is_efi_data(resource_size_t phys_addr,
> + unsigned long size)
> +{
> + u64 paddr;
> +
> + /* Check if the address is part of EFI boot/runtime data */
> + if (efi_enabled(EFI_BOOT)) {
Save indentation level:
if (!efi_enabled(EFI_BOOT))
return false;
> + paddr = boot_params.efi_info.efi_memmap_hi;
> + paddr <<= 32;
> + paddr |= boot_params.efi_info.efi_memmap;
> + if (phys_addr == paddr)
> + return true;
> +
> + paddr = boot_params.efi_info.efi_systab_hi;
> + paddr <<= 32;
> + paddr |= boot_params.efi_info.efi_systab;
So those two above look like could be two global vars which are
initialized somewhere in the EFI init path:
efi_memmap_phys and efi_systab_phys or so.
Matt ?
And then you won't need to create that paddr each time on the fly. I
mean, it's not a lot of instructions but still...
> + if (phys_addr == paddr)
> + return true;
> +
> + if (efi_table_address_match(phys_addr))
> + return true;
> +
> + switch (efi_mem_type(phys_addr)) {
> + case EFI_BOOT_SERVICES_DATA:
> + case EFI_RUNTIME_SERVICES_DATA:
> + return true;
> + default:
> + break;
> + }
> + }
> +
> + return false;
> +}
> +
> +/*
> + * Examine the physical address to determine if it is boot data by checking
> + * it against the boot params setup_data chain.
> + */
> +static bool memremap_is_setup_data(resource_size_t phys_addr,
> + unsigned long size)
> +{
> + struct setup_data *data;
> + u64 paddr, paddr_next;
> +
> + paddr = boot_params.hdr.setup_data;
> + while (paddr) {
> + bool is_setup_data = false;
You don't need that bool:
static bool memremap_is_setup_data(resource_size_t phys_addr,
unsigned long size)
{
struct setup_data *data;
u64 paddr, paddr_next;
paddr = boot_params.hdr.setup_data;
while (paddr) {
if (phys_addr == paddr)
return true;
data = memremap(paddr, sizeof(*data), MEMREMAP_WB | MEMREMAP_DEC);
paddr_next = data->next;
if ((phys_addr > paddr) && (phys_addr < (paddr + data->len))) {
memunmap(data);
return true;
}
memunmap(data);
paddr = paddr_next;
}
return false;
}
Flow is a bit clearer.
> +/*
> + * Examine the physical address to determine if it is boot data by checking
> + * it against the boot params setup_data chain (early boot version).
> + */
> +static bool __init early_memremap_is_setup_data(resource_size_t phys_addr,
> + unsigned long size)
> +{
> + struct setup_data *data;
> + u64 paddr, paddr_next;
> +
> + paddr = boot_params.hdr.setup_data;
> + while (paddr) {
> + bool is_setup_data = false;
> +
> + if (phys_addr == paddr)
> + return true;
> +
> + data = early_memremap_decrypted(paddr, sizeof(*data));
> +
> + paddr_next = data->next;
> +
> + if ((phys_addr > paddr) && (phys_addr < (paddr + data->len)))
> + is_setup_data = true;
> +
> + early_memunmap(data, sizeof(*data));
> +
> + if (is_setup_data)
> + return true;
> +
> + paddr = paddr_next;
> + }
> +
> + return false;
> +}
This one is begging to be unified with memremap_is_setup_data() to both
call a __ worker function.
> +
> +/*
> + * Architecture function to determine if RAM remap is allowed. By default, a
> + * RAM remap will map the data as encrypted. Determine if a RAM remap should
> + * not be done so that the data will be mapped decrypted.
> + */
> +bool arch_memremap_do_ram_remap(resource_size_t phys_addr, unsigned long size,
> + unsigned long flags)
So this function doesn't do anything - it replies to a yes/no question.
So the name should not say "do" but sound like a question. Maybe:
if (arch_memremap_can_remap( ... ))
or so...
> +{
> + if (!sme_active())
> + return true;
> +
> + if (flags & MEMREMAP_ENC)
> + return true;
> +
> + if (flags & MEMREMAP_DEC)
> + return false;
So this looks strange to me: both flags MEMREMAP_ENC and _DEC override
setup and efi data checking. But we want to remap setup and EFI data
*always* decrypted because that data was not encrypted as, as you say,
firmware doesn't run with SME active.
So my simple logic says that EFI stuff should *always* be mapped DEC,
regardless of flags. Ditto for setup data. So that check below should
actually *override* the flags checks and go before them, no?
> +
> + if (memremap_is_setup_data(phys_addr, size) ||
> + memremap_is_efi_data(phys_addr, size) ||
> + memremap_should_map_decrypted(phys_addr, size))
> + return false;
> +
> + return true;
> +}
> +
> +/*
> + * Architecture override of __weak function to adjust the protection attributes
> + * used when remapping memory. By default, early_memremp() will map the data
early_memremAp() - a is missing.
--
Regards/Gruss,
Boris.
Good mailing practices for 400: avoid top-posting and trim the reply.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox