* Re: [PATCH v3 0/4] cgroup/rdma: add rdma.peak and rdma.events[.local]
From: Tejun Heo @ 2026-05-14 21:27 UTC (permalink / raw)
To: Tao Cui; +Cc: hannes, mkoutny, cgroups
In-Reply-To: <20260514065034.387197-1-cuitao@kylinos.cn>
Hello,
> Tao Cui (4):
> cgroup/rdma: add rdma.peak for per-device peak usage tracking
> cgroup/rdma: add rdma.events to track resource limit exhaustion
> cgroup/rdma: add rdma.events.local for per-cgroup allocation failure
> attribution
> cgroup/rdma: document rdma.peak, rdma.events and rdma.events.local
Applied 1-4 to cgroup/for-7.2.
One follow-up: the new event counters use READ_ONCE() on reads but plain
++ on writes, and all accesses are under rdmacg_mutex. Please send a
follow-up patch dropping the READ_ONCE()s.
Thanks.
--
tejun
^ permalink raw reply
* Re: [PATCH v3 0/4] cgroup/rdma: add rdma.peak and rdma.events[.local]
From: Tejun Heo @ 2026-05-14 21:26 UTC (permalink / raw)
To: Tao Cui; +Cc: hannes, mkoutny, cgroups
In-Reply-To: <20260514065034.387197-1-cuitao@kylinos.cn>
Hello,
> Tao Cui (4):
> cgroup/rdma: add rdma.peak for per-device peak usage tracking
> cgroup/rdma: add rdma.events to track resource limit exhaustion
> cgroup/rdma: add rdma.events.local for per-cgroup allocation failure
> attribution
> cgroup/rdma: document rdma.peak, rdma.events and rdma.events.local
Applied 1-4 to cgroup/for-7.2.
One follow-up: the new event counters use READ_ONCE() on reads but plain
++ on writes, and all accesses are under rdmacg_mutex. Please send a
follow-up patch dropping the READ_ONCE()s.
Thanks.
--
tejun
^ permalink raw reply
* Re: [PATCH RFC 4/5] selinux: Restrict cross-cgroup dma-heap charging
From: Paul Moore @ 2026-05-14 20:44 UTC (permalink / raw)
To: Albert Esteve, Tejun Heo, Johannes Weiner, Michal Koutný,
Jonathan Corbet, Shuah Khan, Sumit Semwal, Christian König,
Michal Hocko, Roman Gushchin, Shakeel Butt, Muchun Song,
Andrew Morton, Benjamin Gaignard, Brian Starkey, John Stultz,
T.J. Mercier, Christian Brauner, James Morris, Serge E. Hallyn,
Stephen Smalley, Ondrej Mosnacek, Shuah Khan
Cc: cgroups, linux-doc, linux-kernel, linux-media, dri-devel,
linaro-mm-sig, linux-mm, linux-security-module, selinux,
linux-kselftest, Albert Esteve, mripard, echanude
In-Reply-To: <20260512-v2_20230123_tjmercier_google_com-v1-4-6326701c3691@redhat.com>
On May 12, 2026 Albert Esteve <aesteve@redhat.com> wrote:
>
> The security_dma_heap_alloc() hook allows security modules
> to control which processes may charge dma-buf allocations
> to another process's cgroup via the charge_pid_fd field of
> DMA_HEAP_IOCTL_ALLOC. Without a policy implementation, the
> hook is a no-op and the restriction is not enforced.
>
> On SELinux-managed systems any domain with access to a
> dma-heap device node can therefore exhaust another cgroup's
> memory budget without restriction.
>
> Implement selinux_dma_heap_alloc() using avc_has_perm() with
> a new dma_heap object class and a charge_to permission. Policy
> authors can then grant cross-cgroup charging selectively,
> for example:
>
> allow allocator_app_t client_app_t:dma_heap charge_to;
>
> Signed-off-by: Albert Esteve <aesteve@redhat.com>
> ---
> security/selinux/hooks.c | 7 +++++++
> security/selinux/include/classmap.h | 1 +
> 2 files changed, 8 insertions(+)
>
> diff --git a/security/selinux/hooks.c b/security/selinux/hooks.c
> index 0f704380a8c81..ea1f410b9f619 100644
> --- a/security/selinux/hooks.c
> +++ b/security/selinux/hooks.c
> @@ -2189,6 +2189,12 @@ static int selinux_capable(const struct cred *cred, struct user_namespace *ns,
> return cred_has_capability(cred, cap, opts, ns == &init_user_ns);
> }
>
> +static int selinux_dma_heap_alloc(const struct cred *from, const struct cred *to)
> +{
> + return avc_has_perm(cred_sid(from), cred_sid(to),
> + SECCLASS_DMA_HEAP, DMA_HEAP__CHARGE_TO, NULL);
> +}
> +
> static int selinux_quotactl(int cmds, int type, int id, const struct super_block *sb)
> {
> const struct cred *cred = current_cred();
> @@ -7541,6 +7547,7 @@ static struct security_hook_list selinux_hooks[] __ro_after_init = {
> LSM_HOOK_INIT(capget, selinux_capget),
> LSM_HOOK_INIT(capset, selinux_capset),
> LSM_HOOK_INIT(capable, selinux_capable),
> + LSM_HOOK_INIT(dma_heap_alloc, selinux_dma_heap_alloc),
> LSM_HOOK_INIT(quotactl, selinux_quotactl),
> LSM_HOOK_INIT(quota_on, selinux_quota_on),
> LSM_HOOK_INIT(syslog, selinux_syslog),
> diff --git a/security/selinux/include/classmap.h b/security/selinux/include/classmap.h
> index 90cb61b164256..d232f7808f6b8 100644
> --- a/security/selinux/include/classmap.h
> +++ b/security/selinux/include/classmap.h
> @@ -181,6 +181,7 @@ const struct security_class_mapping secclass_map[] = {
> { "user_namespace", { "create", NULL } },
> { "memfd_file",
> { COMMON_FILE_PERMS, "execute_no_trans", "entrypoint", NULL } },
> + { "dma_heap", { "charge_to", NULL } },
> /* last one */ { NULL, {} }
> };
While we have seen some one-off patches to add specific resource/cgroups
controls in the past, much like this one, we've yet to see a patchset
that provides a more comprehensive set of resource/cgroup access controls
for SELinux.
I'm not opposed to a patch like this, but I would like to see it as part
of a larger effort to introduce access controls across all of the
existing cgroup control points where it makes sense. In other words,
let's see a design for cgroup access controls so that we can ensure we
have something that is meaningful and makes sense from a policy
developer's perspective.
--
paul-moore.com
^ permalink raw reply
* Re: [PATCH v2 2/3] security: Expand task_setscheduler LSM hook to include CPU affinity mask
From: Paul Moore @ 2026-05-14 20:15 UTC (permalink / raw)
To: Aaron Tomlin
Cc: tsbogend, jmorris, serge, mingo, peterz, juri.lelli,
vincent.guittot, stephen.smalley.work, casey, longman, tj, hannes,
mkoutny, chenridong, dietmar.eggemann, rostedt, bsegall, mgorman,
vschneid, kprateek.nayak, omosnace, kees, neelx, sean, chjohnst,
steve, mproche, nick.lange, cgroups, linux-mips, linux-fsdevel,
linux-security-module, selinux, linux-kernel
In-Reply-To: <bscbywzzx4nmxzbuw2bkzltb7rrmgmzy5u4gqy5pfpmafcnlto@eznniiguusqb>
On Tue, May 12, 2026 at 3:49 PM Aaron Tomlin <atomlin@atomlin.com> wrote:
> On Mon, May 11, 2026 at 04:28:09PM -0400, Paul Moore wrote:
> [ ... ]
> > > Signed-off-by: Aaron Tomlin <atomlin@atomlin.com>
> > > ---
> > > arch/mips/kernel/mips-mt-fpaff.c | 30 +++++++++++++++++-------------
> > > fs/proc/base.c | 2 +-
> > > include/linux/lsm_hook_defs.h | 3 ++-
> > > include/linux/security.h | 11 +++++++----
> > > kernel/cgroup/cpuset.c | 4 ++--
> > > kernel/sched/syscalls.c | 4 ++--
> > > security/commoncap.c | 7 +++++--
> > > security/security.c | 11 ++++++-----
> > > security/selinux/hooks.c | 3 ++-
> > > security/smack/smack_lsm.c | 11 +++++++++--
> > > 10 files changed, 53 insertions(+), 33 deletions(-)
> >
> > I haven't looked too closely at this patch yet, but based on a quick
> > glance, can you help me understand why it is included with the other
> > two patches in one patchset? The other two patches look like stable
> > level kernel bug fixes, while this patch introduces functionality to
> > an existing LSM hook; one of these is not like the others :)
> >
> > Unless there is something critical that I'm missing here, I would
> > suggest splitting this patch out from the other two bugfixes for
> > separate handling. If there is a patch dependency issue you can
> > always mention that in the cover letter.
>
> Hi Paul,
>
> Thank you for taking the time to have a look.
>
> You raise a perfectly valid point.
>
> Please note, the cgroup-related BUG fix will be dropped from the next
> iteration of this series. As per Waiman Long (on Cc), a solution for the
> BUG was already proposed here [1].
That's good news. I saw some discussion on that but didn't follow it
very closely.
> However, I suspect the MIPS-related patch will need to remain coupled with
> this feature patch. Because the first patch fundamentally alters the
> signature of the security_task_setscheduler() hook, the MIPS FPU affinity
> code must be updated concurrently to accommodate the new parameter.
I generally dislike when bug fixes depend on new functionality; it's
backwards in my opinion. I would much rather see the MIPS bug fix
patch submitted as a standalone patch and then have the LSM hook
modification patch come separately, perhaps with a note that it
depends on the bug fix patch.
--
paul-moore.com
^ permalink raw reply
* [PATCH v3] cgroup/dmem: introduce a peak file
From: Thadeu Lima de Souza Cascardo @ 2026-05-14 17:36 UTC (permalink / raw)
To: Tejun Heo, Johannes Weiner, Michal Koutný, Michal Hocko,
Roman Gushchin, Shakeel Butt, Muchun Song, Andrew Morton,
Jonathan Corbet, Shuah Khan, Maarten Lankhorst, Maxime Ripard,
Natalie Vock, Tvrtko Ursulin
Cc: cgroups, linux-kernel, linux-mm, linux-doc, dri-devel, kernel-dev,
Thadeu Lima de Souza Cascardo
Just like we have memory.peak, introduce a dmem.peak, which uses the
page_counter support for that.
For now, make it read-only.
This allows for memory usage monitoring without polling dmem.current when
the information needed is the maximum device memory used. That can be used
for capacity planning, such that dmem.max can be properly setup for a given
workload. It can also be used for debugging to determine whether a given
workload would have caused eviction or system memory use.
Signed-off-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
---
Changes in v3:
- EDITME: describe what is new in this series revision.
- EDITME: use bulletpoints and terse descriptions.
- Link to v2: https://patch.msgid.link/20260513-dmem_peak-v2-1-dac06999db9e@igalia.com
Changes in v2:
- Make it read-only for now and adjust documentation accordingly.
- Link to v1: https://patch.msgid.link/20260506-dmem_peak-v1-0-8d803eb3449c@igalia.com
---
Documentation/admin-guide/cgroup-v2.rst | 6 ++++++
kernel/cgroup/dmem.c | 15 +++++++++++++++
2 files changed, 21 insertions(+)
diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst
index 6efd0095ed99..d103623b2be4 100644
--- a/Documentation/admin-guide/cgroup-v2.rst
+++ b/Documentation/admin-guide/cgroup-v2.rst
@@ -2808,6 +2808,12 @@ DMEM Interface Files
The semantics are the same as for the memory cgroup controller, and are
calculated in the same way.
+ dmem.peak
+ A read-only nested-keyed file that exists on non-root cgroups.
+
+ The max device memory usage recorded for the cgroup and its
+ descendants since the creation of the cgroup for each region.
+
dmem.capacity
A read-only file that describes maximum region capacity.
It only exists on the root cgroup. Not all memory can be
diff --git a/kernel/cgroup/dmem.c b/kernel/cgroup/dmem.c
index 4753a67d0f0f..6430c7ce1e03 100644
--- a/kernel/cgroup/dmem.c
+++ b/kernel/cgroup/dmem.c
@@ -182,6 +182,11 @@ static u64 get_resource_current(struct dmem_cgroup_pool_state *pool)
return pool ? page_counter_read(&pool->cnt) : 0;
}
+static u64 get_resource_peak(struct dmem_cgroup_pool_state *pool)
+{
+ return pool ? READ_ONCE(pool->cnt.watermark) : 0;
+}
+
static void reset_all_resource_limits(struct dmem_cgroup_pool_state *rpool)
{
set_resource_min(rpool, 0);
@@ -808,6 +813,11 @@ static int dmemcg_limit_show(struct seq_file *sf, void *v,
return 0;
}
+static int dmem_cgroup_region_peak_show(struct seq_file *sf, void *v)
+{
+ return dmemcg_limit_show(sf, v, get_resource_peak);
+}
+
static int dmem_cgroup_region_current_show(struct seq_file *sf, void *v)
{
return dmemcg_limit_show(sf, v, get_resource_current);
@@ -856,6 +866,11 @@ static struct cftype files[] = {
.name = "current",
.seq_show = dmem_cgroup_region_current_show,
},
+ {
+ .name = "peak",
+ .seq_show = dmem_cgroup_region_peak_show,
+ .flags = CFTYPE_NOT_ON_ROOT,
+ },
{
.name = "min",
.write = dmem_cgroup_region_min_write,
---
base-commit: d3b0a7f21119f5a66cb76aa28fb8cc13206aaf7d
change-id: 20260409-dmem_peak-3abc1be95072
Best regards,
--
Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
^ permalink raw reply related
* [PATCH cgroup/for-next 4/4] cgroup/cpuset: Support multiple source/destination cpusets for cpuset_*attach()
From: Waiman Long @ 2026-05-14 17:02 UTC (permalink / raw)
To: Chen Ridong, Tejun Heo, Johannes Weiner, Michal Koutný
Cc: cgroups, linux-kernel, Dietmar Eggemann, Aaron Tomlin, Juri Lelli,
Waiman Long
In-Reply-To: <20260514170240.575156-1-longman@redhat.com>
With cgroup v2, the cgroup_taskset structure passed into the cgroup
can_attach() and attach() methods can contain task migration data with
multiple destination or source cpusets when the cpuset controller is
enabled or disabled respectively.
Since cpuset is threaded, another possible way to cause many-to-one
migration is to move the whole process with multiple threads in different
cpuset enabled threaded cgroups into another cpuset enabled cgroup.
Alternatively, multiple processs from different cpusets can be written
into cgroup.proc as a single operation.
The current cpuset_can_attach() and cpuset_attach() functions still
expect task migration is from one source cpuset to one destination
cpuset. This has been the case since cpuset was enabled for cgroup v2
in commit 4ec22e9c5a90 ("cpuset: Enable cpuset controller in default
hierarchy").
This problem is less an issue when enabling the cpuset controller as all
the newly created child cpusets will have exactly the same set of CPUs
and memory nodes except when deadline tasks are involved in migration
as the deadline task accounting data can be off.
It can be more problematic when the cpuset controller is disabled as
their set of CPUs and memory nodes may differ from their parent or with
the moving of multi-threaded process from different threaded cgroups.
Fix that by tracking the set of source (old) and destination cpusets
in singly linked lists and iterating them all to properly update the
internal data. Also keep the current cs and oldcs variables up-to-date
with the css and task iterators. cpuset_attach_old_cs is now dropped
as the old cpusets are now being tracked.
To ensure proper DL tasks accounting, the nr_migrate_dl_tasks in both
the source and destination cpusets are decremented/incremented with
their values added to nr_deadline_tasks when the migration is successful.
Fixes: 4ec22e9c5a90 ("cpuset: Enable cpuset controller in default hierarchy")
Signed-off-by: Waiman Long <longman@redhat.com>
---
kernel/cgroup/cpuset-internal.h | 6 +
kernel/cgroup/cpuset.c | 206 +++++++++++++++++++++++---------
2 files changed, 158 insertions(+), 54 deletions(-)
diff --git a/kernel/cgroup/cpuset-internal.h b/kernel/cgroup/cpuset-internal.h
index f7aaf01f7cd5..4c2772a7fd5e 100644
--- a/kernel/cgroup/cpuset-internal.h
+++ b/kernel/cgroup/cpuset-internal.h
@@ -161,6 +161,12 @@ struct cpuset {
*/
bool remote_partition;
+ /*
+ * cpuset_can_attach() and cpuset_attach() specific data
+ */
+ bool attach_node_in_llist;
+ struct llist_node attach_node;
+
/*
* number of SCHED_DEADLINE tasks attached to this cpuset, so that we
* know when to rebuild associated root domain bandwidth information.
diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 8ced1fa0900f..c46454b29d74 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -37,6 +37,7 @@
#include <linux/wait.h>
#include <linux/workqueue.h>
#include <linux/task_work.h>
+#include <linux/llist.h>
DEFINE_STATIC_KEY_FALSE(cpusets_pre_enable_key);
DEFINE_STATIC_KEY_FALSE(cpusets_enabled_key);
@@ -2967,7 +2968,8 @@ static int update_prstate(struct cpuset *cs, int new_prs)
/*
* cpuset_can_attach() and cpuset_attach() specific internal data
*/
-static struct cpuset *cpuset_attach_old_cs;
+static LLIST_HEAD(src_cs_head);
+static LLIST_HEAD(dst_cs_head);
static bool attach_cpus_updated;
static bool attach_mems_updated;
@@ -2980,9 +2982,10 @@ static bool attach_mems_updated;
* Also set the boolean flag passed in by @psetsched depending on if
* security_task_setscheduler() call is needed and @oldcs is not NULL.
*/
-static int cpuset_can_attach_check(struct cpuset *cs, struct cpuset *oldcs,
- bool *psetsched)
+static int cpuset_can_attach_check(struct cpuset *cs, struct cpuset *oldcs, bool *psetsched)
{
+ bool cpu_match, mem_match;
+
if (cpumask_empty(cs->effective_cpus) ||
(!is_in_v2_mode() && nodes_empty(cs->mems_allowed)))
return -ENOSPC;
@@ -2993,15 +2996,34 @@ static int cpuset_can_attach_check(struct cpuset *cs, struct cpuset *oldcs,
/*
* Update attach specific data
*/
- attach_cpus_updated = !cpumask_equal(cs->effective_cpus, oldcs->effective_cpus);
- attach_mems_updated = !nodes_equal(cs->effective_mems, oldcs->effective_mems);
+ if (!cs->attach_node_in_llist) {
+ llist_add(&cs->attach_node, &dst_cs_head);
+ cs->attach_node_in_llist = true;
+ }
+ if (!oldcs->attach_node_in_llist) {
+ llist_add(&oldcs->attach_node, &src_cs_head);
+ oldcs->attach_node_in_llist = true;
+ }
+
+ cpu_match = cpumask_equal(cs->effective_cpus, oldcs->effective_cpus);
+ mem_match = nodes_equal(cs->effective_mems, oldcs->effective_mems);
+
+ /*
+ * Set the updated flags whenever there is a mismatch in any of the
+ * src/dst pairs.
+ */
+ if (!attach_cpus_updated)
+ attach_cpus_updated = !cpu_match;
+
+ if (!attach_mems_updated)
+ attach_mems_updated = !mem_match;
/*
* Skip rights over task setsched check in v2 when nothing changes,
* migration permission derives from hierarchy ownership in
* cgroup_procs_write_permission()).
*/
- *psetsched = !cpuset_v2() || attach_cpus_updated || attach_mems_updated;
+ *psetsched = !cpuset_v2() || !cpu_match || !mem_match;
/*
* A v1 cpuset with tasks will have no CPU left only when CPU hotplug
@@ -3016,33 +3038,105 @@ static int cpuset_can_attach_check(struct cpuset *cs, struct cpuset *oldcs,
return 0;
}
-static int alloc_dl_bw(struct cpuset *cs)
+/*
+ * If reset_dl_bw is set, reset the previous dl_bw_alloc() call. Otherwise,
+ * update nr_deadline_tasks according to nr_migrate_dl_tasks in both source
+ * and destination cpusets.
+ */
+static void clear_attach_data(bool reset_dl_bw)
+{
+ struct cpuset *cs, *next;
+
+ llist_for_each_entry_safe(cs, next, src_cs_head.first, attach_node) {
+ cs->attach_node.next = NULL;
+ cs->attach_node_in_llist = false;
+ if (cs->nr_migrate_dl_tasks && !reset_dl_bw)
+ cs->nr_deadline_tasks += cs->nr_migrate_dl_tasks;
+ cs->nr_migrate_dl_tasks = 0;
+ }
+
+ llist_for_each_entry_safe(cs, next, dst_cs_head.first, attach_node) {
+ cs->attach_node.next = NULL;
+ cs->attach_node_in_llist = false;
+ if (reset_dl_bw && cs->dl_bw_cpu >= 0)
+ dl_bw_free(cs->dl_bw_cpu, cs->sum_migrate_dl_bw);
+ if (cs->nr_migrate_dl_tasks && !reset_dl_bw)
+ cs->nr_deadline_tasks += cs->nr_migrate_dl_tasks;
+ cs->nr_migrate_dl_tasks = 0;
+ cs->sum_migrate_dl_bw = 0;
+ cs->dl_bw_cpu = -1;
+ }
+
+ src_cs_head.first = NULL;
+ dst_cs_head.first = NULL;
+ attach_cpus_updated = false;
+ attach_mems_updated = false;
+}
+
+static int alloc_dl_bw(void)
{
+ struct cpuset *cs;
int cpu, ret;
- if (!cs->sum_migrate_dl_bw)
- return 0;
+ llist_for_each_entry(cs, dst_cs_head.first, attach_node) {
+ if (!cs->sum_migrate_dl_bw)
+ continue;
- cpu = cpumask_any_and(cpu_active_mask, cs->effective_cpus);
- if (unlikely(cpu >= nr_cpu_ids))
- return -EINVAL;
+ cpu = cpumask_any_and(cpu_active_mask, cs->effective_cpus);
+ if (unlikely(cpu >= nr_cpu_ids))
+ return -EINVAL;
- ret = dl_bw_alloc(cpu, cs->sum_migrate_dl_bw);
- if (ret)
- return ret;
+ ret = dl_bw_alloc(cpu, cs->sum_migrate_dl_bw);
+ if (ret)
+ return ret;
- cs->dl_bw_cpu = cpu;
+ cs->dl_bw_cpu = cpu;
+ }
return 0;
}
-static void reset_migrate_dl_data(struct cpuset *cs)
+static void set_attach_in_progress(void)
{
- cs->nr_migrate_dl_tasks = 0;
- cs->sum_migrate_dl_bw = 0;
- cs->dl_bw_cpu = -1;
+ struct cpuset *cs;
+
+ /*
+ * Mark attach is in progress. This makes validate_change() fail
+ * changes which zero cpus/mems_allowed.
+ */
+ llist_for_each_entry(cs, dst_cs_head.first, attach_node)
+ cs->attach_in_progress++;
}
-/* Called by cgroups to determine if a cpuset is usable; cpuset_mutex held */
+static void reset_attach_in_progress(void)
+{
+ struct cpuset *cs;
+
+ llist_for_each_entry(cs, dst_cs_head.first, attach_node)
+ dec_attach_in_progress_locked(cs);
+}
+
+/*
+ * Called by cgroups to determine if a cpuset is usable; cpuset_mutex held.
+ *
+ * With cgroup v2, enabling of cpuset controller in a cgroup subtree can
+ * cause @tset to contain task migration data from one parent cpuset to multiple
+ * child cpusets. Not much is needed to be done here other than tracking the
+ * number of DL tasks in each cpuset as the CPUs and memory nodes of the child
+ * cpusets are exactly the same as the parent.
+ *
+ * Conversely, disabling of cpuset controller can cause @tset to contain task
+ * migration data from multiple child cpusets to one parent cpuset. Here, the
+ * CPUs and memory nodes of the child cpusets may be different from the parent,
+ * but must be a subset of its parent.
+ *
+ * Another possible many-to-one migration is the moving of the whole
+ * multithreaded process with threads in different cpusets to another cpuset.
+ * Alternatively, multiple processes from multiple cpusets can be moved to
+ * another cpuset in a single operation.
+ *
+ * For all other use cases including cgroup v1, @tset task migration data
+ * should be from one source cpuset to one destination cpuset.
+ */
static int cpuset_can_attach(struct cgroup_taskset *tset)
{
struct cgroup_subsys_state *css;
@@ -3052,8 +3146,7 @@ static int cpuset_can_attach(struct cgroup_taskset *tset)
int ret;
/* used later by cpuset_attach() */
- cpuset_attach_old_cs = task_cs(cgroup_taskset_first(tset, &css));
- oldcs = cpuset_attach_old_cs;
+ oldcs = task_cs(cgroup_taskset_first(tset, &css));
cs = css_cs(css);
mutex_lock(&cpuset_mutex);
@@ -3064,6 +3157,16 @@ static int cpuset_can_attach(struct cgroup_taskset *tset)
goto out_unlock;
cgroup_taskset_for_each(task, css, tset) {
+ struct cpuset *newcs = css_cs(css);
+ struct cpuset *new_oldcs = task_cs(task);
+
+ if ((newcs != cs) || (new_oldcs != oldcs)) {
+ cs = newcs;
+ oldcs = new_oldcs;
+ ret = cpuset_can_attach_check(cs, oldcs, &setsched_check);
+ if (ret)
+ goto out_unlock;
+ }
ret = task_can_attach(task);
if (ret)
goto out_unlock;
@@ -3081,23 +3184,18 @@ static int cpuset_can_attach(struct cgroup_taskset *tset)
* contribute to sum_migrate_dl_bw.
*/
cs->nr_migrate_dl_tasks++;
+ oldcs->nr_migrate_dl_tasks--;
if (dl_task_needs_bw_move(task, cs->effective_cpus))
cs->sum_migrate_dl_bw += task->dl.dl_bw;
}
}
- ret = alloc_dl_bw(cs);
-
+ ret = alloc_dl_bw();
out_unlock:
if (ret)
- reset_migrate_dl_data(cs);
+ clear_attach_data(true);
else
- /*
- * Mark attach is in progress. This makes validate_change() fail
- * changes which zero cpus/mems_allowed.
- */
- cs->attach_in_progress++;
-
+ set_attach_in_progress();
mutex_unlock(&cpuset_mutex);
return ret;
}
@@ -3111,14 +3209,8 @@ static void cpuset_cancel_attach(struct cgroup_taskset *tset)
cs = css_cs(css);
mutex_lock(&cpuset_mutex);
- dec_attach_in_progress_locked(cs);
-
- if (cs->dl_bw_cpu >= 0)
- dl_bw_free(cs->dl_bw_cpu, cs->sum_migrate_dl_bw);
-
- if (cs->nr_migrate_dl_tasks)
- reset_migrate_dl_data(cs);
-
+ reset_attach_in_progress();
+ clear_attach_data(true);
mutex_unlock(&cpuset_mutex);
}
@@ -3172,8 +3264,7 @@ static void cpuset_attach(struct cgroup_taskset *tset)
struct task_struct *task;
struct task_struct *leader;
struct cgroup_subsys_state *css;
- struct cpuset *cs;
- struct cpuset *oldcs = cpuset_attach_old_cs;
+ struct cpuset *cs, *oldcs;
bool queue_task_work = false;
cgroup_taskset_first(tset, &css);
@@ -3184,9 +3275,9 @@ static void cpuset_attach(struct cgroup_taskset *tset)
/*
* In the default hierarchy, enabling cpuset in the child cgroups
- * will trigger a number of cpuset_attach() calls with no change
- * in effective cpus and mems. In that case, we can optimize out
- * by skipping the task iteration and update.
+ * will trigger a cpuset_attach() call with no change in effective cpus
+ * and mems. In that case, we can optimize out by skipping the task
+ * iteration and update.
*/
if (cpuset_v2() && !attach_cpus_updated && !attach_mems_updated) {
cpuset_attach_nodemask_to = cs->effective_mems;
@@ -3195,8 +3286,16 @@ static void cpuset_attach(struct cgroup_taskset *tset)
guarantee_online_mems(cs, &cpuset_attach_nodemask_to);
- cgroup_taskset_for_each(task, css, tset)
+ cgroup_taskset_for_each(task, css, tset) {
+ struct cpuset *newcs = css_cs(css);
+
+ if (newcs != cs) {
+ cs->old_mems_allowed = cs->effective_mems;
+ cs = newcs;
+ guarantee_online_mems(cs, &cpuset_attach_nodemask_to);
+ }
cpuset_attach_task(cs, task);
+ }
/*
* Change mm for all threadgroup leaders. This is expensive and may
@@ -3208,6 +3307,11 @@ static void cpuset_attach(struct cgroup_taskset *tset)
if (!is_memory_migrate(cs))
goto out;
+ /*
+ * Only v1 supports memory_migrate and there should only be one source
+ * and one destination cpuset.
+ */
+ oldcs = llist_entry(src_cs_head.first, struct cpuset, attach_node);
cgroup_taskset_for_each_leader(leader, css, tset) {
struct mm_struct *mm = get_task_mm(leader);
@@ -3231,14 +3335,8 @@ static void cpuset_attach(struct cgroup_taskset *tset)
out:
cs->old_mems_allowed = cpuset_attach_nodemask_to;
- if (cs->nr_migrate_dl_tasks) {
- cs->nr_deadline_tasks += cs->nr_migrate_dl_tasks;
- oldcs->nr_deadline_tasks -= cs->nr_migrate_dl_tasks;
- reset_migrate_dl_data(cs);
- }
-
- dec_attach_in_progress_locked(cs);
-
+ reset_attach_in_progress();
+ clear_attach_data(false);
mutex_unlock(&cpuset_mutex);
}
--
2.54.0
^ permalink raw reply related
* [PATCH cgroup/for-next 3/4] cgroup/cpuset: Optimize cpuset_attach_task()
From: Waiman Long @ 2026-05-14 17:02 UTC (permalink / raw)
To: Chen Ridong, Tejun Heo, Johannes Weiner, Michal Koutný
Cc: cgroups, linux-kernel, Dietmar Eggemann, Aaron Tomlin, Juri Lelli,
Waiman Long
In-Reply-To: <20260514170240.575156-1-longman@redhat.com>
Within cpuset_attach(), cpuset_attach_task() is called only if either the
CPU and/or the memory setting are updated. If only one of the settings
is updated, cpuset_attach_task() still updates both CPU and memory node
setting of each task. Further optimize it by checking attach_cpus_updated
and attach_mems_updated for v2 to skip the unnecessary update.
While at it, also move the mpol_rebind_mm() call for mm group leader
to cpuset_attach_task(). This change shouldn't affect the cpuset_fork()
caller as the newly cloned task isn't the group leader. For that caller,
it is assumed that both CPU and memory nodes are updated to keep the
existing behavior.
Signed-off-by: Waiman Long <longman@redhat.com>
---
kernel/cgroup/cpuset.c | 36 ++++++++++++++++++++++++++----------
1 file changed, 26 insertions(+), 10 deletions(-)
diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 68392cf6429b..8ced1fa0900f 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -3132,8 +3132,13 @@ static nodemask_t cpuset_attach_nodemask_to;
static void cpuset_attach_task(struct cpuset *cs, struct task_struct *task)
{
+ struct mm_struct *mm;
+
lockdep_assert_cpuset_lock_held();
+ if (cpuset_v2() && !attach_cpus_updated)
+ goto update_mem;
+
if (cs != &top_cpuset)
guarantee_active_cpus(task, cpus_attach);
else
@@ -3145,8 +3150,21 @@ static void cpuset_attach_task(struct cpuset *cs, struct task_struct *task)
*/
WARN_ON_ONCE(set_cpus_allowed_ptr(task, cpus_attach));
+update_mem:
+ if (cpuset_v2() && !attach_mems_updated)
+ return;
+
cpuset_change_task_nodemask(task, &cpuset_attach_nodemask_to);
cpuset1_update_task_spread_flags(cs, task);
+
+ if (task != task->group_leader)
+ return;
+
+ mm = get_task_mm(task);
+ if (mm) {
+ mpol_rebind_mm(mm, &cs->effective_mems);
+ mmput(mm);
+ }
}
static void cpuset_attach(struct cgroup_taskset *tset)
@@ -3187,15 +3205,13 @@ static void cpuset_attach(struct cgroup_taskset *tset)
* not set.
*/
cpuset_attach_nodemask_to = cs->effective_mems;
- if (!is_memory_migrate(cs) && !attach_mems_updated)
+ if (!is_memory_migrate(cs))
goto out;
cgroup_taskset_for_each_leader(leader, css, tset) {
struct mm_struct *mm = get_task_mm(leader);
if (mm) {
- mpol_rebind_mm(mm, &cpuset_attach_nodemask_to);
-
/*
* old_mems_allowed is the same with mems_allowed
* here, except if this task is being moved
@@ -3204,18 +3220,15 @@ static void cpuset_attach(struct cgroup_taskset *tset)
* @old_mems_allowed is the right nodesets that we
* migrate mm from.
*/
- if (is_memory_migrate(cs)) {
- cpuset_migrate_mm(mm, &oldcs->old_mems_allowed,
- &cpuset_attach_nodemask_to);
- queue_task_work = true;
- } else
- mmput(mm);
+ cpuset_migrate_mm(mm, &oldcs->old_mems_allowed,
+ &cpuset_attach_nodemask_to);
+ queue_task_work = true;
}
}
-out:
if (queue_task_work)
schedule_flush_migrate_mm();
+out:
cs->old_mems_allowed = cpuset_attach_nodemask_to;
if (cs->nr_migrate_dl_tasks) {
@@ -3666,7 +3679,10 @@ static void cpuset_fork(struct task_struct *task)
/* CLONE_INTO_CGROUP */
mutex_lock(&cpuset_mutex);
guarantee_online_mems(cs, &cpuset_attach_nodemask_to);
+ /* Assume CPUs and memory nodes are updated */
+ attach_cpus_updated = attach_mems_updated = true;
cpuset_attach_task(cs, task);
+ attach_cpus_updated = attach_mems_updated = false;
dec_attach_in_progress_locked(cs);
mutex_unlock(&cpuset_mutex);
--
2.54.0
^ permalink raw reply related
* [PATCH cgroup/for-next 2/4] cgroup/cpuset: Expand the scope of cpuset_can_attach_check()
From: Waiman Long @ 2026-05-14 17:02 UTC (permalink / raw)
To: Chen Ridong, Tejun Heo, Johannes Weiner, Michal Koutný
Cc: cgroups, linux-kernel, Dietmar Eggemann, Aaron Tomlin, Juri Lelli,
Waiman Long
In-Reply-To: <20260514170240.575156-1-longman@redhat.com>
Expand the scope of cpuset_can_attach_check() by including the setting
of setsched flag inside cpuset_can_attach_check() with the new @oldcs
and @psetsched argument. As cpuset_can_attach_check() is also called
from cpuset_can_fork(), set the new arguments to NULL from that caller.
While at it, expose the source and destination cpuset cpu/memory check
results in the new attach_cpus_updated and attach_mems_updated static
flags so that these flags can be used directly from cpuset_attach()
without the need to do the same computations again.
No functional change is expected.
Signed-off-by: Waiman Long <longman@redhat.com>
---
kernel/cgroup/cpuset.c | 69 +++++++++++++++++++++++++-----------------
1 file changed, 41 insertions(+), 28 deletions(-)
diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 9de3c907436f..68392cf6429b 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -2964,19 +2964,55 @@ static int update_prstate(struct cpuset *cs, int new_prs)
return 0;
}
+/*
+ * cpuset_can_attach() and cpuset_attach() specific internal data
+ */
static struct cpuset *cpuset_attach_old_cs;
+static bool attach_cpus_updated;
+static bool attach_mems_updated;
/*
* Check to see if a cpuset can accept a new task
* For v1, cpus_allowed and mems_allowed can't be empty.
* For v2, effective_cpus can't be empty.
* Note that in v1, effective_cpus = cpus_allowed.
+ *
+ * Also set the boolean flag passed in by @psetsched depending on if
+ * security_task_setscheduler() call is needed and @oldcs is not NULL.
*/
-static int cpuset_can_attach_check(struct cpuset *cs)
+static int cpuset_can_attach_check(struct cpuset *cs, struct cpuset *oldcs,
+ bool *psetsched)
{
if (cpumask_empty(cs->effective_cpus) ||
(!is_in_v2_mode() && nodes_empty(cs->mems_allowed)))
return -ENOSPC;
+
+ if (!oldcs)
+ return 0;
+
+ /*
+ * Update attach specific data
+ */
+ attach_cpus_updated = !cpumask_equal(cs->effective_cpus, oldcs->effective_cpus);
+ attach_mems_updated = !nodes_equal(cs->effective_mems, oldcs->effective_mems);
+
+ /*
+ * Skip rights over task setsched check in v2 when nothing changes,
+ * migration permission derives from hierarchy ownership in
+ * cgroup_procs_write_permission()).
+ */
+ *psetsched = !cpuset_v2() || attach_cpus_updated || attach_mems_updated;
+
+ /*
+ * A v1 cpuset with tasks will have no CPU left only when CPU hotplug
+ * brings the last online CPU offline as users are not allowed to empty
+ * cpuset.cpus when there are active tasks inside. When that happens,
+ * we should allow tasks to migrate out without security check to make
+ * sure they will be able to run after migration.
+ */
+ if (!is_in_v2_mode() && cpumask_empty(oldcs->effective_cpus))
+ *psetsched = false;
+
return 0;
}
@@ -3023,29 +3059,10 @@ static int cpuset_can_attach(struct cgroup_taskset *tset)
mutex_lock(&cpuset_mutex);
/* Check to see if task is allowed in the cpuset */
- ret = cpuset_can_attach_check(cs);
+ ret = cpuset_can_attach_check(cs, oldcs, &setsched_check);
if (ret)
goto out_unlock;
- /*
- * Skip rights over task setsched check in v2 when nothing changes,
- * migration permission derives from hierarchy ownership in
- * cgroup_procs_write_permission()).
- */
- setsched_check = !cpuset_v2() ||
- !cpumask_equal(cs->effective_cpus, oldcs->effective_cpus) ||
- !nodes_equal(cs->effective_mems, oldcs->effective_mems);
-
- /*
- * A v1 cpuset with tasks will have no CPU left only when CPU hotplug
- * brings the last online CPU offline as users are not allowed to empty
- * cpuset.cpus when there are active tasks inside. When that happens,
- * we should allow tasks to migrate out without security check to make
- * sure they will be able to run after migration.
- */
- if (!is_in_v2_mode() && cpumask_empty(oldcs->effective_cpus))
- setsched_check = false;
-
cgroup_taskset_for_each(task, css, tset) {
ret = task_can_attach(task);
if (ret)
@@ -3139,7 +3156,6 @@ static void cpuset_attach(struct cgroup_taskset *tset)
struct cgroup_subsys_state *css;
struct cpuset *cs;
struct cpuset *oldcs = cpuset_attach_old_cs;
- bool cpus_updated, mems_updated;
bool queue_task_work = false;
cgroup_taskset_first(tset, &css);
@@ -3147,9 +3163,6 @@ static void cpuset_attach(struct cgroup_taskset *tset)
lockdep_assert_cpus_held(); /* see cgroup_attach_lock() */
mutex_lock(&cpuset_mutex);
- cpus_updated = !cpumask_equal(cs->effective_cpus,
- oldcs->effective_cpus);
- mems_updated = !nodes_equal(cs->effective_mems, oldcs->effective_mems);
/*
* In the default hierarchy, enabling cpuset in the child cgroups
@@ -3157,7 +3170,7 @@ static void cpuset_attach(struct cgroup_taskset *tset)
* in effective cpus and mems. In that case, we can optimize out
* by skipping the task iteration and update.
*/
- if (cpuset_v2() && !cpus_updated && !mems_updated) {
+ if (cpuset_v2() && !attach_cpus_updated && !attach_mems_updated) {
cpuset_attach_nodemask_to = cs->effective_mems;
goto out;
}
@@ -3174,7 +3187,7 @@ static void cpuset_attach(struct cgroup_taskset *tset)
* not set.
*/
cpuset_attach_nodemask_to = cs->effective_mems;
- if (!is_memory_migrate(cs) && !mems_updated)
+ if (!is_memory_migrate(cs) && !attach_mems_updated)
goto out;
cgroup_taskset_for_each_leader(leader, css, tset) {
@@ -3589,7 +3602,7 @@ static int cpuset_can_fork(struct task_struct *task, struct css_set *cset)
mutex_lock(&cpuset_mutex);
/* Check to see if task is allowed in the cpuset */
- ret = cpuset_can_attach_check(cs);
+ ret = cpuset_can_attach_check(cs, NULL, NULL);
if (ret)
goto out_unlock;
--
2.54.0
^ permalink raw reply related
* [PATCH cgroup/for-next 1/4] cgroup/cpuset: Add an alloc_dl_bw() helper
From: Waiman Long @ 2026-05-14 17:02 UTC (permalink / raw)
To: Chen Ridong, Tejun Heo, Johannes Weiner, Michal Koutný
Cc: cgroups, linux-kernel, Dietmar Eggemann, Aaron Tomlin, Juri Lelli,
Waiman Long
In-Reply-To: <20260514170240.575156-1-longman@redhat.com>
Extract the DL bandwidth allocation code in cpuset_attach() to a new
alloc_dl_bw() helper to simplify code.
No functional change is expected.
Signed-off-by: Waiman Long <longman@redhat.com>
---
kernel/cgroup/cpuset.c | 50 +++++++++++++++++++++++-------------------
1 file changed, 28 insertions(+), 22 deletions(-)
diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index bcefc9f50ac5..9de3c907436f 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -2980,6 +2980,25 @@ static int cpuset_can_attach_check(struct cpuset *cs)
return 0;
}
+static int alloc_dl_bw(struct cpuset *cs)
+{
+ int cpu, ret;
+
+ if (!cs->sum_migrate_dl_bw)
+ return 0;
+
+ cpu = cpumask_any_and(cpu_active_mask, cs->effective_cpus);
+ if (unlikely(cpu >= nr_cpu_ids))
+ return -EINVAL;
+
+ ret = dl_bw_alloc(cpu, cs->sum_migrate_dl_bw);
+ if (ret)
+ return ret;
+
+ cs->dl_bw_cpu = cpu;
+ return 0;
+}
+
static void reset_migrate_dl_data(struct cpuset *cs)
{
cs->nr_migrate_dl_tasks = 0;
@@ -2994,7 +3013,7 @@ static int cpuset_can_attach(struct cgroup_taskset *tset)
struct cpuset *cs, *oldcs;
struct task_struct *task;
bool setsched_check;
- int cpu, ret;
+ int ret;
/* used later by cpuset_attach() */
cpuset_attach_old_cs = task_cs(cgroup_taskset_first(tset, &css));
@@ -3050,31 +3069,18 @@ static int cpuset_can_attach(struct cgroup_taskset *tset)
}
}
- if (!cs->sum_migrate_dl_bw)
- goto out_success;
-
- cpu = cpumask_any_and(cpu_active_mask, cs->effective_cpus);
- if (unlikely(cpu >= nr_cpu_ids)) {
- ret = -EINVAL;
- goto out_unlock;
- }
-
- ret = dl_bw_alloc(cpu, cs->sum_migrate_dl_bw);
- if (ret)
- goto out_unlock;
-
- cs->dl_bw_cpu = cpu;
-
-out_success:
- /*
- * Mark attach is in progress. This makes validate_change() fail
- * changes which zero cpus/mems_allowed.
- */
- cs->attach_in_progress++;
+ ret = alloc_dl_bw(cs);
out_unlock:
if (ret)
reset_migrate_dl_data(cs);
+ else
+ /*
+ * Mark attach is in progress. This makes validate_change() fail
+ * changes which zero cpus/mems_allowed.
+ */
+ cs->attach_in_progress++;
+
mutex_unlock(&cpuset_mutex);
return ret;
}
--
2.54.0
^ permalink raw reply related
* [PATCH cgroup/for-next 0/4] cgroup/cpuset: Support multiple source/destination cpusets for cpuset_*attach()
From: Waiman Long @ 2026-05-14 17:02 UTC (permalink / raw)
To: Chen Ridong, Tejun Heo, Johannes Weiner, Michal Koutný
Cc: cgroups, linux-kernel, Dietmar Eggemann, Aaron Tomlin, Juri Lelli,
Waiman Long
Sashiko AI review of another cpuset patch had found that cpuset_attach()
and cpuset_can_attach() can be passed a cgroup_taskset with tasks
migrating from one source cpuset to multiple destination cpusets and
vice versa. Further testing of the cpuset code indicates that this is
indeed the case when the v2 cpuset controller is enabled or disabled.
Unfortunately, cpuset_attach() and cpuset_can_attach() still assume that
there will be one source and one destinaton cpuset which may result in
inocrrect behavior. This patch series is created to fix this issue. The
first 3 patches are just preparatory patches to make it easier to review
the last patch which fixes this problem.
Waiman Long (4):
cgroup/cpuset: Add an alloc_dl_bw() helper
cgroup/cpuset: Expand the scope of cpuset_can_attach_check()
cgroup/cpuset: Optimize cpuset_attach_task()
cgroup/cpuset: Support multiple source/destination cpusets for
cpuset_*attach()
kernel/cgroup/cpuset-internal.h | 6 +
kernel/cgroup/cpuset.c | 315 +++++++++++++++++++++++---------
2 files changed, 230 insertions(+), 91 deletions(-)
--
2.54.0
^ permalink raw reply
* Re: [GIT PULL] cgroup: Fixes for v7.1-rc3
From: pr-tracker-bot @ 2026-05-14 16:40 UTC (permalink / raw)
To: Tejun Heo
Cc: Linus Torvalds, Johannes Weiner, Michal Koutný, Waiman Long,
cgroups, linux-kernel
In-Reply-To: <6038eeed9424f26cc6bec8c0a01c8e43@kernel.org>
The pull request you sent on Wed, 13 May 2026 10:59:45 -1000:
> git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup.git tags/cgroup-for-7.1-rc3-fixes
has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/0913b580f8490caaaf08dd1591e0bc07ac2720cb
Thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/prtracker.html
^ permalink raw reply
* Re: [linus:master] [mm] 01b9da291c: stress-ng.switch.ops_per_sec 67.7% regression
From: Shakeel Butt @ 2026-05-14 13:40 UTC (permalink / raw)
To: Qi Zheng, kernel test robot
Cc: oe-lkp, lkp, linux-kernel, Andrew Morton, David Carlier,
Allen Pais, Axel Rasmussen, Baoquan He, Chengming Zhou,
Chen Ridong, David Hildenbrand, Hamza Mahfooz, Harry Yoo,
Hugh Dickins, Imran Khan, Johannes Weiner, Kamalesh Babulal,
Lance Yang, Liam Howlett, Lorenzo Stoakes, Michal Hocko,
Michal Koutný, Mike Rapoport, Muchun Song, Muchun Song,
Nhat Pham, Roman Gushchin, Suren Baghdasaryan, Usama Arif,
Vlastimil Babka, Wei Xu, Yosry Ahmed, Yuanchu Xie, Zi Yan,
Usama Arif, cgroups, linux-mm
In-Reply-To: <46e9f5cf-34cb-466d-a53a-5778768af4d9@linux.dev>
May 14, 2026 at 12:46 AM, "Qi Zheng" <qi.zheng@linux.dev mailto:qi.zheng@linux.dev?to=%22Qi%20Zheng%22%20%3Cqi.zheng%40linux.dev%3E > wrote:
>
> On 5/13/26 10:27 PM, Shakeel Butt wrote:
>
> >
> > On Wed, May 13, 2026 at 06:49:45AM -0700, Shakeel Butt wrote:
> >
> > >
> > > On Wed, May 13, 2026 at 10:10:34AM +0800, Qi Zheng wrote:
> > >
> > On 5/13/26 12:03 AM, Shakeel Butt wrote:
> > On Tue, May 12, 2026 at 08:56:52PM +0800, kernel test robot wrote:
> >
> > Hello,
> >
> > kernel test robot noticed a 67.7% regression of stress-ng.switch.ops_per_sec on:
> >
> > commit: 01b9da291c4969354807b52956f4aae1f41b4924 ("mm: memcontrol: convert objcg to be per-memcg per-node type")
> > https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
> >
> > This is most probably due to shuffling of struct mem_cgroup and struct
> > mem_cgroup_per_node members.
> >
> > Another possibility is that after objcg was split into per-node, the
> > slab accounting fast path is still designed assuming only one current
> > objcg per CPU:
> >
> > struct obj_stock_pcp {
> > struct obj_cgroup *cached_objcg;
> > };
> >
> > So it's may cause the following thrashing:
> >
> > CPU stock cached = memcg/node0 objcg
> > free object tagged = memcg/node1 objcg
> > => __refill_obj_stock --> objcg mismatch
> > => drain_obj_stock()
> > => cache switches to node1 objcg
> >
> > next local allocation tagged = node0 objcg
> > => mismatch again
> > => drain_obj_stock()
> >
> > >
> > > Actually I think this is the issue, we have ping pong threads running on
> > > different nodes where though theu are in same cgroup but their current->obcg is
> > > for local node and thus this ping pong is thrashing the per-cpu objcg stock.
> > >
> > > The easier fix would be to compare objcg->memcg instead of just objcg during
> > > draining and caching. In addition we can add support for multiple objcg per-cpu
> > > stock caching.
> > >
> > Something like the following:
> > From d756abe831a905d6fe32bad9a984fc619dafb7e0 Mon Sep 17 00:00:00 2001
> > From: Shakeel Butt <shakeel.butt@linux.dev>
> > Date: Wed, 13 May 2026 07:24:55 -0700
> > Subject: [PATCH] mm/memcontrol: skip obj_stock drain when refilled objcg
> > shares memcg
> > Signed-off-by: Shakeel Butt <shakeel.butt@linux.dev>
> > ---
> > mm/memcontrol.c | 14 +++++++++++++-
> > 1 file changed, 13 insertions(+), 1 deletion(-)
> > diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> > index d978e18b9b2d..01ed7a8e18ac 100644
> > --- a/mm/memcontrol.c
> > +++ b/mm/memcontrol.c
> > @@ -3318,6 +3318,7 @@ static void __refill_obj_stock(struct obj_cgroup *objcg,
> > unsigned int nr_bytes,
> > bool allow_uncharge)
> > {
> > + struct obj_cgroup *cached;
> > unsigned int nr_pages = 0;
> > > if (!stock) {
> > @@ -3327,7 +3328,18 @@ static void __refill_obj_stock(struct obj_cgroup *objcg,
> > goto out;
> > }
> > > - if (READ_ONCE(stock->cached_objcg) != objcg) { /* reset if necessary */
> > + cached = READ_ONCE(stock->cached_objcg);
> > + if (cached != objcg &&
> > + (!cached || obj_cgroup_memcg(cached) != obj_cgroup_memcg(objcg))) {
> > drain_obj_stock(stock);
> > obj_cgroup_get(objcg);
> > stock->nr_bytes = atomic_read(&objcg->nr_charged_bytes)
> >
> This change looks like it should be able to fix the ping-pong issue, but
> I stiil haven't reproduced the performance regression locally. I'll
> continue testing it.
Same here, couldn't reproduce locally. It seems like we had to craft a scenario
where the pair pingpong threads get their current->objcg from different nodes.
I will try that.
>
> Hi kernel-test-robot, could you help check if the patch above fixes the
> issue on your end?
>
In the meantime, Oliver, can you please help in testing this patch?
^ permalink raw reply
* Re: [PATCH v3 12/12] mm, swap: merge zeromap into swap table
From: Kairui Song @ 2026-05-14 9:34 UTC (permalink / raw)
To: YoungJun Park
Cc: linux-mm, Andrew Morton, David Hildenbrand, Zi Yan, Baolin Wang,
Barry Song, Hugh Dickins, Chris Li, Kemeng Shi, Nhat Pham,
Baoquan He, Johannes Weiner, Chengming Zhou, Roman Gushchin,
Shakeel Butt, Muchun Song, Qi Zheng, linux-kernel, cgroups,
Yosry Ahmed, Lorenzo Stoakes, Dev Jain, Lance Yang, Michal Hocko,
Michal Hocko, Suren Baghdasaryan, Axel Rasmussen
In-Reply-To: <agTCeEuMWJtYN9EF@yjaykim-PowerEdge-T330>
On Thu, May 14, 2026 at 2:33 AM YoungJun Park <youngjun.park@lge.com> wrote:
>
> On Tue, Apr 21, 2026 at 02:16:56PM +0800, Kairui Song via B4 Relay wrote:
>
> Nice! LGTM
>
> Reviewed-by: Youngjun Park <youngjun.park@lge.com>
>
> A few nitpicks follow. take them if you find them useful. :)
Thanks!
>
> > +static inline void __swap_table_clear_zero(struct swap_cluster_info *ci,
> > + unsigned int ci_off)
> > +{
> > +
>
> trailing blank line.
>
> > +#if SWAP_TABLE_HAS_ZEROFLAG
> > + unsigned long swp_tb = __swap_table_get(ci, ci_off);
> > +
> > + VM_WARN_ON(!swp_tb_is_countable(swp_tb));
> > + swp_tb &= ~SWP_TB_ZERO_FLAG;
> > + __swap_table_set(ci, ci_off, swp_tb);
> > +#else
> > + lockdep_assert_held(&ci->lock);
> > + __clear_bit(ci_off, ci->zero_bitmap);
> > +#endif
> > +}
> ...
> > +
> > table = (struct swap_table *)rcu_access_pointer(ci->table);
> > if (!table)
> > return;
> > @@ -470,6 +475,13 @@ static int swap_cluster_alloc_table(struct swap_cluster_info *ci, gfp_t gfp)
> > if (!ci->memcg_table)
> > ret = -ENOMEM;
> > #endif
> > +
> > +#if !SWAP_TABLE_HAS_ZEROFLAG
> > + ci->zero_bitmap = bitmap_zalloc(SWAPFILE_CLUSTER, gfp);
> > + if (!ci->zero_bitmap)
> > + ret = -ENOMEM;
> > +#endif
> > +
>
> memcg_table above uses `if (!ci->memcg_table)` before
> kzalloc, but ci->zero_bitmap is assigned unconditionally. Both
> are NULL on entry today (swap_cluster_free_table nullifies them),
> so either form works but the asymmetry reads oddly. Either
> drop the memcg guard (with an entry VM_WARN_ON asserting both
> NULL by design), or mirror the guard here.
Good idea. Let me turn them into VM_WARN_ON. They should never be
non-NULL right now, and proceeding after a WARN shouldn't be
catastrophic.
I also notice I better let them do something like goto err_free on the
failure case, where err_free aborts early and frees everything. If any
allocation fails, the entire cluster should be freed for cleanliness
instead of proceeding. I will update this part.
^ permalink raw reply
* [tj-cgroup:for-7.1-fixes] BUILD SUCCESS 345f40166694e60db6d5cf02233814bb27ac5dec
From: kernel test robot @ 2026-05-14 9:28 UTC (permalink / raw)
To: Tejun Heo; +Cc: cgroups
tree/branch: https://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup.git for-7.1-fixes
branch HEAD: 345f40166694e60db6d5cf02233814bb27ac5dec cgroup/cpuset: Return only actually allocated CPUs during partition invalidation
elapsed time: 871m
configs tested: 222
configs skipped: 8
The following configs have been built successfully.
More configs may be tested in the coming days.
tested configs:
alpha allnoconfig gcc-15.2.0
alpha allyesconfig gcc-15.2.0
alpha defconfig gcc-15.2.0
arc allmodconfig clang-16
arc allnoconfig gcc-15.2.0
arc allyesconfig clang-23
arc defconfig gcc-15.2.0
arc randconfig-001 gcc-8.5.0
arc randconfig-001-20260514 clang-23
arc randconfig-001-20260514 gcc-8.5.0
arc randconfig-002 gcc-8.5.0
arc randconfig-002-20260514 clang-23
arc randconfig-002-20260514 gcc-8.5.0
arm allnoconfig gcc-15.2.0
arm allyesconfig clang-16
arm defconfig gcc-15.2.0
arm randconfig-001 gcc-8.5.0
arm randconfig-001-20260514 clang-23
arm randconfig-001-20260514 gcc-8.5.0
arm randconfig-002 gcc-8.5.0
arm randconfig-002-20260514 clang-23
arm randconfig-002-20260514 gcc-8.5.0
arm randconfig-003 gcc-8.5.0
arm randconfig-003-20260514 clang-23
arm randconfig-003-20260514 gcc-8.5.0
arm randconfig-004 gcc-8.5.0
arm randconfig-004-20260514 clang-23
arm randconfig-004-20260514 gcc-8.5.0
arm vf610m4_defconfig gcc-15.2.0
arm64 allmodconfig clang-23
arm64 allnoconfig gcc-15.2.0
arm64 defconfig gcc-15.2.0
arm64 randconfig-001-20260514 clang-23
arm64 randconfig-002-20260514 clang-23
arm64 randconfig-003-20260514 clang-23
arm64 randconfig-004-20260514 clang-23
csky allmodconfig gcc-15.2.0
csky allnoconfig gcc-15.2.0
csky defconfig gcc-15.2.0
csky randconfig-001-20260514 clang-23
csky randconfig-002-20260514 clang-23
hexagon allmodconfig gcc-15.2.0
hexagon allnoconfig gcc-15.2.0
hexagon defconfig gcc-15.2.0
hexagon randconfig-001 gcc-11.5.0
hexagon randconfig-001-20260514 gcc-10.5.0
hexagon randconfig-002 gcc-11.5.0
hexagon randconfig-002-20260514 gcc-10.5.0
i386 allmodconfig clang-20
i386 allnoconfig gcc-15.2.0
i386 allyesconfig clang-20
i386 buildonly-randconfig-001 gcc-14
i386 buildonly-randconfig-001-20260514 gcc-14
i386 buildonly-randconfig-002 gcc-14
i386 buildonly-randconfig-002-20260514 gcc-14
i386 buildonly-randconfig-003 gcc-14
i386 buildonly-randconfig-003-20260514 gcc-14
i386 buildonly-randconfig-004 gcc-14
i386 buildonly-randconfig-004-20260514 gcc-14
i386 buildonly-randconfig-005 gcc-14
i386 buildonly-randconfig-005-20260514 gcc-14
i386 buildonly-randconfig-006 gcc-14
i386 buildonly-randconfig-006-20260514 gcc-14
i386 defconfig gcc-15.2.0
i386 randconfig-001-20260514 clang-20
i386 randconfig-002-20260514 clang-20
i386 randconfig-003-20260514 clang-20
i386 randconfig-004-20260514 clang-20
i386 randconfig-005-20260514 clang-20
i386 randconfig-006-20260514 clang-20
i386 randconfig-007-20260514 clang-20
i386 randconfig-011-20260514 clang-20
i386 randconfig-012-20260514 clang-20
i386 randconfig-013-20260514 clang-20
i386 randconfig-014-20260514 clang-20
i386 randconfig-015-20260514 clang-20
i386 randconfig-016-20260514 clang-20
i386 randconfig-017-20260514 clang-20
loongarch allmodconfig clang-23
loongarch allnoconfig gcc-15.2.0
loongarch defconfig clang-19
loongarch randconfig-001 gcc-11.5.0
loongarch randconfig-001-20260514 gcc-10.5.0
loongarch randconfig-002 gcc-11.5.0
loongarch randconfig-002-20260514 gcc-10.5.0
m68k allmodconfig gcc-15.2.0
m68k allnoconfig gcc-15.2.0
m68k allyesconfig clang-16
m68k defconfig clang-19
microblaze allnoconfig gcc-15.2.0
microblaze allyesconfig gcc-15.2.0
microblaze defconfig clang-19
mips allmodconfig gcc-15.2.0
mips allnoconfig gcc-15.2.0
mips allyesconfig gcc-15.2.0
mips bcm63xx_defconfig clang-23
mips maltasmvp_defconfig gcc-15.2.0
mips qi_lb60_defconfig clang-23
nios2 allmodconfig clang-23
nios2 allnoconfig clang-23
nios2 defconfig clang-19
nios2 randconfig-001 gcc-11.5.0
nios2 randconfig-001-20260514 gcc-10.5.0
nios2 randconfig-002 gcc-11.5.0
nios2 randconfig-002-20260514 gcc-10.5.0
openrisc allmodconfig clang-23
openrisc allnoconfig clang-23
openrisc defconfig gcc-15.2.0
parisc allmodconfig gcc-15.2.0
parisc allnoconfig clang-23
parisc allyesconfig clang-19
parisc defconfig gcc-15.2.0
parisc randconfig-001 gcc-13.4.0
parisc randconfig-001-20260514 gcc-13.4.0
parisc randconfig-002 gcc-13.4.0
parisc randconfig-002-20260514 gcc-13.4.0
parisc64 defconfig clang-19
powerpc allmodconfig gcc-15.2.0
powerpc allnoconfig clang-23
powerpc randconfig-001 gcc-13.4.0
powerpc randconfig-001-20260514 gcc-13.4.0
powerpc randconfig-002 gcc-13.4.0
powerpc randconfig-002-20260514 gcc-13.4.0
powerpc tqm8541_defconfig clang-23
powerpc64 randconfig-001 gcc-13.4.0
powerpc64 randconfig-001-20260514 gcc-13.4.0
powerpc64 randconfig-002 gcc-13.4.0
powerpc64 randconfig-002-20260514 gcc-13.4.0
riscv allmodconfig clang-23
riscv allnoconfig clang-23
riscv allyesconfig clang-16
riscv defconfig gcc-15.2.0
riscv randconfig-001-20260514 gcc-14.3.0
riscv randconfig-002-20260514 gcc-14.3.0
s390 allmodconfig clang-19
s390 allnoconfig clang-23
s390 allyesconfig gcc-15.2.0
s390 defconfig gcc-15.2.0
s390 randconfig-001-20260514 gcc-14.3.0
s390 randconfig-002-20260514 gcc-14.3.0
sh allmodconfig gcc-15.2.0
sh allnoconfig clang-23
sh allyesconfig clang-19
sh defconfig gcc-14
sh randconfig-001-20260514 gcc-14.3.0
sh randconfig-002-20260514 gcc-14.3.0
sparc allnoconfig clang-23
sparc defconfig gcc-15.2.0
sparc randconfig-001 gcc-15.2.0
sparc randconfig-001-20260514 gcc-15.2.0
sparc randconfig-002 gcc-15.2.0
sparc randconfig-002-20260514 gcc-15.2.0
sparc64 allmodconfig clang-23
sparc64 defconfig gcc-14
sparc64 randconfig-001 gcc-15.2.0
sparc64 randconfig-001-20260514 gcc-15.2.0
sparc64 randconfig-002 gcc-15.2.0
sparc64 randconfig-002-20260514 gcc-15.2.0
um allmodconfig clang-19
um allnoconfig clang-23
um allyesconfig gcc-15.2.0
um defconfig gcc-14
um i386_defconfig gcc-14
um randconfig-001 gcc-15.2.0
um randconfig-001-20260514 gcc-15.2.0
um randconfig-002 gcc-15.2.0
um randconfig-002-20260514 gcc-15.2.0
um x86_64_defconfig gcc-14
x86_64 allmodconfig clang-20
x86_64 allnoconfig clang-23
x86_64 allyesconfig clang-20
x86_64 buildonly-randconfig-001 clang-20
x86_64 buildonly-randconfig-001-20260514 clang-20
x86_64 buildonly-randconfig-002 clang-20
x86_64 buildonly-randconfig-002-20260514 clang-20
x86_64 buildonly-randconfig-003 clang-20
x86_64 buildonly-randconfig-003-20260514 clang-20
x86_64 buildonly-randconfig-004 clang-20
x86_64 buildonly-randconfig-004-20260514 clang-20
x86_64 buildonly-randconfig-005 clang-20
x86_64 buildonly-randconfig-005-20260514 clang-20
x86_64 buildonly-randconfig-006 clang-20
x86_64 buildonly-randconfig-006-20260514 clang-20
x86_64 defconfig gcc-14
x86_64 kexec clang-20
x86_64 randconfig-001 gcc-14
x86_64 randconfig-001-20260514 gcc-14
x86_64 randconfig-002 gcc-14
x86_64 randconfig-002-20260514 gcc-14
x86_64 randconfig-003 gcc-14
x86_64 randconfig-003-20260514 gcc-14
x86_64 randconfig-004 gcc-14
x86_64 randconfig-004-20260514 gcc-14
x86_64 randconfig-005 gcc-14
x86_64 randconfig-005-20260514 gcc-14
x86_64 randconfig-006 gcc-14
x86_64 randconfig-006-20260514 gcc-14
x86_64 randconfig-011-20260514 clang-20
x86_64 randconfig-012-20260514 clang-20
x86_64 randconfig-013-20260514 clang-20
x86_64 randconfig-014-20260514 clang-20
x86_64 randconfig-015-20260514 clang-20
x86_64 randconfig-016-20260514 clang-20
x86_64 randconfig-071-20260514 clang-20
x86_64 randconfig-072-20260514 clang-20
x86_64 randconfig-073-20260514 clang-20
x86_64 randconfig-074-20260514 clang-20
x86_64 randconfig-075-20260514 clang-20
x86_64 randconfig-076-20260514 clang-20
x86_64 rhel-9.4 clang-20
x86_64 rhel-9.4-bpf gcc-14
x86_64 rhel-9.4-func clang-20
x86_64 rhel-9.4-kselftests clang-20
x86_64 rhel-9.4-kunit gcc-14
x86_64 rhel-9.4-ltp gcc-14
x86_64 rhel-9.4-rust clang-20
xtensa allnoconfig clang-23
xtensa allyesconfig clang-23
xtensa randconfig-001 gcc-15.2.0
xtensa randconfig-001-20260514 gcc-15.2.0
xtensa randconfig-002 gcc-15.2.0
xtensa randconfig-002-20260514 gcc-15.2.0
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply
* [tj-cgroup:for-next] BUILD SUCCESS a234764e334b01b4b4a631b1c94df3458f3f57cb
From: kernel test robot @ 2026-05-14 9:27 UTC (permalink / raw)
To: Tejun Heo; +Cc: cgroups
tree/branch: https://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup.git for-next
branch HEAD: a234764e334b01b4b4a631b1c94df3458f3f57cb Merge branch 'for-7.1-fixes' into for-next
elapsed time: 869m
configs tested: 114
configs skipped: 14
The following configs have been built successfully.
More configs may be tested in the coming days.
tested configs:
alpha allnoconfig gcc-15.2.0
alpha allyesconfig gcc-15.2.0
arc allmodconfig gcc-15.2.0
arc allnoconfig gcc-15.2.0
arc allyesconfig gcc-15.2.0
arm allnoconfig clang-23
arm allyesconfig gcc-15.2.0
arm64 allnoconfig gcc-15.2.0
arm64 randconfig-001-20260514 clang-23
arm64 randconfig-002-20260514 gcc-8.5.0
arm64 randconfig-003-20260514 clang-23
arm64 randconfig-004-20260514 clang-23
csky allmodconfig gcc-15.2.0
csky allnoconfig gcc-15.2.0
csky randconfig-001-20260514 gcc-15.2.0
csky randconfig-002-20260514 gcc-15.2.0
hexagon allmodconfig clang-17
hexagon allnoconfig clang-23
hexagon randconfig-001-20260514 clang-16
hexagon randconfig-002-20260514 clang-23
i386 allmodconfig gcc-14
i386 allnoconfig gcc-14
i386 allyesconfig gcc-14
i386 buildonly-randconfig-004-20260514 clang-20
i386 randconfig-015 gcc-14
i386 randconfig-016 gcc-14
loongarch allmodconfig clang-19
loongarch allnoconfig clang-23
loongarch randconfig-001-20260514 gcc-14.3.0
loongarch randconfig-002-20260514 clang-23
m68k allmodconfig gcc-15.2.0
m68k allnoconfig gcc-15.2.0
m68k allyesconfig gcc-15.2.0
microblaze allnoconfig gcc-15.2.0
microblaze allyesconfig gcc-15.2.0
mips allnoconfig gcc-15.2.0
mips allyesconfig gcc-15.2.0
nios2 allmodconfig gcc-11.5.0
nios2 allnoconfig gcc-11.5.0
nios2 randconfig-001-20260514 gcc-11.5.0
nios2 randconfig-002-20260514 gcc-10.5.0
openrisc allnoconfig gcc-15.2.0
parisc allmodconfig gcc-15.2.0
parisc allnoconfig gcc-15.2.0
parisc allyesconfig gcc-15.2.0
parisc randconfig-001-20260514 gcc-13.4.0
parisc randconfig-002-20260514 gcc-15.2.0
powerpc allnoconfig gcc-15.2.0
powerpc randconfig-001-20260514 gcc-8.5.0
powerpc randconfig-002-20260514 gcc-13.4.0
powerpc64 randconfig-002-20260514 gcc-8.5.0
riscv allmodconfig clang-23
riscv allnoconfig gcc-15.2.0
riscv allyesconfig clang-16
riscv randconfig-001-20260514 gcc-12.5.0
riscv randconfig-002-20260514 clang-20
s390 allmodconfig clang-18
s390 allnoconfig clang-23
s390 allyesconfig gcc-15.2.0
s390 randconfig-001-20260514 gcc-11.5.0
s390 randconfig-002-20260514 gcc-8.5.0
sh allmodconfig gcc-15.2.0
sh allnoconfig gcc-15.2.0
sh allyesconfig gcc-15.2.0
sh defconfig gcc-15.2.0
sh randconfig-001-20260514 gcc-10.5.0
sh randconfig-002-20260514 gcc-14.3.0
sparc allnoconfig gcc-15.2.0
sparc randconfig-001-20260514 gcc-15.2.0
sparc randconfig-002-20260514 gcc-13.4.0
sparc64 allmodconfig clang-23
sparc64 defconfig clang-20
sparc64 randconfig-001-20260514 clang-23
sparc64 randconfig-002-20260514 gcc-10.5.0
um allmodconfig clang-19
um allnoconfig clang-23
um allyesconfig gcc-14
um defconfig clang-23
um i386_defconfig gcc-14
um randconfig-001-20260514 gcc-14
um randconfig-002-20260514 gcc-14
um x86_64_defconfig clang-23
x86_64 allnoconfig clang-20
x86_64 allyesconfig clang-20
x86_64 buildonly-randconfig-001-20260514 gcc-14
x86_64 buildonly-randconfig-002-20260514 clang-20
x86_64 buildonly-randconfig-003-20260514 gcc-14
x86_64 buildonly-randconfig-004-20260514 clang-20
x86_64 buildonly-randconfig-005-20260514 clang-20
x86_64 buildonly-randconfig-006-20260514 gcc-14
x86_64 defconfig gcc-14
x86_64 randconfig-001-20260514 clang-20
x86_64 randconfig-002-20260514 clang-20
x86_64 randconfig-003-20260514 gcc-14
x86_64 randconfig-004-20260514 gcc-14
x86_64 randconfig-005-20260514 clang-20
x86_64 randconfig-006-20260514 gcc-14
x86_64 randconfig-011-20260514 clang-20
x86_64 randconfig-012-20260514 gcc-14
x86_64 randconfig-013-20260514 clang-20
x86_64 randconfig-014-20260514 clang-20
x86_64 randconfig-015-20260514 clang-20
x86_64 randconfig-016-20260514 clang-20
x86_64 randconfig-071-20260514 clang-20
x86_64 randconfig-072-20260514 gcc-14
x86_64 randconfig-073-20260514 clang-20
x86_64 randconfig-074-20260514 gcc-14
x86_64 randconfig-075-20260514 clang-20
x86_64 randconfig-076-20260514 gcc-13
x86_64 rhel-9.4-rust clang-20
xtensa allnoconfig gcc-15.2.0
xtensa audio_kc705_defconfig gcc-15.2.0
xtensa randconfig-001-20260514 gcc-13.4.0
xtensa randconfig-002-20260514 gcc-8.5.0
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply
* Re: [PATCH 3/3] mm/zswap: Add per-memcg stat for proactive writeback
From: Hao Jia @ 2026-05-14 8:21 UTC (permalink / raw)
To: Nhat Pham
Cc: akpm, tj, hannes, shakeel.butt, mhocko, yosry, mkoutny,
chengming.zhou, muchun.song, roman.gushchin, cgroups, linux-mm,
linux-kernel, linux-doc, Hao Jia
In-Reply-To: <CAKEwX=OigngmcNo1OU-apCFG2hebt5yZwXQxZQHqgC7SwH_HAQ@mail.gmail.com>
On 2026/5/14 05:21, Nhat Pham wrote:
> On Mon, May 11, 2026 at 3:52 AM Hao Jia <jiahao.kernel@gmail.com> wrote:
>>
>> From: Hao Jia <jiahao1@lixiang.com>
>>
>> Currently, zswap writeback can be triggered by either the pool limit
>> being hit or by the proactive writeback mechanism. However, the
>> existing 'zswpwb' metric in memory.stat and /proc/vmstat counts all
>> written back pages, making it difficult to distinguish between pages
>> written back due to the pool limit and those written back proactively.
>>
>> Add a new statistic 'zswpwb_proactive' to memory.stat and /proc/vmstat.
>> This counter tracks the number of pages written back due to proactive
>> writeback. This allows users to better monitor and tune the proactive
>> writeback mechanism.
>>
>> Signed-off-by: Hao Jia <jiahao1@lixiang.com>
>> ---
>> Documentation/admin-guide/cgroup-v2.rst | 4 ++++
>> include/linux/vm_event_item.h | 1 +
>> mm/memcontrol.c | 1 +
>> mm/vmstat.c | 1 +
>> mm/zswap.c | 11 +++++++++--
>> 5 files changed, 16 insertions(+), 2 deletions(-)
>>
>> diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst
>> index 05b664b3b3e8..29a189b18efc 100644
>> --- a/Documentation/admin-guide/cgroup-v2.rst
>> +++ b/Documentation/admin-guide/cgroup-v2.rst
>> @@ -1734,6 +1734,10 @@ The following nested keys are defined.
>> zswpwb
>> Number of pages written from zswap to swap.
>>
>> + zswpwb_proactive
>> + Number of pages written from zswap to swap by proactive
>> + writeback. This is a subset of zswpwb.
>> +
>> zswap_incomp
>> Number of incompressible pages currently stored in zswap
>> without compression. These pages could not be compressed to
>
> nit: once we have reached consensus on an interface, can you add
> documentation for the new knob in cgroup v2 doc and zswap doc too, and
> how it interacts with the other interface (memory.zswap.writeback,
> shrinker_enabled sysfs knob).
>
> A kselftest would be very much appreciated too :)
Thanks, will do in v2
Thanks,
Hao
^ permalink raw reply
* Re: [PATCH 2/3] mm/zswap: Implement proactive writeback
From: Hao Jia @ 2026-05-14 8:15 UTC (permalink / raw)
To: Nhat Pham
Cc: Yosry Ahmed, akpm, tj, hannes, shakeel.butt, mhocko, mkoutny,
chengming.zhou, muchun.song, roman.gushchin, cgroups, linux-mm,
linux-kernel, linux-doc, Hao Jia, Alexandre Ghiti
In-Reply-To: <CAKEwX=M=6AQVYA7ROM0YOP7irpxbdMrEOAHKGKYo0Qgr+-uhSw@mail.gmail.com>
On 2026/5/14 05:09, Nhat Pham wrote:
> On Wed, May 13, 2026 at 1:04 AM Hao Jia <jiahao.kernel@gmail.com> wrote:
>>
>>
>>
>> On 2026/5/12 23:47, Nhat Pham wrote:
>>> On Tue, May 12, 2026 at 2:32 AM Hao Jia <jiahao.kernel@gmail.com> wrote:
>>>>
>>>>
>>>>
>>>> On 2026/5/12 03:57, Yosry Ahmed wrote:
>>>>> On Mon, May 11, 2026 at 12:49 PM Nhat Pham <nphamcs@gmail.com> wrote:
>>>>>>
>>>>>> On Mon, May 11, 2026 at 3:52 AM Hao Jia <jiahao.kernel@gmail.com> wrote:
>>>>>>>
>>>>>>> From: Hao Jia <jiahao1@lixiang.com>
>>>>>>>
>>>>>>> Zswap currently writes back pages to backing swap devices reactively,
>>>>>>> triggered either by memory pressure via the shrinker or by the pool
>>>>>>> reaching its size limit. This reactive approach offers no precise
>>>>>>> control over when writeback happens, which can disturb latency-sensitive
>>>>>>> workloads, and it cannot direct writeback at a specific memory cgroup.
>>>>>>> However, there are scenarios where users might want to proactively
>>>>>>> write back cold pages from zswap to the backing swap device, for
>>>>>>> example, to free up memory for other applications or to prepare for
>>>>>>> upcoming memory-intensive workloads.
>>>>>>>
>>>>>>> Therefore, implement a proactive writeback mechanism for zswap by
>>>>>>> adding a new cgroup interface file memory.zswap.proactive_writeback
>>>>>>> within the memory controller.
>>>>>>
>>>>
>>>> Thanks Nhat, Yosry — let me address both comments together.
>>>>
>>>>>>
>>>>>> We already have memory.reclaim, no? Would that not work to create
>>>>>> headroom generally for your use case? Is there a reason why we are
>>>>>> treating zswap memory as special here?
>>>>>
>>>>
>>>> Apologies for the lack of detailed explanation in the patch description,
>>>> which led to the confusion.
>>>>
>>>> While we are already utilizing memory.reclaim, it does not fully address
>>>> our requirements.
>>>>
>>>> Our deployment runs a userspace proactive reclaimer that drives
>>>> memory.reclaim based on the system's runtime state (memory/CPU/IO
>>>> pressure, refault rate, ...) and workload-specific
>>>> policy. That first stage compresses cold anon pages into zswap. Entries
>>>> that then remain in zswap past a policy-defined age threshold are
>>>> considered "twice cold", and the reclaimer wants
>>>> to write them back to the backing swap device at a moment of its own
>>>> choosing, to further reclaim the DRAM still held by the compressed data.
>>>>
>>>> This is the "second-level offloading" pattern described in Meta's TMO
>>>> paper [1]. zswap proactive writeback is what this series introduces to
>>>> address that second-level offloading stage.
>>>>
>>>> [1] https://www.pdl.cmu.edu/ftp/NVM/tmo_asplos22.pdf
>>>
>>> Yeah that's what we've been trying to work on as well :) We are
>>> working on a couple of improvements to the mechanism side of this path
>>> (cc Alex) - hopefully it will help your use case too!
>>>
>>> Anyway, back to my original inquiry: I understand your use case. It's
>>> pretty similar to our goal. What I'm not getting is why is
>>> memory.reclaim (which you already use) not sufficient for zswap ->
>>> disk swap offloading too?
>>>
>>> Zswap objects are organized into LRU and exposed to the shrinker
>>> interface. Echo-ing to memory.reclaim should also offload some zswap
>>> entries, correct? Are there still cold zswap entries that escape this,
>>> somehow?
>>>
>>
>> Yes, the memory.reclaim path does drive some zswap writeback, but
>> it is not enough for our case.
>>
>> 1. For a memcg that has reached steady state (a common case being
>> when memory.current is below the policy target), the userspace
>> reclaimer may not invoke memory.reclaim on it for a long time,
>> and so no second-level offloading happens through
>> memory.reclaim. In this state we want
>> memory.zswap.proactive_writeback to write back entries that
>> have sat in zswap past an age threshold, to further reclaim
>> the DRAM still held by the compressed data.
>>
>> 2. Even when memory.reclaim is running, the fraction of zswap
>> residency that ends up reaching the backing swap device is
>> still very small for many of our workloads, and the userspace
>> reclaimer has no way to participate in or control the
>> granularity of zswap writeback. So in our deployment we prefer
>> to leave the zswap shrinker disabled, decouple LRU -> zswap
>> from zswap -> swap, and use a dedicated proactive-writeback
>> interface that lifts the writeback policy into userspace where
>> it can evolve independently of the kernel.
>
> I see. It's interesting - we've been dealing with the opposite
> problems (reclaiming too much from zswap) that it's refreshing to see
> the other end of the spectrum :) We should invest more into this to
> see why we are not reclaiming enough, but I see the value of adding a
> knob to hit zswap exclusively.
>
> Regarding age-based reclaim, I agree with Yosry here. Let us try to
> land an interface to do targeted reclaim on compressed memory first. I
> do see the value of age information: with it, you can track zswap
> entries ages and the distribution of refault ages, and only reclaim
> the tail. However, I wonder if you can just build a system that adapt
> the reclaim request size based on PSI, refault rate etc. similar to
> how you're adjusting memory.reclaim on uncompressed memories with a
> senpai-like system. Something along the line of - if we are swapping
> in too much from disk (or if IO pressure is high), back off, and if
> not, stealing a bit more from zswap pool (perhaps with a bigger step
> size), etc. Is there a reason why zswap cannot adopt a similar
> strategy?
I'm not sure, as we haven't tested the case of tuning proactive zswap
writeback without using age. As you pointed out, age provides a
deterministic target that allows the userspace reclaimer to converge
faster in a closed-loop, which helps avoid performance jitters.
That said, using age as a zswap writeback parameter indeed warrants
further independent discussion. So I'll remove the age-related parts in v2.
Thanks,
Hao
^ permalink raw reply
* Re: [PATCH 2/3] mm/zswap: Implement proactive writeback
From: Hao Jia @ 2026-05-14 8:13 UTC (permalink / raw)
To: Nhat Pham, Yosry Ahmed, hannes, mhocko, tj
Cc: akpm, shakeel.butt, mkoutny, chengming.zhou, muchun.song,
roman.gushchin, cgroups, linux-mm, linux-kernel, linux-doc,
Hao Jia, Alexandre Ghiti
In-Reply-To: <CAKEwX=OY_nws-vf3VgnD54G205TK2YjkoAwRCyB9jvW=Oz3PpQ@mail.gmail.com>
On 2026/5/14 04:53, Nhat Pham wrote:
> On Wed, May 13, 2026 at 11:55 AM Yosry Ahmed <yosry@kernel.org> wrote:
>>
>>>> Zswap objects are organized into LRU and exposed to the shrinker
>>>> interface. Echo-ing to memory.reclaim should also offload some zswap
>>>> entries, correct? Are there still cold zswap entries that escape this,
>>>> somehow?
>>>>
>>>
>>> Yes, the memory.reclaim path does drive some zswap writeback, but
>>> it is not enough for our case.
>>>
>>> 1. For a memcg that has reached steady state (a common case being
>>> when memory.current is below the policy target), the userspace
>>> reclaimer may not invoke memory.reclaim on it for a long time,
>>> and so no second-level offloading happens through
>>> memory.reclaim. In this state we want
>>> memory.zswap.proactive_writeback to write back entries that
>>> have sat in zswap past an age threshold, to further reclaim
>>> the DRAM still held by the compressed data.
>>>
>>> 2. Even when memory.reclaim is running, the fraction of zswap
>>> residency that ends up reaching the backing swap device is
>>> still very small for many of our workloads, and the userspace
>>> reclaimer has no way to participate in or control the
>>> granularity of zswap writeback. So in our deployment we prefer
>>> to leave the zswap shrinker disabled, decouple LRU -> zswap
>>> from zswap -> swap, and use a dedicated proactive-writeback
>>> interface that lifts the writeback policy into userspace where
>>> it can evolve independently of the kernel.
>>
>> To be honest I see the point of proactively reclaiming compressed
>> memory in zswap. If you use memory.reclaim, you are also reclaiming
>> hotter memory in the process, and you are not necessarily getting as
>> much writeback as you want. The memory in zswap is a more conservative
>> choice for proactive reclaim because it's memory that's guaranteed to
>> be cold(ish) and not being accessed.
>>
>> That being said, the interface is not great any way you cut it :/
>>
>> I don't like the 'memory.zswap.proactive_writeback' name, maybe we can
>> stay consistent by doing 'memory.zswap.reclaim', but that just as
>> easily reads as "reclaim using zswap". Maybe
>> 'memory.zswap.do_writeback' or something, idk.
>>
>> I also don't like having two proactive reclaim interfaces, so a voice
>> in my head wants to tie this into 'memory.reclaim' somehow, but that
>> includes adding a pretty specific argument (e.g. 'memory.reclaim
>> zswap_writeback_only=1'.
>>
>> I don't like any of these options, and we also need to consider what
>> the memcg maintainers think. I see the use case of proactive writeback
>> but I am struggling to come up with a clean interface.
>>
>> I also think we should take the 'age' aspect out of the conversation
>> for now, it can be a separate discussion. Well, unless we decide to
>> tie it to memory.reclaim. If memory.reclaim broadly supports age-based
>> reclaim then zswap writeback can be a natural part of that without
>> requiring a specific interface.
>
> Yeah perhaps extending memory.reclaim is best... Sort of analogous to
> the way we have swappiness to balance file v.s anon....
Thanks for the suggestions, Yosry and Nhat.
My only concern is that if we eventually need to add more parameters to
zswap_writeback (such as age or others) in the future, would it make the
parameter parsing and the functionality of memory.reclaim overly complex?
As you mentioned, if the memcg maintainers have no objections, I will
attempt to implement it in v2.
How about something like this?
echo "100M zswap_writeback_only" > memory.reclaim
Thanks,
Hao
^ permalink raw reply
* Re: [linus:master] [mm] 01b9da291c: stress-ng.switch.ops_per_sec 67.7% regression
From: Qi Zheng @ 2026-05-14 7:46 UTC (permalink / raw)
To: Shakeel Butt, kernel test robot
Cc: oe-lkp, lkp, linux-kernel, Andrew Morton, David Carlier,
Allen Pais, Axel Rasmussen, Baoquan He, Chengming Zhou,
Chen Ridong, David Hildenbrand, Hamza Mahfooz, Harry Yoo,
Hugh Dickins, Imran Khan, Johannes Weiner, Kamalesh Babulal,
Lance Yang, Liam Howlett, Lorenzo Stoakes, Michal Hocko,
Michal Koutný, Mike Rapoport, Muchun Song, Muchun Song,
Nhat Pham, Roman Gushchin, Suren Baghdasaryan, Usama Arif,
Vlastimil Babka, Wei Xu, Yosry Ahmed, Yuanchu Xie, Zi Yan,
Usama Arif, cgroups, linux-mm
In-Reply-To: <agSJ4ulNDZ17ah8H@linux.dev>
On 5/13/26 10:27 PM, Shakeel Butt wrote:
> On Wed, May 13, 2026 at 06:49:45AM -0700, Shakeel Butt wrote:
>> On Wed, May 13, 2026 at 10:10:34AM +0800, Qi Zheng wrote:
>>>
>>>
>>> On 5/13/26 12:03 AM, Shakeel Butt wrote:
>>>> On Tue, May 12, 2026 at 08:56:52PM +0800, kernel test robot wrote:
>>>>>
>>>>>
>>>>> Hello,
>>>>>
>>>>> kernel test robot noticed a 67.7% regression of stress-ng.switch.ops_per_sec on:
>>>>>
>>>>>
>>>>> commit: 01b9da291c4969354807b52956f4aae1f41b4924 ("mm: memcontrol: convert objcg to be per-memcg per-node type")
>>>>> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
>>>>
>>>> This is most probably due to shuffling of struct mem_cgroup and struct
>>>> mem_cgroup_per_node members.
>>>
>>> Another possibility is that after objcg was split into per-node, the
>>> slab accounting fast path is still designed assuming only one current
>>> objcg per CPU:
>>>
>>> struct obj_stock_pcp {
>>> struct obj_cgroup *cached_objcg;
>>> };
>>>
>>> So it's may cause the following thrashing:
>>>
>>> CPU stock cached = memcg/node0 objcg
>>> free object tagged = memcg/node1 objcg
>>> => __refill_obj_stock --> objcg mismatch
>>> => drain_obj_stock()
>>> => cache switches to node1 objcg
>>>
>>> next local allocation tagged = node0 objcg
>>> => mismatch again
>>> => drain_obj_stock()
>>
>> Actually I think this is the issue, we have ping pong threads running on
>> different nodes where though theu are in same cgroup but their current->obcg is
>> for local node and thus this ping pong is thrashing the per-cpu objcg stock.
>>
>> The easier fix would be to compare objcg->memcg instead of just objcg during
>> draining and caching. In addition we can add support for multiple objcg per-cpu
>> stock caching.
>
> Something like the following:
>
> From d756abe831a905d6fe32bad9a984fc619dafb7e0 Mon Sep 17 00:00:00 2001
> From: Shakeel Butt <shakeel.butt@linux.dev>
> Date: Wed, 13 May 2026 07:24:55 -0700
> Subject: [PATCH] mm/memcontrol: skip obj_stock drain when refilled objcg
> shares memcg
>
> Signed-off-by: Shakeel Butt <shakeel.butt@linux.dev>
> ---
> mm/memcontrol.c | 14 +++++++++++++-
> 1 file changed, 13 insertions(+), 1 deletion(-)
>
> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> index d978e18b9b2d..01ed7a8e18ac 100644
> --- a/mm/memcontrol.c
> +++ b/mm/memcontrol.c
> @@ -3318,6 +3318,7 @@ static void __refill_obj_stock(struct obj_cgroup *objcg,
> unsigned int nr_bytes,
> bool allow_uncharge)
> {
> + struct obj_cgroup *cached;
> unsigned int nr_pages = 0;
>
> if (!stock) {
> @@ -3327,7 +3328,18 @@ static void __refill_obj_stock(struct obj_cgroup *objcg,
> goto out;
> }
>
> - if (READ_ONCE(stock->cached_objcg) != objcg) { /* reset if necessary */
> + cached = READ_ONCE(stock->cached_objcg);
> + if (cached != objcg &&
> + (!cached || obj_cgroup_memcg(cached) != obj_cgroup_memcg(objcg))) {
> drain_obj_stock(stock);
> obj_cgroup_get(objcg);
> stock->nr_bytes = atomic_read(&objcg->nr_charged_bytes)
This change looks like it should be able to fix the ping-pong issue, but
I stiil haven't reproduced the performance regression locally. I'll
continue testing it.
Hi kernel-test-robot, could you help check if the patch above fixes the
issue on your end?
Thanks,
Qi
^ permalink raw reply
* Re: [RFC PATCH v5 20/29] sched/deadline: Allow deeper hierarchies of RT cgroups
From: luca abeni @ 2026-05-14 7:25 UTC (permalink / raw)
To: Tejun Heo
Cc: Yuri Andriaccio, Peter Zijlstra, Yuri Andriaccio, Ingo Molnar,
Juri Lelli, Vincent Guittot, Dietmar Eggemann, Steven Rostedt,
Ben Segall, Mel Gorman, Valentin Schneider, linux-kernel, hannes,
mkoutny, cgroups
In-Reply-To: <b549b3cb062f2823ba6d4723b7b9260b@kernel.org>
Hi Tejun,
On Tue, 12 May 2026 08:19:02 -1000
Tejun Heo <tj@kernel.org> wrote:
> Hello,
>
> How is a delegated subtree prevented from setting cpu.rt.min = 'root'
> and escaping its ancestors' cpu.rt.max budget?
If I understand well (please correct me :), the following strategy
should address this concern (and the ones expressed in successive
emails):
- cpu.rt.max can be "runtime, period" (or "runtime, period, deadline")
or "root". "root" gives the current behaviour when RT cgroup
scheduling is not enabled (so, no need to disable it at build time :)
- if cpu.rt.max is "root", the cgroup's FIFO/RR tasks are scheduled in
the root cgroup
- if cpu.rt.max is "root", the children cgroups can only have "root"
in cpu.rt.max
- if cpu.rt.max is not "root", then cpu.rt.min (or cpu.rt.internal)
is "runtime, period" and describes the dl server for this cgroup's
FIFO/RR tasks.
- The default value for cpu.rt.min is copied from cpu.rt.max, so as a
default all the CPU utilization of the cgroup is dedicated it its RT
tasks
- the admission test is: cpu.rt.min utilization plus the sum of the
children's cpu.rt.max utilizations must be <= cpu.rt.max utilization;
children can have cpu.rt.max="root" only if cpu.rt.max="root"
Can this work? I think it avoids escaping the parents' cpu.rt.max,
allows for a reasonable default (no-one should be forced to disable this
feature), and should respect all the requirements... Or am I missing
something?
Thanks,
Luca
^ permalink raw reply
* [PATCH v3 3/4] cgroup/rdma: add rdma.events.local for per-cgroup allocation failure attribution
From: Tao Cui @ 2026-05-14 6:50 UTC (permalink / raw)
To: tj, hannes, mkoutny, cgroups; +Cc: Tao Cui
In-Reply-To: <20260514065034.387197-1-cuitao@kylinos.cn>
Add per-cgroup local event counters to track RDMA resource limit
exhaustion from the perspective of individual cgroups. The
rdma.events.local file reports two per-resource counters:
- max: number of times this cgroup's limit was the one that blocked
an allocation in the subtree
- alloc_fail: number of allocation attempts originating from this
cgroup that failed due to an ancestor's limit
This mirrors the design of pids.events.local, where events are
attributed to the cgroup that imposed the limit, not necessarily the
cgroup where the allocation was attempted.
Also extend rdma.events with a hierarchical alloc_fail counter that
tracks allocation failures propagating upward from the requesting
cgroup, complementing the existing max counter, so that rdma.events
and rdma.events.local share the same output format.
Signed-off-by: Tao Cui <cuitao@kylinos.cn>
---
include/linux/cgroup_rdma.h | 3 +-
kernel/cgroup/rdma.c | 143 +++++++++++++++++++++++++++---------
2 files changed, 109 insertions(+), 37 deletions(-)
diff --git a/include/linux/cgroup_rdma.h b/include/linux/cgroup_rdma.h
index ac691fe7d3f5..404e746552ca 100644
--- a/include/linux/cgroup_rdma.h
+++ b/include/linux/cgroup_rdma.h
@@ -25,8 +25,9 @@ struct rdma_cgroup {
*/
struct list_head rpools;
- /* Handle for rdma.events */
+ /* Handles for rdma.events[.local] */
struct cgroup_file events_file;
+ struct cgroup_file events_local_file;
};
struct rdmacg_device {
diff --git a/kernel/cgroup/rdma.c b/kernel/cgroup/rdma.c
index 927bbf1eb949..7c238a9d64d4 100644
--- a/kernel/cgroup/rdma.c
+++ b/kernel/cgroup/rdma.c
@@ -82,8 +82,11 @@ struct rdmacg_resource_pool {
/* total number counts which are set to max */
int num_max_cnt;
- /* per-resource hierarchical max event counters */
+ /* per-resource event counters */
u64 events_max[RDMACG_RESOURCE_MAX];
+ u64 events_alloc_fail[RDMACG_RESOURCE_MAX];
+ u64 events_local_max[RDMACG_RESOURCE_MAX];
+ u64 events_local_alloc_fail[RDMACG_RESOURCE_MAX];
};
static struct rdma_cgroup *css_rdmacg(struct cgroup_subsys_state *css)
@@ -131,6 +134,26 @@ static void free_cg_rpool_locked(struct rdmacg_resource_pool *rpool)
kfree(rpool);
}
+static bool rpool_has_persistent_state(struct rdmacg_resource_pool *rpool)
+{
+ int i;
+
+ /*
+ * Keep the rpool alive if any peak value is non-zero,
+ * so that rdma.peak persists as a historical high-
+ * watermark even after all resources are freed.
+ */
+ for (i = 0; i < RDMACG_RESOURCE_MAX; i++) {
+ if (rpool->resources[i].peak ||
+ READ_ONCE(rpool->events_max[i]) ||
+ READ_ONCE(rpool->events_local_max[i]) ||
+ READ_ONCE(rpool->events_alloc_fail[i]) ||
+ READ_ONCE(rpool->events_local_alloc_fail[i]))
+ return true;
+ }
+ return false;
+}
+
static struct rdmacg_resource_pool *
find_cg_rpool_locked(struct rdma_cgroup *cg,
struct rdmacg_device *device)
@@ -209,37 +232,30 @@ uncharge_cg_locked(struct rdma_cgroup *cg,
rpool->usage_sum--;
if (rpool->usage_sum == 0 &&
rpool->num_max_cnt == RDMACG_RESOURCE_MAX) {
- int i;
-
- /*
- * Keep the rpool alive if any peak value is non-zero,
- * so that rdma.peak persists as a historical high-
- * watermark even after all resources are freed.
- */
- for (i = 0; i < RDMACG_RESOURCE_MAX; i++) {
- if (rpool->resources[i].peak ||
- READ_ONCE(rpool->events_max[i]))
- return;
+ if (!rpool_has_persistent_state(rpool)) {
+ /*
+ * No user of the rpool and all entries are set to max, so
+ * safe to delete this rpool.
+ */
+ free_cg_rpool_locked(rpool);
}
- /*
- * No user of the rpool and all entries are set to max, so
- * safe to delete this rpool.
- */
- free_cg_rpool_locked(rpool);
}
}
/**
- * rdmacg_event_locked - fire hierarchical max event when resource limit is hit
+ * rdmacg_event_locked - fire event when resource allocation exceeds limit
+ * @cg: requesting cgroup
* @over_cg: cgroup whose limit was exceeded
* @device: rdma device
* @index: resource type index
*
- * Must be called under rdmacg_mutex. Propagates max event counts
- * from @over_cg (including itself) upward to all ancestors with
- * an rpool and notifies userspace.
+ * Must be called under rdmacg_mutex. Updates event counters in the
+ * resource pools of @cg and @over_cg, propagates hierarchical max
+ * events from @over_cg (including itself) upward, and notifies
+ * userspace via cgroup_file_notify().
*/
-static void rdmacg_event_locked(struct rdma_cgroup *over_cg,
+static void rdmacg_event_locked(struct rdma_cgroup *cg,
+ struct rdma_cgroup *over_cg,
struct rdmacg_device *device,
enum rdmacg_resource_type index)
{
@@ -248,6 +264,21 @@ static void rdmacg_event_locked(struct rdma_cgroup *over_cg,
lockdep_assert_held(&rdmacg_mutex);
+ /* Increment local alloc_fail in requesting cgroup */
+ rpool = find_cg_rpool_locked(cg, device);
+ if (rpool) {
+ rpool->events_local_alloc_fail[index]++;
+ cgroup_file_notify(&cg->events_local_file);
+ }
+
+ /* Increment local max in the over-limit cgroup */
+ rpool = find_cg_rpool_locked(over_cg, device);
+ if (rpool) {
+ rpool->events_local_max[index]++;
+ cgroup_file_notify(&over_cg->events_local_file);
+ }
+
+ /* Propagate hierarchical max events upward */
for (p = over_cg; parent_rdmacg(p); p = parent_rdmacg(p)) {
rpool = get_cg_rpool_locked(p, device);
if (!IS_ERR(rpool)) {
@@ -255,6 +286,14 @@ static void rdmacg_event_locked(struct rdma_cgroup *over_cg,
cgroup_file_notify(&p->events_file);
}
}
+ /* Propagate hierarchical alloc_fail from requesting cgroup upward */
+ for (p = cg; parent_rdmacg(p); p = parent_rdmacg(p)) {
+ rpool = get_cg_rpool_locked(p, device);
+ if (!IS_ERR(rpool)) {
+ rpool->events_alloc_fail[index]++;
+ cgroup_file_notify(&p->events_file);
+ }
+ }
}
/**
@@ -368,7 +407,7 @@ int rdmacg_try_charge(struct rdma_cgroup **rdmacg,
err:
if (ret == -EAGAIN)
- rdmacg_event_locked(p, device, index);
+ rdmacg_event_locked(cg, p, device, index);
mutex_unlock(&rdmacg_mutex);
rdmacg_uncharge_hierarchy(cg, device, p, index);
return ret;
@@ -525,18 +564,13 @@ static ssize_t rdmacg_resource_set_max(struct kernfs_open_file *of,
if (rpool->usage_sum == 0 &&
rpool->num_max_cnt == RDMACG_RESOURCE_MAX) {
- int i;
-
- for (i = 0; i < RDMACG_RESOURCE_MAX; i++) {
- if (rpool->resources[i].peak ||
- READ_ONCE(rpool->events_max[i]))
- goto dev_err;
+ if (!rpool_has_persistent_state(rpool)) {
+ /*
+ * No user of the rpool and all entries are set to max, so
+ * safe to delete this rpool.
+ */
+ free_cg_rpool_locked(rpool);
}
- /*
- * No user of the rpool and all entries are set to max, so
- * safe to delete this rpool.
- */
- free_cg_rpool_locked(rpool);
}
dev_err:
@@ -618,9 +652,40 @@ static int rdmacg_events_show(struct seq_file *sf, void *v)
seq_printf(sf, "%s ", device->name);
for (i = 0; i < RDMACG_RESOURCE_MAX; i++) {
- seq_printf(sf, "%s.max=%llu",
+ seq_printf(sf, "%s.max=%llu %s.alloc_fail=%llu",
+ rdmacg_resource_names[i],
+ rpool ? READ_ONCE(rpool->events_max[i]) : 0ULL,
+ rdmacg_resource_names[i],
+ rpool ? READ_ONCE(rpool->events_alloc_fail[i]) : 0ULL);
+ if (i < RDMACG_RESOURCE_MAX - 1)
+ seq_putc(sf, ' ');
+ }
+ seq_putc(sf, '\n');
+ }
+
+ mutex_unlock(&rdmacg_mutex);
+ return 0;
+}
+
+static int rdmacg_events_local_show(struct seq_file *sf, void *v)
+{
+ struct rdma_cgroup *cg = css_rdmacg(seq_css(sf));
+ struct rdmacg_resource_pool *rpool;
+ struct rdmacg_device *device;
+ int i;
+
+ mutex_lock(&rdmacg_mutex);
+
+ list_for_each_entry(device, &rdmacg_devices, dev_node) {
+ rpool = find_cg_rpool_locked(cg, device);
+
+ seq_printf(sf, "%s ", device->name);
+ for (i = 0; i < RDMACG_RESOURCE_MAX; i++) {
+ seq_printf(sf, "%s.max=%llu %s.alloc_fail=%llu",
+ rdmacg_resource_names[i],
+ rpool ? READ_ONCE(rpool->events_local_max[i]) : 0ULL,
rdmacg_resource_names[i],
- rpool ? READ_ONCE(rpool->events_max[i]) : 0ULL);
+ rpool ? READ_ONCE(rpool->events_local_alloc_fail[i]) : 0ULL);
if (i < RDMACG_RESOURCE_MAX - 1)
seq_putc(sf, ' ');
}
@@ -657,6 +722,12 @@ static struct cftype rdmacg_files[] = {
.file_offset = offsetof(struct rdma_cgroup, events_file),
.flags = CFTYPE_NOT_ON_ROOT,
},
+ {
+ .name = "events.local",
+ .seq_show = rdmacg_events_local_show,
+ .file_offset = offsetof(struct rdma_cgroup, events_local_file),
+ .flags = CFTYPE_NOT_ON_ROOT,
+ },
{ } /* terminate */
};
--
2.43.0
^ permalink raw reply related
* [PATCH v3 4/4] cgroup/rdma: document rdma.peak, rdma.events and rdma.events.local
From: Tao Cui @ 2026-05-14 6:50 UTC (permalink / raw)
To: tj, hannes, mkoutny, cgroups; +Cc: Tao Cui
In-Reply-To: <20260514065034.387197-1-cuitao@kylinos.cn>
Add interface file documentation for the new rdma cgroup files to
Documentation/admin-guide/cgroup-v2.rst.
Signed-off-by: Tao Cui <cuitao@kylinos.cn>
---
Documentation/admin-guide/cgroup-v2.rst | 53 +++++++++++++++++++++++++
1 file changed, 53 insertions(+)
diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst
index 6efd0095ed99..993446ab66d0 100644
--- a/Documentation/admin-guide/cgroup-v2.rst
+++ b/Documentation/admin-guide/cgroup-v2.rst
@@ -2785,6 +2785,59 @@ RDMA Interface Files
mlx4_0 hca_handle=1 hca_object=20
ocrdma1 hca_handle=1 hca_object=23
+ rdma.peak
+ A read-only nested-keyed file that exists for all the cgroups
+ except root. It shows the historical high watermark of
+ resource usage per device since the cgroup was created.
+
+ An example for mlx4 and ocrdma device follows::
+
+ mlx4_0 hca_handle=1 hca_object=20
+ ocrdma1 hca_handle=0 hca_object=23
+
+ rdma.events
+ A read-only nested-keyed file which exists on non-root
+ cgroups. The following nested keys are defined.
+
+ max
+ The number of times a process in this cgroup or its
+ descendants attempted an RDMA resource allocation that
+ was rejected because a rdma.max limit in the subtree
+ was reached. This is a hierarchical counter: the event
+ is propagated upward to all ancestor cgroups. A value
+ change in this file generates a file modified event.
+
+ alloc_fail
+ The number of RDMA resource allocation attempts that
+ originated in this cgroup or its descendants and failed
+ due to a rdma.max limit being reached. This is a
+ hierarchical counter propagated upward.
+
+ An example for mlx4 device follows::
+
+ mlx4_0 hca_handle.max=5 hca_handle.alloc_fail=3 hca_object.max=0 hca_object.alloc_fail=0
+
+ rdma.events.local
+ Similar to rdma.events but the fields in the file are local
+ to the cgroup i.e. not hierarchical. The file modified event
+ generated on this file reflects only the local events.
+
+ The following nested keys are defined.
+
+ max
+ The number of times a process in this cgroup or its
+ descendants attempted an RDMA resource allocation that
+ was rejected because this cgroup's own rdma.max limit
+ was reached.
+ alloc_fail
+ The number of RDMA resource allocation attempts
+ originating from this cgroup that failed due to this
+ cgroup's or an ancestor's rdma.max limit.
+
+ An example for mlx4 device follows::
+
+ mlx4_0 hca_handle.max=5 hca_handle.alloc_fail=0 hca_object.max=0 hca_object.alloc_fail=0
+
DMEM
----
--
2.43.0
^ permalink raw reply related
* [PATCH v3 2/4] cgroup/rdma: add rdma.events to track resource limit exhaustion
From: Tao Cui @ 2026-05-14 6:50 UTC (permalink / raw)
To: tj, hannes, mkoutny, cgroups; +Cc: Tao Cui
In-Reply-To: <20260514065034.387197-1-cuitao@kylinos.cn>
Add per-device hierarchical event counters to track when RDMA resource
limits are exceeded. The rdma.events file reports max event counts
propagated upward from the cgroup whose limit was hit to all ancestors.
This mirrors the design of pids.events, where events are attributed to
the cgroup that imposed the limit, not necessarily the cgroup where the
allocation was attempted. Userspace can monitor this file via
poll/epoll for real-time notification of resource exhaustion.
Signed-off-by: Tao Cui <cuitao@kylinos.cn>
---
include/linux/cgroup_rdma.h | 3 ++
kernel/cgroup/rdma.c | 72 +++++++++++++++++++++++++++++++++++--
2 files changed, 73 insertions(+), 2 deletions(-)
diff --git a/include/linux/cgroup_rdma.h b/include/linux/cgroup_rdma.h
index 80edae03c313..ac691fe7d3f5 100644
--- a/include/linux/cgroup_rdma.h
+++ b/include/linux/cgroup_rdma.h
@@ -24,6 +24,9 @@ struct rdma_cgroup {
* that belongs to this cgroup.
*/
struct list_head rpools;
+
+ /* Handle for rdma.events */
+ struct cgroup_file events_file;
};
struct rdmacg_device {
diff --git a/kernel/cgroup/rdma.c b/kernel/cgroup/rdma.c
index 4e3bf0bade18..927bbf1eb949 100644
--- a/kernel/cgroup/rdma.c
+++ b/kernel/cgroup/rdma.c
@@ -81,6 +81,9 @@ struct rdmacg_resource_pool {
u64 usage_sum;
/* total number counts which are set to max */
int num_max_cnt;
+
+ /* per-resource hierarchical max event counters */
+ u64 events_max[RDMACG_RESOURCE_MAX];
};
static struct rdma_cgroup *css_rdmacg(struct cgroup_subsys_state *css)
@@ -214,7 +217,8 @@ uncharge_cg_locked(struct rdma_cgroup *cg,
* watermark even after all resources are freed.
*/
for (i = 0; i < RDMACG_RESOURCE_MAX; i++) {
- if (rpool->resources[i].peak)
+ if (rpool->resources[i].peak ||
+ READ_ONCE(rpool->events_max[i]))
return;
}
/*
@@ -225,6 +229,34 @@ uncharge_cg_locked(struct rdma_cgroup *cg,
}
}
+/**
+ * rdmacg_event_locked - fire hierarchical max event when resource limit is hit
+ * @over_cg: cgroup whose limit was exceeded
+ * @device: rdma device
+ * @index: resource type index
+ *
+ * Must be called under rdmacg_mutex. Propagates max event counts
+ * from @over_cg (including itself) upward to all ancestors with
+ * an rpool and notifies userspace.
+ */
+static void rdmacg_event_locked(struct rdma_cgroup *over_cg,
+ struct rdmacg_device *device,
+ enum rdmacg_resource_type index)
+{
+ struct rdmacg_resource_pool *rpool;
+ struct rdma_cgroup *p;
+
+ lockdep_assert_held(&rdmacg_mutex);
+
+ for (p = over_cg; parent_rdmacg(p); p = parent_rdmacg(p)) {
+ rpool = get_cg_rpool_locked(p, device);
+ if (!IS_ERR(rpool)) {
+ rpool->events_max[index]++;
+ cgroup_file_notify(&p->events_file);
+ }
+ }
+}
+
/**
* rdmacg_uncharge_hierarchy - hierarchically uncharge rdma resource count
* @cg: pointer to cg to uncharge and all parents in hierarchy
@@ -335,6 +367,8 @@ int rdmacg_try_charge(struct rdma_cgroup **rdmacg,
return 0;
err:
+ if (ret == -EAGAIN)
+ rdmacg_event_locked(p, device, index);
mutex_unlock(&rdmacg_mutex);
rdmacg_uncharge_hierarchy(cg, device, p, index);
return ret;
@@ -494,7 +528,8 @@ static ssize_t rdmacg_resource_set_max(struct kernfs_open_file *of,
int i;
for (i = 0; i < RDMACG_RESOURCE_MAX; i++) {
- if (rpool->resources[i].peak)
+ if (rpool->resources[i].peak ||
+ READ_ONCE(rpool->events_max[i]))
goto dev_err;
}
/*
@@ -569,6 +604,33 @@ static int rdmacg_resource_read(struct seq_file *sf, void *v)
return 0;
}
+static int rdmacg_events_show(struct seq_file *sf, void *v)
+{
+ struct rdma_cgroup *cg = css_rdmacg(seq_css(sf));
+ struct rdmacg_resource_pool *rpool;
+ struct rdmacg_device *device;
+ int i;
+
+ mutex_lock(&rdmacg_mutex);
+
+ list_for_each_entry(device, &rdmacg_devices, dev_node) {
+ rpool = find_cg_rpool_locked(cg, device);
+
+ seq_printf(sf, "%s ", device->name);
+ for (i = 0; i < RDMACG_RESOURCE_MAX; i++) {
+ seq_printf(sf, "%s.max=%llu",
+ rdmacg_resource_names[i],
+ rpool ? READ_ONCE(rpool->events_max[i]) : 0ULL);
+ if (i < RDMACG_RESOURCE_MAX - 1)
+ seq_putc(sf, ' ');
+ }
+ seq_putc(sf, '\n');
+ }
+
+ mutex_unlock(&rdmacg_mutex);
+ return 0;
+}
+
static struct cftype rdmacg_files[] = {
{
.name = "max",
@@ -589,6 +651,12 @@ static struct cftype rdmacg_files[] = {
.private = RDMACG_RESOURCE_TYPE_PEAK,
.flags = CFTYPE_NOT_ON_ROOT,
},
+ {
+ .name = "events",
+ .seq_show = rdmacg_events_show,
+ .file_offset = offsetof(struct rdma_cgroup, events_file),
+ .flags = CFTYPE_NOT_ON_ROOT,
+ },
{ } /* terminate */
};
--
2.43.0
^ permalink raw reply related
* [PATCH v3 1/4] cgroup/rdma: add rdma.peak for per-device peak usage tracking
From: Tao Cui @ 2026-05-14 6:50 UTC (permalink / raw)
To: tj, hannes, mkoutny, cgroups; +Cc: Tao Cui
In-Reply-To: <20260514065034.387197-1-cuitao@kylinos.cn>
rdma.peak tracks the high watermark of resource usage per device,
giving a better baseline on which to set rdma.max. Polling
rdma.current isn't feasible since it would miss short-lived spikes.
This interface is analogous to memory.peak.
Signed-off-by: Tao Cui <cuitao@kylinos.cn>
---
kernel/cgroup/rdma.c | 40 ++++++++++++++++++++++++++++++++++++++++
1 file changed, 40 insertions(+)
diff --git a/kernel/cgroup/rdma.c b/kernel/cgroup/rdma.c
index 3df7c38ce481..4e3bf0bade18 100644
--- a/kernel/cgroup/rdma.c
+++ b/kernel/cgroup/rdma.c
@@ -44,6 +44,7 @@ static LIST_HEAD(rdmacg_devices);
enum rdmacg_file_type {
RDMACG_RESOURCE_TYPE_MAX,
RDMACG_RESOURCE_TYPE_STAT,
+ RDMACG_RESOURCE_TYPE_PEAK,
};
/*
@@ -60,6 +61,7 @@ static char const *rdmacg_resource_names[] = {
struct rdmacg_resource {
int max;
int usage;
+ int peak;
};
/*
@@ -204,6 +206,17 @@ uncharge_cg_locked(struct rdma_cgroup *cg,
rpool->usage_sum--;
if (rpool->usage_sum == 0 &&
rpool->num_max_cnt == RDMACG_RESOURCE_MAX) {
+ int i;
+
+ /*
+ * Keep the rpool alive if any peak value is non-zero,
+ * so that rdma.peak persists as a historical high-
+ * watermark even after all resources are freed.
+ */
+ for (i = 0; i < RDMACG_RESOURCE_MAX; i++) {
+ if (rpool->resources[i].peak)
+ return;
+ }
/*
* No user of the rpool and all entries are set to max, so
* safe to delete this rpool.
@@ -310,6 +323,12 @@ int rdmacg_try_charge(struct rdma_cgroup **rdmacg,
}
}
}
+ /* Update peak only after all charges succeed */
+ for (p = cg; p; p = parent_rdmacg(p)) {
+ rpool = find_cg_rpool_locked(p, device);
+ if (rpool && rpool->resources[index].usage > rpool->resources[index].peak)
+ rpool->resources[index].peak = rpool->resources[index].usage;
+ }
mutex_unlock(&rdmacg_mutex);
*rdmacg = cg;
@@ -472,6 +491,12 @@ static ssize_t rdmacg_resource_set_max(struct kernfs_open_file *of,
if (rpool->usage_sum == 0 &&
rpool->num_max_cnt == RDMACG_RESOURCE_MAX) {
+ int i;
+
+ for (i = 0; i < RDMACG_RESOURCE_MAX; i++) {
+ if (rpool->resources[i].peak)
+ goto dev_err;
+ }
/*
* No user of the rpool and all entries are set to max, so
* safe to delete this rpool.
@@ -506,6 +531,8 @@ static void print_rpool_values(struct seq_file *sf,
value = rpool->resources[i].max;
else
value = S32_MAX;
+ } else if (sf_type == RDMACG_RESOURCE_TYPE_PEAK) {
+ value = rpool ? rpool->resources[i].peak : 0;
} else {
if (rpool)
value = rpool->resources[i].usage;
@@ -556,6 +583,12 @@ static struct cftype rdmacg_files[] = {
.private = RDMACG_RESOURCE_TYPE_STAT,
.flags = CFTYPE_NOT_ON_ROOT,
},
+ {
+ .name = "peak",
+ .seq_show = rdmacg_resource_read,
+ .private = RDMACG_RESOURCE_TYPE_PEAK,
+ .flags = CFTYPE_NOT_ON_ROOT,
+ },
{ } /* terminate */
};
@@ -575,6 +608,13 @@ rdmacg_css_alloc(struct cgroup_subsys_state *parent)
static void rdmacg_css_free(struct cgroup_subsys_state *css)
{
struct rdma_cgroup *cg = css_rdmacg(css);
+ struct rdmacg_resource_pool *rpool, *tmp;
+
+ /* Clean up rpools kept alive by non-zero peak values */
+ mutex_lock(&rdmacg_mutex);
+ list_for_each_entry_safe(rpool, tmp, &cg->rpools, cg_node)
+ free_cg_rpool_locked(rpool);
+ mutex_unlock(&rdmacg_mutex);
kfree(cg);
}
--
2.43.0
^ permalink raw reply related
* [PATCH v3 0/4] cgroup/rdma: add rdma.peak and rdma.events[.local]
From: Tao Cui @ 2026-05-14 6:50 UTC (permalink / raw)
To: tj, hannes, mkoutny, cgroups; +Cc: Tao Cui
Hi,
This is v3 of the RDMA cgroup observability series. Thanks to the
reviewers for the detailed feedback on v1 and v2.
This series adds new cgroup interface files to the RDMA controller
to improve observability of resource usage and limit enforcement:
- rdma.peak: per-device high watermark of resource usage
- rdma.events: hierarchical max and alloc_fail event counters
- rdma.events.local: per-cgroup local max and alloc_fail counters
rdma.peak tracks the historical high watermark so administrators can
determine a sensible rdma.max based on actual peak demand rather than
guesswork. This is directly analogous to memory.peak.
rdma.events and rdma.events.local provide per-device counters that
track how often resource limits block allocations, and can be monitored
via poll/epoll for real-time alerting. Both files expose the same
keys (max and alloc_fail); rdma.events aggregates hierarchically while
rdma.events.local shows per-cgroup values. This follows the
pids.events / pids.events.local design.
Patch overview:
Patch 1 introduces rdma.peak, adding a per-resource peak field to track
the high watermark of usage, updated only after a full hierarchical
charge succeeds, and extends rpool lifetime to preserve non-zero
peak values.
Patch 2 adds rdma.events, which introduces rdmacg_event_locked() to
propagate hierarchical max counters upward from the over-limit
cgroup using get_cg_rpool_locked() to ensure full hierarchical
coverage even for ancestors without a prior rpool, with poll/epoll
notification via cgroup_file_notify().
Patch 3 adds rdma.events.local and hierarchical alloc_fail, extending
the event framework with per-cgroup local counters (local_max for
the over-limit cgroup, local_alloc_fail for the requesting cgroup)
and a hierarchical alloc_fail counter propagated from the requestor
upward. It also extracts the duplicated rpool-keep predicate into
a rpool_has_persistent_state() helper and replaces the non-error
goto dev_err in rdmacg_resource_set_max() with an if-guard.
Patch 4 documents all three new interface files in cgroup-v2.rst.
Tao Cui (4):
cgroup/rdma: add rdma.peak for per-device peak usage tracking
cgroup/rdma: add rdma.events to track resource limit exhaustion
cgroup/rdma: add rdma.events.local for per-cgroup allocation failure
attribution
cgroup/rdma: document rdma.peak, rdma.events and rdma.events.local
Documentation/admin-guide/cgroup-v2.rst | 54 +++++++
include/linux/cgroup_rdma.h | 4 +
kernel/cgroup/rdma.c | 199 ++++++++++++++++++++++--
3 files changed, 247 insertions(+), 10 deletions(-)
---
Changes in v3:
- Switch rdmacg_event_locked() from find_ to get_cg_rpool_locked()
in hierarchical propagation loops (events_max and events_alloc_fail)
to ensure full hierarchical coverage; the rpool-keep check now
covers event counters, so spurious-rpool concern from v1 no longer
applies.
- Extract the duplicated rpool-keep predicate (peak + 4 event
counters) into rpool_has_persistent_state() helper.
- Replace the non-error goto dev_err in rdmacg_resource_set_max()
with an if-guard so dev_err is only used for real error paths.
- Fix commit message of rdma.events.local patch to mention the
rdma.events hierarchical alloc_fail extension.
- Use %llu and drop (s64) cast in rdmacg_events_show() and
rdmacg_events_local_show() to match u64 counter type.
Changes in v2:
- Fix peak updated before full hierarchical charge succeeds.
- Use find_cg_rpool_locked() to avoid creating spurious rpools.
- Replace atomic64_t with u64 + READ_ONCE (all under rdmacg_mutex).
- Use key=value output format, remove trailing spaces.
- Always list all devices, show zero for devices without an rpool.
- Extend rpool-free condition to preserve non-zero event counters.
- Rename "failcnt" to "alloc_fail" (cgroup v2 naming convention).
- Fix alloc_fail semantics: local to the requesting cgroup only.
- Add hierarchical alloc_fail to rdma.events for key consistency.
- Add documentation in Documentation/admin-guide/cgroup-v2.rst.
v1:
https://lore.kernel.org/all/20260512031719.273507-1-cuitao@kylinos.cn/
v2:
https://lore.kernel.org/all/20260513104956.373216-1-cuitao@kylinos.cn/
--
2.43.0
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox