* [PATCH v3 0/4] cgroup/rdma: add rdma.peak and rdma.events[.local]
From: Tao Cui @ 2026-05-14 6:50 UTC (permalink / raw)
To: tj, hannes, mkoutny, cgroups; +Cc: Tao Cui
Hi,
This is v3 of the RDMA cgroup observability series. Thanks to the
reviewers for the detailed feedback on v1 and v2.
This series adds new cgroup interface files to the RDMA controller
to improve observability of resource usage and limit enforcement:
- rdma.peak: per-device high watermark of resource usage
- rdma.events: hierarchical max and alloc_fail event counters
- rdma.events.local: per-cgroup local max and alloc_fail counters
rdma.peak tracks the historical high watermark so administrators can
determine a sensible rdma.max based on actual peak demand rather than
guesswork. This is directly analogous to memory.peak.
rdma.events and rdma.events.local provide per-device counters that
track how often resource limits block allocations, and can be monitored
via poll/epoll for real-time alerting. Both files expose the same
keys (max and alloc_fail); rdma.events aggregates hierarchically while
rdma.events.local shows per-cgroup values. This follows the
pids.events / pids.events.local design.
Patch overview:
Patch 1 introduces rdma.peak, adding a per-resource peak field to track
the high watermark of usage, updated only after a full hierarchical
charge succeeds, and extends rpool lifetime to preserve non-zero
peak values.
Patch 2 adds rdma.events, which introduces rdmacg_event_locked() to
propagate hierarchical max counters upward from the over-limit
cgroup using get_cg_rpool_locked() to ensure full hierarchical
coverage even for ancestors without a prior rpool, with poll/epoll
notification via cgroup_file_notify().
Patch 3 adds rdma.events.local and hierarchical alloc_fail, extending
the event framework with per-cgroup local counters (local_max for
the over-limit cgroup, local_alloc_fail for the requesting cgroup)
and a hierarchical alloc_fail counter propagated from the requestor
upward. It also extracts the duplicated rpool-keep predicate into
a rpool_has_persistent_state() helper and replaces the non-error
goto dev_err in rdmacg_resource_set_max() with an if-guard.
Patch 4 documents all three new interface files in cgroup-v2.rst.
Tao Cui (4):
cgroup/rdma: add rdma.peak for per-device peak usage tracking
cgroup/rdma: add rdma.events to track resource limit exhaustion
cgroup/rdma: add rdma.events.local for per-cgroup allocation failure
attribution
cgroup/rdma: document rdma.peak, rdma.events and rdma.events.local
Documentation/admin-guide/cgroup-v2.rst | 54 +++++++
include/linux/cgroup_rdma.h | 4 +
kernel/cgroup/rdma.c | 199 ++++++++++++++++++++++--
3 files changed, 247 insertions(+), 10 deletions(-)
---
Changes in v3:
- Switch rdmacg_event_locked() from find_ to get_cg_rpool_locked()
in hierarchical propagation loops (events_max and events_alloc_fail)
to ensure full hierarchical coverage; the rpool-keep check now
covers event counters, so spurious-rpool concern from v1 no longer
applies.
- Extract the duplicated rpool-keep predicate (peak + 4 event
counters) into rpool_has_persistent_state() helper.
- Replace the non-error goto dev_err in rdmacg_resource_set_max()
with an if-guard so dev_err is only used for real error paths.
- Fix commit message of rdma.events.local patch to mention the
rdma.events hierarchical alloc_fail extension.
- Use %llu and drop (s64) cast in rdmacg_events_show() and
rdmacg_events_local_show() to match u64 counter type.
Changes in v2:
- Fix peak updated before full hierarchical charge succeeds.
- Use find_cg_rpool_locked() to avoid creating spurious rpools.
- Replace atomic64_t with u64 + READ_ONCE (all under rdmacg_mutex).
- Use key=value output format, remove trailing spaces.
- Always list all devices, show zero for devices without an rpool.
- Extend rpool-free condition to preserve non-zero event counters.
- Rename "failcnt" to "alloc_fail" (cgroup v2 naming convention).
- Fix alloc_fail semantics: local to the requesting cgroup only.
- Add hierarchical alloc_fail to rdma.events for key consistency.
- Add documentation in Documentation/admin-guide/cgroup-v2.rst.
v1:
https://lore.kernel.org/all/20260512031719.273507-1-cuitao@kylinos.cn/
v2:
https://lore.kernel.org/all/20260513104956.373216-1-cuitao@kylinos.cn/
--
2.43.0
^ permalink raw reply
* Re: [PATCH v3 04/12] mm, swap: add support for stable large allocation in swap cache directly
From: Kairui Song @ 2026-05-14 6:11 UTC (permalink / raw)
To: YoungJun Park
Cc: linux-mm, Andrew Morton, David Hildenbrand, Zi Yan, Baolin Wang,
Barry Song, Hugh Dickins, Chris Li, Kemeng Shi, Nhat Pham,
Baoquan He, Johannes Weiner, Chengming Zhou, Roman Gushchin,
Shakeel Butt, Muchun Song, Qi Zheng, linux-kernel, cgroups,
Yosry Ahmed, Lorenzo Stoakes, Dev Jain, Lance Yang, Michal Hocko,
Michal Hocko, Suren Baghdasaryan, Axel Rasmussen
In-Reply-To: <agS0RbXpVMcYMI3g@yjaykim-PowerEdge-T330>
On Thu, May 14, 2026 at 1:53 AM YoungJun Park <youngjun.park@lge.com> wrote:
>
> On Tue, Apr 21, 2026 at 02:16:48PM +0800, Kairui Song via B4 Relay wrote:
> ...
> > static struct folio *swap_cache_read_folio(swp_entry_t entry, gfp_t gfp,
> > struct mempolicy *mpol, pgoff_t ilx,
> > struct swap_iocb **plug, bool readahead)
> > {
> > - struct swap_info_struct *si = __swap_entry_to_info(entry);
> > struct folio *folio;
> >
> > /* Check the swap cache again for readahead path. */
> > @@ -594,16 +700,12 @@ static struct folio *swap_cache_read_folio(swp_entry_t entry, gfp_t gfp,
> > if (folio)
> > return folio;
> > - /* Skip allocation for unused and bad swap slot for readahead. */
> > - if (!swap_entry_swapped(si, entry))
> > - return NULL;
> > -
>
> Hello Kairui
>
> With the swap_entry_swapped() check gone, the swap_cache_get_folio()
> above the do-while is now just a duplicate of the loop's first
> iteration. Might as well drop it (and the now-stale "again for readahead
> path" comment) here.
Ah, very nice observation! Thanks, will drop it.
^ permalink raw reply
* Re: [PATCH] cpuset: Fix multi-source deadline task accounting and bandwidth bypass
From: Waiman Long @ 2026-05-14 4:26 UTC (permalink / raw)
To: Aaron Tomlin
Cc: Dietmar Eggemann, tj, hannes, mkoutny, chenridong, neelx, cgroups,
linux-kernel
In-Reply-To: <djbtucfusnpngys2nritqsi7stjq243zchel45ahfgaruba7el@4rtk534mfq4j>
On 5/13/26 7:39 PM, Aaron Tomlin wrote:
> On Wed, May 13, 2026 at 07:19:18PM -0400, Waiman Long wrote:
>> Multiple source or destination cpusets in task migration can only happens
>> when the cpuset controller is enabled or disabled in a cgroup subtree. If
>> there are DL tasks in 2 or more child cgroups, enabling or disabling of the
>> cpuset controller for those child cgroups may lead to incorrect DL task
>> accounting. This patch will probably fix the DL accounting aspect. However,
>> there are also other issues unrelated to DL tasks that need to be addressed
>> as well. So this patch is incomplete in this regard. I am working on a patch
>> series to address these issues. Hopefully I can send it out in a day or 2.
>>
> Hi Longman,
>
> Acknowledged.
>
> Also, the Sashiko AI review reported: "TOCTOU race on dl_task(task) during
> rollback causes state corruption."
>
> A concurrent sched_setscheduler() could alter the scheduling class of a
> task between the initial pass and a rollback. This assertion seems valid to
> me. Currently, neither cgroup_mutex or cpuset_mutex prevents scheduling
> class changes.
>
> Should I let you handle this too?
No, you can handle it if you want. I am more familiar with the cpuset
code, but scheduler is much more complex. I don't think I have enough
understanding of the code to handle it correctly.
Cheers,
Longman
^ permalink raw reply
* Re: [PATCH RFC] memcg: add per-cgroup dirty page controls (dirty_ratio, dirty_min)
From: Alireza Haghdoost @ 2026-05-14 4:10 UTC (permalink / raw)
To: Jan Kara
Cc: Johannes Weiner, Michal Hocko, Roman Gushchin, Shakeel Butt,
Muchun Song, Andrew Morton, Matthew Wilcox (Oracle), cgroups,
linux-mm, linux-kernel, linux-fsdevel, Kshitij Doshi
In-Reply-To: <pbwzmyglpz33d3k63aopi5vlghz4jmur2k2g4res6mhktuujhh@rmqooz6bqaao>
On Wed 06-05-26 16:21:00, Jan Kara wrote:
> Things like motivation actually belong to the changelog itself, measured
> results how the patch helps as well. On the other hand stuff like history
> is largely irrelevant here, frankly I don't have a bandwidth to carefully
> read the huge amount of text LLM has generated below so please try to make
> it more concise for next time.
Understood. Will trim for the non-RFC posting; apologies for the
volume.
> ... I quite don't see how a multisecond stalls you are describing would
> happen [...] If we are below freerun in the memcg, the task dirtying
> folios from that memcg shouldn't be throttled at all, once we get above
> freerun we throttle by maximum of throttling delay decided from global
> and memcg situation [...]
The stall is reachable even with the victim's memcg well below its
own freerun. The freerun shortcut in balance_dirty_pages() is an AND,
not OR:
if (gdtc->freerun && (!mdtc || mdtc->freerun))
goto free_running;
Once gdtc is over freerun (because the noisy neighbour pushed it
there) the shortcut does not fire, even when mdtc->freerun is true.
After the shortcut fails, the per-task pause is computed from the
dtc with the smaller pos_ratio:
if (mdtc->pos_ratio < gdtc->pos_ratio)
sdtc = mdtc;
When global is the worse domain, the victim sleeps against global
state, not memcg state.
> So can you perhaps share more details about the configuration where
> you observe these delays to innocent tasks due to another task
> dirtying a lot of memory? How many page cache in total and dirty
> pages are there in each memcg [...]? Is the delayed task really
> throttled in balance_dirty_pages()?
Yes. Re-ran the reproducer: stock 7.0-rc5, ext4 on virtio-blk
throttled to 256 KB/s, dirty_bytes=32M, dirty_background_bytes=16M
(freerun = 24 MB), noisy = single fio job doing unlimited buffered
randwrite, victim = single fio job doing 4 KiB sequential write
rate-limited to 500 KB/s.
Per-memcg snapshot during the contended phase, ~10 s into the run:
noisy memcg victim memcg global
memory.current 47 MB 21 MB --
file (cache) 38 MB 14 MB --
file_dirty 26 MB 1.7 MB 27 MB
file_writeback 1.5 MB 4.0 MB 5.3 MB
Victim memcg holds 1.7 MB dirty, far below any reasonable per-memcg
freerun. Global dirty (NR_FILE_DIRTY + NR_WRITEBACK ~ 32 MB) is over
the 24 MB freerun ceiling, driven entirely by noisy.
The victim writer (fio with psync) is in fact sleeping in
balance_dirty_pages(). One stack snapshot during a stall:
[<0>] balance_dirty_pages+0x5c5/0xac0
[<0>] balance_dirty_pages_ratelimited_flags+0x2a1/0x380
[<0>] generic_perform_write+0x194/0x280
[<0>] ext4_buffered_write_iter+0x63/0x110
[<0>] vfs_write+0x28d/0x450
[<0>] __x64_sys_pwrite64+0x8c/0xc0
[<0>] do_syscall_64+0xfa/0x520
[<0>] entry_SYSCALL_64_after_hwframe+0x77/0x7f
Sampling /proc/<pid>/wchan at 100 Hz across the contended phase
yields the histogram:
104 balance_dirty_pages
88 hrtimer_nanosleep (the fio rate-limit sleep between writes)
12 RUNNING
4 p9_client_rpc (virtfs, host-guest filesystem RPC)
3 d_alloc_parallel
The vast majority of non-rate-limit samples have the writer parked
in balance_dirty_pages(). Victim per-IO clat in this run reaches a
3 s tail (worst single 4 KiB pwrite blocked ~3.0 s) while its own
memcg holds < 2 MB dirty.
I'm happy to share the full traces and the reproducer if useful.
Thanks for the review,
Alireza
^ permalink raw reply
* Re: [PATCH v2 10/10] sched/eevdf: Move to a single runqueue
From: John Stultz @ 2026-05-14 3:14 UTC (permalink / raw)
To: K Prateek Nayak
Cc: Peter Zijlstra, mingo, longman, chenridong, juri.lelli,
vincent.guittot, dietmar.eggemann, rostedt, bsegall, mgorman,
vschneid, tj, hannes, mkoutny, cgroups, linux-kernel, qyousef
In-Reply-To: <f28220a8-955f-4bf2-9981-855816519ea6@amd.com>
On Wed, May 13, 2026 at 7:53 PM K Prateek Nayak <kprateek.nayak@amd.com> wrote:
>
> On 5/14/2026 7:06 AM, John Stultz wrote:
> > So looking at the callstack when I see the failure:
> > proxy_find_task()
> > proxy_force_return()
> > proxy_resched_idle() <- sets rq->donor to idle
> > attach_one_task()
> > wakeup_preempt()
> > wakeup_preempt_fair()
>
> After this point, I would have expected we called idle class's
> wakeup_preempt() since that is the donor context ...
Ah, that's a good point! (I was getting muddied by the rq->idle having
SCHED_NORMAL policy value and assuming that was why we were in the
fair code).
> > update_protect_slice() <- called with the donor's se
> > calc_delta_fair()
> > __calc_delta() <- div by zero
> >
> > Basically we end up in wakeup_preempt_fair() with rq->donor ==
> > rq->idle because we earlier called proxy_resched_idle().
>
> Could you check if following makes things better:
>
> (Only build tested)
>
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index 3ae5f19c1b7e..77f4ebe8f5c7 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -6653,6 +6653,7 @@ static inline void proxy_set_task_cpu(struct task_struct *p, int cpu)
> static inline struct task_struct *proxy_resched_idle(struct rq *rq)
> {
> put_prev_set_next_task(rq, rq->donor, rq->idle);
> + rq->next_class = &idle_sched_class;
> rq_set_donor(rq, rq->idle);
> set_tsk_need_resched(rq->idle);
> return rq->idle;
Yeah, that looks to avoid the problem and is a fair bit cleaner. I
missed the introduction of the rq->next_class detail!
Thanks for pointing this out!
I'll do some testing against the full series and get a patch sent out here soon.
-john
^ permalink raw reply
* Re: [PATCH v2 10/10] sched/eevdf: Move to a single runqueue
From: K Prateek Nayak @ 2026-05-14 2:53 UTC (permalink / raw)
To: John Stultz, Peter Zijlstra
Cc: mingo, longman, chenridong, juri.lelli, vincent.guittot,
dietmar.eggemann, rostedt, bsegall, mgorman, vschneid, tj, hannes,
mkoutny, cgroups, linux-kernel, qyousef
In-Reply-To: <CANDhNCqsZVsWygBA7m2F_w2r3DnQkFDXfd95Lc4ny-zjQQE7Qg@mail.gmail.com>
Hello John,
On 5/14/2026 7:06 AM, John Stultz wrote:
> So looking at the callstack when I see the failure:
> proxy_find_task()
> proxy_force_return()
> proxy_resched_idle() <- sets rq->donor to idle
> attach_one_task()
> wakeup_preempt()
> wakeup_preempt_fair()
After this point, I would have expected we called idle class's
wakeup_preempt() since that is the donor context ...
> update_protect_slice() <- called with the donor's se
> calc_delta_fair()
> __calc_delta() <- div by zero
>
> Basically we end up in wakeup_preempt_fair() with rq->donor ==
> rq->idle because we earlier called proxy_resched_idle().
Could you check if following makes things better:
(Only build tested)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 3ae5f19c1b7e..77f4ebe8f5c7 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -6653,6 +6653,7 @@ static inline void proxy_set_task_cpu(struct task_struct *p, int cpu)
static inline struct task_struct *proxy_resched_idle(struct rq *rq)
{
put_prev_set_next_task(rq, rq->donor, rq->idle);
+ rq->next_class = &idle_sched_class;
rq_set_donor(rq, rq->idle);
set_tsk_need_resched(rq->idle);
return rq->idle;
---
I'm just getting started for the day so it'll be a while before I
actually get to test this on top of flat cgroup bits which I haven't yet
run with SCHED_PROXY_EXEC enabled.
--
Thanks and Regards,
Prateek
^ permalink raw reply related
* Re: [PATCH v2 10/10] sched/eevdf: Move to a single runqueue
From: John Stultz @ 2026-05-14 1:36 UTC (permalink / raw)
To: Peter Zijlstra
Cc: mingo, longman, chenridong, juri.lelli, vincent.guittot,
dietmar.eggemann, rostedt, bsegall, mgorman, vschneid, tj, hannes,
mkoutny, cgroups, linux-kernel, kprateek.nayak, qyousef
In-Reply-To: <CANDhNCqWJ=Q3LxazK_ioo_39aFfR+yVbPEV+MQHC8_QvadhuTg@mail.gmail.com>
On Tue, May 12, 2026 at 10:00 PM John Stultz <jstultz@google.com> wrote:
> On Tue, May 12, 2026 at 9:51 PM John Stultz <jstultz@google.com> wrote:
> >
> > On Mon, May 11, 2026 at 5:07 AM Peter Zijlstra <peterz@infradead.org> wrote:
> > >
> > > Change fair/cgroup to a single runqueue.
> > >
> ...
> >
> > I know Vincent was having some perf troubles with this patch, but
> > booting on a 64 vCPU qemu environment, I'm seeing:
> >
> > [ 5.688490] Oops: divide error: 0000 [#1] SMP NOPTI
> > [ 5.689457] CPU: 47 UID: 0 PID: 0 Comm: swapper/47 Not tainted
> > 7.1.0-rc2-00026-g82a8ec6fb3f9 #38 PREEMPT(full)
> > [ 5.689457] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
> > BIOS 1.17.0-debian-1.17.0-1 04/01/2014
> > [ 5.689457] RIP: 0010:wakeup_preempt_fair+0x1b7/0x430
> > [ 5.689457] Code: 74 0b 48 8b 52 28 48 39 d0 48 0f 47 c2 48 8b b9
> > 90 00 00 00 48 8b b1 08 01 00 00 48 81 ff 00 00 10 00 74 09 48 c1 e0
> > 14 31 9
> > [ 5.689457] RSP: 0000:ffffc9000021fd70 EFLAGS: 00010046
> > [ 5.689457] RAX: 000002ab98000000 RBX: ffff8881b8e2db40 RCX: ffffffff83022a80
> > [ 5.689457] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
> > [ 5.689457] RBP: 0000000000000001 R08: ffff88810cb14380 R09: ffffffff83022b00
> > [ 5.689457] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000002
> > [ 5.689457] R13: 0000000000000000 R14: ffff88810cb14300 R15: ffff8881b8e2da00
> > [ 5.689457] FS: 0000000000000000(0000) GS:ffff888235c2e000(0000)
> > knlGS:0000000000000000
> > [ 5.689457] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > [ 5.689457] CR2: 0000000000000000 CR3: 000000000304c001 CR4: 0000000000370ef0
> > [ 5.689457] Call Trace:
> > [ 5.689457] <TASK>
> > [ 5.689457] wakeup_preempt+0xa8/0xd0
> > [ 5.689457] attach_one_task+0xec/0x150
> > [ 5.689457] __schedule+0x1ad8/0x21c0
> > [ 5.689457] schedule_idle+0x22/0x40
> > [ 5.689457] cpu_startup_entry+0x29/0x30
> > [ 5.689457] start_secondary+0xf7/0x100
> > [ 5.689457] common_startup_64+0x13e/0x148
> > [ 5.689457] </TASK>
> > [ 5.689457] Dumping ftrace buffer:
> > [ 5.689457] (ftrace buffer empty)
> > [ 5.689457] ---[ end trace 0000000000000000 ]---
> > [ 5.689457] RIP: 0010:wakeup_preempt_fair+0x1b7/0x430
> > [ 5.689457] Code: 74 0b 48 8b 52 28 48 39 d0 48 0f 47 c2 48 8b b9
> > 90 00 00 00 48 8b b1 08 01 00 00 48 81 ff 00 00 10 00 74 09 48 c1 e0
> > 14 31 9
> > [ 5.689457] RSP: 0000:ffffc9000021fd70 EFLAGS: 00010046
> > [ 5.689457] RAX: 000002ab98000000 RBX: ffff8881b8e2db40 RCX: ffffffff83022a80
> > [ 5.689457] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
> > [ 5.689457] RBP: 0000000000000001 R08: ffff88810cb14380 R09: ffffffff83022b00
> > [ 5.689457] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000002
> > [ 5.689457] R13: 0000000000000000 R14: ffff88810cb14300 R15: ffff8881b8e2da00
> > [ 5.689457] FS: 0000000000000000(0000) GS:ffff888235c2e000(0000)
> > knlGS:0000000000000000
> > [ 5.689457] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > [ 5.689457] CR2: 0000000000000000 CR3: 000000000304c001 CR4: 0000000000370ef0
> > [ 5.689457] Kernel panic - not syncing: Fatal exception
> >
> > Which I bisected down to this last patch in the series.
> >
> > faddr2line gave me:
> > __calc_delta at kernel/sched/fair.c:290
> > (inlined by) calc_delta_fair at kernel/sched/fair.c:300
> > (inlined by) update_protect_slice at kernel/sched/fair.c:1070
> > (inlined by) wakeup_preempt_fair at kernel/sched/fair.c:9193
> >
> > This usually trips as the ww_mutex selftest starts at bootup.
> >
> > Unfortunately I still see it with the add-on changes you proposed to K
> > Prateek's feedback here.
> >
> > I'll try to narrow it down further tomorrow.
>
> As karma would have it, this does seem to depend on CONFIG_SCHED_PROXY_EXEC. :)
> I'm guessing the switch in calc_delta_fair() to use se->h_load is
> uncovering something proxy isn't handling properly with that value.
>
So looking at the callstack when I see the failure:
proxy_find_task()
proxy_force_return()
proxy_resched_idle() <- sets rq->donor to idle
attach_one_task()
wakeup_preempt()
wakeup_preempt_fair()
update_protect_slice() <- called with the donor's se
calc_delta_fair()
__calc_delta() <- div by zero
Basically we end up in wakeup_preempt_fair() with rq->donor ==
rq->idle because we earlier called proxy_resched_idle().
Without proxy, if we call wakeup_preempt_fair() when rq->donor (and
rq->curr) is rq->idle, we usually end up taking the `if
(test_tsk_need_resched(rq->curr))` early exit and we don't hit this.
But with proxy, rq->curr isn't idle at this point. So we end up
continuing on. Despite the se_is_idle(se) checks (where se is the
&donor->se), those don't catch because rq->idle (maybe unintuitvely)
has a SCHED_NORMAL policy.
So we end up getting down to update_protect_slice() with rq->idle as
the se and the idle h_load.weight is zero.
Not sure what the best approach might be, but adding:
if (donor == rq->idle) {
/* don't give rq->idle slice protection */
preempt_action = PREEMPT_WAKEUP_SHORT;
goto preempt;
}
similar to the `if (cse_is_idle && !pse_is_idle)` check seems to resolve this.
Anyway, if you have thoughts on better approach, I'd be happy to work
up a patch to add on top of this one.
thanks
-john
^ permalink raw reply
* Re: [PATCH v3 05/12] mm, swap: unify large folio allocation
From: Baolin Wang @ 2026-05-14 1:30 UTC (permalink / raw)
To: Kairui Song
Cc: linux-mm, Andrew Morton, David Hildenbrand, Zi Yan, Barry Song,
Hugh Dickins, Chris Li, Kemeng Shi, Nhat Pham, Baoquan He,
Johannes Weiner, Youngjun Park, Chengming Zhou, Roman Gushchin,
Shakeel Butt, Muchun Song, Qi Zheng, linux-kernel, cgroups,
Yosry Ahmed, Lorenzo Stoakes, Dev Jain, Lance Yang, Michal Hocko,
Michal Hocko, Suren Baghdasaryan, Axel Rasmussen
In-Reply-To: <CAMgjq7CPDk4hP-5hnV4Bth523W_Z+9hvg7Wo_upE-5rYCA0Mpw@mail.gmail.com>
On 5/13/26 2:47 PM, Kairui Song wrote:
> On Tue, May 12, 2026 at 6:14 PM Baolin Wang
> <baolin.wang@linux.alibaba.com> wrote:
>>
>> On 4/21/26 2:16 PM, Kairui Song via B4 Relay wrote:
>>> From: Kairui Song <kasong@tencent.com>
>>>
>>> Now that direct large order allocation is supported in the swap cache,
>>> both anon and shmem can use it instead of implementing their own methods.
>>> This unifies the fallback and swap cache check, which also reduces the
>>> TOCTOU race window of swap cache state: previously, high order swapin
>>> required checking swap cache states first, then allocating and falling
>>> back separately. Now all these steps happen in the same compact loop.
>>>
>>> Order fallback and statistics are also unified, callers just need to
>>> check and pass the acceptable order bitmask.
>>>
>>> There is basically no behavior change. This only makes things more
>>> unified and prepares for later commits. Cgroup and zero map checks can
>>> also be moved into the compact loop, further reducing race windows and
>>> redundancy
>>>
>>> Signed-off-by: Kairui Song <kasong@tencent.com>
>>> ---
>>> mm/memory.c | 77 ++++++------------------------
>>> mm/shmem.c | 94 +++++++++---------------------------
>>> mm/swap.h | 30 ++----------
>>> mm/swap_state.c | 145 ++++++++++----------------------------------------------
>>> mm/swapfile.c | 3 +-
>>> 5 files changed, 67 insertions(+), 282 deletions(-)
>>>
>>> diff --git a/mm/memory.c b/mm/memory.c
>>> index ea6568571131..404734a5bcff 100644
>>> --- a/mm/memory.c
>>> +++ b/mm/memory.c
>>> @@ -4593,26 +4593,6 @@ static vm_fault_t handle_pte_marker(struct vm_fault *vmf)
>>> return VM_FAULT_SIGBUS;
>>> }
>>>
>>> -static struct folio *__alloc_swap_folio(struct vm_fault *vmf)
>>> -{
>>> - struct vm_area_struct *vma = vmf->vma;
>>> - struct folio *folio;
>>> - softleaf_t entry;
>>> -
>>> - folio = vma_alloc_folio(GFP_HIGHUSER_MOVABLE, 0, vma, vmf->address);
>>> - if (!folio)
>>> - return NULL;
>>> -
>>> - entry = softleaf_from_pte(vmf->orig_pte);
>>> - if (mem_cgroup_swapin_charge_folio(folio, vma->vm_mm,
>>> - GFP_KERNEL, entry)) {
>>> - folio_put(folio);
>>> - return NULL;
>>> - }
>>> -
>>> - return folio;
>>> -}
>>> -
>>> #ifdef CONFIG_TRANSPARENT_HUGEPAGE
>>> /*
>>> * Check if the PTEs within a range are contiguous swap entries
>>> @@ -4642,8 +4622,6 @@ static bool can_swapin_thp(struct vm_fault *vmf, pte_t *ptep, int nr_pages)
>>> */
>>> if (unlikely(swap_zeromap_batch(entry, nr_pages, NULL) != nr_pages))
>>> return false;
>>> - if (unlikely(non_swapcache_batch(entry, nr_pages) != nr_pages))
>>> - return false;
>>>
>>> return true;
>>> }
>>> @@ -4671,16 +4649,14 @@ static inline unsigned long thp_swap_suitable_orders(pgoff_t swp_offset,
>>> return orders;
>>> }
>>>
>>> -static struct folio *alloc_swap_folio(struct vm_fault *vmf)
>>> +static unsigned long thp_swapin_suitable_orders(struct vm_fault *vmf)
>>> {
>>> struct vm_area_struct *vma = vmf->vma;
>>> unsigned long orders;
>>> - struct folio *folio;
>>> unsigned long addr;
>>> softleaf_t entry;
>>> spinlock_t *ptl;
>>> pte_t *pte;
>>> - gfp_t gfp;
>>> int order;
>>>
>>> /*
>>> @@ -4688,7 +4664,7 @@ static struct folio *alloc_swap_folio(struct vm_fault *vmf)
>>> * maintain the uffd semantics.
>>> */
>>> if (unlikely(userfaultfd_armed(vma)))
>>> - goto fallback;
>>> + return 0;
>>>
>>> /*
>>> * A large swapped out folio could be partially or fully in zswap. We
>>> @@ -4696,7 +4672,7 @@ static struct folio *alloc_swap_folio(struct vm_fault *vmf)
>>> * folio.
>>> */
>>> if (!zswap_never_enabled())
>>> - goto fallback;
>>> + return 0;
>>>
>>> entry = softleaf_from_pte(vmf->orig_pte);
>>> /*
>>> @@ -4710,12 +4686,12 @@ static struct folio *alloc_swap_folio(struct vm_fault *vmf)
>>> vmf->address, orders);
>>>
>>> if (!orders)
>>> - goto fallback;
>>> + return 0;
>>>
>>> pte = pte_offset_map_lock(vmf->vma->vm_mm, vmf->pmd,
>>> vmf->address & PMD_MASK, &ptl);
>>> if (unlikely(!pte))
>>> - goto fallback;
>>> + return 0;
>>>
>>> /*
>>> * For do_swap_page, find the highest order where the aligned range is
>>> @@ -4731,29 +4707,12 @@ static struct folio *alloc_swap_folio(struct vm_fault *vmf)
>>>
>>> pte_unmap_unlock(pte, ptl);
>>>
>>> - /* Try allocating the highest of the remaining orders. */
>>> - gfp = vma_thp_gfp_mask(vma);
>>> - while (orders) {
>>> - addr = ALIGN_DOWN(vmf->address, PAGE_SIZE << order);
>>> - folio = vma_alloc_folio(gfp, order, vma, addr);
>>> - if (folio) {
>>> - if (!mem_cgroup_swapin_charge_folio(folio, vma->vm_mm,
>>> - gfp, entry))
>>> - return folio;
>>> - count_mthp_stat(order, MTHP_STAT_SWPIN_FALLBACK_CHARGE);
>>> - folio_put(folio);
>>> - }
>>> - count_mthp_stat(order, MTHP_STAT_SWPIN_FALLBACK);
>>> - order = next_order(&orders, order);
>>> - }
>>> -
>>> -fallback:
>>> - return __alloc_swap_folio(vmf);
>>> + return orders;
>>> }
>>> #else /* !CONFIG_TRANSPARENT_HUGEPAGE */
>>> -static struct folio *alloc_swap_folio(struct vm_fault *vmf)
>>> +static unsigned long thp_swapin_suitable_orders(struct vm_fault *vmf)
>>> {
>>> - return __alloc_swap_folio(vmf);
>>> + return 0;
>>> }
>>> #endif /* CONFIG_TRANSPARENT_HUGEPAGE */
>>>
>>> @@ -4859,21 +4818,13 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
>>> if (folio)
>>> swap_update_readahead(folio, vma, vmf->address);
>>> if (!folio) {
>>> - if (data_race(si->flags & SWP_SYNCHRONOUS_IO)) {
>>> - folio = alloc_swap_folio(vmf);
>>> - if (folio) {
>>> - /*
>>> - * folio is charged, so swapin can only fail due
>>> - * to raced swapin and return NULL.
>>> - */
>>> - swapcache = swapin_folio(entry, folio);
>>> - if (swapcache != folio)
>>> - folio_put(folio);
>>> - folio = swapcache;
>>> - }
>>> - } else {
>>> + /* Swapin bypasses readahead for SWP_SYNCHRONOUS_IO devices */
>>> + if (data_race(si->flags & SWP_SYNCHRONOUS_IO))
>>> + folio = swapin_entry(entry, GFP_HIGHUSER_MOVABLE,
>>> + thp_swapin_suitable_orders(vmf),
>>> + vmf, NULL, 0);
>>> + else
>>> folio = swapin_readahead(entry, GFP_HIGHUSER_MOVABLE, vmf);
>>> - }
>>>
>>> if (!folio) {
>>> /*
>>> diff --git a/mm/shmem.c b/mm/shmem.c
>>> index 5916acf594a8..17e3da11bb1d 100644
>>> --- a/mm/shmem.c
>>> +++ b/mm/shmem.c
>>> @@ -159,7 +159,7 @@ static unsigned long shmem_default_max_inodes(void)
>>>
>>> static int shmem_swapin_folio(struct inode *inode, pgoff_t index,
>>> struct folio **foliop, enum sgp_type sgp, gfp_t gfp,
>>> - struct vm_area_struct *vma, vm_fault_t *fault_type);
>>> + struct vm_fault *vmf, vm_fault_t *fault_type);
>>>
>>> static inline struct shmem_sb_info *SHMEM_SB(struct super_block *sb)
>>> {
>>> @@ -2017,68 +2017,24 @@ static struct folio *shmem_alloc_and_add_folio(struct vm_fault *vmf,
>>> }
>>>
>>> static struct folio *shmem_swap_alloc_folio(struct inode *inode,
>>> - struct vm_area_struct *vma, pgoff_t index,
>>> + struct vm_fault *vmf, pgoff_t index,
>>> swp_entry_t entry, int order, gfp_t gfp)
>>> {
>>> + pgoff_t ilx;
>>> + struct folio *folio;
>>> + struct mempolicy *mpol;
>>> + unsigned long orders = BIT(order);
>>> struct shmem_inode_info *info = SHMEM_I(inode);
>>> - struct folio *new, *swapcache;
>>> - int nr_pages = 1 << order;
>>> - gfp_t alloc_gfp = gfp;
>>> -
>>> - if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE)) {
>>> - if (WARN_ON_ONCE(order))
>>> - return ERR_PTR(-EINVAL);
>>> - } else if (order) {
>>> - /*
>>> - * If uffd is active for the vma, we need per-page fault
>>> - * fidelity to maintain the uffd semantics, then fallback
>>> - * to swapin order-0 folio, as well as for zswap case.
>>> - * Any existing sub folio in the swap cache also blocks
>>> - * mTHP swapin.
>>> - */
>>> - if ((vma && unlikely(userfaultfd_armed(vma))) ||
>>> - !zswap_never_enabled() ||
>>> - non_swapcache_batch(entry, nr_pages) != nr_pages)
>>> - goto fallback;
>>>
>>> - alloc_gfp = thp_limit_gfp_mask(vma_thp_gfp_mask(vma), gfp);
>>> - }
>>> -retry:
>>> - new = shmem_alloc_folio(alloc_gfp, order, info, index);
>>> - if (!new) {
>>> - new = ERR_PTR(-ENOMEM);
>>> - goto fallback;
>>> - }
>>> + if ((vmf && unlikely(userfaultfd_armed(vmf->vma))) ||
>>> + !zswap_never_enabled())
>>> + orders = 0;
>>>
>>> - if (mem_cgroup_swapin_charge_folio(new, vma ? vma->vm_mm : NULL,
>>> - alloc_gfp, entry)) {
>>> - folio_put(new);
>>> - new = ERR_PTR(-ENOMEM);
>>> - goto fallback;
>>> - }
>>> + mpol = shmem_get_pgoff_policy(info, index, order, &ilx);
>>> + folio = swapin_entry(entry, gfp, orders, vmf, mpol, ilx);
>>> + mpol_cond_put(mpol);
>>>
>>> - swapcache = swapin_folio(entry, new);
>>> - if (swapcache != new) {
>>> - folio_put(new);
>>> - if (!swapcache) {
>>> - /*
>>> - * The new folio is charged already, swapin can
>>> - * only fail due to another raced swapin.
>>> - */
>>> - new = ERR_PTR(-EEXIST);
>>> - goto fallback;
>>> - }
>>> - }
>>> - return swapcache;
>>> -fallback:
>>> - /* Order 0 swapin failed, nothing to fallback to, abort */
>>> - if (!order)
>>> - return new;
>>> - entry.val += index - round_down(index, nr_pages);
>>> - alloc_gfp = gfp;
>>> - nr_pages = 1;
>>> - order = 0;
>>> - goto retry;
>>> + return folio;
>>> }
>>
>> IIUC, in the __swap_cache_alloc() implementation in patch 4, when shmem
>> swapin falls back to order 0, it doesn't adjust the swap entry value
>> like here. Because the original swap entry may not correspond to the
>> swap entry for the order 0 index.
>>
>> Of course, I haven't tested this yet, just pointing it out for you to
>> double check.
>
> Thanks for pointing it out. No worry, we have the below change in this
> commit already:
>
> /* Direct swapin skipping swap cache & readahead */
> - folio = shmem_swap_alloc_folio(inode, vma, index,
> - index_entry, order, gfp);
> - if (IS_ERR(folio)) {
> - error = PTR_ERR(folio);
> - folio = NULL;
> - goto failed;
> - }
> + folio = shmem_swap_alloc_folio(inode, vmf, index,
> + swap, order, gfp);
>
> It's using swap instead of index_entry now, so __swap_cache_alloc will
> do the round down for large order instead and skip the round_down if
> ordedr is zero. So we are fine here.
OK. I overlooked this change here, then the shmem part looks good to me.
I will do some testing once this patchset lands in the mm-new branch.
^ permalink raw reply
* Re: [PATCH] cpuset: Fix multi-source deadline task accounting and bandwidth bypass
From: Aaron Tomlin @ 2026-05-13 23:39 UTC (permalink / raw)
To: Waiman Long
Cc: Dietmar Eggemann, tj, hannes, mkoutny, chenridong, neelx, cgroups,
linux-kernel
In-Reply-To: <7ae7fe29-6405-41e3-9f3b-6c1d0255dc9e@redhat.com>
[-- Attachment #1: Type: text/plain, Size: 1150 bytes --]
On Wed, May 13, 2026 at 07:19:18PM -0400, Waiman Long wrote:
> Multiple source or destination cpusets in task migration can only happens
> when the cpuset controller is enabled or disabled in a cgroup subtree. If
> there are DL tasks in 2 or more child cgroups, enabling or disabling of the
> cpuset controller for those child cgroups may lead to incorrect DL task
> accounting. This patch will probably fix the DL accounting aspect. However,
> there are also other issues unrelated to DL tasks that need to be addressed
> as well. So this patch is incomplete in this regard. I am working on a patch
> series to address these issues. Hopefully I can send it out in a day or 2.
>
Hi Longman,
Acknowledged.
Also, the Sashiko AI review reported: "TOCTOU race on dl_task(task) during
rollback causes state corruption."
A concurrent sched_setscheduler() could alter the scheduling class of a
task between the initial pass and a rollback. This assertion seems valid to
me. Currently, neither cgroup_mutex or cpuset_mutex prevents scheduling
class changes.
Should I let you handle this too?
Kind regards,
--
Aaron Tomlin
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply
* Re: [PATCH] cpuset: Fix multi-source deadline task accounting and bandwidth bypass
From: Waiman Long @ 2026-05-13 23:19 UTC (permalink / raw)
To: Dietmar Eggemann, Aaron Tomlin, tj, hannes, mkoutny
Cc: chenridong, neelx, cgroups, linux-kernel
In-Reply-To: <ddc8040f-2186-4c72-a69e-26b388cb7249@arm.com>
On 5/13/26 12:22 PM, Dietmar Eggemann wrote:
> On 12.05.26 03:03, Aaron Tomlin wrote:
>> During a batch migration where threads in a taskset originate from
>> multiple source cpusets (e.g., via cgroup.procs), cpuset_can_attach()
>> and cpuset_attach() currently evaluate the source cpuset exactly once
>> by caching the first task's oldcs.
>>
>> This creates two distinct critical flaws for SCHED_DEADLINE tasks:
>>
>> 1. oldcs->nr_deadline_tasks is decremented solely on the first
>> source cpuset. If tasks originated from other cpusets, their
>> counts are permanently leaked, and the first cpuset permanently
>> underflows.
>>
>> 2. cpumask_intersects() is evaluated strictly against the first
>> task's source cpuset. This allows tasks originating from
>> entirely isolated root domains to silently bypass the
>> dl_bw_alloc() admission control.
>>
>> This patch refactors the deadline accounting to evaluate task_cs(task)
>> on a per-task basis during the cgroup_taskset_for_each() loops. To
>> achieve accurate accounting before the core cgroup migration actually
>> executes, the permanent nr_deadline_tasks increments/decrements are
>> shifted into cpuset_can_attach(). If the migration aborts, the counts
>> are gracefully reverted via an internal rollback loop or the
>> cpuset_cancel_attach() callback.
> Is there a testcase to provoke this issue in the current code?
>
> I tried to move a process with 6 DL tasks from one cpuset to another by:
>
> echo $PID > /sys/fs/cgroup/B/cgroup.procs
>
> but in this case old_cs is the same for all these tasks.
>
> [ 1991.852034] cgroup_migrate() (7) leader=[dl_batch_cgroup 823] threadgroup=1
> [ 1991.852068] cgroup_migrate_execute() tset->nr_tasks=7
> [ 1991.852238] cpuset_can_attach() (4) [dl_batch_cgroup 832] nr_migrate_dl_tasks=1 sum_migrate_dl_bw=104857 old_cs=ffff0000c4955200
> [ 1991.852246] cpuset_can_attach() (4) [dl_batch_cgroup 833] nr_migrate_dl_tasks=2 sum_migrate_dl_bw=209714 old_cs=ffff0000c4955200
> [ 1991.852248] cpuset_can_attach() (4) [dl_batch_cgroup 834] nr_migrate_dl_tasks=3 sum_migrate_dl_bw=314571 old_cs=ffff0000c4955200
> [ 1991.852249] cpuset_can_attach() (4) [dl_batch_cgroup 835] nr_migrate_dl_tasks=4 sum_migrate_dl_bw=419428 old_cs=ffff0000c4955200
> [ 1991.852249] cpuset_can_attach() (4) [dl_batch_cgroup 836] nr_migrate_dl_tasks=5 sum_migrate_dl_bw=524285 old_cs=ffff0000c4955200
> [ 1991.852250] cpuset_can_attach() (4) [dl_batch_cgroup 837] nr_migrate_dl_tasks=6 sum_migrate_dl_bw=629142 old_cs=ffff0000c4955200
> [ 1991.852328] cpuset_attach() (5) cs=ffff0000c1e9fc00 oldcs=ffff0000c4955200 cs->nr_deadline_tasks=6 oldcs->nr_deadline_tasks=6 cs->nr_migrate_dl_tasks=6
>
> dl_batch_cgroup 823 823 19 - 0 TS
> dl_batch_cgroup 823 832 140 0 - DLN
> dl_batch_cgroup 823 833 140 0 - DLN
> dl_batch_cgroup 823 834 140 0 - DLN
> dl_batch_cgroup 823 835 140 0 - DLN
> dl_batch_cgroup 823 836 140 0 - DLN
> dl_batch_cgroup 823 837 140 0 - DLN
>
> [...]
Multiple source or destination cpusets in task migration can only
happens when the cpuset controller is enabled or disabled in a cgroup
subtree. If there are DL tasks in 2 or more child cgroups, enabling or
disabling of the cpuset controller for those child cgroups may lead to
incorrect DL task accounting. This patch will probably fix the DL
accounting aspect. However, there are also other issues unrelated to DL
tasks that need to be addressed as well. So this patch is incomplete in
this regard. I am working on a patch series to address these issues.
Hopefully I can send it out in a day or 2.
Cheers,
Longman
^ permalink raw reply
* Re: [PATCH] cpuset: Fix multi-source deadline task accounting and bandwidth bypass
From: Aaron Tomlin @ 2026-05-13 23:09 UTC (permalink / raw)
To: Dietmar Eggemann
Cc: longman, tj, hannes, mkoutny, chenridong, neelx, cgroups,
linux-kernel
In-Reply-To: <ddc8040f-2186-4c72-a69e-26b388cb7249@arm.com>
[-- Attachment #1: Type: text/plain, Size: 4187 bytes --]
On Wed, May 13, 2026 at 06:22:25PM +0200, Dietmar Eggemann wrote:
> Is there a testcase to provoke this issue in the current code?
>
> I tried to move a process with 6 DL tasks from one cpuset to another by:
>
> echo $PID > /sys/fs/cgroup/B/cgroup.procs
>
> but in this case old_cs is the same for all these tasks.
>
> [ 1991.852034] cgroup_migrate() (7) leader=[dl_batch_cgroup 823] threadgroup=1
> [ 1991.852068] cgroup_migrate_execute() tset->nr_tasks=7
> [ 1991.852238] cpuset_can_attach() (4) [dl_batch_cgroup 832] nr_migrate_dl_tasks=1 sum_migrate_dl_bw=104857 old_cs=ffff0000c4955200
> [ 1991.852246] cpuset_can_attach() (4) [dl_batch_cgroup 833] nr_migrate_dl_tasks=2 sum_migrate_dl_bw=209714 old_cs=ffff0000c4955200
> [ 1991.852248] cpuset_can_attach() (4) [dl_batch_cgroup 834] nr_migrate_dl_tasks=3 sum_migrate_dl_bw=314571 old_cs=ffff0000c4955200
> [ 1991.852249] cpuset_can_attach() (4) [dl_batch_cgroup 835] nr_migrate_dl_tasks=4 sum_migrate_dl_bw=419428 old_cs=ffff0000c4955200
> [ 1991.852249] cpuset_can_attach() (4) [dl_batch_cgroup 836] nr_migrate_dl_tasks=5 sum_migrate_dl_bw=524285 old_cs=ffff0000c4955200
> [ 1991.852250] cpuset_can_attach() (4) [dl_batch_cgroup 837] nr_migrate_dl_tasks=6 sum_migrate_dl_bw=629142 old_cs=ffff0000c4955200
> [ 1991.852328] cpuset_attach() (5) cs=ffff0000c1e9fc00 oldcs=ffff0000c4955200 cs->nr_deadline_tasks=6 oldcs->nr_deadline_tasks=6 cs->nr_migrate_dl_tasks=6
>
> dl_batch_cgroup 823 823 19 - 0 TS
> dl_batch_cgroup 823 832 140 0 - DLN
> dl_batch_cgroup 823 833 140 0 - DLN
> dl_batch_cgroup 823 834 140 0 - DLN
> dl_batch_cgroup 823 835 140 0 - DLN
> dl_batch_cgroup 823 836 140 0 - DLN
> dl_batch_cgroup 823 837 140 0 - DLN
>
> [...]
Hi Dietmar,
Thank you for your feedback.
When you write a PID to cgroup.procs, the cgroup core gathers all threads
in that threadgroup into a single cgroup_taskset. If those threads were
spawned normally and never individually moved, they will all share the
exact same old_cs, which is why your test yielded identical source cpusets.
To provoke this specific BUG, you have to split the threads across
different cgroups before you trigger the batch migration that pulls them
all back together.
Here is the test case to reproduce the multi-source edge case:
1. Create two source cpusets and one target cpuset
mkdir /sys/fs/cgroup/SRC_A
mkdir /sys/fs/cgroup/SRC_B
mkdir /sys/fs/cgroup/TARGET
2. Start your Multithreaded DL Application
Run your dl_batch_cgroup app. Let's assume it has PID 1000 and
spawns two SCHED_DEADLINE threads: TID 1001 and TID 1002.
3. Split the Threads
Instead of moving the whole process, move the individual threads
into different source cpusets using the thread-level interface
echo 1001 > /sys/fs/cgroup/SRC_A/cgroup.threads
echo 1002 > /sys/fs/cgroup/SRC_B/cgroup.threads
At this point, SRC_A has nr_deadline_tasks = 1 and SRC_B has
nr_deadline_tasks = 1.
4. Trigger the Batch Migration
Now, trigger a process-level migration by writing the main
threadgroup ID to the target cpuset's cgroup.procs file.
echo 1000 > /sys/fs/cgroup/TARGET/cgroup.procs
Now, when you execute Step 4, the cgroup core gathers TID 1001 and 1002 into a
single cgroup_taskset. Because they originated from different cgroups, they
have different old_cs pointers.
However, the unpatched cpuset_can_attach() loops through the taskset, finds
the oldcs of the first task (e.g., SRC_A), and caches it. It then counts
that there are 2 migrating DL tasks (i.e., cs->nr_migrate_dl_tasks = 2).
In cpuset_attach(), it blindly subtracts the total migrating DL count from
the cached oldcs:
oldcs->nr_deadline_tasks -= cs->nr_migrate_dl_tasks;
/*
* SRC_A count becomes 1 - 2 = -1 (Underflow)
* SRC_B count remains 1 (Permanent leak)
*/
The patch resolves this by evaluating task_cs(task) individually for every
single task as the loop iterates through the cgroup_taskset.
Kind regards,
--
Aaron Tomlin
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply
* Re: [PATCH 3/3] mm/zswap: Add per-memcg stat for proactive writeback
From: Nhat Pham @ 2026-05-13 21:21 UTC (permalink / raw)
To: Hao Jia
Cc: akpm, tj, hannes, shakeel.butt, mhocko, yosry, mkoutny,
chengming.zhou, muchun.song, roman.gushchin, cgroups, linux-mm,
linux-kernel, linux-doc, Hao Jia
In-Reply-To: <20260511105149.75584-4-jiahao.kernel@gmail.com>
On Mon, May 11, 2026 at 3:52 AM Hao Jia <jiahao.kernel@gmail.com> wrote:
>
> From: Hao Jia <jiahao1@lixiang.com>
>
> Currently, zswap writeback can be triggered by either the pool limit
> being hit or by the proactive writeback mechanism. However, the
> existing 'zswpwb' metric in memory.stat and /proc/vmstat counts all
> written back pages, making it difficult to distinguish between pages
> written back due to the pool limit and those written back proactively.
>
> Add a new statistic 'zswpwb_proactive' to memory.stat and /proc/vmstat.
> This counter tracks the number of pages written back due to proactive
> writeback. This allows users to better monitor and tune the proactive
> writeback mechanism.
>
> Signed-off-by: Hao Jia <jiahao1@lixiang.com>
> ---
> Documentation/admin-guide/cgroup-v2.rst | 4 ++++
> include/linux/vm_event_item.h | 1 +
> mm/memcontrol.c | 1 +
> mm/vmstat.c | 1 +
> mm/zswap.c | 11 +++++++++--
> 5 files changed, 16 insertions(+), 2 deletions(-)
>
> diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst
> index 05b664b3b3e8..29a189b18efc 100644
> --- a/Documentation/admin-guide/cgroup-v2.rst
> +++ b/Documentation/admin-guide/cgroup-v2.rst
> @@ -1734,6 +1734,10 @@ The following nested keys are defined.
> zswpwb
> Number of pages written from zswap to swap.
>
> + zswpwb_proactive
> + Number of pages written from zswap to swap by proactive
> + writeback. This is a subset of zswpwb.
> +
> zswap_incomp
> Number of incompressible pages currently stored in zswap
> without compression. These pages could not be compressed to
nit: once we have reached consensus on an interface, can you add
documentation for the new knob in cgroup v2 doc and zswap doc too, and
how it interacts with the other interface (memory.zswap.writeback,
shrinker_enabled sysfs knob).
A kselftest would be very much appreciated too :)
^ permalink raw reply
* Re: [PATCH 2/3] mm/zswap: Implement proactive writeback
From: Nhat Pham @ 2026-05-13 21:09 UTC (permalink / raw)
To: Hao Jia
Cc: Yosry Ahmed, akpm, tj, hannes, shakeel.butt, mhocko, mkoutny,
chengming.zhou, muchun.song, roman.gushchin, cgroups, linux-mm,
linux-kernel, linux-doc, Hao Jia, Alexandre Ghiti
In-Reply-To: <6fc7fdf0-368c-5129-038e-623f9db2aa88@gmail.com>
On Wed, May 13, 2026 at 1:04 AM Hao Jia <jiahao.kernel@gmail.com> wrote:
>
>
>
> On 2026/5/12 23:47, Nhat Pham wrote:
> > On Tue, May 12, 2026 at 2:32 AM Hao Jia <jiahao.kernel@gmail.com> wrote:
> >>
> >>
> >>
> >> On 2026/5/12 03:57, Yosry Ahmed wrote:
> >>> On Mon, May 11, 2026 at 12:49 PM Nhat Pham <nphamcs@gmail.com> wrote:
> >>>>
> >>>> On Mon, May 11, 2026 at 3:52 AM Hao Jia <jiahao.kernel@gmail.com> wrote:
> >>>>>
> >>>>> From: Hao Jia <jiahao1@lixiang.com>
> >>>>>
> >>>>> Zswap currently writes back pages to backing swap devices reactively,
> >>>>> triggered either by memory pressure via the shrinker or by the pool
> >>>>> reaching its size limit. This reactive approach offers no precise
> >>>>> control over when writeback happens, which can disturb latency-sensitive
> >>>>> workloads, and it cannot direct writeback at a specific memory cgroup.
> >>>>> However, there are scenarios where users might want to proactively
> >>>>> write back cold pages from zswap to the backing swap device, for
> >>>>> example, to free up memory for other applications or to prepare for
> >>>>> upcoming memory-intensive workloads.
> >>>>>
> >>>>> Therefore, implement a proactive writeback mechanism for zswap by
> >>>>> adding a new cgroup interface file memory.zswap.proactive_writeback
> >>>>> within the memory controller.
> >>>>
> >>
> >> Thanks Nhat, Yosry — let me address both comments together.
> >>
> >>>>
> >>>> We already have memory.reclaim, no? Would that not work to create
> >>>> headroom generally for your use case? Is there a reason why we are
> >>>> treating zswap memory as special here?
> >>>
> >>
> >> Apologies for the lack of detailed explanation in the patch description,
> >> which led to the confusion.
> >>
> >> While we are already utilizing memory.reclaim, it does not fully address
> >> our requirements.
> >>
> >> Our deployment runs a userspace proactive reclaimer that drives
> >> memory.reclaim based on the system's runtime state (memory/CPU/IO
> >> pressure, refault rate, ...) and workload-specific
> >> policy. That first stage compresses cold anon pages into zswap. Entries
> >> that then remain in zswap past a policy-defined age threshold are
> >> considered "twice cold", and the reclaimer wants
> >> to write them back to the backing swap device at a moment of its own
> >> choosing, to further reclaim the DRAM still held by the compressed data.
> >>
> >> This is the "second-level offloading" pattern described in Meta's TMO
> >> paper [1]. zswap proactive writeback is what this series introduces to
> >> address that second-level offloading stage.
> >>
> >> [1] https://www.pdl.cmu.edu/ftp/NVM/tmo_asplos22.pdf
> >
> > Yeah that's what we've been trying to work on as well :) We are
> > working on a couple of improvements to the mechanism side of this path
> > (cc Alex) - hopefully it will help your use case too!
> >
> > Anyway, back to my original inquiry: I understand your use case. It's
> > pretty similar to our goal. What I'm not getting is why is
> > memory.reclaim (which you already use) not sufficient for zswap ->
> > disk swap offloading too?
> >
> > Zswap objects are organized into LRU and exposed to the shrinker
> > interface. Echo-ing to memory.reclaim should also offload some zswap
> > entries, correct? Are there still cold zswap entries that escape this,
> > somehow?
> >
>
> Yes, the memory.reclaim path does drive some zswap writeback, but
> it is not enough for our case.
>
> 1. For a memcg that has reached steady state (a common case being
> when memory.current is below the policy target), the userspace
> reclaimer may not invoke memory.reclaim on it for a long time,
> and so no second-level offloading happens through
> memory.reclaim. In this state we want
> memory.zswap.proactive_writeback to write back entries that
> have sat in zswap past an age threshold, to further reclaim
> the DRAM still held by the compressed data.
>
> 2. Even when memory.reclaim is running, the fraction of zswap
> residency that ends up reaching the backing swap device is
> still very small for many of our workloads, and the userspace
> reclaimer has no way to participate in or control the
> granularity of zswap writeback. So in our deployment we prefer
> to leave the zswap shrinker disabled, decouple LRU -> zswap
> from zswap -> swap, and use a dedicated proactive-writeback
> interface that lifts the writeback policy into userspace where
> it can evolve independently of the kernel.
I see. It's interesting - we've been dealing with the opposite
problems (reclaiming too much from zswap) that it's refreshing to see
the other end of the spectrum :) We should invest more into this to
see why we are not reclaiming enough, but I see the value of adding a
knob to hit zswap exclusively.
Regarding age-based reclaim, I agree with Yosry here. Let us try to
land an interface to do targeted reclaim on compressed memory first. I
do see the value of age information: with it, you can track zswap
entries ages and the distribution of refault ages, and only reclaim
the tail. However, I wonder if you can just build a system that adapt
the reclaim request size based on PSI, refault rate etc. similar to
how you're adjusting memory.reclaim on uncompressed memories with a
senpai-like system. Something along the line of - if we are swapping
in too much from disk (or if IO pressure is high), back off, and if
not, stealing a bit more from zswap pool (perhaps with a bigger step
size), etc. Is there a reason why zswap cannot adopt a similar
strategy?
^ permalink raw reply
* Re: [PATCHSET cgroup/for-7.2] cgroup: Per-css kill_css_finish deferral
From: Tejun Heo @ 2026-05-13 21:01 UTC (permalink / raw)
To: Johannes Weiner, Michal Koutný
Cc: Sebastian Andrzej Siewior, Petr Malat, Bert Karwatzki,
kernel test robot, Martin Pitt, cgroups, linux-kernel
In-Reply-To: <20260505005121.1230198-1-tj@kernel.org>
Michal, any thoughts?
Thanks.
--
tejun
^ permalink raw reply
* [GIT PULL] cgroup: Fixes for v7.1-rc3
From: Tejun Heo @ 2026-05-13 20:59 UTC (permalink / raw)
To: Linus Torvalds
Cc: Johannes Weiner, Michal Koutný, Waiman Long, cgroups,
linux-kernel
Hello, Linus.
The following changes since commit d8769544bde51b0ac980d10f8fe9f9fed6c95995:
docs: cgroup-v1: Update charge-commit section (2026-05-04 11:02:12 -1000)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup.git tags/cgroup-for-7.1-rc3-fixes
for you to fetch changes up to 345f40166694e60db6d5cf02233814bb27ac5dec:
cgroup/cpuset: Return only actually allocated CPUs during partition invalidation (2026-05-13 08:54:53 -1000)
----------------------------------------------------------------
cgroup: Fixes for v7.1-rc3
- cpuset fixes:
- Partition invalidation could return CPUs still in use by sibling
partitions, producing overlapping effective_cpus.
- cpuset_can_attach() over-reserved DL bandwidth on moves that stayed
within the same root domain.
- Pending DL migration state leaked into later attaches when a later
can_attach() check failed.
- Reorder PF_EXITING and __GFP_HARDWALL checks so dying tasks can
allocate from any node and exit quickly.
- dmem: propagate -ENOMEM instead of spinning forever when the fallback
pool allocation also fails.
- selftests/cgroup: percpu test error-path leak, bogus numeric
comparison of cpuset strings, and a zero-length read() that silently
passed OOM-kill tests.
----------------------------------------------------------------
Chen Wandun (1):
cgroup/cpuset: move PF_EXITING check before __GFP_HARDWALL in cpuset_current_node_allowed()
Guopeng Zhang (3):
cgroup/dmem: Return -ENOMEM on failed pool preallocation
cgroup/cpuset: Reset DL migration state on can_attach() failure
cgroup/cpuset: Reserve DL bandwidth only for root-domain moves
Hongfu Li (2):
selftests/cgroup: Fix cg_read_strcmp() empty string comparison
selftests/cgroup: Fix string comparison in write_test
Yu Miao (1):
selftests/cgroup: Fix error path leaks in test_percpu_basic
sunshaojie (1):
cgroup/cpuset: Return only actually allocated CPUs during partition invalidation
include/linux/sched/deadline.h | 9 ++++
kernel/cgroup/cpuset-internal.h | 1 +
kernel/cgroup/cpuset.c | 56 ++++++++++++----------
kernel/cgroup/dmem.c | 1 +
kernel/sched/deadline.c | 13 +++--
tools/testing/selftests/cgroup/lib/cgroup_util.c | 5 +-
.../selftests/cgroup/test_cpuset_v1_base.sh | 2 +-
tools/testing/selftests/cgroup/test_kmem.c | 10 ++--
8 files changed, 63 insertions(+), 34 deletions(-)
Thanks.
--
tejun
^ permalink raw reply
* Re: [PATCH 2/3] mm/zswap: Implement proactive writeback
From: Nhat Pham @ 2026-05-13 20:53 UTC (permalink / raw)
To: Yosry Ahmed
Cc: Hao Jia, akpm, tj, hannes, shakeel.butt, mhocko, mkoutny,
chengming.zhou, muchun.song, roman.gushchin, cgroups, linux-mm,
linux-kernel, linux-doc, Hao Jia, Alexandre Ghiti
In-Reply-To: <CAO9r8zPvgB-MG2ufmdn4HoS+QEPBAehU9u7fQmYs+47NF-C9aw@mail.gmail.com>
On Wed, May 13, 2026 at 11:55 AM Yosry Ahmed <yosry@kernel.org> wrote:
>
> > > Zswap objects are organized into LRU and exposed to the shrinker
> > > interface. Echo-ing to memory.reclaim should also offload some zswap
> > > entries, correct? Are there still cold zswap entries that escape this,
> > > somehow?
> > >
> >
> > Yes, the memory.reclaim path does drive some zswap writeback, but
> > it is not enough for our case.
> >
> > 1. For a memcg that has reached steady state (a common case being
> > when memory.current is below the policy target), the userspace
> > reclaimer may not invoke memory.reclaim on it for a long time,
> > and so no second-level offloading happens through
> > memory.reclaim. In this state we want
> > memory.zswap.proactive_writeback to write back entries that
> > have sat in zswap past an age threshold, to further reclaim
> > the DRAM still held by the compressed data.
> >
> > 2. Even when memory.reclaim is running, the fraction of zswap
> > residency that ends up reaching the backing swap device is
> > still very small for many of our workloads, and the userspace
> > reclaimer has no way to participate in or control the
> > granularity of zswap writeback. So in our deployment we prefer
> > to leave the zswap shrinker disabled, decouple LRU -> zswap
> > from zswap -> swap, and use a dedicated proactive-writeback
> > interface that lifts the writeback policy into userspace where
> > it can evolve independently of the kernel.
>
> To be honest I see the point of proactively reclaiming compressed
> memory in zswap. If you use memory.reclaim, you are also reclaiming
> hotter memory in the process, and you are not necessarily getting as
> much writeback as you want. The memory in zswap is a more conservative
> choice for proactive reclaim because it's memory that's guaranteed to
> be cold(ish) and not being accessed.
>
> That being said, the interface is not great any way you cut it :/
>
> I don't like the 'memory.zswap.proactive_writeback' name, maybe we can
> stay consistent by doing 'memory.zswap.reclaim', but that just as
> easily reads as "reclaim using zswap". Maybe
> 'memory.zswap.do_writeback' or something, idk.
>
> I also don't like having two proactive reclaim interfaces, so a voice
> in my head wants to tie this into 'memory.reclaim' somehow, but that
> includes adding a pretty specific argument (e.g. 'memory.reclaim
> zswap_writeback_only=1'.
>
> I don't like any of these options, and we also need to consider what
> the memcg maintainers think. I see the use case of proactive writeback
> but I am struggling to come up with a clean interface.
>
> I also think we should take the 'age' aspect out of the conversation
> for now, it can be a separate discussion. Well, unless we decide to
> tie it to memory.reclaim. If memory.reclaim broadly supports age-based
> reclaim then zswap writeback can be a natural part of that without
> requiring a specific interface.
Yeah perhaps extending memory.reclaim is best... Sort of analogous to
the way we have swappiness to balance file v.s anon....
^ permalink raw reply
* Re: [PATCH v2] cgroup/dmem: introduce a peak file
From: Tejun Heo @ 2026-05-13 20:44 UTC (permalink / raw)
To: Thadeu Lima de Souza Cascardo
Cc: Johannes Weiner, Michal Koutný, Michal Hocko, Roman Gushchin,
Shakeel Butt, Muchun Song, Andrew Morton, Jonathan Corbet,
Shuah Khan, Maarten Lankhorst, Maxime Ripard, Natalie Vock,
Tvrtko Ursulin, cgroups, linux-kernel, linux-mm, linux-doc,
dri-devel, kernel-dev
In-Reply-To: <20260513-dmem_peak-v2-1-dac06999db9e@igalia.com>
Hello,
The patch looks fine to me, but please flesh out the motivation in the
commit description - what's the use case, why do we want this?
Thanks.
--
tejun
^ permalink raw reply
* Re: [PATCH v2 0/4] cgroup/rdma: add rdma.peak and rdma.events[.local]
From: Tejun Heo @ 2026-05-13 20:27 UTC (permalink / raw)
To: Tao Cui; +Cc: hannes, mkoutny, cgroups
In-Reply-To: <20260513104956.373216-1-cuitao@kylinos.cn>
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 1511 bytes --]
Hello,
v1 points are fully addressed. A few more on v2:
* In rdmacg_resource_set_max(), the new "rpool has peak/events" check
uses `goto dev_err` to skip free_cg_rpool_locked(). It works
because ret is still 0 at that point, but dev_err is the error
label and this isn't an error path. Restructure so the free is
guarded by an if, or rename the label.
* By the end of patch 3, the rpool-keep predicate is five lines
duplicated in uncharge_cg_locked() and rdmacg_resource_set_max().
Worth extracting into a rpool_has_persistent_state() helper — a
sixth counter later then changes one site, not two.
* Switching rdmacg_event_locked() from get_ to find_ avoids the
spurious-rpool problem I raised in v1, but it also means
ancestors of over_cg without a prior rpool for this device
silently drop the hierarchical event. Now that the rpool-keep
check covers event counters, get_ + keep-alive would give full
hierarchical coverage without the issue from v1 (rpools getting
freed on the next uncharge). The struct is small and rpool
presence isn't user-observable. Worth reconsidering — or, if you
keep find_, note the caveat in the rdma.events documentation.
* Patch 3 also extends rdma.events with hierarchical alloc_fail
but the commit message only describes rdma.events.local. Mention
the rdma.events change.
* In rdmacg_events_show() / rdmacg_events_local_show(), the
`(s64)READ_ONCE(u64) ... %lld` pattern can drop the cast and use
%llu.
Thanks.
--
tejun
^ permalink raw reply
* Re: [PATCH] cgroup/cpuset: Return only actually allocated CPUs during partition invalidation
From: Tejun Heo @ 2026-05-13 19:52 UTC (permalink / raw)
To: Sun Shaojie
Cc: Waiman Long, Chen Ridong, Johannes Weiner, Michal Koutný,
cgroups, linux-kernel
In-Reply-To: <20260512090034.183133-1-sunshaojie@kylinos.cn>
Hello,
On Tue, May 12, 2026 at 05:00:34PM +0800, Sun Shaojie wrote:
> From: sunshaojie <sunshaojie@kylinos.cn>
...
> Fixes: 0c7f293efc87 ("cgroup/cpuset: Add cpuset.cpus.exclusive.effective for v2")
> Signed-off-by: sunshaojie <sunshaojie@kylinos.cn>
I applied as-is to cgroup/for-7.1-fixes but from next time on please
capitalize and put a space between first and last name.
Thanks.
--
tejun
^ permalink raw reply
* Re: [RFC PATCH v5 20/29] sched/deadline: Allow deeper hierarchies of RT cgroups
From: Tejun Heo @ 2026-05-13 19:10 UTC (permalink / raw)
To: Yuri Andriaccio
Cc: luca abeni, Peter Zijlstra, Yuri Andriaccio, Ingo Molnar,
Juri Lelli, Vincent Guittot, Dietmar Eggemann, Steven Rostedt,
Ben Segall, Mel Gorman, Valentin Schneider, linux-kernel, hannes,
mkoutny, cgroups
In-Reply-To: <0d3336a7-ae42-4359-bfe7-48a7d6796d06@santannapisa.it>
Hello,
On Wed, May 13, 2026 at 02:08:52PM +0200, Yuri Andriaccio wrote:
> > How is a delegated subtree prevented from setting cpu.rt.min = 'root' and
> > escaping its ancestors' cpu.rt.max budget?
>
> Is it strictly required that a child cgroup must have 'less runtime' than
> its parent? To be more precise I mean scheduling tasks on the root runqueue
> instead of using dl-servers. Small note: given that HCBS cgroups use
> dl-servers, and thus run at higher priority than FIFO/RR scheduled on the
> root runqueue, if a cgroup rt.min is 'root' would yes escape its ancestor
> budget but it may also possibly get starved because of the priority levels.
The high-level invariant that we must maintain is that any given cgroup has
control over resource usages in its subtree. If that doesn't work, the whole
thing is not very useful.
e.g. There are multiple containers in the system and each wants to manage
its own internal resource distribution, which is a relatively common
scenario in server deployments. This is implemented by putting each
container in a cgroup and deletating the sub-tree to the nested container
manager. At the host level, you don't know or have control over what's going
on in each container but you can control how much each container consumes in
total so that each gets what it's allotted and doesn't get in the way of
others.
While delegation scenario is a clear example, even in regular usages, it
gets really confusing if hierarchical resource distribution isn't actually
hierarchical. If you let a child escape to root at its own discretion, might
as well just not have all the complexities with hierarchical resource
control.
> If we require that child cgroups cannot escape their parent's bandwidth,
> even when using 'root', then the cpu.rt.max file must be disallowed in the
> root cgroup (removing the possibility to reserve bandwidth for HCBS, and so
> doing the admission test similarly to when SCHED_DEADLINE tasks are
> executed), and cpu.rt.max would use either 'root' if the whole subtree must
> be scheduled onto the root runqueue or a <runtime> <period> combination to
> reserve bandwidth for the whole subtree. The cpu.rt.min would then only be
> used to reserve internal bandwidth for the cgroup itself. This also means
> that a whole subtree either uses HCBS everywhere or the root runqueue
> everywhere.
>
> > If the users on the system already started using rt, how do you enable the
> > controller from the top down with budgets already being used down in the
> > hierarchy?
>
> In my original idea rt tasks would only interfere with their own cgroup
> configuration, but not with the subtree or their parents. When cpu.rt.min =
> 'root', you are free to change cpu.rt.max values to whatever you like in any
> place of the hierarchy, and tasks inside the rt.min = 'root' cgroup would
> not be affected as they are run in the root runqueue.
>
> If you want to switch a cgroup from/to 'root' and HCBS, you'd have to either
> move all the RT tasks out of the cgroup, set rt.min, and then move them back
> in, or change temporarily their scheduling policy to non-rt (SCHED_OTHER,
> SCHED_DEADLINE, whatever) and then back.
>
> Hopefully I've answered your questions. Which solution do you think makes
> the most sense?
I'm not sure either makes sense. There's not much point in having
hierarchical controller in the first one (just require direct system-level
distribution) and I don't think the second one is very useable. I mean, try
to imagine being a user. You have to hunt down all rt tasks and twiddle
every one one way or another to change some config and then have to worry
about racing forks and class changes. At that point, you might as well just
control it centrally without the hirarchical stuff. You'd have to be really
dedicated or desparate, which means that not many are going to use it which
then brings up the question why are we doing this at all?
I wonder whether this can just be a regular max interface - ie. limit
maximum reservation in the subtree rather than exact reservation allocation.
Then cgroup can report total reservations in the subtree and admission
control can just reject anyting going over.
Thanks.
--
tejun
^ permalink raw reply
* [PATCH v2] cgroup/dmem: introduce a peak file
From: Thadeu Lima de Souza Cascardo @ 2026-05-13 18:58 UTC (permalink / raw)
To: Tejun Heo, Johannes Weiner, Michal Koutný, Michal Hocko,
Roman Gushchin, Shakeel Butt, Muchun Song, Andrew Morton,
Jonathan Corbet, Shuah Khan, Maarten Lankhorst, Maxime Ripard,
Natalie Vock, Tvrtko Ursulin
Cc: cgroups, linux-kernel, linux-mm, linux-doc, dri-devel, kernel-dev,
Thadeu Lima de Souza Cascardo
Just like we have memory.peak, introduce a dmem.peak, which uses the
page_counter support for that.
For now, make it read-only.
Signed-off-by: Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
---
Changes in v2:
- Make it read-only for now and adjust documentation accordingly.
- Link to v1: https://patch.msgid.link/20260506-dmem_peak-v1-0-8d803eb3449c@igalia.com
---
Documentation/admin-guide/cgroup-v2.rst | 6 ++++++
kernel/cgroup/dmem.c | 15 +++++++++++++++
2 files changed, 21 insertions(+)
diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst
index 6efd0095ed99..d103623b2be4 100644
--- a/Documentation/admin-guide/cgroup-v2.rst
+++ b/Documentation/admin-guide/cgroup-v2.rst
@@ -2808,6 +2808,12 @@ DMEM Interface Files
The semantics are the same as for the memory cgroup controller, and are
calculated in the same way.
+ dmem.peak
+ A read-only nested-keyed file that exists on non-root cgroups.
+
+ The max device memory usage recorded for the cgroup and its
+ descendants since the creation of the cgroup for each region.
+
dmem.capacity
A read-only file that describes maximum region capacity.
It only exists on the root cgroup. Not all memory can be
diff --git a/kernel/cgroup/dmem.c b/kernel/cgroup/dmem.c
index 4753a67d0f0f..6430c7ce1e03 100644
--- a/kernel/cgroup/dmem.c
+++ b/kernel/cgroup/dmem.c
@@ -182,6 +182,11 @@ static u64 get_resource_current(struct dmem_cgroup_pool_state *pool)
return pool ? page_counter_read(&pool->cnt) : 0;
}
+static u64 get_resource_peak(struct dmem_cgroup_pool_state *pool)
+{
+ return pool ? READ_ONCE(pool->cnt.watermark) : 0;
+}
+
static void reset_all_resource_limits(struct dmem_cgroup_pool_state *rpool)
{
set_resource_min(rpool, 0);
@@ -808,6 +813,11 @@ static int dmemcg_limit_show(struct seq_file *sf, void *v,
return 0;
}
+static int dmem_cgroup_region_peak_show(struct seq_file *sf, void *v)
+{
+ return dmemcg_limit_show(sf, v, get_resource_peak);
+}
+
static int dmem_cgroup_region_current_show(struct seq_file *sf, void *v)
{
return dmemcg_limit_show(sf, v, get_resource_current);
@@ -856,6 +866,11 @@ static struct cftype files[] = {
.name = "current",
.seq_show = dmem_cgroup_region_current_show,
},
+ {
+ .name = "peak",
+ .seq_show = dmem_cgroup_region_peak_show,
+ .flags = CFTYPE_NOT_ON_ROOT,
+ },
{
.name = "min",
.write = dmem_cgroup_region_min_write,
---
base-commit: d3b0a7f21119f5a66cb76aa28fb8cc13206aaf7d
change-id: 20260409-dmem_peak-3abc1be95072
Best regards,
--
Thadeu Lima de Souza Cascardo <cascardo@igalia.com>
^ permalink raw reply related
* Re: [PATCH v2] cgroup/cpuset: Return only actually allocated CPUs during partition invalidation
From: Tejun Heo @ 2026-05-13 18:57 UTC (permalink / raw)
To: Sun Shaojie
Cc: Waiman Long, Chen Ridong, Johannes Weiner, Michal Koutný,
cgroups, linux-kernel
In-Reply-To: <20260513103738.442779-1-sunshaojie@kylinos.cn>
Hello,
On Wed, May 13, 2026 at 06:37:38PM +0800, Sun Shaojie wrote:
> From: sunshaojie <sunshaojie@kylinos.cn>
>
> In update_parent_effective_cpumask() with partcmd_invalidate, the CPUs
> to return to the parent are computed as:
...
Applied to cgroup/for-7.1-fixes with the following changes:
- s/Test-by/Tested-by/ on Chen Ridong's tag.
- Added Reviewed-by: Waiman Long <longman@redhat.com>.
- Added Cc: stable@vger.kernel.org # v7.0+ since 2a3602030d80 shipped
in v7.0.
Thanks.
--
tejun
^ permalink raw reply
* Re: [PATCH] selftests/cgroup: Fix error path leaks in test_percpu_basic
From: Tejun Heo @ 2026-05-13 18:56 UTC (permalink / raw)
To: Yu Miao
Cc: Johannes Weiner, Michal Hocko, Roman Gushchin, Shakeel Butt,
Michal Koutný, Shuah Khan, Muchun Song, cgroups, linux-mm,
linux-kselftest, linux-kernel
In-Reply-To: <20260513023907.179097-1-yumiao@kylinos.cn>
Hello,
On Wed, May 13, 2026 at 10:39:07AM +0800, Yu Miao wrote:
> [PATCH] selftests/cgroup: Fix error path leaks in test_percpu_basic
Applied to cgroup/for-7.1-fixes with the following Fixes: tag added:
Fixes: 90631e1dea55 ("kselftests: cgroup: add perpcu memory accounting test")
Thanks.
--
tejun
^ permalink raw reply
* Re: [PATCH 2/3] mm/zswap: Implement proactive writeback
From: Yosry Ahmed @ 2026-05-13 18:54 UTC (permalink / raw)
To: Hao Jia
Cc: Nhat Pham, akpm, tj, hannes, shakeel.butt, mhocko, mkoutny,
chengming.zhou, muchun.song, roman.gushchin, cgroups, linux-mm,
linux-kernel, linux-doc, Hao Jia, Alexandre Ghiti
In-Reply-To: <6fc7fdf0-368c-5129-038e-623f9db2aa88@gmail.com>
> > Zswap objects are organized into LRU and exposed to the shrinker
> > interface. Echo-ing to memory.reclaim should also offload some zswap
> > entries, correct? Are there still cold zswap entries that escape this,
> > somehow?
> >
>
> Yes, the memory.reclaim path does drive some zswap writeback, but
> it is not enough for our case.
>
> 1. For a memcg that has reached steady state (a common case being
> when memory.current is below the policy target), the userspace
> reclaimer may not invoke memory.reclaim on it for a long time,
> and so no second-level offloading happens through
> memory.reclaim. In this state we want
> memory.zswap.proactive_writeback to write back entries that
> have sat in zswap past an age threshold, to further reclaim
> the DRAM still held by the compressed data.
>
> 2. Even when memory.reclaim is running, the fraction of zswap
> residency that ends up reaching the backing swap device is
> still very small for many of our workloads, and the userspace
> reclaimer has no way to participate in or control the
> granularity of zswap writeback. So in our deployment we prefer
> to leave the zswap shrinker disabled, decouple LRU -> zswap
> from zswap -> swap, and use a dedicated proactive-writeback
> interface that lifts the writeback policy into userspace where
> it can evolve independently of the kernel.
To be honest I see the point of proactively reclaiming compressed
memory in zswap. If you use memory.reclaim, you are also reclaiming
hotter memory in the process, and you are not necessarily getting as
much writeback as you want. The memory in zswap is a more conservative
choice for proactive reclaim because it's memory that's guaranteed to
be cold(ish) and not being accessed.
That being said, the interface is not great any way you cut it :/
I don't like the 'memory.zswap.proactive_writeback' name, maybe we can
stay consistent by doing 'memory.zswap.reclaim', but that just as
easily reads as "reclaim using zswap". Maybe
'memory.zswap.do_writeback' or something, idk.
I also don't like having two proactive reclaim interfaces, so a voice
in my head wants to tie this into 'memory.reclaim' somehow, but that
includes adding a pretty specific argument (e.g. 'memory.reclaim
zswap_writeback_only=1'.
I don't like any of these options, and we also need to consider what
the memcg maintainers think. I see the use case of proactive writeback
but I am struggling to come up with a clean interface.
I also think we should take the 'age' aspect out of the conversation
for now, it can be a separate discussion. Well, unless we decide to
tie it to memory.reclaim. If memory.reclaim broadly supports age-based
reclaim then zswap writeback can be a natural part of that without
requiring a specific interface.
^ permalink raw reply
* Re: [PATCH RFC 2/5] dma-heap: charge dma-buf memory via explicit memcg
From: Albert Esteve @ 2026-05-13 18:39 UTC (permalink / raw)
To: T.J. Mercier
Cc: Christian König, Tejun Heo, Johannes Weiner,
Michal Koutný, Jonathan Corbet, Shuah Khan, Sumit Semwal,
Michal Hocko, Roman Gushchin, Shakeel Butt, Muchun Song,
Andrew Morton, Benjamin Gaignard, Brian Starkey, John Stultz,
Christian Brauner, Paul Moore, James Morris, Serge E. Hallyn,
Stephen Smalley, Ondrej Mosnacek, Shuah Khan, cgroups, linux-doc,
linux-kernel, linux-media, dri-devel, linaro-mm-sig, linux-mm,
linux-security-module, selinux, linux-kselftest, mripard,
echanude
In-Reply-To: <CABdmKX3R5faNgFva-HHVhtTcxJ0_BK9Rei3iTQcA+SRwdKv1Aw@mail.gmail.com>
On Wed, May 13, 2026 at 6:39 PM T.J. Mercier <tjmercier@google.com> wrote:
>
> On Wed, May 13, 2026 at 5:41 AM Albert Esteve <aesteve@redhat.com> wrote:
> >
> > On Tue, May 12, 2026 at 12:14 PM Christian König
> > <christian.koenig@amd.com> wrote:
> > >
> > > On 5/12/26 11:10, Albert Esteve wrote:
> > > > On embedded platforms a central process often allocates dma-buf
> > > > memory on behalf of client applications. Without a way to
> > > > attribute the charge to the requesting client's cgroup, the
> > > > cost lands on the allocator, making per-cgroup memory limits
> > > > ineffective for the actual consumers.
> > > >
> > > > Add charge_pid_fd to struct dma_heap_allocation_data. When set to
> > > > a valid pidfd, DMA_HEAP_IOCTL_ALLOC resolves the target task's
> > > > memcg and charges the buffer there via mem_cgroup_charge_dmabuf()
> > > > inside dma_heap_buffer_alloc(). Without charge_pid_fd, and with
> > > > the mem_accounting module parameter enabled, the buffer is charged
> > > > to the allocator's own cgroup.
> > > >
> > > > Additionally, commit 3c227be90659 ("dma-buf: system_heap: account for
> > > > system heap allocation in memcg") adds __GFP_ACCOUNT to system-heap
> > > > page allocations. Keeping __GFP_ACCOUNT would charge the same pages
> > > > twice (once to kmem, once to MEMCG_DMABUF), thus remove it and route
> > > > all accounting through a single MEMCG_DMABUF path.
> > > >
> > > > Usage examples:
> > > >
> > > > 1. Central allocator charging to a client at allocation time.
> > > > The allocator knows the client's PID (e.g., from binder's
> > > > sender_pid) and uses pidfd to attribute the charge:
> > > >
> > > > pid_t client_pid = txn->sender_pid;
> > > > int pidfd = pidfd_open(client_pid, 0);
> > > >
> > > > struct dma_heap_allocation_data alloc = {
> > > > .len = buffer_size,
> > > > .fd_flags = O_RDWR | O_CLOEXEC,
> > > > .charge_pid_fd = pidfd,
> > > > };
> > > > ioctl(heap_fd, DMA_HEAP_IOCTL_ALLOC, &alloc);
> > > > close(pidfd);
> > > > /* alloc.fd is now charged to client's cgroup */
> > > >
> > > > 2. Default allocation (no pidfd, mem_accounting=1).
> > > > When charge_pid_fd is not set and the mem_accounting module
> > > > parameter is enabled, the buffer is charged to the allocator's
> > > > own cgroup:
> > > >
> > > > struct dma_heap_allocation_data alloc = {
> > > > .len = buffer_size,
> > > > .fd_flags = O_RDWR | O_CLOEXEC,
> > > > };
> > > > ioctl(heap_fd, DMA_HEAP_IOCTL_ALLOC, &alloc);
> > > > /* charged to current process's cgroup */
> > > >
> > > > Current limitations:
> > > >
> > > > - Single-owner model: a dma-buf carries one memcg charge regardless of
> > > > how many processes share it. Means only the first owner (and exporter)
> > > > of the shared buffer bears the charge.
> > > > - Only memcg accounting supported. While this makes sense for system
> > > > heap buffers, other heaps (e.g., CMA heaps) will require selectively
> > > > charging also for the dmem controller.
> > >
> > > Well that doesn't looks soo bad, it at least seems to tackle the problem at hand for Android and some of other embedded use cases.
> > >
> > > I'm just not sure if this is future prove and will work for all use cases, e.g. cloud gaming, native context for automotive etc...
> > >
> > > Essentially the problem boils down to two limitations:
> > > 1) a piece of memory can only be charged to one cgroup, the framework doesn't has a concept of charging shared memory to multiple groups
> > > 2) when memory references in the form of file descriptors are passed between applications we have no way of changing the accounting to a different cgroup
> > >
> > > The passing of the memory reference already has a well defined uAPI and if we could solve those two limitations we not only solve the problem without introducing new uAPI (with potential new security risks) but also solve it for all other use cases which uses file descriptors as well as. E.g. memfd, accel and GPU drivers etc...
> >
> > Honestly, adding a hook to fd-passing uAPI to manage charge transfers
> > sounds like a promising solution requiring no uAPI changes. However,
> > it still does not cover all paths, e.g., dup() or fork(). And shared
> > memory sounds like a hard one to tackle, where deciding the best
> > policy is more a per-usecase thing and would probably require
> > userspace configuration.
>
> I'm curious if anyone knows of a use case where FDs aren't involved at
> all? It's possible to fork() or clone() with only a dmabuf mapping and
> no FD. That sounds strange, and I'm not sure there's a real usecase
> for transferring ownership with that approach, but figured I'd at
> least pose the question.
Yeah, that's a good point. I do not really have a usecase myself for
fork(), just thought of it as a posible gap/uncovered path.
>
> > All in all, charge_pid_fd covers a
> > well-defined and immediately practical subset. The UAPI cost is small
> > and the mechanism is explicit about what it does and doesn't solve. A
> > general solution, if it ever converges, would likely supersede
> > charge_pid_fd for most cases, which is a fine outcome if it solves the
> > problem more completely.
> >
> > Either way, if you have a specific approach in mind for solving any of
> > the above limitations, I'd be happy to look into it further.
> >
> > BR,
> > Albert.
> >
> > >
> > > On the other hand it is really nice to finally see this tackled for at least DMA-buf heaps. On the GPU side I have seen just another try of a driver doing some kind of special driver specific accounting to solve this just a few weeks ago. And to be honest such single driver island approach have the tendency to break more often that they are working correctly.
> > >
> > > Regards,
> > > Christian.
> > >
> > > >
> > > > Signed-off-by: Albert Esteve <aesteve@redhat.com>
> > > > ---
> > > > Documentation/admin-guide/cgroup-v2.rst | 5 ++--
> > > > drivers/dma-buf/dma-buf.c | 16 ++++---------
> > > > drivers/dma-buf/dma-heap.c | 42 ++++++++++++++++++++++++++++++---
> > > > drivers/dma-buf/heaps/system_heap.c | 2 --
> > > > include/uapi/linux/dma-heap.h | 6 +++++
> > > > 5 files changed, 53 insertions(+), 18 deletions(-)
> > > >
> > > > diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst
> > > > index 8bdbc2e866430..824d269531eb1 100644
> > > > --- a/Documentation/admin-guide/cgroup-v2.rst
> > > > +++ b/Documentation/admin-guide/cgroup-v2.rst
> > > > @@ -1636,8 +1636,9 @@ The following nested keys are defined.
> > > > structures.
> > > >
> > > > dmabuf (npn)
> > > > - Amount of memory used for exported DMA buffers allocated by the cgroup.
> > > > - Stays with the allocating cgroup regardless of how the buffer is shared.
> > > > + Amount of memory used for exported DMA buffers allocated by or on
> > > > + behalf of the cgroup. Stays with the allocating cgroup regardless
> > > > + of how the buffer is shared.
> > > >
> > > > workingset_refault_anon
> > > > Number of refaults of previously evicted anonymous pages.
> > > > diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
> > > > index ce02377f48908..23fb758b78297 100644
> > > > --- a/drivers/dma-buf/dma-buf.c
> > > > +++ b/drivers/dma-buf/dma-buf.c
> > > > @@ -181,8 +181,11 @@ static void dma_buf_release(struct dentry *dentry)
> > > > */
> > > > BUG_ON(dmabuf->cb_in.active || dmabuf->cb_out.active);
> > > >
> > > > - mem_cgroup_uncharge_dmabuf(dmabuf->memcg, PAGE_ALIGN(dmabuf->size) / PAGE_SIZE);
> > > > - mem_cgroup_put(dmabuf->memcg);
> > > > + if (dmabuf->memcg) {
> > > > + mem_cgroup_uncharge_dmabuf(dmabuf->memcg,
> > > > + PAGE_ALIGN(dmabuf->size) / PAGE_SIZE);
> > > > + mem_cgroup_put(dmabuf->memcg);
> > > > + }
> > > >
> > > > dmabuf->ops->release(dmabuf);
> > > >
> > > > @@ -764,13 +767,6 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info)
> > > > dmabuf->resv = resv;
> > > > }
> > > >
> > > > - dmabuf->memcg = get_mem_cgroup_from_mm(current->mm);
> > > > - if (!mem_cgroup_charge_dmabuf(dmabuf->memcg, PAGE_ALIGN(dmabuf->size) / PAGE_SIZE,
> > > > - GFP_KERNEL)) {
> > > > - ret = -ENOMEM;
> > > > - goto err_memcg;
> > > > - }
> > > > -
> > > > file->private_data = dmabuf;
> > > > file->f_path.dentry->d_fsdata = dmabuf;
> > > > dmabuf->file = file;
> > > > @@ -781,8 +777,6 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info)
> > > >
> > > > return dmabuf;
> > > >
> > > > -err_memcg:
> > > > - mem_cgroup_put(dmabuf->memcg);
> > > > err_file:
> > > > fput(file);
> > > > err_module:
> > > > diff --git a/drivers/dma-buf/dma-heap.c b/drivers/dma-buf/dma-heap.c
> > > > index ac5f8685a6494..ff6e259afcdc0 100644
> > > > --- a/drivers/dma-buf/dma-heap.c
> > > > +++ b/drivers/dma-buf/dma-heap.c
> > > > @@ -7,13 +7,17 @@
> > > > */
> > > >
> > > > #include <linux/cdev.h>
> > > > +#include <linux/cgroup.h>
> > > > #include <linux/device.h>
> > > > #include <linux/dma-buf.h>
> > > > #include <linux/dma-heap.h>
> > > > +#include <linux/memcontrol.h>
> > > > +#include <linux/sched/mm.h>
> > > > #include <linux/err.h>
> > > > #include <linux/export.h>
> > > > #include <linux/list.h>
> > > > #include <linux/nospec.h>
> > > > +#include <linux/pidfd.h>
> > > > #include <linux/syscalls.h>
> > > > #include <linux/uaccess.h>
> > > > #include <linux/xarray.h>
> > > > @@ -55,10 +59,12 @@ MODULE_PARM_DESC(mem_accounting,
> > > > "Enable cgroup-based memory accounting for dma-buf heap allocations (default=false).");
> > > >
> > > > static int dma_heap_buffer_alloc(struct dma_heap *heap, size_t len,
> > > > - u32 fd_flags,
> > > > - u64 heap_flags)
> > > > + u32 fd_flags, u64 heap_flags,
> > > > + struct mem_cgroup *charge_to)
> > > > {
> > > > struct dma_buf *dmabuf;
> > > > + unsigned int nr_pages;
> > > > + struct mem_cgroup *memcg = charge_to;
> > > > int fd;
> > > >
> > > > /*
> > > > @@ -73,6 +79,22 @@ static int dma_heap_buffer_alloc(struct dma_heap *heap, size_t len,
> > > > if (IS_ERR(dmabuf))
> > > > return PTR_ERR(dmabuf);
> > > >
> > > > + nr_pages = len / PAGE_SIZE;
> > > > +
> > > > + if (memcg)
> > > > + css_get(&memcg->css);
> > > > + else if (mem_accounting)
> > > > + memcg = get_mem_cgroup_from_mm(current->mm);
> > > > +
> > > > + if (memcg) {
> > > > + if (!mem_cgroup_charge_dmabuf(memcg, nr_pages, GFP_KERNEL)) {
> > > > + mem_cgroup_put(memcg);
> > > > + dma_buf_put(dmabuf);
> > > > + return -ENOMEM;
> > > > + }
> > > > + dmabuf->memcg = memcg;
> > > > + }
> > > > +
> > > > fd = dma_buf_fd(dmabuf, fd_flags);
> > > > if (fd < 0) {
> > > > dma_buf_put(dmabuf);
> > > > @@ -102,6 +124,9 @@ static long dma_heap_ioctl_allocate(struct file *file, void *data)
> > > > {
> > > > struct dma_heap_allocation_data *heap_allocation = data;
> > > > struct dma_heap *heap = file->private_data;
> > > > + struct mem_cgroup *memcg = NULL;
> > > > + struct task_struct *task;
> > > > + unsigned int pidfd_flags;
> > > > int fd;
> > > >
> > > > if (heap_allocation->fd)
> > > > @@ -113,9 +138,20 @@ static long dma_heap_ioctl_allocate(struct file *file, void *data)
> > > > if (heap_allocation->heap_flags & ~DMA_HEAP_VALID_HEAP_FLAGS)
> > > > return -EINVAL;
> > > >
> > > > + if (heap_allocation->charge_pid_fd) {
> > > > + task = pidfd_get_task(heap_allocation->charge_pid_fd, &pidfd_flags);
> > > > + if (IS_ERR(task))
> > > > + return PTR_ERR(task);
> > > > +
> > > > + memcg = get_mem_cgroup_from_mm(task->mm);
> > > > + put_task_struct(task);
> > > > + }
> > > > +
> > > > fd = dma_heap_buffer_alloc(heap, heap_allocation->len,
> > > > heap_allocation->fd_flags,
> > > > - heap_allocation->heap_flags);
> > > > + heap_allocation->heap_flags,
> > > > + memcg);
> > > > + mem_cgroup_put(memcg);
> > > > if (fd < 0)
> > > > return fd;
> > > >
> > > > diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/system_heap.c
> > > > index 03c2b87cb1112..95d7688167b93 100644
> > > > --- a/drivers/dma-buf/heaps/system_heap.c
> > > > +++ b/drivers/dma-buf/heaps/system_heap.c
> > > > @@ -385,8 +385,6 @@ static struct page *alloc_largest_available(unsigned long size,
> > > > if (max_order < orders[i])
> > > > continue;
> > > > flags = order_flags[i];
> > > > - if (mem_accounting)
> > > > - flags |= __GFP_ACCOUNT;
> > > > page = alloc_pages(flags, orders[i]);
> > > > if (!page)
> > > > continue;
> > > > diff --git a/include/uapi/linux/dma-heap.h b/include/uapi/linux/dma-heap.h
> > > > index a4cf716a49fa6..e02b0f8cbc6a1 100644
> > > > --- a/include/uapi/linux/dma-heap.h
> > > > +++ b/include/uapi/linux/dma-heap.h
> > > > @@ -29,6 +29,10 @@
> > > > * handle to the allocated dma-buf
> > > > * @fd_flags: file descriptor flags used when allocating
> > > > * @heap_flags: flags passed to heap
> > > > + * @charge_pid_fd: optional pidfd of the process whose cgroup should be
> > > > + * charged for this allocation; 0 means charge the calling
> > > > + * process's cgroup
> > > > + * @__padding: reserved, must be zero
> > > > *
> > > > * Provided by userspace as an argument to the ioctl
> > > > */
> > > > @@ -37,6 +41,8 @@ struct dma_heap_allocation_data {
> > > > __u32 fd;
> > > > __u32 fd_flags;
> > > > __u64 heap_flags;
> > > > + __u32 charge_pid_fd;
> > > > + __u32 __padding;
> > > > };
> > > >
> > > > #define DMA_HEAP_IOC_MAGIC 'H'
> > > >
> > >
> >
>
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox