* [PATCHSET cgroup/for-7.2] cgroup: Per-css kill_css_finish deferral
From: Tejun Heo @ 2026-05-05 0:51 UTC (permalink / raw)
To: Johannes Weiner, Michal Koutný
Cc: Sebastian Andrzej Siewior, Petr Malat, Bert Karwatzki,
kernel test robot, Martin Pitt, cgroups, linux-kernel, Tejun Heo
Hello,
Follow-up to 93618edf7538 ("cgroup: Defer css percpu_ref kill on rmdir
until cgroup is depopulated") in cgroup/for-7.1-fixes, assumed merged
into cgroup/for-7.2.
That commit fixed the rmdir race by deferring kill_css_finish() at the
cgroup level so ->css_offline() runs only after PF_EXITING tasks have
left the cgroup. cgroup_apply_control_disable() has the same race shape
(PF_EXITING tasks pinning the dying controller's css while
->css_offline() runs), but fixing it requires switching
cgroup_lock_and_drain_offline()'s wait predicate from
percpu_ref_is_dying() to css_is_dying() to cover the deferral window -
too invasive for -stable, hence -7.2.
This series:
- Replaces the cgroup-level deferral with a per-subsys-css mechanism
so each controller css independently defers kill_css_finish() until
its own subtree drains.
- Pairs smp_mb()s in kill_css_sync() and css_update_populated() to
interlock the synchronous- and deferred-fire decisions.
- Wires cgroup_apply_control_disable() through the per-css deferral
and switches drain_offline to wait on css_is_dying.
After the predicate switch, a +ctrl re-enable issued while a deferred
-ctrl is still draining blocks in TASK_UNINTERRUPTIBLE on offline_waitq
until the dying css drains. Pre-existing for rmdir; the apply path now
joins it.
Verified by 200001 iterations of repro-a72f73c4dd9b, per-commit
deterministic repros for the bug-chain commits, 5292 iterations of
stress-disable-control, and targeted ftrace coverage of rmdir,
apply_disable, and nested-destroy paths. No warnings or stalls.
Based on cgroup/for-7.2 (d8769544bde5) with cgroup/for-7.1-fixes
(93618edf7538) assumed merged.
Patches:
[PATCH 1/5] cgroup: Inline cgroup_has_tasks() in cgroup.h
[PATCH 2/5] cgroup: Annotate unlocked nr_populated_* accesses with READ_ONCE/WRITE_ONCE
[PATCH 3/5] cgroup: Move populated counters to cgroup_subsys_state
[PATCH 4/5] cgroup: Add per-subsys-css kill_css_finish deferral
[PATCH 5/5] cgroup: Defer kill_css_finish() in cgroup_apply_control_disable()
Git tree: git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup.git cgroup-drain-for-7.2
include/linux/cgroup-defs.h | 30 ++++---
include/linux/cgroup.h | 27 ++++++-
kernel/cgroup/cgroup.c | 188 +++++++++++++++++++++++++-------------------
kernel/cgroup/cpuset-v1.c | 2 +-
kernel/cgroup/cpuset.c | 2 +-
5 files changed, 148 insertions(+), 101 deletions(-)
Thanks.
--
tejun
^ permalink raw reply
* Re: [PATCH] docs: cgroup-v1: Update charge-commit section
From: Tejun Heo @ 2026-05-04 21:05 UTC (permalink / raw)
To: T.J. Mercier
Cc: Johannes Weiner, Michal Koutný, cgroups, Jonathan Corbet,
Shuah Khan, linux-doc, linux-kernel
In-Reply-To: <20260430201142.240387-1-tjmercier@google.com>
Hello,
Applied to cgroup/for-7.1-fixes.
Thanks.
--
tejun
^ permalink raw reply
* Re: [PATCH v2 cgroup/for-7.1-fixes sched_ext/for-7.1-fixes] cgroup: Defer css percpu_ref kill on rmdir until cgroup is depopulated
From: Tejun Heo @ 2026-05-04 18:54 UTC (permalink / raw)
To: Martin Pitt
Cc: regressions, cgroups, Johannes Weiner, Michal Koutný,
Sebastian Andrzej Siewior, David Vernet, Andrea Righi,
Changwoo Min, Emil Tsalapatis, sched-ext, linux-kernel
In-Reply-To: <ad3b4597f3df81914b871618535370db@kernel.org>
Hello,
Applied to cgroup/for-7.1-fixes.
Thanks.
--
tejun
^ permalink raw reply
* Re: [LSF/MM/BPF TOPIC][RFC PATCH v4 00/27] Private Memory Nodes (w/ Compressed RAM)
From: Arun George/Arun George @ 2026-05-04 13:08 UTC (permalink / raw)
To: Gregory Price
Cc: lsf-pc, linux-kernel, linux-cxl, cgroups, linux-mm,
linux-trace-kernel, damon, kernel-team, gregkh, rafael, dakr,
dave, jonathan.cameron, dave.jiang, alison.schofield,
vishal.l.verma, ira.weiny, longman, akpm, david, lorenzo.stoakes,
Liam.Howlett, vbabka, rppt, surenb, mhocko, osalvador, ziy,
matthew.brost, joshua.hahnjy, rakie.kim, byungchul, ying.huang,
apopple, axelrasmussen, yuanchu, weixugc, yury.norov, linux,
mhiramat, mathieu.desnoyers, tj, hannes, mkoutny, jackmanb, sj,
baolin.wang, npache, ryan.roberts, dev.jain, baohua, lance.yang,
muchun.song, xu.xin16, chengming.zhou, jannh, linmiaohe,
nao.horiguchi, pfalcato, rientjes, shakeel.butt, riel, harry.yoo,
cl, roman.gushchin, chrisl, kasong, shikemeng, nphamcs, bhe,
zhengqi.arch, terry.bowman, gost.dev, arungeorge05, cpgs
In-Reply-To: <afIKxG5mJZE6QgpR@gourry-fedora-PF4VCD3F>
On 29-04-2026 07:12 pm, Gregory Price wrote:
>>
>> Great! I believe "writable budget" could be an interesting idea which
>> can solve the 'bus error' sort of scenarios due to device not capable of
>> taking any more writes. The write budget could be replenished using the
>> control path and writes will not go ahead without the budget available,
>> right?>
>>
>
> Write budget is simple
>
> budget=1 (up to 1 page can be writable
> 1) swap 1 page -> cram alloc 1 page, put VSWAP_CRAM in PTE
> 2) read-fault -> cram upgrades VSWAP_CRAM to R/O PTE
> 3) write-fault ->
> a) if (writable_cnt < budget) { budget++; mkwrite(pte); }
> b) else: normal swap semantic -> promote to normal memory
>
> Meanwhile - use ballooning and a simple shrinker to dynamically size the
> region to respond to real compression ratio.
>
>
> All said an done - you get something close to zswap but with R/O
> mappings for all entries, and optional R/W-mappings for administrators
> who know something about their workload and can afford to take the risk
> of some amount of capacity being written to uncontended in exchange for
> performance.
>
> The writable-budget is a risk-dial: How much do you trust your workload
> to now spew un/poorly-compressible memory? The write-budget is a direct
> measure of that. (so take P99.99999 compression ratios, and you can make
> a good chunk of that writable).
>
> ~Gregory
>
>
I believe we are converging. Agree to most points you mentioned.
I see this problem statement can be solved by 'write-control + write
budget' approach similar to what you have described, whether we take
swap path or not.
But I see this 'write budget' (budget in terms of number of write
operations that can be handled by the device, not capacity) to be
provided by the device in control plane; not by the workloads in the host.
The budget can be communicated by the device in the device control plane
periodically (to be handled in the specific cram back-end driver; may be
interpreting the device back-pressure indications into a write budget
value). Even if the control plane breaks down, the host does not run
into issues except that it will not write further.
I assume you see this value coming from the workloads. This might be a
place where I have a different opinion.
There are multiple advantages of this value coming from the device:
1) We can modulate the write budget depending on the actual
compressibility in the device (and so workloads data). We don't have to
do estimation based on the workloads.
2) We don't have to do the capacity modulation - as in ballooning or
shrinker.
3) Even if the control path is broken, host can write only till the
available 'write budget'; so it won't get into 'bus error' situations.
~Arun George
^ permalink raw reply
* [PATCH] mm: swap_cgroup: fix NULL deref in lookup_swap_cgroup_id on swapless host
From: Jose Fernandez (Anthropic) @ 2026-05-04 12:55 UTC (permalink / raw)
To: Johannes Weiner, Michal Hocko, Roman Gushchin, Shakeel Butt,
Muchun Song, Andrew Morton, Hugh Dickins, David Hildenbrand,
Barry Song
Cc: cgroups, linux-mm, linux-kernel, Kairui Song, stable,
syzbot+e12bd9ca48157add237a, Jose Fernandez (Anthropic)
lookup_swap_cgroup_id() passes swap_cgroup_ctrl[type].map to
__swap_cgroup_id_lookup() without checking that the type was ever
registered via swap_cgroup_swapon(). On a swapless host every
ctrl->map is NULL, so __swap_cgroup_id_lookup() dereferences
NULL + a scaled swp_offset().
Since commit bea67dcc5eea ("mm: attempt to batch free swap entries
for zap_pte_range()"), zap_pte_range() -> swap_pte_batch() calls
lookup_swap_cgroup_id() on any non-present, non-none PTE that
decodes as a real swap entry, without first validating it against
swap_info[]. A single PTE corrupted into a type-0 swap entry takes
the host down at process exit.
We hit this in production on a swapless 6.12.58 host: ~1s of
"get_swap_device: Bad swap file entry 3f800204222bb" (do_swap_page()
being correctly defensive about the same entry) followed by
BUG: unable to handle page fault for address: 000003f800204220
RIP: 0010:lookup_swap_cgroup_id+0x2b/0x60
Call Trace:
swap_pte_batch+0xbf/0x230
zap_pte_range+0x4c8/0x780
unmap_page_range+0x190/0x3e0
exit_mmap+0xd9/0x3c0
do_exit+0x20c/0x4b0
syzbot has reported the identical stack.
The source of the PTE corruption is a separate bug; this change
makes the teardown path as robust as the fault path already is.
Every other caller of lookup_swap_cgroup_id() is downstream of a
get_swap_device() that has already validated the entry, so the new
branch is cold.
Fixes: bea67dcc5eea ("mm: attempt to batch free swap entries for zap_pte_range()")
Cc: stable@vger.kernel.org
Reported-by: syzbot+e12bd9ca48157add237a@syzkaller.appspotmail.com
Link: https://lore.kernel.org/r/69859728.050a0220.3b3015.0033.GAE@google.com
Assisted-by: Claude:unspecified
Signed-off-by: Jose Fernandez (Anthropic) <jose.fernandez@linux.dev>
---
mm/swap_cgroup.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/mm/swap_cgroup.c b/mm/swap_cgroup.c
index de779fed8c210..95c38e54dd587 100644
--- a/mm/swap_cgroup.c
+++ b/mm/swap_cgroup.c
@@ -124,6 +124,8 @@ unsigned short lookup_swap_cgroup_id(swp_entry_t ent)
return 0;
ctrl = &swap_cgroup_ctrl[swp_type(ent)];
+ if (unlikely(!ctrl->map))
+ return 0;
return __swap_cgroup_id_lookup(ctrl->map, swp_offset(ent));
}
---
base-commit: 254f49634ee16a731174d2ae34bc50bd5f45e731
change-id: 20260504-swap-cgroup-fix-7-0-ed0fcdb8f103
Best regards,
--
Jose Fernandez (Anthropic) <jose.fernandez@linux.dev>
^ permalink raw reply related
* [PATCH v3 4/4] selftests: cgroup: handle vmtest-dmem -b to test locally built kernel
From: Albert Esteve @ 2026-05-04 8:39 UTC (permalink / raw)
To: Tejun Heo, Johannes Weiner, Michal Koutný, Shuah Khan
Cc: linux-kernel, cgroups, linux-kselftest, Albert Esteve, mripard,
Eric Chanudet
In-Reply-To: <20260504-kunit_cgroups-v3-0-4eac90b76f91@redhat.com>
From: Eric Chanudet <echanude@redhat.com>
Currently vmtest-dmem.sh relies on the host's running kernel or a
pre-built one when booting the virtme-ng VM, with no option to
configure and build a local kernel tree directly.
This adds friction to the development cycle: the user must manually
run vng --kconfig with the correct config fragment, build the kernel,
and pass the result to the script.
Add a -b flag that automates this workflow. When set, handle_build()
configures the kernel using vng --kconfig with the selftest config
fragment, builds it with make -j$(nproc), and vm_start() passes the
local tree to vng --run so the VM boots the freshly built kernel.
Signed-off-by: Eric Chanudet <echanude@redhat.com>
Signed-off-by: Albert Esteve <aesteve@redhat.com>
---
tools/testing/selftests/cgroup/vmtest-dmem.sh | 35 ++++++++++++++++++++++++++-
1 file changed, 34 insertions(+), 1 deletion(-)
diff --git a/tools/testing/selftests/cgroup/vmtest-dmem.sh b/tools/testing/selftests/cgroup/vmtest-dmem.sh
index 9524dbddb06b7..b6e4777285c1b 100755
--- a/tools/testing/selftests/cgroup/vmtest-dmem.sh
+++ b/tools/testing/selftests/cgroup/vmtest-dmem.sh
@@ -23,6 +23,7 @@ readonly WAIT_TOTAL=$((WAIT_PERIOD * WAIT_PERIOD_MAX))
readonly QEMU_PIDFILE="$(mktemp /tmp/qemu_dmem_vmtest_XXXX.pid)"
readonly QEMU_OPTS=" --pidfile ${QEMU_PIDFILE} "
+BUILD=0
QEMU="qemu-system-$(uname -m)"
VERBOSE=0
SHELL_MODE=0
@@ -31,6 +32,7 @@ GUEST_TREE="${GUEST_TREE:-$KERNEL_CHECKOUT}"
usage() {
echo
echo "$0 [OPTIONS]"
+ echo " -b Build kernel from source tree before booting"
echo " -q <qemu> QEMU binary/path (default: ${QEMU})"
echo " -s Start interactive shell in VM"
echo " -v Verbose output (vng boot logs on stdout)"
@@ -72,17 +74,46 @@ check_deps() {
done
}
+handle_build() {
+ if [[ ! "${BUILD}" -eq 1 ]]; then
+ return
+ fi
+
+ if [[ ! -d "${KERNEL_CHECKOUT}" ]]; then
+ echo "-b requires vmtest-dmem.sh called from the kernel source tree" >&2
+ exit 1
+ fi
+
+ pushd "${KERNEL_CHECKOUT}" &>/dev/null
+
+ if ! vng --kconfig --config "${SCRIPT_DIR}"/config; then
+ die "failed to generate .config for kernel source tree (${KERNEL_CHECKOUT})"
+ fi
+
+ if ! make -j"$(nproc)"; then
+ die "failed to build kernel from source tree (${KERNEL_CHECKOUT})"
+ fi
+
+ popd &>/dev/null
+}
+
vm_start() {
local logfile=/dev/null
local verbose_opt=""
+ local kernel_opt=""
if [[ "${VERBOSE}" -eq 1 ]]; then
verbose_opt="--verbose"
logfile=/dev/stdout
fi
+ if [[ "${BUILD}" -eq 1 ]]; then
+ kernel_opt="${KERNEL_CHECKOUT}"
+ fi
+
vng \
--run \
+ ${kernel_opt} \
${verbose_opt} \
--qemu-opts="${QEMU_OPTS}" \
--qemu="$(command -v "${QEMU}")" \
@@ -165,10 +196,11 @@ run_test() {
vm_ssh -- "cd '${GUEST_TREE}' && ./tools/testing/selftests/cgroup/test_dmem"
}
-while getopts ":hvq:s" o; do
+while getopts ":hvq:sb" o; do
case "${o}" in
v) VERBOSE=1 ;;
q) QEMU="${OPTARG}" ;;
+ b) BUILD=1 ;;
s) SHELL_MODE=1 ;;
h|*) usage ;;
esac
@@ -177,6 +209,7 @@ done
trap cleanup EXIT
check_deps
+handle_build
echo "Booting virtme-ng VM..."
vm_start
vm_wait_for_ssh
--
2.53.0
^ permalink raw reply related
* [PATCH v3 3/4] selftests: cgroup: Add vmtest-dmem runner based on hid vmtest
From: Albert Esteve @ 2026-05-04 8:39 UTC (permalink / raw)
To: Tejun Heo, Johannes Weiner, Michal Koutný, Shuah Khan
Cc: linux-kernel, cgroups, linux-kselftest, Albert Esteve, mripard
In-Reply-To: <20260504-kunit_cgroups-v3-0-4eac90b76f91@redhat.com>
Currently, test_dmem relies on the dmem_selftest helper module
and a VM setup that may not have the helper preinstalled.
This makes automated coverage of dmem charge paths harder in
virtme-based runs.
Add tools/testing/selftests/cgroup/vmtest-dmem.sh, modeled
after the existing selftests vmtest runners
(notably tools/testing/selftests/hid/vmtest.sh),
to provide a repeatable VM workflow for dmem tests.
The script boots a virtme-ng guest, validates dmem
controller availability, ensures the dmem helper path is
present, and runs tools/testing/selftests/cgroup/test_dmem.
If the helper is not available as a loaded module, it
attempts module build/load for the running guest kernel
before executing the test binary.
The runner also supports interactive shell mode and
reuses the same verbosity and exit-code conventions
used by other vmtest scripts, so it integrates with existing
kselftest workflows.
Signed-off-by: Albert Esteve <aesteve@redhat.com>
---
tools/testing/selftests/cgroup/vmtest-dmem.sh | 197 ++++++++++++++++++++++++++
1 file changed, 197 insertions(+)
diff --git a/tools/testing/selftests/cgroup/vmtest-dmem.sh b/tools/testing/selftests/cgroup/vmtest-dmem.sh
new file mode 100755
index 0000000000000..9524dbddb06b7
--- /dev/null
+++ b/tools/testing/selftests/cgroup/vmtest-dmem.sh
@@ -0,0 +1,197 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (c) 2026 Red Hat, Inc.
+#
+# Run cgroup test_dmem inside a virtme-ng VM.
+# Dependencies:
+# * virtme-ng
+# * busybox-static (used by virtme-ng)
+# * qemu (used by virtme-ng)
+
+set -euo pipefail
+
+readonly SCRIPT_DIR="$(cd -P -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
+readonly KERNEL_CHECKOUT="$(realpath "${SCRIPT_DIR}"/../../../../)"
+
+source "${SCRIPT_DIR}"/../kselftest/ktap_helpers.sh
+
+readonly SSH_GUEST_PORT="${SSH_GUEST_PORT:-22}"
+readonly WAIT_PERIOD=3
+readonly WAIT_PERIOD_MAX=80
+readonly WAIT_TOTAL=$((WAIT_PERIOD * WAIT_PERIOD_MAX))
+readonly QEMU_PIDFILE="$(mktemp /tmp/qemu_dmem_vmtest_XXXX.pid)"
+readonly QEMU_OPTS=" --pidfile ${QEMU_PIDFILE} "
+
+QEMU="qemu-system-$(uname -m)"
+VERBOSE=0
+SHELL_MODE=0
+GUEST_TREE="${GUEST_TREE:-$KERNEL_CHECKOUT}"
+
+usage() {
+ echo
+ echo "$0 [OPTIONS]"
+ echo " -q <qemu> QEMU binary/path (default: ${QEMU})"
+ echo " -s Start interactive shell in VM"
+ echo " -v Verbose output (vng boot logs on stdout)"
+ echo
+
+ exit 1
+}
+
+die() {
+ echo "$*" >&2
+ exit "${KSFT_FAIL}"
+}
+
+cleanup() {
+ if [[ -s "${QEMU_PIDFILE}" ]]; then
+ pkill -SIGTERM -F "${QEMU_PIDFILE}" >/dev/null 2>&1 || true
+ fi
+
+ # If failure occurred during or before qemu start up, then we need
+ # to clean this up ourselves.
+ if [[ -e "${QEMU_PIDFILE}" ]]; then
+ rm -f "${QEMU_PIDFILE}"
+ fi
+}
+
+vm_ssh() {
+ stdbuf -oL ssh -q \
+ -F "${HOME}/.cache/virtme-ng/.ssh/virtme-ng-ssh.conf" \
+ -l root "virtme-ng%${SSH_GUEST_PORT}" \
+ "$@"
+}
+
+check_deps() {
+ for dep in vng "${QEMU}" busybox pkill ssh; do
+ if ! command -v "${dep}" >/dev/null 2>&1; then
+ echo "skip: dependency ${dep} not found"
+ exit "${KSFT_SKIP}"
+ fi
+ done
+}
+
+vm_start() {
+ local logfile=/dev/null
+ local verbose_opt=""
+
+ if [[ "${VERBOSE}" -eq 1 ]]; then
+ verbose_opt="--verbose"
+ logfile=/dev/stdout
+ fi
+
+ vng \
+ --run \
+ ${verbose_opt} \
+ --qemu-opts="${QEMU_OPTS}" \
+ --qemu="$(command -v "${QEMU}")" \
+ --user root \
+ --ssh "${SSH_GUEST_PORT}" \
+ --rw &>"${logfile}" &
+
+ local vng_pid=$!
+ local elapsed=0
+
+ while [[ ! -s "${QEMU_PIDFILE}" ]]; do
+ kill -0 "${vng_pid}" 2>/dev/null || die "vng exited early; failed to boot VM"
+ [[ "${elapsed}" -ge "${WAIT_TOTAL}" ]] && die "timed out waiting for VM boot"
+ sleep 1
+ elapsed=$((elapsed + 1))
+ done
+}
+
+vm_wait_for_ssh() {
+ local i=0
+ while true; do
+ vm_ssh -- true && break
+ i=$((i + 1))
+ [[ "${i}" -gt "${WAIT_PERIOD_MAX}" ]] && die "timed out waiting for guest ssh"
+ sleep "${WAIT_PERIOD}"
+ done
+}
+
+check_guest_requirements() {
+ local cfg_ok
+ cfg_ok="$(vm_ssh -- " \
+ grep -q dmem /sys/fs/cgroup/cgroup.controllers && \
+ grep -q memory /sys/fs/cgroup/cgroup.controllers;
+ echo \$?
+ ")"
+ [[ "${cfg_ok}" == "0" ]] || die "guest kernel missing CONFIG_CGROUP_DMEM"
+}
+
+setup_guest_dmem_helper() {
+ local kdir
+
+ vm_ssh -- "mountpoint -q /sys/kernel/debug || \
+ mount -t debugfs none /sys/kernel/debug" || true
+
+ # Already available (built-in or loaded).
+ if vm_ssh -- "[[ -e /sys/kernel/debug/dmem_selftest/charge ]]"; then
+ echo "dmem_selftest ready"
+ return 0
+ fi
+
+ # Fast path: try installed module.
+ vm_ssh -- "modprobe -q dmem_selftest 2>/dev/null || true"
+ if vm_ssh -- "[[ -e /sys/kernel/debug/dmem_selftest/charge ]]"; then
+ echo "dmem_selftest ready"
+ return 0
+ fi
+
+ # Fallback: build only this module against running guest kernel,
+ # then insert it.
+ kdir="$(vm_ssh -- "echo /lib/modules/\$(uname -r)/build")"
+ if vm_ssh -- "[[ -d '${kdir}' ]]"; then
+ echo "Building dmem_selftest.ko against running guest kernel..."
+ vm_ssh -- "make -C '${kdir}' \
+ M='${GUEST_TREE}/kernel/cgroup' \
+ CONFIG_DMEM_SELFTEST=m modules"
+ vm_ssh -- "insmod '${GUEST_TREE}/kernel/cgroup/dmem_selftest.ko' \
+ 2>/dev/null || modprobe -q dmem_selftest 2>/dev/null || true"
+ fi
+
+ if vm_ssh -- "[[ -e /sys/kernel/debug/dmem_selftest/charge ]]"; then
+ echo "dmem_selftest ready"
+ return 0
+ fi
+
+ die "dmem_selftest unavailable (modprobe/build+insmod failed)"
+}
+
+run_test() {
+ vm_ssh -- "cd '${GUEST_TREE}' && make -C tools/testing/selftests TARGETS=cgroup"
+ vm_ssh -- "cd '${GUEST_TREE}' && ./tools/testing/selftests/cgroup/test_dmem"
+}
+
+while getopts ":hvq:s" o; do
+ case "${o}" in
+ v) VERBOSE=1 ;;
+ q) QEMU="${OPTARG}" ;;
+ s) SHELL_MODE=1 ;;
+ h|*) usage ;;
+ esac
+done
+
+trap cleanup EXIT
+
+check_deps
+echo "Booting virtme-ng VM..."
+vm_start
+vm_wait_for_ssh
+echo "VM is reachable via SSH."
+
+if [[ "${SHELL_MODE}" -eq 1 ]]; then
+ echo "Starting interactive shell in VM. Exit to stop VM."
+ vm_ssh -t -- "cd '${GUEST_TREE}' && exec bash --noprofile --norc"
+ exit "${KSFT_PASS}"
+fi
+
+check_guest_requirements
+setup_guest_dmem_helper
+
+echo "Running cgroup/test_dmem in VM..."
+run_test
+echo "PASS: test_dmem completed"
+exit "${KSFT_PASS}"
--
2.53.0
^ permalink raw reply related
* [PATCH v3 2/4] selftests: cgroup: Add dmem selftest coverage
From: Albert Esteve @ 2026-05-04 8:39 UTC (permalink / raw)
To: Tejun Heo, Johannes Weiner, Michal Koutný, Shuah Khan
Cc: linux-kernel, cgroups, linux-kselftest, Albert Esteve, mripard
In-Reply-To: <20260504-kunit_cgroups-v3-0-4eac90b76f91@redhat.com>
Currently, tools/testing/selftests/cgroup/ does not include
a dmem-specific test binary. This leaves dmem charge and
limit behavior largely unvalidated in kselftest coverage.
Add test_dmem and wire it into the cgroup selftests Makefile.
The new test exercises dmem controller behavior through the
dmem_selftest debugfs interface for the dmem_selftest region.
The test adds three complementary checks:
- test_dmem_max creates a nested hierarchy with per-leaf
dmem.max values and verifies that over-limit charges
fail while in-limit charges succeed with bounded rounding
in dmem.current.
- test_dmem_min and test_dmem_low verify that charging
from a cgroup with the corresponding protection knob
set updates dmem.current as expected.
- test_dmem_charge_byte_granularity validates accounting
bounds for non-page-aligned charge sizes and
uncharge-to-zero behavior.
This provides deterministic userspace coverage for dmem
accounting and hard-limit enforcement using a test helper
module, without requiring subsystem-specific production
drivers.
Signed-off-by: Albert Esteve <aesteve@redhat.com>
---
tools/testing/selftests/cgroup/.gitignore | 1 +
tools/testing/selftests/cgroup/Makefile | 2 +
tools/testing/selftests/cgroup/config | 2 +
tools/testing/selftests/cgroup/test_dmem.c | 492 +++++++++++++++++++++++++++++
4 files changed, 497 insertions(+)
diff --git a/tools/testing/selftests/cgroup/.gitignore b/tools/testing/selftests/cgroup/.gitignore
index 952e4448bf070..ea2322598217d 100644
--- a/tools/testing/selftests/cgroup/.gitignore
+++ b/tools/testing/selftests/cgroup/.gitignore
@@ -2,6 +2,7 @@
test_core
test_cpu
test_cpuset
+test_dmem
test_freezer
test_hugetlb_memcg
test_kill
diff --git a/tools/testing/selftests/cgroup/Makefile b/tools/testing/selftests/cgroup/Makefile
index e01584c2189ac..e1a5e9316620e 100644
--- a/tools/testing/selftests/cgroup/Makefile
+++ b/tools/testing/selftests/cgroup/Makefile
@@ -10,6 +10,7 @@ TEST_GEN_FILES := wait_inotify
TEST_GEN_PROGS = test_core
TEST_GEN_PROGS += test_cpu
TEST_GEN_PROGS += test_cpuset
+TEST_GEN_PROGS += test_dmem
TEST_GEN_PROGS += test_freezer
TEST_GEN_PROGS += test_hugetlb_memcg
TEST_GEN_PROGS += test_kill
@@ -26,6 +27,7 @@ include lib/libcgroup.mk
$(OUTPUT)/test_core: $(LIBCGROUP_O)
$(OUTPUT)/test_cpu: $(LIBCGROUP_O)
$(OUTPUT)/test_cpuset: $(LIBCGROUP_O)
+$(OUTPUT)/test_dmem: $(LIBCGROUP_O)
$(OUTPUT)/test_freezer: $(LIBCGROUP_O)
$(OUTPUT)/test_hugetlb_memcg: $(LIBCGROUP_O)
$(OUTPUT)/test_kill: $(LIBCGROUP_O)
diff --git a/tools/testing/selftests/cgroup/config b/tools/testing/selftests/cgroup/config
index 39f979690dd3b..5728278310a31 100644
--- a/tools/testing/selftests/cgroup/config
+++ b/tools/testing/selftests/cgroup/config
@@ -1,6 +1,8 @@
CONFIG_CGROUPS=y
CONFIG_CGROUP_CPUACCT=y
+CONFIG_CGROUP_DMEM=y
CONFIG_CGROUP_FREEZER=y
CONFIG_CGROUP_SCHED=y
+CONFIG_DMEM_SELFTEST=m
CONFIG_MEMCG=y
CONFIG_PAGE_COUNTER=y
diff --git a/tools/testing/selftests/cgroup/test_dmem.c b/tools/testing/selftests/cgroup/test_dmem.c
new file mode 100644
index 0000000000000..0a1b9561a1b9e
--- /dev/null
+++ b/tools/testing/selftests/cgroup/test_dmem.c
@@ -0,0 +1,492 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Test the dmem (device memory) cgroup controller.
+ *
+ * Depends on dmem_selftest kernel module.
+ */
+
+#define _GNU_SOURCE
+
+#include <linux/limits.h>
+#include <errno.h>
+#include <fcntl.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <unistd.h>
+
+#include "kselftest.h"
+#include "cgroup_util.h"
+
+/* kernel/cgroup/dmem_selftest.c */
+#define DM_SELFTEST_REGION "dmem_selftest"
+#define DM_SELFTEST_CHARGE "/sys/kernel/debug/dmem_selftest/charge"
+#define DM_SELFTEST_UNCHARGE "/sys/kernel/debug/dmem_selftest/uncharge"
+
+/*
+ * Parse the first line of dmem.capacity (root):
+ * "<name> <size_in_bytes>"
+ * Returns 1 if a region was found, 0 if capacity is empty, -1 on read error.
+ */
+static int parse_first_region(const char *root, char *name, size_t name_len,
+ unsigned long long *size_out)
+{
+ char buf[4096];
+ char nm[256];
+ unsigned long long sz;
+
+ if (cg_read(root, "dmem.capacity", buf, sizeof(buf)) < 0)
+ return -1;
+
+ if (sscanf(buf, "%255s %llu", nm, &sz) < 2)
+ return 0;
+
+ if (name_len <= strlen(nm))
+ return -1;
+
+ strcpy(name, nm);
+ *size_out = sz;
+ return 1;
+}
+
+/*
+ * Read the numeric limit for @region_name from a multiline
+ * dmem.{min,low,max} file. Returns bytes,
+ * or -1 if the line is "<name> max", or -2 if missing/err.
+ */
+static long long dmem_read_limit_for_region(const char *cgroup, const char *ctrl,
+ const char *region_name)
+{
+ char buf[4096];
+ char *line, *saveptr = NULL;
+ char fname[256];
+ char fval[64];
+
+ if (cg_read(cgroup, ctrl, buf, sizeof(buf)) < 0)
+ return -2;
+
+ for (line = strtok_r(buf, "\n", &saveptr); line;
+ line = strtok_r(NULL, "\n", &saveptr)) {
+ if (!line[0])
+ continue;
+ if (sscanf(line, "%255s %63s", fname, fval) != 2)
+ continue;
+ if (strcmp(fname, region_name))
+ continue;
+ if (!strcmp(fval, "max"))
+ return -1;
+ return strtoll(fval, NULL, 0);
+ }
+ return -2;
+}
+
+static long long dmem_read_limit(const char *cgroup, const char *ctrl)
+{
+ return dmem_read_limit_for_region(cgroup, ctrl, DM_SELFTEST_REGION);
+}
+
+static int dmem_write_limit(const char *cgroup, const char *ctrl,
+ const char *val)
+{
+ char wr[512];
+
+ snprintf(wr, sizeof(wr), "%s %s", DM_SELFTEST_REGION, val);
+ return cg_write(cgroup, ctrl, wr);
+}
+
+static int dmem_selftest_charge_bytes(unsigned long long bytes)
+{
+ char wr[32];
+
+ snprintf(wr, sizeof(wr), "%llu", bytes);
+ return write_text(DM_SELFTEST_CHARGE, wr, strlen(wr));
+}
+
+static int dmem_selftest_uncharge(void)
+{
+ return write_text(DM_SELFTEST_UNCHARGE, "\n", 1);
+}
+
+/*
+ * First, this test creates the following hierarchy:
+ * A
+ * A/B dmem.max=1M
+ * A/B/C dmem.max=75K
+ * A/B/D dmem.max=25K
+ * A/B/E dmem.max=8K
+ * A/B/F dmem.max=0
+ *
+ * Then for each leaf cgroup it tries to charge above dmem.max
+ * and expects the charge request to fail and dmem.current to
+ * remain unchanged.
+ *
+ * For leaves with non-zero dmem.max, it additionally charges a
+ * smaller amount and verifies accounting grows within one PAGE_SIZE
+ * rounding bound, then uncharges and verifies dmem.current returns
+ * to the previous value.
+ *
+ */
+static int test_dmem_max(const char *root)
+{
+ static const char * const leaf_max[] = { "75K", "25K", "8K", "0" };
+ static const unsigned long long fail_sz[] = {
+ (75ULL * 1024ULL) + 1ULL,
+ (25ULL * 1024ULL) + 1ULL,
+ (8ULL * 1024ULL) + 1ULL,
+ 1ULL
+ };
+ static const unsigned long long pass_sz[] = {
+ 4096ULL, 4096ULL, 4096ULL, 0ULL
+ };
+ char *parent[2] = {NULL};
+ char *children[4] = {NULL};
+ unsigned long long cap;
+ char region[256];
+ long long page_size;
+ long long cur_before, cur_after;
+ int ret = KSFT_FAIL;
+ int charged = 0;
+ int in_child = 0;
+ long long v;
+ int i;
+
+ if (access(DM_SELFTEST_CHARGE, W_OK) != 0)
+ return KSFT_SKIP;
+
+ if (parse_first_region(root, region, sizeof(region), &cap) != 1)
+ return KSFT_SKIP;
+ if (strcmp(region, DM_SELFTEST_REGION) != 0)
+ return KSFT_SKIP;
+
+ page_size = sysconf(_SC_PAGESIZE);
+ if (page_size <= 0)
+ goto cleanup;
+
+ parent[0] = cg_name(root, "dmem_prot_0");
+ if (!parent[0])
+ goto cleanup;
+
+ parent[1] = cg_name(parent[0], "dmem_prot_1");
+ if (!parent[1])
+ goto cleanup;
+
+ if (cg_create(parent[0]))
+ goto cleanup;
+
+ if (cg_write(parent[0], "cgroup.subtree_control", "+dmem"))
+ goto cleanup;
+
+ if (cg_create(parent[1]))
+ goto cleanup;
+
+ if (cg_write(parent[1], "cgroup.subtree_control", "+dmem"))
+ goto cleanup;
+
+ for (i = 0; i < 4; i++) {
+ children[i] = cg_name_indexed(parent[1], "dmem_child", i);
+ if (!children[i])
+ goto cleanup;
+ if (cg_create(children[i]))
+ goto cleanup;
+ }
+
+ if (dmem_write_limit(parent[1], "dmem.max", "1M"))
+ goto cleanup;
+ for (i = 0; i < 4; i++)
+ if (dmem_write_limit(children[i], "dmem.max", leaf_max[i]))
+ goto cleanup;
+
+ v = dmem_read_limit(parent[1], "dmem.max");
+ if (!values_close(v, 1024LL * 1024LL, 3))
+ goto cleanup;
+ v = dmem_read_limit(children[0], "dmem.max");
+ if (!values_close(v, 75LL * 1024LL, 3))
+ goto cleanup;
+ v = dmem_read_limit(children[1], "dmem.max");
+ if (!values_close(v, 25LL * 1024LL, 3))
+ goto cleanup;
+ v = dmem_read_limit(children[2], "dmem.max");
+ if (!values_close(v, 8LL * 1024LL, 3))
+ goto cleanup;
+ v = dmem_read_limit(children[3], "dmem.max");
+ if (v != 0)
+ goto cleanup;
+
+ for (i = 0; i < 4; i++) {
+ if (cg_enter_current(children[i]))
+ goto cleanup;
+ in_child = 1;
+
+ cur_before = dmem_read_limit(children[i], "dmem.current");
+ if (cur_before < 0)
+ goto cleanup;
+
+ if (dmem_selftest_charge_bytes(fail_sz[i]) >= 0) {
+ charged = 1;
+ goto cleanup;
+ }
+
+ cur_after = dmem_read_limit(children[i], "dmem.current");
+ if (cur_after != cur_before)
+ goto cleanup;
+
+ if (pass_sz[i] > 0) {
+ if (dmem_selftest_charge_bytes(pass_sz[i]) < 0)
+ goto cleanup;
+ charged = 1;
+
+ cur_after = dmem_read_limit(children[i], "dmem.current");
+ if (cur_after < cur_before + (long long)pass_sz[i])
+ goto cleanup;
+ if (cur_after > cur_before + (long long)pass_sz[i] + page_size)
+ goto cleanup;
+
+ if (dmem_selftest_uncharge() < 0)
+ goto cleanup;
+ charged = 0;
+
+ cur_after = dmem_read_limit(children[i], "dmem.current");
+ if (cur_after != cur_before)
+ goto cleanup;
+ }
+
+ if (cg_enter_current(root))
+ goto cleanup;
+ in_child = 0;
+ }
+
+ ret = KSFT_PASS;
+
+cleanup:
+ if (charged)
+ dmem_selftest_uncharge();
+ if (in_child)
+ cg_enter_current(root);
+ for (i = 3; i >= 0; i--) {
+ if (!children[i])
+ continue;
+ cg_destroy(children[i]);
+ free(children[i]);
+ }
+ for (i = 1; i >= 0; i--) {
+ if (!parent[i])
+ continue;
+ cg_destroy(parent[i]);
+ free(parent[i]);
+ }
+ return ret;
+}
+
+/*
+ * This test sets dmem.min and dmem.low on a child cgroup, then charge
+ * from that context and verify dmem.current tracks the charged bytes
+ * (within one page rounding).
+ */
+static int test_dmem_charge_with_attr(const char *root, bool min)
+{
+ char region[256];
+ unsigned long long cap;
+ const unsigned long long charge_sz = 12345ULL;
+ const char *attribute = min ? "dmem.min" : "dmem.low";
+ int ret = KSFT_FAIL;
+ char *cg = NULL;
+ long long cur;
+ long long page_size;
+ int charged = 0;
+ int in_child = 0;
+
+ if (access(DM_SELFTEST_CHARGE, W_OK) != 0)
+ return KSFT_SKIP;
+
+ if (parse_first_region(root, region, sizeof(region), &cap) != 1)
+ return KSFT_SKIP;
+ if (strcmp(region, DM_SELFTEST_REGION) != 0)
+ return KSFT_SKIP;
+
+ page_size = sysconf(_SC_PAGESIZE);
+ if (page_size <= 0)
+ goto cleanup;
+
+ cg = cg_name(root, "test_dmem_attr");
+ if (!cg)
+ goto cleanup;
+
+ if (cg_create(cg))
+ goto cleanup;
+
+ if (cg_enter_current(cg))
+ goto cleanup;
+ in_child = 1;
+
+ if (dmem_write_limit(cg, attribute, "16K"))
+ goto cleanup;
+
+ if (dmem_selftest_charge_bytes(charge_sz) < 0)
+ goto cleanup;
+ charged = 1;
+
+ cur = dmem_read_limit(cg, "dmem.current");
+ if (cur < (long long)charge_sz)
+ goto cleanup;
+ if (cur > (long long)charge_sz + page_size)
+ goto cleanup;
+
+ if (dmem_selftest_uncharge() < 0)
+ goto cleanup;
+ charged = 0;
+
+ cur = dmem_read_limit(cg, "dmem.current");
+ if (cur != 0)
+ goto cleanup;
+
+ ret = KSFT_PASS;
+
+cleanup:
+ if (charged)
+ dmem_selftest_uncharge();
+ if (in_child)
+ cg_enter_current(root);
+ cg_destroy(cg);
+ free(cg);
+ return ret;
+}
+
+static int test_dmem_min(const char *root)
+{
+ return test_dmem_charge_with_attr(root, true);
+}
+
+static int test_dmem_low(const char *root)
+{
+ return test_dmem_charge_with_attr(root, false);
+}
+
+/*
+ * This test charges non-page-aligned byte sizes and verify dmem.current
+ * stays consistent: it must account at least the requested bytes and
+ * never exceed one kernel page of rounding overhead. Then uncharge must
+ * return usage to 0.
+ */
+static int test_dmem_charge_byte_granularity(const char *root)
+{
+ static const unsigned long long sizes[] = { 1ULL, 4095ULL, 4097ULL, 12345ULL };
+ char *cg = NULL;
+ unsigned long long cap;
+ char region[256];
+ long long cur;
+ long long page_size;
+ int ret = KSFT_FAIL;
+ int charged = 0;
+ int in_child = 0;
+ size_t i;
+
+ if (access(DM_SELFTEST_CHARGE, W_OK) != 0)
+ return KSFT_SKIP;
+
+ if (parse_first_region(root, region, sizeof(region), &cap) != 1)
+ return KSFT_SKIP;
+ if (strcmp(region, DM_SELFTEST_REGION) != 0)
+ return KSFT_SKIP;
+
+ page_size = sysconf(_SC_PAGESIZE);
+ if (page_size <= 0)
+ goto cleanup;
+
+ cg = cg_name(root, "dmem_dbg_byte_gran");
+ if (!cg)
+ goto cleanup;
+
+ if (cg_create(cg))
+ goto cleanup;
+
+ if (dmem_write_limit(cg, "dmem.max", "8M"))
+ goto cleanup;
+
+ if (cg_enter_current(cg))
+ goto cleanup;
+ in_child = 1;
+
+ for (i = 0; i < ARRAY_SIZE(sizes); i++) {
+ if (dmem_selftest_charge_bytes(sizes[i]) < 0)
+ goto cleanup;
+ charged = 1;
+
+ cur = dmem_read_limit(cg, "dmem.current");
+ if (cur < (long long)sizes[i])
+ goto cleanup;
+ if (cur > (long long)sizes[i] + page_size)
+ goto cleanup;
+
+ if (dmem_selftest_uncharge() < 0)
+ goto cleanup;
+ charged = 0;
+
+ cur = dmem_read_limit(cg, "dmem.current");
+ if (cur != 0)
+ goto cleanup;
+ }
+
+ ret = KSFT_PASS;
+
+cleanup:
+ if (charged)
+ dmem_selftest_uncharge();
+ if (in_child)
+ cg_enter_current(root);
+ if (cg) {
+ cg_destroy(cg);
+ free(cg);
+ }
+ return ret;
+}
+
+#define T(x) { x, #x }
+struct dmem_test {
+ int (*fn)(const char *root);
+ const char *name;
+} tests[] = {
+ T(test_dmem_max),
+ T(test_dmem_min),
+ T(test_dmem_low),
+ T(test_dmem_charge_byte_granularity),
+};
+#undef T
+
+int main(int argc, char **argv)
+{
+ char root[PATH_MAX];
+ int i;
+
+ ksft_print_header();
+ ksft_set_plan(ARRAY_SIZE(tests));
+
+ if (cg_find_unified_root(root, sizeof(root), NULL))
+ ksft_exit_skip("cgroup v2 isn't mounted\n");
+
+ if (cg_read_strstr(root, "cgroup.controllers", "dmem"))
+ ksft_exit_skip("dmem controller isn't available (CONFIG_CGROUP_DMEM?)\n");
+
+ if (cg_read_strstr(root, "cgroup.subtree_control", "dmem"))
+ if (cg_write(root, "cgroup.subtree_control", "+dmem"))
+ ksft_exit_skip("Failed to enable dmem controller\n");
+
+ for (i = 0; i < ARRAY_SIZE(tests); i++) {
+ switch (tests[i].fn(root)) {
+ case KSFT_PASS:
+ ksft_test_result_pass("%s\n", tests[i].name);
+ break;
+ case KSFT_SKIP:
+ ksft_test_result_skip(
+ "%s (need CONFIG_DMEM_SELFTEST, modprobe dmem_selftest)\n",
+ tests[i].name);
+ break;
+ default:
+ ksft_test_result_fail("%s\n", tests[i].name);
+ break;
+ }
+ }
+
+ ksft_finished();
+}
--
2.53.0
^ permalink raw reply related
* [PATCH v3 1/4] cgroup: Add dmem_selftest module
From: Albert Esteve @ 2026-05-04 8:39 UTC (permalink / raw)
To: Tejun Heo, Johannes Weiner, Michal Koutný, Shuah Khan
Cc: linux-kernel, cgroups, linux-kselftest, Albert Esteve, mripard
In-Reply-To: <20260504-kunit_cgroups-v3-0-4eac90b76f91@redhat.com>
Currently, dmem charging is driver-driven through direct
calls to dmem_cgroup_try_charge(), so cgroup selftests
do not have a generic way to trigger charge and uncharge
paths from userspace.
This limits any selftest coverage to configuration/readout
checks unless a specific driver exposing charge hooks is
present in the test environment.
Add kernel/cgroup/dmem_selftest.c as a helper module
(CONFIG_DMEM_SELFTEST) that registers a synthetic dmem region
(dmem_selftest) and exposes debugfs control files:
/sys/kernel/debug/dmem_selftest/charge
/sys/kernel/debug/dmem_selftest/uncharge
Writing a size to charge triggers dmem_cgroup_try_charge() for
the calling task's cgroup (the module calls kstrtou64()).
Writing to uncharge releases the outstanding charge via
dmem_cgroup_uncharge(). Only a single outstanding charge
is supported.
This provides a deterministic, driver-independent mechanism
for exercising dmem accounting paths in selftests.
Signed-off-by: Albert Esteve <aesteve@redhat.com>
---
init/Kconfig | 12 +++
kernel/cgroup/Makefile | 1 +
kernel/cgroup/dmem_selftest.c | 199 ++++++++++++++++++++++++++++++++++++++++++
3 files changed, 212 insertions(+)
diff --git a/init/Kconfig b/init/Kconfig
index 444ce811ea674..060ba8ca49333 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -1238,6 +1238,18 @@ config CGROUP_DMEM
As an example, it allows you to restrict VRAM usage for applications
in the DRM subsystem.
+config DMEM_SELFTEST
+ tristate "dmem cgroup selftest helper module"
+ depends on CGROUP_DMEM && DEBUG_FS
+ default n
+ help
+ Builds a small loadable module that registers a dmem region named
+ "dmem_selftest" and exposes debugfs files under
+ /sys/kernel/debug/dmem_selftest/ so kselftests can trigger
+ dmem charge/uncharge operations from userspace.
+
+ Say N unless you run dmem selftests or develop the dmem controller.
+
config CGROUP_FREEZER
bool "Freezer controller"
help
diff --git a/kernel/cgroup/Makefile b/kernel/cgroup/Makefile
index ede31601a363a..febc36e60f9f9 100644
--- a/kernel/cgroup/Makefile
+++ b/kernel/cgroup/Makefile
@@ -8,4 +8,5 @@ obj-$(CONFIG_CPUSETS) += cpuset.o
obj-$(CONFIG_CPUSETS_V1) += cpuset-v1.o
obj-$(CONFIG_CGROUP_MISC) += misc.o
obj-$(CONFIG_CGROUP_DMEM) += dmem.o
+obj-$(CONFIG_DMEM_SELFTEST) += dmem_selftest.o
obj-$(CONFIG_CGROUP_DEBUG) += debug.o
diff --git a/kernel/cgroup/dmem_selftest.c b/kernel/cgroup/dmem_selftest.c
new file mode 100644
index 0000000000000..15520807e2dc8
--- /dev/null
+++ b/kernel/cgroup/dmem_selftest.c
@@ -0,0 +1,199 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Kselftest helper for the dmem cgroup controller.
+ *
+ * Registers a dmem region and debugfs files so tests can trigger charges
+ * from the calling task's cgroup.
+ *
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/cgroup_dmem.h>
+#include <linux/debugfs.h>
+#include <linux/fs.h>
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/mutex.h>
+#include <linux/module.h>
+#include <linux/string.h>
+#include <linux/uaccess.h>
+
+#include "../../tools/testing/selftests/kselftest_module.h"
+
+#define DM_SELFTEST_REGION_NAME "dmem_selftest"
+#define DM_SELFTEST_REGION_SIZE (256ULL * 1024 * 1024)
+
+KSTM_MODULE_GLOBALS();
+
+static struct dmem_cgroup_region *selftest_region;
+static struct dentry *dbg_dir;
+
+static struct dmem_cgroup_pool_state *charged_pool;
+static u64 charged_size;
+static DEFINE_MUTEX(charge_lock);
+
+static ssize_t dmem_selftest_charge_write(struct file *file, const char __user *user_buf,
+ size_t count, loff_t *ppos)
+{
+ struct dmem_cgroup_pool_state *pool = NULL, *limit = NULL;
+ u64 size;
+ char buf[32];
+ int ret;
+
+ if (!selftest_region)
+ return -ENODEV;
+
+ if (count == 0 || count >= sizeof(buf))
+ return -EINVAL;
+
+ if (copy_from_user(buf, user_buf, count))
+ return -EFAULT;
+ buf[count] = '\0';
+
+ ret = kstrtou64(strim(buf), 0, &size);
+ if (ret)
+ return ret;
+ if (!size)
+ return -EINVAL;
+
+ mutex_lock(&charge_lock);
+ if (charged_pool) {
+ mutex_unlock(&charge_lock);
+ return -EBUSY;
+ }
+
+ ret = dmem_cgroup_try_charge(selftest_region, size, &pool, &limit);
+ if (ret == -EAGAIN && limit)
+ dmem_cgroup_pool_state_put(limit);
+ if (ret) {
+ mutex_unlock(&charge_lock);
+ return ret;
+ }
+
+ charged_pool = pool;
+ charged_size = size;
+ mutex_unlock(&charge_lock);
+
+ return count;
+}
+
+static ssize_t dmem_selftest_uncharge_write(struct file *file, const char __user *user_buf,
+ size_t count, loff_t *ppos)
+{
+ if (!count)
+ return -EINVAL;
+
+ mutex_lock(&charge_lock);
+ if (!charged_pool) {
+ mutex_unlock(&charge_lock);
+ return -EINVAL;
+ }
+
+ dmem_cgroup_uncharge(charged_pool, charged_size);
+ charged_pool = NULL;
+ charged_size = 0;
+ mutex_unlock(&charge_lock);
+
+ return count;
+}
+
+static const struct file_operations dmem_selftest_charge_fops = {
+ .write = dmem_selftest_charge_write,
+ .llseek = noop_llseek,
+};
+
+static const struct file_operations dmem_selftest_uncharge_fops = {
+ .write = dmem_selftest_uncharge_write,
+ .llseek = noop_llseek,
+};
+
+static int __init dmem_selftest_register(void)
+{
+ int ret = 0;
+
+ selftest_region = dmem_cgroup_register_region(
+ DM_SELFTEST_REGION_SIZE, DM_SELFTEST_REGION_NAME);
+ if (IS_ERR(selftest_region))
+ return PTR_ERR(selftest_region);
+ if (!selftest_region)
+ return -EINVAL;
+
+ dbg_dir = debugfs_create_dir("dmem_selftest", NULL);
+ if (IS_ERR(dbg_dir)) {
+ ret = PTR_ERR(dbg_dir);
+ goto dbgfs_error;
+ }
+
+ debugfs_create_file("charge", 0200, dbg_dir, NULL, &dmem_selftest_charge_fops);
+ debugfs_create_file("uncharge", 0200, dbg_dir, NULL, &dmem_selftest_uncharge_fops);
+
+ pr_info("region '%s' registered; debugfs at dmem_selftest/{charge,uncharge}\n",
+ DM_SELFTEST_REGION_NAME);
+ return ret;
+
+dbgfs_error:
+ dmem_cgroup_unregister_region(selftest_region);
+ dbg_dir = NULL;
+ selftest_region = NULL;
+ return ret;
+}
+
+static void dmem_selftest_remove(void)
+{
+ debugfs_remove_recursive(dbg_dir);
+ dbg_dir = NULL;
+
+ if (selftest_region) {
+ dmem_cgroup_unregister_region(selftest_region);
+ selftest_region = NULL;
+ }
+}
+
+static void __init selftest(void)
+{
+ KSTM_CHECK_ZERO(!selftest_region);
+ KSTM_CHECK_ZERO(IS_ERR_OR_NULL(dbg_dir));
+}
+
+static int __init dmem_selftest_init(void)
+{
+ int report_rc;
+ int err;
+
+ err = dmem_selftest_register();
+ if (err)
+ return err;
+
+ pr_info("loaded.\n");
+ add_taint(TAINT_TEST, LOCKDEP_STILL_OK);
+ selftest();
+ report_rc = kstm_report(total_tests, failed_tests, skipped_tests);
+ if (report_rc) {
+ dmem_selftest_remove();
+ return report_rc;
+ }
+
+ return 0;
+}
+
+static void __exit dmem_selftest_exit(void)
+{
+ pr_info("unloaded.\n");
+
+ dmem_selftest_remove();
+
+ mutex_lock(&charge_lock);
+ if (charged_pool) {
+ dmem_cgroup_uncharge(charged_pool, charged_size);
+ charged_pool = NULL;
+ }
+ mutex_unlock(&charge_lock);
+}
+
+module_init(dmem_selftest_init);
+module_exit(dmem_selftest_exit);
+
+MODULE_AUTHOR("Albert Esteve <aesteve@redhat.com>");
+MODULE_DESCRIPTION("Kselftest helper for cgroup dmem controller");
+MODULE_LICENSE("GPL");
--
2.53.0
^ permalink raw reply related
* [PATCH v3 0/4] cgroup: dmem: add selftest helper, coverage, and VM runner
From: Albert Esteve @ 2026-05-04 8:39 UTC (permalink / raw)
To: Tejun Heo, Johannes Weiner, Michal Koutný, Shuah Khan
Cc: linux-kernel, cgroups, linux-kselftest, Albert Esteve, mripard,
Eric Chanudet
Hi all,
This small series adds practical test coverage for the dmem
cgroup controller.
The motivation came from following the recent dmem API discussion in
thread [1]. That discussion considered changing the dmem API and
adding a new knob. Currently there are no dedicated tests covering
dmem behaviour, which makes such changes riskier.
Adding selftests has an additional challenge: dmem charging paths
are driver-driven today, so regression testing is harder unless a
suitable driver is present in the test environment.
This series addresses that by adding:
- a kernel-side selftest helper module to trigger charge/uncharge
from userspace in a controlled way,
- cgroup selftests covering dmem accounting and protection semantics
(including dmem.max enforcement and byte-granularity checks),
- a virtme-based VM runner for repeatable execution of the dmem tests.
The goal is to make dmem behavior easier to validate when evolving the API
and implementation, while keeping tests deterministic and driver-independent.
Thanks.
[1] - https://lore.kernel.org/all/aZoHfloupKvF2oSu@fedora/
Signed-off-by: Albert Esteve <aesteve@redhat.com>
---
Changes in v3:
- Set charged flag on unexpected over-limit charge success.
- Add CONFIG_DMEM_SELFTEST=m to selftest config.
- Simplify -v to a boolean; single -v was a no-op.
- Unquote kernel_opt to avoid empty-string arg to vng.
- Document -b in usage() output.
- Link to v2: https://lore.kernel.org/r/20260421-kunit_cgroups-v2-0-bb6675d8249c@redhat.com
Changes in v2:
- Fix debugfs_create_dir() error check
- Fix module teardown race: call dmem_selftest_remove() before
uncharging so debugfs files are torn down
- Use IS_ERR_OR_NULL() in selftest() sanity check
- Add CONFIG_CGROUP_DMEM=y to the cgroup selftest config
- Replace config-file parsing in check_guest_requirements() with
a direct check of /sys/fs/cgroup/cgroup.controllers
- Add new patch 4 (from Eric Chanudet): vmtest-dmem.sh -b flag
to configure and build a local kernel tree
- Link to v1: https://lore.kernel.org/r/20260327-kunit_cgroups-v1-0-971b3c739a00@redhat.com
---
Albert Esteve (3):
cgroup: Add dmem_selftest module
selftests: cgroup: Add dmem selftest coverage
selftests: cgroup: Add vmtest-dmem runner based on hid vmtest
Eric Chanudet (1):
selftests: cgroup: handle vmtest-dmem -b to test locally built kernel
init/Kconfig | 12 +
kernel/cgroup/Makefile | 1 +
kernel/cgroup/dmem_selftest.c | 199 +++++++++++
tools/testing/selftests/cgroup/.gitignore | 1 +
tools/testing/selftests/cgroup/Makefile | 2 +
tools/testing/selftests/cgroup/config | 2 +
tools/testing/selftests/cgroup/test_dmem.c | 492 ++++++++++++++++++++++++++
tools/testing/selftests/cgroup/vmtest-dmem.sh | 230 ++++++++++++
8 files changed, 939 insertions(+)
---
base-commit: 80234b5ab240f52fa45d201e899e207b9265ef91
change-id: 20260318-kunit_cgroups-7fb0b9e64017
Best regards,
--
Albert Esteve <aesteve@redhat.com>
^ permalink raw reply
* Re: [PATCH v2 cgroup/for-7.1-fixes sched_ext/for-7.1-fixes] cgroup: Defer css percpu_ref kill on rmdir until cgroup is depopulated
From: Sebastian Andrzej Siewior @ 2026-05-04 7:28 UTC (permalink / raw)
To: Tejun Heo
Cc: Martin Pitt, regressions, cgroups, Johannes Weiner,
Michal Koutný, David Vernet, Andrea Righi, Changwoo Min,
Emil Tsalapatis, sched-ext, linux-kernel
In-Reply-To: <ad3b4597f3df81914b871618535370db@kernel.org>
On 2026-05-01 08:31:22 [-1000], Tejun Heo wrote:
> A chain of commits going back to v7.0 reworked rmdir to satisfy the
> controller invariant that a subsystem's ->css_offline() must not run while
> tasks are still doing kernel-side work in the cgroup.
…
>
> Fixes: 1b164b876c36 ("cgroup: Wait for dying tasks to leave on rmdir")
> Cc: stable@vger.kernel.org # v7.0+
> Reported-by: Martin Pitt <martin@piware.de>
> Link: https://lore.kernel.org/all/afHNg2VX2jy9bW7y@piware.de/
> Link: https://lore.kernel.org/all/35e0670adb4abeab13da2c321582af9f@kernel.org/
> Signed-off-by: Tejun Heo <tj@kernel.org>
> Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
I gave it a quick spin, everything looks fine.
Acked-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Sebastian
^ permalink raw reply
* Re: [PATCH] cgroup: Defer css percpu_ref kill on rmdir until cgroup is depopulated
From: kernel test robot @ 2026-05-03 22:45 UTC (permalink / raw)
To: Tejun Heo, Martin Pitt
Cc: oe-kbuild-all, regressions, cgroups, lizefan.x, hannes,
Sebastian Andrzej Siewior, linux-kernel, Tejun Heo
In-Reply-To: <20260501022943.3714461-1-tj@kernel.org>
Hi Tejun,
kernel test robot noticed the following build errors:
[auto build test ERROR on tj-cgroup/for-next]
[also build test ERROR on linus/master next-20260430]
[cannot apply to v7.1-rc1]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Tejun-Heo/cgroup-Defer-css-percpu_ref-kill-on-rmdir-until-cgroup-is-depopulated/20260503-165802
base: https://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup.git for-next
patch link: https://lore.kernel.org/r/20260501022943.3714461-1-tj%40kernel.org
patch subject: [PATCH] cgroup: Defer css percpu_ref kill on rmdir until cgroup is depopulated
config: m68k-allmodconfig (https://download.01.org/0day-ci/archive/20260504/202605040655.KI0GsBVb-lkp@intel.com/config)
compiler: m68k-linux-gcc (GCC) 15.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20260504/202605040655.KI0GsBVb-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202605040655.KI0GsBVb-lkp@intel.com/
All errors (new ones prefixed by >>):
kernel/cgroup/cgroup.c: In function 'cgroup_apply_control_disable':
>> kernel/cgroup/cgroup.c:3391:33: error: implicit declaration of function 'kill_css_sync'; did you mean 'kill_fasync'? [-Wimplicit-function-declaration]
3391 | kill_css_sync(css);
| ^~~~~~~~~~~~~
| kill_fasync
>> kernel/cgroup/cgroup.c:3392:33: error: implicit declaration of function 'kill_css_finish' [-Wimplicit-function-declaration]
3392 | kill_css_finish(css);
| ^~~~~~~~~~~~~~~
kernel/cgroup/cgroup.c: At top level:
kernel/cgroup/cgroup.c:6047:13: warning: conflicting types for 'kill_css_sync'; have 'void(struct cgroup_subsys_state *)'
6047 | static void kill_css_sync(struct cgroup_subsys_state *css)
| ^~~~~~~~~~~~~
>> kernel/cgroup/cgroup.c:6047:13: error: static declaration of 'kill_css_sync' follows non-static declaration
kernel/cgroup/cgroup.c:3391:33: note: previous implicit declaration of 'kill_css_sync' with type 'void(struct cgroup_subsys_state *)'
3391 | kill_css_sync(css);
| ^~~~~~~~~~~~~
kernel/cgroup/cgroup.c:6087:13: warning: conflicting types for 'kill_css_finish'; have 'void(struct cgroup_subsys_state *)'
6087 | static void kill_css_finish(struct cgroup_subsys_state *css)
| ^~~~~~~~~~~~~~~
>> kernel/cgroup/cgroup.c:6087:13: error: static declaration of 'kill_css_finish' follows non-static declaration
kernel/cgroup/cgroup.c:3392:33: note: previous implicit declaration of 'kill_css_finish' with type 'void(struct cgroup_subsys_state *)'
3392 | kill_css_finish(css);
| ^~~~~~~~~~~~~~~
vim +3391 kernel/cgroup/cgroup.c
3359
3360 /**
3361 * cgroup_apply_control_disable - kill or hide csses according to control
3362 * @cgrp: root of the target subtree
3363 *
3364 * Walk @cgrp's subtree and kill and hide csses so that they match
3365 * cgroup_ss_mask() and cgroup_visible_mask().
3366 *
3367 * A css is hidden when the userland requests it to be disabled while other
3368 * subsystems are still depending on it. The css must not actively control
3369 * resources and be in the vanilla state if it's made visible again later.
3370 * Controllers which may be depended upon should provide ->css_reset() for
3371 * this purpose.
3372 */
3373 static void cgroup_apply_control_disable(struct cgroup *cgrp)
3374 {
3375 struct cgroup *dsct;
3376 struct cgroup_subsys_state *d_css;
3377 struct cgroup_subsys *ss;
3378 int ssid;
3379
3380 cgroup_for_each_live_descendant_post(dsct, d_css, cgrp) {
3381 for_each_subsys(ss, ssid) {
3382 struct cgroup_subsys_state *css = cgroup_css(dsct, ss);
3383
3384 if (!css)
3385 continue;
3386
3387 WARN_ON_ONCE(percpu_ref_is_dying(&css->refcnt));
3388
3389 if (css->parent &&
3390 !(cgroup_ss_mask(dsct) & (1 << ss->id))) {
> 3391 kill_css_sync(css);
> 3392 kill_css_finish(css);
3393 } else if (!css_visible(css)) {
3394 css_clear_dir(css);
3395 if (ss->css_reset)
3396 ss->css_reset(css);
3397 }
3398 }
3399 }
3400 }
3401
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply
* Re: [PATCH] cgroup: Defer css percpu_ref kill on rmdir until cgroup is depopulated
From: kernel test robot @ 2026-05-03 20:15 UTC (permalink / raw)
To: Tejun Heo, Martin Pitt
Cc: oe-kbuild-all, regressions, cgroups, lizefan.x, hannes,
Sebastian Andrzej Siewior, linux-kernel, Tejun Heo
In-Reply-To: <20260501022943.3714461-1-tj@kernel.org>
Hi Tejun,
kernel test robot noticed the following build warnings:
[auto build test WARNING on tj-cgroup/for-next]
[also build test WARNING on linus/master next-20260430]
[cannot apply to v7.1-rc1]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Tejun-Heo/cgroup-Defer-css-percpu_ref-kill-on-rmdir-until-cgroup-is-depopulated/20260503-165802
base: https://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup.git for-next
patch link: https://lore.kernel.org/r/20260501022943.3714461-1-tj%40kernel.org
patch subject: [PATCH] cgroup: Defer css percpu_ref kill on rmdir until cgroup is depopulated
config: powerpc-randconfig-r071-20260504 (https://download.01.org/0day-ci/archive/20260504/202605040408.yt7xcKug-lkp@intel.com/config)
compiler: powerpc-linux-gcc (GCC) 8.5.0
smatch: v0.5.0-9065-ge9cc34fd
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20260504/202605040408.yt7xcKug-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202605040408.yt7xcKug-lkp@intel.com/
All warnings (new ones prefixed by >>):
kernel/cgroup/cgroup.c: In function 'cgroup_apply_control_disable':
kernel/cgroup/cgroup.c:3391:5: error: implicit declaration of function 'kill_css_sync'; did you mean 'kill_fasync'? [-Werror=implicit-function-declaration]
kill_css_sync(css);
^~~~~~~~~~~~~
kill_fasync
kernel/cgroup/cgroup.c:3392:5: error: implicit declaration of function 'kill_css_finish'; did you mean 'kill_cad_pid'? [-Werror=implicit-function-declaration]
kill_css_finish(css);
^~~~~~~~~~~~~~~
kill_cad_pid
kernel/cgroup/cgroup.c: At top level:
>> kernel/cgroup/cgroup.c:6047:13: warning: conflicting types for 'kill_css_sync'
static void kill_css_sync(struct cgroup_subsys_state *css)
^~~~~~~~~~~~~
kernel/cgroup/cgroup.c:6047:13: error: static declaration of 'kill_css_sync' follows non-static declaration
kernel/cgroup/cgroup.c:3391:5: note: previous implicit declaration of 'kill_css_sync' was here
kill_css_sync(css);
^~~~~~~~~~~~~
>> kernel/cgroup/cgroup.c:6087:13: warning: conflicting types for 'kill_css_finish'
static void kill_css_finish(struct cgroup_subsys_state *css)
^~~~~~~~~~~~~~~
kernel/cgroup/cgroup.c:6087:13: error: static declaration of 'kill_css_finish' follows non-static declaration
kernel/cgroup/cgroup.c:3392:5: note: previous implicit declaration of 'kill_css_finish' was here
kill_css_finish(css);
^~~~~~~~~~~~~~~
cc1: some warnings being treated as errors
vim +/kill_css_sync +6047 kernel/cgroup/cgroup.c
6040
6041 /**
6042 * kill_css_sync - synchronous half of css teardown
6043 * @css: css being killed
6044 *
6045 * See cgroup_destroy_locked().
6046 */
> 6047 static void kill_css_sync(struct cgroup_subsys_state *css)
6048 {
6049 struct cgroup_subsys *ss = css->ss;
6050
6051 lockdep_assert_held(&cgroup_mutex);
6052
6053 if (css->flags & CSS_DYING)
6054 return;
6055
6056 /*
6057 * Call css_killed(), if defined, before setting the CSS_DYING flag
6058 */
6059 if (css->ss->css_killed)
6060 css->ss->css_killed(css);
6061
6062 css->flags |= CSS_DYING;
6063
6064 /*
6065 * This must happen before css is disassociated with its cgroup.
6066 * See seq_css() for details.
6067 */
6068 css_clear_dir(css);
6069
6070 css->cgroup->nr_dying_subsys[ss->id]++;
6071 /*
6072 * Parent css and cgroup cannot be freed until after the freeing
6073 * of child css, see css_free_rwork_fn().
6074 */
6075 while ((css = css->parent)) {
6076 css->nr_descendants--;
6077 css->cgroup->nr_dying_subsys[ss->id]++;
6078 }
6079 }
6080
6081 /**
6082 * kill_css_finish - deferred half of css teardown
6083 * @css: css being killed
6084 *
6085 * See cgroup_destroy_locked().
6086 */
> 6087 static void kill_css_finish(struct cgroup_subsys_state *css)
6088 {
6089 lockdep_assert_held(&cgroup_mutex);
6090
6091 /*
6092 * Skip on re-entry: cgroup_apply_control_disable() may have killed @css
6093 * earlier. cgroup_destroy_locked() can still walk it because
6094 * offline_css() (which NULLs cgrp->subsys[ssid]) runs async.
6095 */
6096 if (percpu_ref_is_dying(&css->refcnt))
6097 return;
6098
6099 /*
6100 * Killing would put the base ref, but we need to keep it alive until
6101 * after ->css_offline().
6102 */
6103 css_get(css);
6104
6105 /*
6106 * cgroup core guarantees that, by the time ->css_offline() is invoked,
6107 * no new css reference will be given out via css_tryget_online(). We
6108 * can't simply call percpu_ref_kill() and proceed to offlining css's
6109 * because percpu_ref_kill() doesn't guarantee that the ref is seen as
6110 * killed on all CPUs on return.
6111 *
6112 * Use percpu_ref_kill_and_confirm() to get notifications as each css is
6113 * confirmed to be seen as killed on all CPUs.
6114 */
6115 percpu_ref_kill_and_confirm(&css->refcnt, css_killed_ref_fn);
6116 }
6117
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply
* Re: [PATCH] cgroup: Defer css percpu_ref kill on rmdir until cgroup is depopulated
From: kernel test robot @ 2026-05-03 19:30 UTC (permalink / raw)
To: Tejun Heo, Martin Pitt
Cc: oe-kbuild-all, regressions, cgroups, lizefan.x, hannes,
Sebastian Andrzej Siewior, linux-kernel, Tejun Heo
In-Reply-To: <20260501022943.3714461-1-tj@kernel.org>
Hi Tejun,
kernel test robot noticed the following build warnings:
[auto build test WARNING on tj-cgroup/for-next]
[also build test WARNING on linus/master next-20260430]
[cannot apply to v7.1-rc1]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Tejun-Heo/cgroup-Defer-css-percpu_ref-kill-on-rmdir-until-cgroup-is-depopulated/20260503-165802
base: https://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup.git for-next
patch link: https://lore.kernel.org/r/20260501022943.3714461-1-tj%40kernel.org
patch subject: [PATCH] cgroup: Defer css percpu_ref kill on rmdir until cgroup is depopulated
config: m68k-allmodconfig (https://download.01.org/0day-ci/archive/20260504/202605040315.QbFTzfWy-lkp@intel.com/config)
compiler: m68k-linux-gcc (GCC) 15.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20260504/202605040315.QbFTzfWy-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202605040315.QbFTzfWy-lkp@intel.com/
All warnings (new ones prefixed by >>):
kernel/cgroup/cgroup.c: In function 'cgroup_apply_control_disable':
kernel/cgroup/cgroup.c:3391:33: error: implicit declaration of function 'kill_css_sync'; did you mean 'kill_fasync'? [-Wimplicit-function-declaration]
3391 | kill_css_sync(css);
| ^~~~~~~~~~~~~
| kill_fasync
kernel/cgroup/cgroup.c:3392:33: error: implicit declaration of function 'kill_css_finish' [-Wimplicit-function-declaration]
3392 | kill_css_finish(css);
| ^~~~~~~~~~~~~~~
kernel/cgroup/cgroup.c: At top level:
>> kernel/cgroup/cgroup.c:6047:13: warning: conflicting types for 'kill_css_sync'; have 'void(struct cgroup_subsys_state *)'
6047 | static void kill_css_sync(struct cgroup_subsys_state *css)
| ^~~~~~~~~~~~~
kernel/cgroup/cgroup.c:6047:13: error: static declaration of 'kill_css_sync' follows non-static declaration
kernel/cgroup/cgroup.c:3391:33: note: previous implicit declaration of 'kill_css_sync' with type 'void(struct cgroup_subsys_state *)'
3391 | kill_css_sync(css);
| ^~~~~~~~~~~~~
>> kernel/cgroup/cgroup.c:6087:13: warning: conflicting types for 'kill_css_finish'; have 'void(struct cgroup_subsys_state *)'
6087 | static void kill_css_finish(struct cgroup_subsys_state *css)
| ^~~~~~~~~~~~~~~
kernel/cgroup/cgroup.c:6087:13: error: static declaration of 'kill_css_finish' follows non-static declaration
kernel/cgroup/cgroup.c:3392:33: note: previous implicit declaration of 'kill_css_finish' with type 'void(struct cgroup_subsys_state *)'
3392 | kill_css_finish(css);
| ^~~~~~~~~~~~~~~
vim +6047 kernel/cgroup/cgroup.c
6040
6041 /**
6042 * kill_css_sync - synchronous half of css teardown
6043 * @css: css being killed
6044 *
6045 * See cgroup_destroy_locked().
6046 */
> 6047 static void kill_css_sync(struct cgroup_subsys_state *css)
6048 {
6049 struct cgroup_subsys *ss = css->ss;
6050
6051 lockdep_assert_held(&cgroup_mutex);
6052
6053 if (css->flags & CSS_DYING)
6054 return;
6055
6056 /*
6057 * Call css_killed(), if defined, before setting the CSS_DYING flag
6058 */
6059 if (css->ss->css_killed)
6060 css->ss->css_killed(css);
6061
6062 css->flags |= CSS_DYING;
6063
6064 /*
6065 * This must happen before css is disassociated with its cgroup.
6066 * See seq_css() for details.
6067 */
6068 css_clear_dir(css);
6069
6070 css->cgroup->nr_dying_subsys[ss->id]++;
6071 /*
6072 * Parent css and cgroup cannot be freed until after the freeing
6073 * of child css, see css_free_rwork_fn().
6074 */
6075 while ((css = css->parent)) {
6076 css->nr_descendants--;
6077 css->cgroup->nr_dying_subsys[ss->id]++;
6078 }
6079 }
6080
6081 /**
6082 * kill_css_finish - deferred half of css teardown
6083 * @css: css being killed
6084 *
6085 * See cgroup_destroy_locked().
6086 */
> 6087 static void kill_css_finish(struct cgroup_subsys_state *css)
6088 {
6089 lockdep_assert_held(&cgroup_mutex);
6090
6091 /*
6092 * Skip on re-entry: cgroup_apply_control_disable() may have killed @css
6093 * earlier. cgroup_destroy_locked() can still walk it because
6094 * offline_css() (which NULLs cgrp->subsys[ssid]) runs async.
6095 */
6096 if (percpu_ref_is_dying(&css->refcnt))
6097 return;
6098
6099 /*
6100 * Killing would put the base ref, but we need to keep it alive until
6101 * after ->css_offline().
6102 */
6103 css_get(css);
6104
6105 /*
6106 * cgroup core guarantees that, by the time ->css_offline() is invoked,
6107 * no new css reference will be given out via css_tryget_online(). We
6108 * can't simply call percpu_ref_kill() and proceed to offlining css's
6109 * because percpu_ref_kill() doesn't guarantee that the ref is seen as
6110 * killed on all CPUs on return.
6111 *
6112 * Use percpu_ref_kill_and_confirm() to get notifications as each css is
6113 * confirmed to be seen as killed on all CPUs.
6114 */
6115 percpu_ref_kill_and_confirm(&css->refcnt, css_killed_ref_fn);
6116 }
6117
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply
* Re: [PATCH v2 cgroup/for-7.1-fixes sched_ext/for-7.1-fixes] cgroup: Defer css percpu_ref kill on rmdir until cgroup is depopulated
From: Martin Pitt @ 2026-05-03 13:53 UTC (permalink / raw)
To: Tejun Heo
Cc: regressions, cgroups, Johannes Weiner, Michal Koutný,
Sebastian Andrzej Siewior, David Vernet, Andrea Righi,
Changwoo Min, Emil Tsalapatis, sched-ext, linux-kernel
In-Reply-To: <ad3b4597f3df81914b871618535370db@kernel.org>
Hello Tejun and all,
Tejun Heo [2026-05-01 8:31 -1000]:
> A chain of commits going back to v7.0 reworked rmdir to satisfy the
> controller invariant that a subsystem's ->css_offline() must not run while
> tasks are still doing kernel-side work in the cgroup.
> [..]
> v2: Pin cgrp across the deferred destroy work with explicit
> cgroup_get()/cgroup_put() around queue_work() and the work_fn. v1
> wasn't actually broken (ordered cgroup_offline_wq + queue_work order
> in cgroup_task_dead() saved it) but the explicit ref removes the
> dependency on those non-obvious invariants. Also note the
> pre-existing cgroup_apply_control_disable() race in the description;
> a follow-up will defer kill_css_finish() there.
>
> Fixes: 1b164b876c36 ("cgroup: Wait for dying tasks to leave on rmdir")
Tested-by: Martin Pitt <martin@piware.de>
> Could you give v2 a try? Same defer-the-percpu_ref-kill mechanism as
> v1, with an explicit cgroup_get/put around the deferred work to make
> the lifetime invariant obvious (Sashiko bot review on v1; v1 wasn't
> broken but the explicit ref removes a dependency on non-obvious
> ordering). Fix should behave identically to v1 for your reproducer.
Sorry for the delay, I haven't built a kernel in a decade and not ever for
Fedora.
I applied the patch to the Rawhide 7.1.0-0.rc1 kernel, it applies cleanly
there. (I first tried on top of 6.9.14, but there are conflicts.)
https://copr.fedorainfracloud.org/coprs/martinpitt/test-fixes/build/10419932/
Usage on Fedora 44:
dnf copr enable martinpitt/test-fixes
dnf update kernel-core kernel-modules-internal
(This assumes a cloud VM. If you use the full kernel, update the "kernel"
package as well).
I ran the cockpit-podman test that originally triggered the bug, as well as my
reduced variant, against that patched kernel for 50 rounds each, and it has
consistently succeeded.
So this works great, thanks a lot!
Martin
^ permalink raw reply
* [tj-cgroup:for-7.1-fixes] BUILD SUCCESS 981cd338614c96070cf9854679014fd027c1fb1d
From: kernel test robot @ 2026-05-02 16:27 UTC (permalink / raw)
To: Tejun Heo; +Cc: cgroups
tree/branch: https://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup.git for-7.1-fixes
branch HEAD: 981cd338614c96070cf9854679014fd027c1fb1d docs: cgroup: fix typo 'protetion' -> 'protection'
elapsed time: 7049m
configs tested: 197
configs skipped: 2
The following configs have been built successfully.
More configs may be tested in the coming days.
tested configs:
alpha allnoconfig gcc-15.2.0
alpha allyesconfig gcc-15.2.0
alpha defconfig gcc-15.2.0
arc allmodconfig clang-16
arc allnoconfig gcc-15.2.0
arc allyesconfig clang-23
arc defconfig gcc-15.2.0
arc randconfig-001 gcc-8.5.0
arc randconfig-001-20260502 gcc-8.5.0
arc randconfig-002 gcc-8.5.0
arc randconfig-002-20260502 gcc-8.5.0
arm allnoconfig gcc-15.2.0
arm allyesconfig clang-16
arm defconfig gcc-15.2.0
arm randconfig-001 gcc-8.5.0
arm randconfig-001-20260502 gcc-8.5.0
arm randconfig-002 gcc-8.5.0
arm randconfig-002-20260502 gcc-8.5.0
arm randconfig-003 gcc-8.5.0
arm randconfig-003-20260502 gcc-8.5.0
arm randconfig-004 gcc-8.5.0
arm randconfig-004-20260502 gcc-8.5.0
arm64 allmodconfig clang-23
arm64 allnoconfig gcc-15.2.0
arm64 defconfig gcc-15.2.0
arm64 randconfig-001 gcc-10.5.0
arm64 randconfig-001-20260502 gcc-10.5.0
arm64 randconfig-002 gcc-10.5.0
arm64 randconfig-002-20260502 gcc-10.5.0
arm64 randconfig-003 gcc-10.5.0
arm64 randconfig-003-20260502 gcc-10.5.0
arm64 randconfig-004 gcc-10.5.0
arm64 randconfig-004-20260502 gcc-10.5.0
csky allmodconfig gcc-15.2.0
csky allnoconfig gcc-15.2.0
csky defconfig gcc-15.2.0
csky randconfig-001 gcc-10.5.0
csky randconfig-001-20260502 gcc-10.5.0
csky randconfig-002 gcc-10.5.0
csky randconfig-002-20260502 gcc-10.5.0
hexagon allmodconfig gcc-15.2.0
hexagon allnoconfig gcc-15.2.0
hexagon defconfig gcc-15.2.0
hexagon randconfig-001-20260501 clang-23
hexagon randconfig-001-20260502 clang-23
hexagon randconfig-002-20260501 clang-23
hexagon randconfig-002-20260502 clang-23
i386 allmodconfig clang-20
i386 allnoconfig gcc-15.2.0
i386 allyesconfig clang-20
i386 buildonly-randconfig-001-20260502 gcc-14
i386 buildonly-randconfig-002-20260502 gcc-14
i386 buildonly-randconfig-003-20260502 gcc-14
i386 buildonly-randconfig-004-20260502 gcc-14
i386 buildonly-randconfig-005-20260502 gcc-14
i386 buildonly-randconfig-006-20260502 gcc-14
i386 defconfig gcc-15.2.0
i386 randconfig-011-20260501 clang-20
i386 randconfig-012-20260501 clang-20
i386 randconfig-013-20260501 clang-20
i386 randconfig-014-20260501 clang-20
i386 randconfig-015-20260501 clang-20
i386 randconfig-016-20260501 clang-20
i386 randconfig-017-20260501 clang-20
loongarch allmodconfig clang-23
loongarch allnoconfig gcc-15.2.0
loongarch defconfig clang-19
loongarch randconfig-001-20260501 clang-23
loongarch randconfig-001-20260502 clang-23
loongarch randconfig-002-20260501 clang-23
loongarch randconfig-002-20260502 clang-23
m68k allmodconfig gcc-15.2.0
m68k allnoconfig gcc-15.2.0
m68k allyesconfig clang-16
m68k defconfig clang-19
m68k mvme147_defconfig gcc-15.2.0
microblaze allnoconfig gcc-15.2.0
microblaze allyesconfig gcc-15.2.0
microblaze defconfig clang-19
mips allmodconfig gcc-15.2.0
mips allnoconfig gcc-15.2.0
mips allyesconfig gcc-15.2.0
nios2 allmodconfig clang-23
nios2 allnoconfig clang-23
nios2 defconfig clang-19
nios2 randconfig-001-20260501 clang-23
nios2 randconfig-001-20260502 clang-23
nios2 randconfig-002-20260501 clang-23
nios2 randconfig-002-20260502 clang-23
openrisc allmodconfig clang-23
openrisc allnoconfig clang-23
openrisc defconfig gcc-15.2.0
parisc allmodconfig gcc-15.2.0
parisc allnoconfig clang-23
parisc allyesconfig clang-19
parisc defconfig gcc-15.2.0
parisc randconfig-001 gcc-12.5.0
parisc randconfig-001-20260501 gcc-12.5.0
parisc randconfig-001-20260502 gcc-12.5.0
parisc randconfig-002 gcc-12.5.0
parisc randconfig-002-20260501 gcc-12.5.0
parisc randconfig-002-20260502 gcc-12.5.0
parisc64 defconfig clang-19
powerpc allmodconfig gcc-15.2.0
powerpc allnoconfig clang-23
powerpc randconfig-001 gcc-12.5.0
powerpc randconfig-001-20260501 gcc-12.5.0
powerpc randconfig-001-20260502 gcc-12.5.0
powerpc randconfig-002 gcc-12.5.0
powerpc randconfig-002-20260501 gcc-12.5.0
powerpc randconfig-002-20260502 gcc-12.5.0
powerpc64 randconfig-001 gcc-12.5.0
powerpc64 randconfig-001-20260501 gcc-12.5.0
powerpc64 randconfig-001-20260502 gcc-12.5.0
powerpc64 randconfig-002 gcc-12.5.0
powerpc64 randconfig-002-20260501 gcc-12.5.0
powerpc64 randconfig-002-20260502 gcc-12.5.0
riscv allmodconfig clang-23
riscv allnoconfig clang-23
riscv allyesconfig clang-16
riscv defconfig gcc-15.2.0
riscv randconfig-001-20260502 gcc-14.3.0
riscv randconfig-002-20260502 gcc-14.3.0
s390 allmodconfig clang-19
s390 allnoconfig clang-23
s390 allyesconfig gcc-15.2.0
s390 defconfig gcc-15.2.0
s390 randconfig-001-20260502 gcc-14.3.0
s390 randconfig-002-20260502 gcc-14.3.0
sh allmodconfig gcc-15.2.0
sh allnoconfig clang-23
sh allyesconfig clang-19
sh defconfig gcc-14
sh randconfig-001-20260502 gcc-14.3.0
sh randconfig-002-20260502 gcc-14.3.0
sparc allnoconfig clang-23
sparc defconfig gcc-15.2.0
sparc randconfig-001-20260502 gcc-8.5.0
sparc randconfig-002-20260502 gcc-8.5.0
sparc64 allmodconfig clang-23
sparc64 defconfig gcc-14
sparc64 randconfig-001-20260502 gcc-8.5.0
sparc64 randconfig-002-20260502 gcc-8.5.0
um allmodconfig clang-19
um allnoconfig clang-23
um allyesconfig gcc-15.2.0
um defconfig gcc-14
um i386_defconfig gcc-14
um randconfig-001-20260502 gcc-8.5.0
um randconfig-002-20260502 gcc-8.5.0
um x86_64_defconfig gcc-14
x86_64 allmodconfig clang-20
x86_64 allnoconfig clang-23
x86_64 allyesconfig clang-20
x86_64 buildonly-randconfig-001 clang-20
x86_64 buildonly-randconfig-001-20260502 clang-20
x86_64 buildonly-randconfig-002 clang-20
x86_64 buildonly-randconfig-002-20260502 clang-20
x86_64 buildonly-randconfig-003 clang-20
x86_64 buildonly-randconfig-003-20260502 clang-20
x86_64 buildonly-randconfig-004 clang-20
x86_64 buildonly-randconfig-004-20260502 clang-20
x86_64 buildonly-randconfig-005 clang-20
x86_64 buildonly-randconfig-005-20260502 clang-20
x86_64 buildonly-randconfig-006 clang-20
x86_64 buildonly-randconfig-006-20260502 clang-20
x86_64 defconfig gcc-14
x86_64 kexec clang-20
x86_64 randconfig-001-20260502 clang-20
x86_64 randconfig-002-20260502 clang-20
x86_64 randconfig-003-20260502 clang-20
x86_64 randconfig-004-20260502 clang-20
x86_64 randconfig-005-20260502 clang-20
x86_64 randconfig-006-20260502 clang-20
x86_64 randconfig-071-20260501 gcc-14
x86_64 randconfig-071-20260502 gcc-14
x86_64 randconfig-072-20260501 gcc-14
x86_64 randconfig-072-20260502 gcc-14
x86_64 randconfig-073-20260501 gcc-14
x86_64 randconfig-073-20260502 gcc-14
x86_64 randconfig-074-20260501 gcc-14
x86_64 randconfig-074-20260502 gcc-14
x86_64 randconfig-075-20260501 gcc-14
x86_64 randconfig-075-20260502 gcc-14
x86_64 randconfig-076-20260501 gcc-14
x86_64 randconfig-076-20260502 gcc-14
x86_64 rhel-9.4 clang-20
x86_64 rhel-9.4-bpf gcc-14
x86_64 rhel-9.4-func clang-20
x86_64 rhel-9.4-kselftests clang-20
x86_64 rhel-9.4-kunit gcc-14
x86_64 rhel-9.4-ltp gcc-14
x86_64 rhel-9.4-rust clang-20
xtensa allnoconfig clang-23
xtensa allyesconfig clang-23
xtensa randconfig-001-20260502 gcc-8.5.0
xtensa randconfig-002-20260502 gcc-8.5.0
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply
* [PATCH RFC] memcg: add per-cgroup dirty page controls (dirty_ratio, dirty_min)
From: Alireza Haghdoost via B4 Relay @ 2026-05-01 22:28 UTC (permalink / raw)
To: Johannes Weiner, Michal Hocko, Roman Gushchin, Shakeel Butt,
Muchun Song, Andrew Morton, Matthew Wilcox (Oracle), Jan Kara
Cc: cgroups, linux-mm, linux-kernel, linux-fsdevel, Kshitij Doshi,
Alireza Haghdoost
From: Alireza Haghdoost <haghdoost@uber.com>
Add two cgroup v2 memory-controller knobs that bring
balance_dirty_pages() throttling into per-cgroup scope so one noisy
writer cannot stall peers sharing the same host:
memory.dirty_ratio Per-cgroup dirty-page ceiling, 0..100 percent of
the cgroup's dirtyable memory. 0 (default) leaves
the cgroup subject to the global threshold only.
memory.dirty_min Guaranteed dirty-page floor, byte value (default 0).
The two knobs compose: dirty_ratio bounds how much dirty memory a
cgroup may accrue, dirty_min guarantees a floor below which it is
never throttled.
Motivation, design trade-offs, cost analysis, validation data, and
open questions are in the cover letter.
Co-developed-by: Kshitij Doshi <kshitijd@uber.com>
Signed-off-by: Kshitij Doshi <kshitijd@uber.com>
Signed-off-by: Alireza Haghdoost <haghdoost@uber.com>
Assisted-by: Cursor:claude-sonnet-4.5
---
Hi all,
This RFC adds two cgroup v2 memory-controller knobs that give operators
per-cgroup control over dirty-page throttling in balance_dirty_pages():
memory.dirty_ratio (per-cgroup ceiling) and memory.dirty_min (guaranteed
floor). A third knob, memory.dirty_weight, is forthcoming in a follow-up
once we have validated the application site (see "Follow-ups" below).
We are posting this as an RFC, as a single squashed patch, to get design
feedback before splitting the prototype into a per-logical-change series.
Motivation
==========
balance_dirty_pages() (BDP) is a global throttle. It sleeps writers
once the host-wide dirty count crosses a single threshold. On a
container host that threshold is shared across cgroups. A cgroup
that dirties pages faster than storage can drain them pushes the
count over the limit. Every writer on the host then parks in
io_schedule_timeout() -- including cgroups that have not dirtied a
single page of their own.
cgroup v2 already has per-memcg dirty accounting, but that accounting
does not translate into per-memcg dirty throttling.
We see this in production: a buffered write-heavy container generates
multi-second stalls for co-located latency-sensitive workloads.
Moreover, dirty-page accumulation from a single noisy neighbor is a
recurring contributor to mount-responsiveness degradation on shared hosts.
Prior work
==========
Per-memcg dirty-page limits have been proposed before. Andrea Righi
posted an initial RFC in February 2010 [1]; Greg Thelen continued the
work through v9 in August 2011 [2]. That series added per-memcg dirty
counters and hooked them into balance_dirty_pages(), but it bolted
per-cgroup limits onto the global writeback path without making
writeback itself cgroup-aware. Without cgroup-aware flusher threads,
a cgroup exceeding its limit triggered writeback of inodes from any
cgroup, giving poor isolation. The series was not merged.
Konstantin Khlebnikov posted "[PATCHSET RFC 0/6] memcg: inode-based
dirty-set controller" in January 2015 [4], which proposed
memory.dirty_ratio (the same interface name this series uses) via an
inode-tagged, filtered-writeback approach. Tejun Heo reviewed it
and rejected it as a "dead end" that duplicated lower-layer policy
without solving the underlying isolation problem; this rejection
directly motivated Tejun's native cgwb rework described below.
Tejun Heo's 48-patch cgroup-writeback rework, merged in Linux 4.2
(commit e4bc13adfd01, "Merge branch 'for-4.2/writeback'"), took the
different approach of restructuring writeback to be natively
cgroup-aware: per-memcg wb_domain (commit 841710aa6e4a), per-memcg
NR_FILE_DIRTY / NR_WRITEBACK accounting, and cgroup-aware flusher
threads [3]. That work deliberately deferred user-facing policy knobs.
This series adds the policy surface that consumes Tejun's
infrastructure. The dirty_min reservation concept is, to our
knowledge, new.
A November 2023 LKML thread by Chengming Zhou [5] independently
identified the identical throttling regression on cgroup v2 (a 5 GB
container constantly throttled because memory.max * dirty_ratio yields
too small a threshold for bursty workloads). Jan Kara participated and
endorsed a bytes-based per-memcg dirty limit; no patches followed that
discussion, confirming the gap this series fills.
[1] https://lwn.net/Articles/408349/
[2] https://lore.kernel.org/lkml/1313597705-6093-1-git-send-email-gthelen@google.com/
[3] https://lwn.net/Articles/648292/
[4] https://lore.kernel.org/all/20150115180242.10450.92.stgit@buzz/
[5] https://lore.kernel.org/all/109029e0-1772-4102-a2a8-ab9076462454@linux.dev/
Proposed interface
==================
Two new cgroup v2 files under the memory controller, absent on the root
cgroup (CFTYPE_NOT_ON_ROOT):
memory.dirty_ratio
Integer [0, 100]. Per-cgroup dirty-page ceiling as a percentage of
the cgroup's dirtyable memory (mdtc->avail: file cache plus
reclaimable slack), the same base the global vm_dirty_ratio scales
against. 0 (the default) disables the per-cgroup ceiling and leaves
the cgroup subject to the global threshold only. A non-zero value
that is stricter than vm_dirty_ratio overrides the global ratio for
this cgroup via min(mdtc->thresh, cg_thresh); because both sides
scale off the same base, the knob can never widen the cgroup past
the global ceiling. A memory.dirty_bytes companion for byte-precise
caps (mirroring vm_dirty_bytes) is noted under "Follow-ups" below.
The prototype reads the value for the immediate memcg only;
hierarchical enforcement (clamping against ancestors, like
memory.max) is not implemented yet. We would like guidance on
whether this is required for v1 or can follow in a subsequent
series.
memory.dirty_min
Byte value (K/M/G suffixes accepted), default 0 (no reservation).
Guaranteed dirty-page floor: while cgroup_dirty < dirty_min,
throttling is bypassed (goto free_running). Lets a latency-sensitive
cgroup buffer a small write burst even under global dirty pressure.
dirty_min is an admission guarantee, so we have to prevent it from
breaking the global dirty invariant. Two aspects:
- Global cap. The sum of dirty_min reservations across all cgroups
must not exceed a fraction of the global dirty threshold (our
working number is 80%), so the system always retains some shared
capacity. The prototype does not enforce this cap yet; we expect
to either reject at write() time or clamp on read in a cheap
precomputed effective_dirty_min. We would appreciate feedback on
which approach the cgroup maintainers prefer.
- Per-cgroup cap. A cgroup should not be able to reserve more dirty
capacity than it can hold. Our tentative rule is
effective_dirty_min = min(dirty_min, memory.max - memory.min),
evaluated at enforcement time so it tracks live memory.max changes,
rather than rejected at write. This is similar to how memory.low
composes with memory.max.
Neither cap is implemented in the prototype; both would land before
a non-RFC posting.
The two knobs compose: dirty_ratio bounds how much dirty memory a
cgroup may accrue, dirty_min guarantees a floor below which it is never
throttled.
Test setup and results
======================
To show the problem and the fix, we built a single reproducer that runs
on an unmodified stock kernel and then on the patched kernel, using the
same setup for both.
Setup: QEMU guest with a virtio-blk disk throttled to 256 KB/s
(bps_wr=262144). Two sibling cgroups, no io.weight; both share disk
bandwidth equally. dirty_bytes=32MB, dirty_background_bytes=16MB
(freerun = 24 MB). Files pre-allocated with fallocate before dirty
pressure. Two phases per run: (1) victim alone (baseline), (2) noisy
fills global dirty to the 32 MB cap, then victim runs contended for
30 s.
- noisy: single fio job, unlimited write rate, fills global dirty pool.
- victim: single fio job, rate-limited to 500 KB/s (128 IOPS target),
4 KiB sequential write().
The high freerun (24 MB) ensures victim's solo dirty accumulation
(244 KB/s x 30 s = 7.3 MB) stays below the threshold. BDP does not fire
during the solo phase on either kernel, giving identical baselines.
Stock kernel results (the problem):
solo (no contention) contended inflation
victim IOPS 125 5.1 24.5x worse
victim p99 0.6 ms 152 ms 253x worse
The contended p99 sits at fio's percentile bucket nearest MAX_PAUSE =
HZ/5 = 200 ms (mm/page-writeback.c:49), the hard kernel ceiling on BDP
sleep. The victim has near-zero dirty pages of its own but is forced to
sleep because balance_dirty_pages() sees gdtc->dirty = NR_FILE_DIRTY +
NR_WRITEBACK above the freerun threshold. Most of noisy's pages are
queued in NR_WRITEBACK waiting for the throttled disk.
memory.events.local on both cgroups shows max 0 throughout the run;
this is not memory pressure inside either cgroup.
Patched kernel results (the fix), with victim/memory.dirty_min = 16 MB:
solo (no contention) contended inflation
victim IOPS 125 125 1.0x
victim p99 0.9 ms 0.7 ms 1.0x
The patched kernel checks cgroup_dirty < dirty_min (4096 pages) before
computing any sleep. Because the rate-limited victim's resident dirty
set stays well below the reservation, the check fires on every write()
-> goto free_running -> write() returns in ~7 us. The victim is fully
protected.
The figures above are the per-kernel medians of N=5 iterations and
reflect a deterministic outcome on every iter: stock had cont_iops in
4.4..6.1 (retention 0.035..0.049) on all five iters, patched had
cont_iops = 125.0 (retention = 1.000) on all five iters. The 5/5 stock
iters all hit BDP's throttled regime; the 5/5 patched iters all
bypassed it via the dirty_min check.
Implementation
==============
The patch touches five files:
- include/linux/memcontrol.h: two new fields on struct mem_cgroup
(dirty_ratio, dirty_min).
- include/linux/writeback.h: a per-pass cg_dirty_cap field on
struct dirty_throttle_control used to publish the memcg clamp to
BDP's setpoint and rate-limit math.
- include/trace/events/writeback.h: cg_dirty_cap added to the
balance_dirty_pages tracepoint so operators can distinguish the
memcg clamp from the global dirty_limit at runtime.
- mm/memcontrol.c: registers the two cgroup v2 files with input
validation.
- mm/page-writeback.c: the throttling changes.
The key changes in page-writeback.c:
- The dirty_ratio clamp lives inside domain_dirty_limits(), keyed
on wb->memcg_css (the inode owner's memcg). Every consumer of
the memcg dtc -- writer throttle, flusher kworker,
cgwb_calc_thresh -- sees the same clamped thresh and bg_thresh.
The clamp uses mult_frac() so a small memcg does not collapse
to a zero ceiling.
- The dirty_min bypass lives in balance_dirty_pages() and is
writer-keyed: the writing task's memcg is looked up under RCU,
and when its dirty+in-flight backlog is below dirty_min the
loop jumps to free_running, bypassing both the global and the
per-memcg BDP gates. dirty_min is an admission guarantee for
the writer's own cgroup, not for inode owners.
- When the clamp engages, mdtc->dirty is replaced with the
memcg-wide NR_FILE_DIRTY + NR_WRITEBACK sum so freerun /
setpoint / rate-limit smoothing see the real backlog and pages
migrating from NR_FILE_DIRTY into NR_WRITEBACK cannot silently
widen the cap.
Fast-path cost when neither knob is set: one rcu_read_lock/unlock
pair plus a READ_ONCE(dirty_min) per balance_dirty_pages()
iteration, and one READ_ONCE(dirty_ratio) per domain_dirty_limits()
call. The memcg counter reads are gated on "knob armed" and do
not fire on the default path. We have not measured the added
cost yet, but we expect it to be in the noise of existing BDP
bookkeeping. A tight pwrite() microbenchmark will confirm this
before a non-RFC posting.
Scope
=====
This series affects the writer-side throttle (balance_dirty_pages())
only. It does not partition the flusher-side writeback queue. A
cgroup's fsync() can still block behind inodes from other cgroups in
writeback_sb_inodes(). We document this limit explicitly and expect
writeback-queue partitioning to be a separate, larger effort.
Interaction with block-layer throttles
======================================
The two knobs are orthogonal to io.max / io.cost. balance_dirty_pages()
runs before the bio reaches the block layer, so dirty_min simply allows
a cgroup to keep accepting write() syscalls up to its reservation; the
actual I/O is still subject to whatever block throttle is in effect. In
the reproducer above, the disk-level bandwidth limit (256 KB/s) is
applied at the QEMU virtio-blk layer, and the protected victim dirties
roughly equal to the rate it can drain, so the block throttle is
exercised on both kernels. We have not yet tested interaction with
guest-side io.max settings; this is on the list before a non-RFC
posting.
Questions for maintainers
=========================
1. Is writer-throttle-only scope (no flusher/writeback-queue work)
acceptable for the first series?
2. Does dirty_ratio belong on struct mem_cgroup (as the prototype
has it) or on the memcg's wb_domain? Routing it through wb_domain
would let us reuse __wb_calc_thresh() and keep all threshold
policy in one place; a possible split is dirty_ratio on wb_domain
(it is threshold policy), dirty_min on the memcg (throttle
bypass). We can go either way.
3. For dirty_min safety caps (per-cgroup and global sum), which
approach do you prefer: reject at write time, or clamp on read in
an effective_dirty_min?
4. Is hierarchical enforcement of dirty_ratio (clamp against
ancestors) required for v1, or can it follow in a subsequent
series?
What's missing before a non-RFC posting
=======================================
- Split the monolithic prototype into a proper series (one patch per
concept + Documentation + selftest).
- Documentation/admin-guide/cgroup-v2.rst entries for both knobs.
- tools/testing/selftests/cgroup/ test for interface surface and
noisy-neighbor protection.
- Implement the per-cgroup and global dirty_min safety caps described
in the memory.dirty_min bullet.
- Fast-path microbenchmark: confirm zero measurable regression for
cgroups that have neither knob set.
- Larger-N validation on real hardware (the current N=5 data is from
a QEMU guest on a throttled virtio-blk).
Follow-ups (out of scope for this series)
=========================================
- memory.dirty_weight: a priority weight knob that scales the BDP
pause length, planned as a separate series. The prototype validated
the interface surface but the application site (post-pause scaling
vs. folding into pos_ratio / dirty_ratelimit) needs to be settled
before we ship it. Happy to discuss in advance of that posting.
- memory.dirty_bytes: a byte-value companion to memory.dirty_ratio,
mirroring the global vm_dirty_bytes. For operators who want a
byte-predictable per-cgroup dirty cap rather than a ratio of the
cgroup's dirtyable memory. We have not prototyped this yet; we
are listing it so reviewers know it is on the roadmap, since
the ratio-only interface omits that use case.
- Writeback-queue partitioning: flusher-side fairness across
cgroups, as noted in Scope above.
Looking forward to feedback.
Thanks,
Alireza Haghdoost <haghdoost@uber.com>
Kshitij Doshi <kshitijd@uber.com>
---
include/linux/memcontrol.h | 10 +++
include/linux/writeback.h | 4 +
include/trace/events/writeback.h | 5 +-
mm/memcontrol.c | 62 ++++++++++++++
mm/page-writeback.c | 179 ++++++++++++++++++++++++++++++++++++---
5 files changed, 249 insertions(+), 11 deletions(-)
diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index dc3fa687759b..45ca949a4c68 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -323,6 +323,16 @@ struct mem_cgroup {
spinlock_t event_list_lock;
#endif /* CONFIG_MEMCG_V1 */
+ /* Per-memcg dirty-page controls (memory.dirty_ratio, memory.dirty_min) */
+ /*
+ * dirty_ratio: [0, 100] percent of dirtyable memory (mdtc->avail),
+ * matching the global vm_dirty_ratio base; 0 inherits the global
+ * threshold.
+ * dirty_min: dirty-page reservation, in pages; 0 disables the bypass.
+ */
+ unsigned int dirty_ratio;
+ unsigned long dirty_min;
+
struct mem_cgroup_per_node *nodeinfo[];
};
diff --git a/include/linux/writeback.h b/include/linux/writeback.h
index 62552a2ce5b9..e37632f728be 100644
--- a/include/linux/writeback.h
+++ b/include/linux/writeback.h
@@ -318,6 +318,10 @@ struct dirty_throttle_control {
unsigned long thresh; /* dirty threshold */
unsigned long bg_thresh; /* dirty background threshold */
unsigned long limit; /* hard dirty limit */
+ unsigned long cg_dirty_cap; /* per-memcg dirty_ratio clamp for
+ * this pass, or PAGE_COUNTER_MAX
+ * when no memcg clamp applies
+ */
unsigned long wb_dirty; /* per-wb counterparts */
unsigned long wb_thresh;
diff --git a/include/trace/events/writeback.h b/include/trace/events/writeback.h
index bdac0d685a98..0bf86b3c903c 100644
--- a/include/trace/events/writeback.h
+++ b/include/trace/events/writeback.h
@@ -672,6 +672,7 @@ TRACE_EVENT(balance_dirty_pages,
__array( char, bdi, 32)
__field(u64, cgroup_ino)
__field(unsigned long, limit)
+ __field(unsigned long, cg_dirty_cap)
__field(unsigned long, setpoint)
__field(unsigned long, dirty)
__field(unsigned long, wb_setpoint)
@@ -691,6 +692,7 @@ TRACE_EVENT(balance_dirty_pages,
strscpy_pad(__entry->bdi, bdi_dev_name(wb->bdi), 32);
__entry->limit = dtc->limit;
+ __entry->cg_dirty_cap = dtc->cg_dirty_cap;
__entry->setpoint = (dtc->limit + freerun) / 2;
__entry->dirty = dtc->dirty;
__entry->wb_setpoint = __entry->setpoint *
@@ -710,13 +712,14 @@ TRACE_EVENT(balance_dirty_pages,
TP_printk("bdi %s: "
- "limit=%lu setpoint=%lu dirty=%lu "
+ "limit=%lu cg_dirty_cap=%lu setpoint=%lu dirty=%lu "
"wb_setpoint=%lu wb_dirty=%lu "
"dirty_ratelimit=%lu task_ratelimit=%lu "
"dirtied=%u dirtied_pause=%u "
"paused=%lu pause=%ld period=%lu think=%ld cgroup_ino=%llu",
__entry->bdi,
__entry->limit,
+ __entry->cg_dirty_cap,
__entry->setpoint,
__entry->dirty,
__entry->wb_setpoint,
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index c3d98ab41f1f..c43fe4f394eb 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -4748,6 +4748,56 @@ static ssize_t memory_max_write(struct kernfs_open_file *of,
return nbytes;
}
+static int memory_dirty_ratio_show(struct seq_file *m, void *v)
+{
+ struct mem_cgroup *memcg = mem_cgroup_from_seq(m);
+
+ seq_printf(m, "%u\n", READ_ONCE(memcg->dirty_ratio));
+ return 0;
+}
+
+static ssize_t memory_dirty_ratio_write(struct kernfs_open_file *of,
+ char *buf, size_t nbytes, loff_t off)
+{
+ struct mem_cgroup *memcg = mem_cgroup_from_css(of_css(of));
+ unsigned int ratio;
+ int err;
+
+ err = kstrtouint(strstrip(buf), 0, &ratio);
+ if (err)
+ return err;
+
+ if (ratio > 100)
+ return -EINVAL;
+
+ WRITE_ONCE(memcg->dirty_ratio, ratio);
+ return nbytes;
+}
+
+static int memory_dirty_min_show(struct seq_file *m, void *v)
+{
+ struct mem_cgroup *memcg = mem_cgroup_from_seq(m);
+
+ /* seq_puts_memcg_tunable automatically multiplies by PAGE_SIZE for the user */
+ return seq_puts_memcg_tunable(m, READ_ONCE(memcg->dirty_min));
+}
+
+static ssize_t memory_dirty_min_write(struct kernfs_open_file *of,
+ char *buf, size_t nbytes, loff_t off)
+{
+ struct mem_cgroup *memcg = mem_cgroup_from_css(of_css(of));
+ unsigned long dirty_min;
+ int err;
+
+ /* page_counter_memparse converts strings like "512M" into a page count */
+ err = page_counter_memparse(strstrip(buf), "max", &dirty_min);
+ if (err)
+ return err;
+
+ WRITE_ONCE(memcg->dirty_min, dirty_min);
+ return nbytes;
+}
+
/*
* Note: don't forget to update the 'samples/cgroup/memcg_event_listener'
* if any new events become available.
@@ -4950,6 +5000,18 @@ static struct cftype memory_files[] = {
.flags = CFTYPE_NS_DELEGATABLE,
.write = memory_reclaim,
},
+ {
+ .name = "dirty_ratio",
+ .flags = CFTYPE_NOT_ON_ROOT,
+ .seq_show = memory_dirty_ratio_show,
+ .write = memory_dirty_ratio_write,
+ },
+ {
+ .name = "dirty_min",
+ .flags = CFTYPE_NOT_ON_ROOT,
+ .seq_show = memory_dirty_min_show,
+ .write = memory_dirty_min_write,
+ },
{ } /* terminate */
};
diff --git a/mm/page-writeback.c b/mm/page-writeback.c
index 88cd53d4ba09..2847b2c1e59a 100644
--- a/mm/page-writeback.c
+++ b/mm/page-writeback.c
@@ -124,14 +124,17 @@ struct wb_domain global_wb_domain;
#define GDTC_INIT(__wb) .wb = (__wb), \
.dom = &global_wb_domain, \
- .wb_completions = &(__wb)->completions
+ .wb_completions = &(__wb)->completions, \
+ .cg_dirty_cap = PAGE_COUNTER_MAX
-#define GDTC_INIT_NO_WB .dom = &global_wb_domain
+#define GDTC_INIT_NO_WB .dom = &global_wb_domain, \
+ .cg_dirty_cap = PAGE_COUNTER_MAX
#define MDTC_INIT(__wb, __gdtc) .wb = (__wb), \
.dom = mem_cgroup_wb_domain(__wb), \
.wb_completions = &(__wb)->memcg_completions, \
- .gdtc = __gdtc
+ .gdtc = __gdtc, \
+ .cg_dirty_cap = PAGE_COUNTER_MAX
static bool mdtc_valid(struct dirty_throttle_control *dtc)
{
@@ -183,8 +186,9 @@ static void wb_min_max_ratio(struct bdi_writeback *wb,
#else /* CONFIG_CGROUP_WRITEBACK */
#define GDTC_INIT(__wb) .wb = (__wb), \
- .wb_completions = &(__wb)->completions
-#define GDTC_INIT_NO_WB
+ .wb_completions = &(__wb)->completions, \
+ .cg_dirty_cap = PAGE_COUNTER_MAX
+#define GDTC_INIT_NO_WB .cg_dirty_cap = PAGE_COUNTER_MAX
#define MDTC_INIT(__wb, __gdtc)
static bool mdtc_valid(struct dirty_throttle_control *dtc)
@@ -392,6 +396,58 @@ static void domain_dirty_limits(struct dirty_throttle_control *dtc)
bg_thresh += bg_thresh / 4 + global_wb_domain.dirty_limit / 32;
thresh += thresh / 4 + global_wb_domain.dirty_limit / 32;
}
+
+ /*
+ * Apply the per-memcg dirty_ratio clamp on mdtc (gdtc != NULL
+ * iff @dtc is a memcg dtc). dirty_ratio is scaled against
+ * the memcg's own dirtyable memory (@available_memory), matching
+ * the semantics of vm_dirty_ratio so the two knobs share a base
+ * and compose via a plain min() on thresh. The clamp is keyed
+ * on wb->memcg_css (the inode-owner's memcg) rather than on
+ * current's memcg, so balance_dirty_pages(), wb_over_bg_thresh()
+ * (flusher kworker context), and cgwb_calc_thresh() all see the
+ * same clamped value.
+ *
+ * Published on dtc->cg_dirty_cap as well so hard_dirty_limit()
+ * callers in balance_dirty_pages() can ignore the slower
+ * dom->dirty_limit smoothing when deriving setpoint/
+ * rate-limit from the clamped ceiling.
+ *
+ * Clamp is applied after the rt/dl boost: dirty_ratio is a
+ * strict override, not widened by priority. bg_thresh is
+ * scaled by the same factor we apply to thresh so the
+ * user-configured bg/thresh ratio survives clamping instead
+ * of snapping to thresh/2 via the bg_thresh >= thresh guard
+ * below. mult_frac() preserves precision for small memcgs
+ * where a plain "(avail / 100) * ratio" would collapse to 0.
+ */
+ if (gdtc) {
+ struct mem_cgroup *memcg =
+ mem_cgroup_from_css(dtc->wb->memcg_css);
+ unsigned int cg_ratio = memcg ?
+ READ_ONCE(memcg->dirty_ratio) : 0;
+
+ /*
+ * dtc is reused across balance_dirty_pages() iterations,
+ * so reset the published clamp every call -- an admin
+ * clearing memory.dirty_ratio mid-flight must take effect
+ * on the next pass.
+ */
+ dtc->cg_dirty_cap = PAGE_COUNTER_MAX;
+
+ if (cg_ratio) {
+ unsigned long cg_thresh = mult_frac(available_memory,
+ cg_ratio, 100);
+
+ if (cg_thresh < thresh) {
+ bg_thresh = mult_frac(bg_thresh, cg_thresh,
+ thresh);
+ thresh = cg_thresh;
+ dtc->cg_dirty_cap = cg_thresh;
+ }
+ }
+ }
+
/*
* Dirty throttling logic assumes the limits in page units fit into
* 32-bits. This gives 16TB dirty limits max which is hopefully enough.
@@ -1065,7 +1121,9 @@ static void wb_position_ratio(struct dirty_throttle_control *dtc)
struct bdi_writeback *wb = dtc->wb;
unsigned long write_bw = READ_ONCE(wb->avg_write_bandwidth);
unsigned long freerun = dirty_freerun_ceiling(dtc->thresh, dtc->bg_thresh);
- unsigned long limit = dtc->limit = hard_dirty_limit(dtc_dom(dtc), dtc->thresh);
+ unsigned long limit = dtc->limit = min(hard_dirty_limit(dtc_dom(dtc),
+ dtc->thresh),
+ dtc->cg_dirty_cap);
unsigned long wb_thresh = dtc->wb_thresh;
unsigned long x_intercept;
unsigned long setpoint; /* dirty pages' target balance point */
@@ -1334,7 +1392,8 @@ static void wb_update_dirty_ratelimit(struct dirty_throttle_control *dtc,
struct bdi_writeback *wb = dtc->wb;
unsigned long dirty = dtc->dirty;
unsigned long freerun = dirty_freerun_ceiling(dtc->thresh, dtc->bg_thresh);
- unsigned long limit = hard_dirty_limit(dtc_dom(dtc), dtc->thresh);
+ unsigned long limit = min(hard_dirty_limit(dtc_dom(dtc), dtc->thresh),
+ dtc->cg_dirty_cap);
unsigned long setpoint = (freerun + limit) / 2;
unsigned long write_bw = wb->avg_write_bandwidth;
unsigned long dirty_ratelimit = wb->dirty_ratelimit;
@@ -1822,22 +1881,122 @@ static int balance_dirty_pages(struct bdi_writeback *wb,
int ret = 0;
for (;;) {
+ unsigned long cg_dirty_min = 0;
+ unsigned long cg_dirty_pages = 0;
unsigned long now = jiffies;
nr_dirty = global_node_page_state(NR_FILE_DIRTY);
balance_domain_limits(gdtc, strictlimit);
+
+ /*
+ * Under RCU, snapshot the current memcg's memory.dirty_min
+ * reservation. When it is non-zero, also snapshot the
+ * memcg-wide dirty backlog. These feed the per-writer
+ * dirty_min bypass below; the dirty_ratio clamp itself
+ * is applied inside domain_dirty_limits() keyed on
+ * wb->memcg_css so balance_dirty_pages(),
+ * wb_over_bg_thresh() (flusher kworker context), and
+ * cgwb_calc_thresh() all see a consistent clamped
+ * threshold.
+ *
+ * rcu_read_lock() is held only for the __rcu dereference
+ * of current->cgroups; the memcg pointer does not escape
+ * the critical section. The counter read matches
+ * domain_dirty_avail(mdtc, true) so the bypass compares
+ * the same dirty+in-flight backlog the global path uses.
+ */
+ rcu_read_lock();
+ {
+ struct mem_cgroup *memcg =
+ mem_cgroup_from_task(current);
+
+ if (memcg) {
+ cg_dirty_min = READ_ONCE(memcg->dirty_min);
+ if (cg_dirty_min)
+ cg_dirty_pages =
+ memcg_page_state(memcg,
+ NR_FILE_DIRTY) +
+ memcg_page_state(memcg,
+ NR_WRITEBACK);
+ }
+ }
+ rcu_read_unlock();
+
if (mdtc) {
/*
- * If @wb belongs to !root memcg, repeat the same
- * basic calculations for the memcg domain.
+ * For !root memcg, repeat the same three-step
+ * sequence as balance_domain_limits(gdtc):
+ * avail -> limits -> freerun. We inline it here
+ * so we can insert the mdtc->dirty override
+ * between step 2 (domain_dirty_limits, which
+ * publishes the per-memcg dirty_ratio clamp on
+ * cg_dirty_cap) and step 3 (domain_dirty_freerun,
+ * which consumes mdtc->dirty along with
+ * thresh/bg_thresh).
+ */
+ domain_dirty_avail(mdtc, true);
+ domain_dirty_limits(mdtc);
+
+ /*
+ * When the dirty_ratio clamp engaged, replace the
+ * per-wb dirty count from mem_cgroup_wb_stats()
+ * with the memcg-wide NR_FILE_DIRTY + NR_WRITEBACK
+ * sum so freerun, the setpoint, and the rate-limit
+ * smoothing see the true memcg backlog instead of
+ * the subset that has migrated to this cgwb (cgwb
+ * migration is lazy and can lag by many seconds),
+ * and so a burst of buffered writes cannot silently
+ * bypass the clamp by shifting pages from
+ * NR_FILE_DIRTY into NR_WRITEBACK.
+ *
+ * Keyed on wb->memcg_css to match the clamp itself.
+ * The cgwb holds a css reference, so the memcg
+ * pointer is stable without additional locking.
+ *
+ * Caveat: memcg_page_state() aggregates across ALL
+ * backing devices owned by this memcg, while mdtc
+ * is scoped to one wb. A writer to a fast BDI may
+ * observe backlog accumulated on slow BDIs in the
+ * same memcg and throttle more than strictly needed.
+ * Accepted for v1; the alternative (summing per-wb
+ * dirty across the memcg's cgwbs) walks the cgwb
+ * list under RCU on a hot path.
*/
- balance_domain_limits(mdtc, strictlimit);
+ if (mdtc->cg_dirty_cap != PAGE_COUNTER_MAX) {
+ struct mem_cgroup *wb_memcg =
+ mem_cgroup_from_css(mdtc->wb->memcg_css);
+
+ if (wb_memcg)
+ mdtc->dirty =
+ memcg_page_state(wb_memcg,
+ NR_FILE_DIRTY) +
+ memcg_page_state(wb_memcg,
+ NR_WRITEBACK);
+ }
+
+ domain_dirty_freerun(mdtc, strictlimit);
}
if (nr_dirty > gdtc->bg_thresh && !writeback_in_progress(wb))
wb_start_background_writeback(wb);
+ /*
+ * dirty_min bypass: when the current memcg's dirty+in-flight
+ * backlog is below its memory.dirty_min reservation, let the
+ * writer proceed without throttling. This check must live
+ * outside the if (mdtc) block because a writer's file may not
+ * yet have been migrated to a cgwb; without cgwb, mdtc is NULL
+ * and the per-memcg block above is skipped entirely.
+ *
+ * cg_dirty_min and cg_dirty_pages come from the per-iteration
+ * snapshot taken above under rcu_read_lock; both are stored
+ * in pages (page_counter_memparse converts bytes -> pages for
+ * dirty_min), so no unit conversion is needed.
+ */
+ if (cg_dirty_min && cg_dirty_pages < cg_dirty_min)
+ goto free_running;
+
/*
* If memcg domain is in effect, @dirty should be under
* both global and memcg freerun ceilings.
---
base-commit: 254f49634ee16a731174d2ae34bc50bd5f45e731
change-id: 20260501-rfc-memcg-dirty-v1-ed4644c3fa8a
Best regards,
--
Alireza Haghdoost <haghdoost@uber.com>
^ permalink raw reply related
* [PATCH v2 cgroup/for-7.1-fixes sched_ext/for-7.1-fixes] cgroup: Defer css percpu_ref kill on rmdir until cgroup is depopulated
From: Tejun Heo @ 2026-05-01 18:31 UTC (permalink / raw)
To: Martin Pitt
Cc: regressions, cgroups, Johannes Weiner, Michal Koutný,
Sebastian Andrzej Siewior, David Vernet, Andrea Righi,
Changwoo Min, Emil Tsalapatis, sched-ext, linux-kernel
A chain of commits going back to v7.0 reworked rmdir to satisfy the
controller invariant that a subsystem's ->css_offline() must not run while
tasks are still doing kernel-side work in the cgroup.
[1] d245698d727a ("cgroup: Defer task cgroup unlink until after the task is done switching out")
[2] a72f73c4dd9b ("cgroup: Don't expose dead tasks in cgroup")
[3] 1b164b876c36 ("cgroup: Wait for dying tasks to leave on rmdir")
[4] 4c56a8ac6869 ("cgroup: Fix cgroup_drain_dying() testing the wrong condition")
[5] 13e786b64bd3 ("cgroup: Increment nr_dying_subsys_* from rmdir context")
[1] moved task cset unlink from do_exit() to finish_task_switch() so a
task's cset link drops only after the task has fully stopped scheduling.
That made tasks past exit_signals() linger on cset->tasks until their final
context switch, which led to a series of problems as what userspace expected
to see after rmdir diverged from what the kernel needs to wait for. [2]-[5]
tried to bridge that divergence: [2] filtered the exiting tasks from
cgroup.procs; [3] had rmdir(2) sleep in TASK_UNINTERRUPTIBLE for them; [4]
fixed the wait's condition; [5] made nr_dying_subsys_* visible
synchronously.
The cgroup_drain_dying() wait in [3] turned out to be a dead end. When the
rmdir caller is also the reaper of a zombie that pins a pidns teardown (e.g.
host PID 1 systemd reaping orphan pids that were re-parented to it during
the same teardown), rmdir blocks in TASK_UNINTERRUPTIBLE waiting for those
pids to free, the pids can't free because PID 1 is the reaper and it's stuck
in rmdir, and the system A-A deadlocks. No internal lock ordering breaks
this; the wait itself is the bug.
The css killing side that drove the original reorder, however, can be made
cleanly asynchronous: ->css_offline() is already async, run from
css_killed_work_fn() driven by percpu_ref_kill_and_confirm(). The fix is to
make that chain start only after all tasks have left the cgroup. rmdir's
user-visible side then returns as soon as cgroup.procs and friends are
empty, while ->css_offline() still runs only after the cgroup is fully
drained.
Verified by the original reproducer (pidns teardown + zombie reaper, runs
under vng) which hangs vanilla and succeeds here, and by per-commit
deterministic repros for [2], [3], [4], [5] with a boot parameter that
widens the post-exit_signals() window so each state is reliably reachable.
Some stress tests on top of that.
cgroup_apply_control_disable() has the same shape of pre-existing race:
when a controller is disabled via subtree_control, kill_css() ran
synchronously while tasks past exit_signals() could still be linked to
the cgroup's csets, and ->css_offline() could fire before they drained.
This patch preserves the existing synchronous behavior at that call site
(kill_css_sync() + kill_css_finish() back-to-back) and a follow-up patch
will defer kill_css_finish() there using a per-css trigger.
This seems like the right approach and I don't see problems with it. The
changes are somewhat invasive but not excessively so, so backporting to
-stable should be okay. If something does turn out to be wrong, the fallback
is to revert the entire chain ([1]-[5]) and rework in the development branch
instead.
v2: Pin cgrp across the deferred destroy work with explicit
cgroup_get()/cgroup_put() around queue_work() and the work_fn. v1
wasn't actually broken (ordered cgroup_offline_wq + queue_work order
in cgroup_task_dead() saved it) but the explicit ref removes the
dependency on those non-obvious invariants. Also note the
pre-existing cgroup_apply_control_disable() race in the description;
a follow-up will defer kill_css_finish() there.
Fixes: 1b164b876c36 ("cgroup: Wait for dying tasks to leave on rmdir")
Cc: stable@vger.kernel.org # v7.0+
Reported-by: Martin Pitt <martin@piware.de>
Link: https://lore.kernel.org/all/afHNg2VX2jy9bW7y@piware.de/
Link: https://lore.kernel.org/all/35e0670adb4abeab13da2c321582af9f@kernel.org/
Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
Hello Martin,
Could you give v2 a try? Same defer-the-percpu_ref-kill mechanism as
v1, with an explicit cgroup_get/put around the deferred work to make
the lifetime invariant obvious (Sashiko bot review on v1; v1 wasn't
broken but the explicit ref removes a dependency on non-obvious
ordering). Fix should behave identically to v1 for your reproducer.
Thanks.
include/linux/cgroup-defs.h | 4
kernel/cgroup/cgroup.c | 250 ++++++++++++++++++++------------------------
2 files changed, 119 insertions(+), 135 deletions(-)
--- a/include/linux/cgroup-defs.h
+++ b/include/linux/cgroup-defs.h
@@ -611,8 +611,8 @@ struct cgroup {
/* used to wait for offlining of csses */
wait_queue_head_t offline_waitq;
- /* used by cgroup_rmdir() to wait for dying tasks to leave */
- wait_queue_head_t dying_populated_waitq;
+ /* defers killing csses after removal until cgroup is depopulated */
+ struct work_struct finish_destroy_work;
/* used to schedule release agent */
struct work_struct release_agent_work;
--- a/kernel/cgroup/cgroup.c
+++ b/kernel/cgroup/cgroup.c
@@ -264,10 +264,12 @@ static void cgroup_finalize_control(stru
static void css_task_iter_skip(struct css_task_iter *it,
struct task_struct *task);
static int cgroup_destroy_locked(struct cgroup *cgrp);
+static void cgroup_finish_destroy(struct cgroup *cgrp);
+static void kill_css_sync(struct cgroup_subsys_state *css);
+static void kill_css_finish(struct cgroup_subsys_state *css);
static struct cgroup_subsys_state *css_create(struct cgroup *cgrp,
struct cgroup_subsys *ss);
static void css_release(struct percpu_ref *ref);
-static void kill_css(struct cgroup_subsys_state *css);
static int cgroup_addrm_files(struct cgroup_subsys_state *css,
struct cgroup *cgrp, struct cftype cfts[],
bool is_add);
@@ -797,6 +799,16 @@ static void cgroup_update_populated(stru
if (was_populated == cgroup_is_populated(cgrp))
break;
+ /*
+ * Subtree just emptied below an offlined cgrp. Fire deferred
+ * destroy. The transition is one-shot.
+ */
+ if (was_populated && !css_is_online(&cgrp->self)) {
+ cgroup_get(cgrp);
+ WARN_ON_ONCE(!queue_work(cgroup_offline_wq,
+ &cgrp->finish_destroy_work));
+ }
+
cgroup1_check_for_release(cgrp);
TRACE_CGROUP_PATH(notify_populated, cgrp,
cgroup_is_populated(cgrp));
@@ -2039,6 +2051,16 @@ static int cgroup_reconfigure(struct fs_
return 0;
}
+static void cgroup_finish_destroy_work_fn(struct work_struct *work)
+{
+ struct cgroup *cgrp = container_of(work, struct cgroup, finish_destroy_work);
+
+ cgroup_lock();
+ cgroup_finish_destroy(cgrp);
+ cgroup_unlock();
+ cgroup_put(cgrp);
+}
+
static void init_cgroup_housekeeping(struct cgroup *cgrp)
{
struct cgroup_subsys *ss;
@@ -2065,7 +2087,7 @@ static void init_cgroup_housekeeping(str
#endif
init_waitqueue_head(&cgrp->offline_waitq);
- init_waitqueue_head(&cgrp->dying_populated_waitq);
+ INIT_WORK(&cgrp->finish_destroy_work, cgroup_finish_destroy_work_fn);
INIT_WORK(&cgrp->release_agent_work, cgroup1_release_agent);
}
@@ -3375,7 +3397,8 @@ static void cgroup_apply_control_disable
if (css->parent &&
!(cgroup_ss_mask(dsct) & (1 << ss->id))) {
- kill_css(css);
+ kill_css_sync(css);
+ kill_css_finish(css);
} else if (!css_visible(css)) {
css_clear_dir(css);
if (ss->css_reset)
@@ -5514,7 +5537,7 @@ static struct cftype cgroup_psi_files[]
* css destruction is four-stage process.
*
* 1. Destruction starts. Killing of the percpu_ref is initiated.
- * Implemented in kill_css().
+ * Implemented in kill_css_finish().
*
* 2. When the percpu_ref is confirmed to be visible as killed on all CPUs
* and thus css_tryget_online() is guaranteed to fail, the css can be
@@ -5993,7 +6016,7 @@ out_unlock:
/*
* This is called when the refcnt of a css is confirmed to be killed.
* css_tryget_online() is now guaranteed to fail. Tell the subsystem to
- * initiate destruction and put the css ref from kill_css().
+ * initiate destruction and put the css ref from kill_css_finish().
*/
static void css_killed_work_fn(struct work_struct *work)
{
@@ -6025,15 +6048,12 @@ static void css_killed_ref_fn(struct per
}
/**
- * kill_css - destroy a css
- * @css: css to destroy
+ * kill_css_sync - synchronous half of css teardown
+ * @css: css being killed
*
- * This function initiates destruction of @css by removing cgroup interface
- * files and putting its base reference. ->css_offline() will be invoked
- * asynchronously once css_tryget_online() is guaranteed to fail and when
- * the reference count reaches zero, @css will be released.
+ * See cgroup_destroy_locked().
*/
-static void kill_css(struct cgroup_subsys_state *css)
+static void kill_css_sync(struct cgroup_subsys_state *css)
{
struct cgroup_subsys *ss = css->ss;
@@ -6056,24 +6076,6 @@ static void kill_css(struct cgroup_subsy
*/
css_clear_dir(css);
- /*
- * Killing would put the base ref, but we need to keep it alive
- * until after ->css_offline().
- */
- css_get(css);
-
- /*
- * cgroup core guarantees that, by the time ->css_offline() is
- * invoked, no new css reference will be given out via
- * css_tryget_online(). We can't simply call percpu_ref_kill() and
- * proceed to offlining css's because percpu_ref_kill() doesn't
- * guarantee that the ref is seen as killed on all CPUs on return.
- *
- * Use percpu_ref_kill_and_confirm() to get notifications as each
- * css is confirmed to be seen as killed on all CPUs.
- */
- percpu_ref_kill_and_confirm(&css->refcnt, css_killed_ref_fn);
-
css->cgroup->nr_dying_subsys[ss->id]++;
/*
* Parent css and cgroup cannot be freed until after the freeing
@@ -6086,44 +6088,88 @@ static void kill_css(struct cgroup_subsy
}
/**
- * cgroup_destroy_locked - the first stage of cgroup destruction
+ * kill_css_finish - deferred half of css teardown
+ * @css: css being killed
+ *
+ * See cgroup_destroy_locked().
+ */
+static void kill_css_finish(struct cgroup_subsys_state *css)
+{
+ lockdep_assert_held(&cgroup_mutex);
+
+ /*
+ * Skip on re-entry: cgroup_apply_control_disable() may have killed @css
+ * earlier. cgroup_destroy_locked() can still walk it because
+ * offline_css() (which NULLs cgrp->subsys[ssid]) runs async.
+ */
+ if (percpu_ref_is_dying(&css->refcnt))
+ return;
+
+ /*
+ * Killing would put the base ref, but we need to keep it alive until
+ * after ->css_offline().
+ */
+ css_get(css);
+
+ /*
+ * cgroup core guarantees that, by the time ->css_offline() is invoked,
+ * no new css reference will be given out via css_tryget_online(). We
+ * can't simply call percpu_ref_kill() and proceed to offlining css's
+ * because percpu_ref_kill() doesn't guarantee that the ref is seen as
+ * killed on all CPUs on return.
+ *
+ * Use percpu_ref_kill_and_confirm() to get notifications as each css is
+ * confirmed to be seen as killed on all CPUs.
+ */
+ percpu_ref_kill_and_confirm(&css->refcnt, css_killed_ref_fn);
+}
+
+/**
+ * cgroup_destroy_locked - destroy @cgrp (called on rmdir)
* @cgrp: cgroup to be destroyed
*
- * css's make use of percpu refcnts whose killing latency shouldn't be
- * exposed to userland and are RCU protected. Also, cgroup core needs to
- * guarantee that css_tryget_online() won't succeed by the time
- * ->css_offline() is invoked. To satisfy all the requirements,
- * destruction is implemented in the following two steps.
- *
- * s1. Verify @cgrp can be destroyed and mark it dying. Remove all
- * userland visible parts and start killing the percpu refcnts of
- * css's. Set up so that the next stage will be kicked off once all
- * the percpu refcnts are confirmed to be killed.
- *
- * s2. Invoke ->css_offline(), mark the cgroup dead and proceed with the
- * rest of destruction. Once all cgroup references are gone, the
- * cgroup is RCU-freed.
- *
- * This function implements s1. After this step, @cgrp is gone as far as
- * the userland is concerned and a new cgroup with the same name may be
- * created. As cgroup doesn't care about the names internally, this
- * doesn't cause any problem.
+ * Tear down @cgrp on behalf of rmdir. Constraints:
+ *
+ * - Userspace: rmdir must succeed when cgroup.procs and friends are empty.
+ *
+ * - Kernel: subsystem ->css_offline() must not run while any task in @cgrp's
+ * subtree is still doing kernel work. A task hidden from cgroup.procs (past
+ * exit_signals() with signal->live cleared) can still schedule, allocate, and
+ * consume resources until its final context switch. Dying descendants in the
+ * subtree can host such tasks too.
+ *
+ * - Kernel: css_tryget_online() must fail by the time ->css_offline() runs.
+ *
+ * The destruction runs in three parts:
+ *
+ * - This function: synchronous user-visible state teardown plus kill_css_sync()
+ * on each subsystem css.
+ *
+ * - cgroup_finish_destroy(): kicks the percpu_ref kill via kill_css_finish() on
+ * each subsystem css. Fires once @cgrp's subtree is fully drained, either
+ * inline here or from cgroup_update_populated().
+ *
+ * - The percpu_ref kill chain: css_killed_ref_fn -> css_killed_work_fn ->
+ * ->css_offline() -> release/free.
+ *
+ * Return 0 on success, -EBUSY if a userspace-visible task or an online child
+ * remains.
*/
static int cgroup_destroy_locked(struct cgroup *cgrp)
- __releases(&cgroup_mutex) __acquires(&cgroup_mutex)
{
struct cgroup *tcgrp, *parent = cgroup_parent(cgrp);
struct cgroup_subsys_state *css;
struct cgrp_cset_link *link;
+ struct css_task_iter it;
+ struct task_struct *task;
int ssid, ret;
lockdep_assert_held(&cgroup_mutex);
- /*
- * Only migration can raise populated from zero and we're already
- * holding cgroup_mutex.
- */
- if (cgroup_is_populated(cgrp))
+ css_task_iter_start(&cgrp->self, 0, &it);
+ task = css_task_iter_next(&it);
+ css_task_iter_end(&it);
+ if (task)
return -EBUSY;
/*
@@ -6147,9 +6193,8 @@ static int cgroup_destroy_locked(struct
link->cset->dead = true;
spin_unlock_irq(&css_set_lock);
- /* initiate massacre of all css's */
for_each_css(css, ssid, cgrp)
- kill_css(css);
+ kill_css_sync(css);
/* clear and remove @cgrp dir, @cgrp has an extra ref on its kn */
css_clear_dir(&cgrp->self);
@@ -6180,79 +6225,27 @@ static int cgroup_destroy_locked(struct
/* put the base reference */
percpu_ref_kill(&cgrp->self.refcnt);
+ if (!cgroup_is_populated(cgrp))
+ cgroup_finish_destroy(cgrp);
+
return 0;
};
/**
- * cgroup_drain_dying - wait for dying tasks to leave before rmdir
- * @cgrp: the cgroup being removed
+ * cgroup_finish_destroy - deferred half of @cgrp destruction
+ * @cgrp: cgroup whose subtree just became empty
*
- * cgroup.procs and cgroup.threads use css_task_iter which filters out
- * PF_EXITING tasks so that userspace doesn't see tasks that have already been
- * reaped via waitpid(). However, cgroup_has_tasks() - which tests whether the
- * cgroup has non-empty css_sets - is only updated when dying tasks pass through
- * cgroup_task_dead() in finish_task_switch(). This creates a window where
- * cgroup.procs reads empty but cgroup_has_tasks() is still true, making rmdir
- * fail with -EBUSY from cgroup_destroy_locked() even though userspace sees no
- * tasks.
- *
- * This function aligns cgroup_has_tasks() with what userspace can observe. If
- * cgroup_has_tasks() but the task iterator sees nothing (all remaining tasks are
- * PF_EXITING), we wait for cgroup_task_dead() to finish processing them. As the
- * window between PF_EXITING and cgroup_task_dead() is short, the wait is brief.
- *
- * This function only concerns itself with this cgroup's own dying tasks.
- * Whether the cgroup has children is cgroup_destroy_locked()'s problem.
- *
- * Each cgroup_task_dead() kicks the waitqueue via cset->cgrp_links, and we
- * retry the full check from scratch.
- *
- * Must be called with cgroup_mutex held.
+ * See cgroup_destroy_locked() for the rationale.
*/
-static int cgroup_drain_dying(struct cgroup *cgrp)
- __releases(&cgroup_mutex) __acquires(&cgroup_mutex)
+static void cgroup_finish_destroy(struct cgroup *cgrp)
{
- struct css_task_iter it;
- struct task_struct *task;
- DEFINE_WAIT(wait);
+ struct cgroup_subsys_state *css;
+ int ssid;
lockdep_assert_held(&cgroup_mutex);
-retry:
- if (!cgroup_has_tasks(cgrp))
- return 0;
- /* Same iterator as cgroup.threads - if any task is visible, it's busy */
- css_task_iter_start(&cgrp->self, 0, &it);
- task = css_task_iter_next(&it);
- css_task_iter_end(&it);
-
- if (task)
- return -EBUSY;
-
- /*
- * All remaining tasks are PF_EXITING and will pass through
- * cgroup_task_dead() shortly. Wait for a kick and retry.
- *
- * cgroup_has_tasks() can't transition from false to true while we're
- * holding cgroup_mutex, but the true to false transition happens
- * under css_set_lock (via cgroup_task_dead()). We must retest and
- * prepare_to_wait() under css_set_lock. Otherwise, the transition
- * can happen between our first test and prepare_to_wait(), and we
- * sleep with no one to wake us.
- */
- spin_lock_irq(&css_set_lock);
- if (!cgroup_has_tasks(cgrp)) {
- spin_unlock_irq(&css_set_lock);
- return 0;
- }
- prepare_to_wait(&cgrp->dying_populated_waitq, &wait,
- TASK_UNINTERRUPTIBLE);
- spin_unlock_irq(&css_set_lock);
- mutex_unlock(&cgroup_mutex);
- schedule();
- finish_wait(&cgrp->dying_populated_waitq, &wait);
- mutex_lock(&cgroup_mutex);
- goto retry;
+ for_each_css(css, ssid, cgrp)
+ kill_css_finish(css);
}
int cgroup_rmdir(struct kernfs_node *kn)
@@ -6264,12 +6257,9 @@ int cgroup_rmdir(struct kernfs_node *kn)
if (!cgrp)
return 0;
- ret = cgroup_drain_dying(cgrp);
- if (!ret) {
- ret = cgroup_destroy_locked(cgrp);
- if (!ret)
- TRACE_CGROUP_PATH(rmdir, cgrp);
- }
+ ret = cgroup_destroy_locked(cgrp);
+ if (!ret)
+ TRACE_CGROUP_PATH(rmdir, cgrp);
cgroup_kn_unlock(kn);
return ret;
@@ -7029,7 +7019,6 @@ void cgroup_task_exit(struct task_struct
static void do_cgroup_task_dead(struct task_struct *tsk)
{
- struct cgrp_cset_link *link;
struct css_set *cset;
unsigned long flags;
@@ -7043,11 +7032,6 @@ static void do_cgroup_task_dead(struct t
if (thread_group_leader(tsk) && atomic_read(&tsk->signal->live))
list_add_tail(&tsk->cg_list, &cset->dying_tasks);
- /* kick cgroup_drain_dying() waiters, see cgroup_rmdir() */
- list_for_each_entry(link, &cset->cgrp_links, cgrp_link)
- if (waitqueue_active(&link->cgrp->dying_populated_waitq))
- wake_up(&link->cgrp->dying_populated_waitq);
-
if (dl_task(tsk))
dec_dl_tasks_cs(tsk);
^ permalink raw reply
* Re: [PATCH v5 00/21] Virtual Swap Space
From: Nhat Pham @ 2026-05-01 14:16 UTC (permalink / raw)
To: Kairui Song
Cc: Yosry Ahmed, Liam R . Howlett, akpm, Alistair Popple,
Axel Rasmussen, Barry Song, Baolin Wang, Baoquan He,
Byungchul Park,
open list:CONTROL GROUP - MEMORY RESOURCE CONTROLLER (MEMCG),
Chengming Zhou, Chris Li, Jonathan Corbet, David Hildenbrand,
Dev Jain, Gregory Price, Johannes Weiner, Hugh Dickins, Jann Horn,
Joshua Hahn, Lance Yang, lenb, linux-doc, LKML, linux-mm,
open list:SUSPEND TO RAM, Lorenzo Stoakes, Matthew Brost,
Michal Hocko, Muchun Song, Mariano Pache, Pavel Machek, Peter Xu,
Peter Zijlstra, Pedro Falcato, Rafael J. Wysocki (Intel),
Rakie Kim, Roman Gushchin, Mike Rapoport, Ryan Roberts,
Shakeel Butt, Kemeng Shi, Suren Baghdasaryan, tglx,
Vlastimil Babka, Wei Xu, Huang, Ying, Yosry Ahmed, Yuanchu Xie,
Qi Zheng, Zi Yan, Meta kernel team, Rik van Riel
In-Reply-To: <CAMgjq7A4+Sac9-CYkig1LFfEh5rq-4vLka8AXREei_m3svzJ7w@mail.gmail.com>
On Fri, Apr 24, 2026 at 8:52 PM Kairui Song <ryncsn@gmail.com> wrote:
>
> On Sat, Apr 25, 2026 at 3:12 AM Yosry Ahmed <yosry@kernel.org> wrote
> > > https://lore.kernel.org/linux-mm/20260421055323.940344-1-youngjun.park@lge.com/
> >
> > Does this do promotion/demotion of swap entries?
>
> Not yet, let's do things step by step.
>
> > > For example just reserve a type (e.g. type 0) as the virtual type?
> > > (type is really a bad naming though).
> > >
> > > The that swap file (or swap mapping) will be
> > >
> > > I was trying that based on this:
> > > https://lore.kernel.org/linux-mm/20260220-swap-table-p4-v1-15-104795d19815@tencent.com/
> > >
> > > It seems to work and the only thing we need is actually just something
> > > like this one in VSS:
> > > https://lore.kernel.org/linux-mm/20260320192735.748051-15-nphamcs@gmail.com/
> > >
> > > This part:
> > > + /* fall back to physical swap device */
> > > + if (!vswap_alloc_swap_slot(folio)) {
> > >
> > > We do a folio_realloc_swap if folio->swap have type 0.
> > >
> > > Which means, if there is no virtual device / mapping / file / space
> > > (I'm not sure how to name it at this point :) ), the ordinary swap
> > > routine is just still there untouched.
> > >
> > > If there is one, and it's being used, then, it is still the ordinary
> > > swap routine, just do an extra allocation (and the extra allocation
> > > strictly follows YoungJun's tier rule), which is same with VSS, but
> > > everything is reused. From a user or high level interface perspective,
> > > this can be designed with no difference as VSS. Just with a few
> > > bonuses: being per memcg / task / runtime optional, zero overhead if
> > > not enabled, and reusing all the infra.
> > >
> > > BTW this deferred allocation (in VSS or dynamic swap mapping, similar
> > > thing) is actually a bit concerning to me as well. It changes the
> > > common swapout routine and maybe worth reconsideration (e.g.
> > > activate_locked_split and mTHP stats is now ignored?), being optional
> > > for now also seems safer.
> >
> > I am not sure if I understand you correctly. I think what you're proposing is:
> >
> > - Page tables either point directly to a swap slot, or to a virtual swap entry.
> > - By default, page tables just point to swap slots maintaining current behavior.
>
> I mean, they are all swap entries, nothing special from the page table
> side. Swap subsystems handle things internally.
>
> > - If we have multiple backends (e.g. zswap or tiering), we use virtual
> > swap entry instead.
>
> Actually that can just follow the swap priority, or tier rule. Even if
> virtual mapping exists, it can be bypassed. e.g. you have a large NBD
> and don't care about either fragmentation or compression for offline
> workload cgroups, then why use a virtual layer for them which could
> double the kmem usage or spend more CPU? Setup is a different issue
> which can be discussed.
I assume NBD == network block device here?
If you use a NBD, I think vswap overhead is not going to be the
bottleneck here :)
And, what about reliability. Say you allocate a slot on the NBD, unmap
the page from the PTEs, then proceed to swap_writeout(). What if the
NBD device is no longer available? What if IO fails? If you already
encode the physical swap slot location in the PTEs, then it's very
expensive to correct this mistake. Whereas with vswap, you can fall
back to another device if you so choose, and all it takes is just a
simple backend change at the vswap layer.
Another issue with the current physical swapfile allocator is that
induces physical contiguity where it's absolutely not needed. I don't
know if this is the case with an NBD, but for flash device for e.g,
obviously contiguity makes thing more efficient, but it would be nice
if we can fallback to discontiguous swapout as a fallback.
I feel like NBD is an argument FOR virtualization, not against.
>
> > - The physical swapfile has clusters and swap tables (status quo).
> > - Virtual swap is implemented with clusters and swap tables in a
> > virtual space, and each table entry points to an underlying swap slot
> > or zswap entry.
> > - If a page table has a physical swap slot, and we need to do tiering,
> > we basically "make it virtual" by making the swap table of the
> > physical swapfile point at a virtual swap entry? or another physical
> > swapfile? Not sure.
>
> They are still ordinary swap entries, nothing special. The virtual
> space is also just a ordinary swap file (or swap mapping), which is
> easy to do:
> https://lore.kernel.org/linux-mm/20260220-swap-table-p4-v1-15-104795d19815@tencent.com/
>
> Then its virtual_table will have a different set of swap entries. (I
> left that part undone though).
>
> > > Right... I mean with two layers you will likely have >16 bytes
> > > overhead, and double lookup.
> >
> > Why >16 bytes? Do we need anything extra other than the reverse
> > mapping? Also why do we need a double lookup?
>
> You will have to store at least the following info: memcg (2 bytes),
> shadow (8 bytes), count (at least 1 bytes), and revert mapping (8
> bytes, since you have to address a full virtual swap space). And some
> type info is also needed. Part of them can be shrinked but still,
> scientifically, merging two layers into one is considered a kind of
> optimization.
Optimization is always a worthwhile pursuit of course. But you have to
gauge it with what we can buy with a more flexbility design, which
might end up buying us more performance win down the line
In the immediate term, vswap buys you a dynamic compressed layer +
maintain the ability to write back.
Looking a bit longer term, I don't think you can do the following
without a layer of indirection here:
1. Compressed writeback.
2. Discontiguous swapouts. I think we need this as a fallback for THP
swapping (see [1] for the discussion).
3. Mixed backend swapin.
4. Optimizing swap IO - if sequential patterns matter for example, you
need the ability to delay or change backend allocation. The current
model is way too inflexible to allow for that.
5. Adding new swap backends. We want to decouple what the MM subsystem
needs (which is minimally captured in the virtual layer), with what
the backend itself wants.
Youngjun's paper is a case study for what you can buy with virtualization:
[1]: https://lore.kernel.org/all/6869b7f0-84e1-fb93-03f1-9442cdfe476b@google.com/
[2]: https://ieeexplore.ieee.org/document/8662047
>
> You need lookup the virtual layer, then the lower layer for many
> decision making, is was discussed before to introduce more cache bit
> or things like that and I think that is getting over complex, reminds
> me of the slot cache or HAS_CACHE thing...:
> https://lore.kernel.org/linux-mm/CAMgjq7DJrtE-jARik849kCufd0qNnZQs7C8fcyzVOKE14-O+Dw@mail.gmail.com/
>
> > I don't think I quite understand it yet, maybe I am the problem :)
>
> Haha, not at all! Blame me for the poor explanation. To be honest, the
> design is still evolving and there are definitely details that need to
> be improved. It's hard to discuss these abstractions purely in theory,
> so it's probably best just keep the works moving forward in a clean
> way, and make things simpler and better be opt-in first.
^ permalink raw reply
* Re: [PATCH v5 00/21] Virtual Swap Space
From: Nhat Pham @ 2026-05-01 13:42 UTC (permalink / raw)
To: Kairui Song
Cc: Yosry Ahmed, Liam R . Howlett, akpm, Alistair Popple,
Axel Rasmussen, Barry Song, Baolin Wang, Baoquan He,
Byungchul Park,
open list:CONTROL GROUP - MEMORY RESOURCE CONTROLLER (MEMCG),
Chengming Zhou, Chris Li, Jonathan Corbet, David Hildenbrand,
Dev Jain, Gregory Price, Johannes Weiner, Hugh Dickins, Jann Horn,
Joshua Hahn, Lance Yang, lenb, linux-doc, LKML, linux-mm,
open list:SUSPEND TO RAM, Lorenzo Stoakes, Matthew Brost,
Michal Hocko, Muchun Song, Mariano Pache, Pavel Machek, Peter Xu,
Peter Zijlstra, Pedro Falcato, Rafael J. Wysocki (Intel),
Rakie Kim, Roman Gushchin, Mike Rapoport, Ryan Roberts,
Shakeel Butt, Kemeng Shi, Suren Baghdasaryan, tglx,
Vlastimil Babka, Wei Xu, Huang, Ying, Yosry Ahmed, Yuanchu Xie,
Qi Zheng, Zi Yan, Meta kernel team, Rik van Riel
In-Reply-To: <CAMgjq7BZpU5K1xHyFiqpsjeFe6CZUouGY_gOGMwbkM2Duq-vGg@mail.gmail.com>
On Tue, Apr 28, 2026 at 7:46 PM Kairui Song <ryncsn@gmail.com> wrote:
>
> On Tue, Apr 28, 2026 at 2:23 AM Yosry Ahmed <yosry@kernel.org> wrote:
> >
> > On Fri, Apr 24, 2026 at 12:52 PM Kairui Song <ryncsn@gmail.com> wrote:
> > >
> > > On Sat, Apr 25, 2026 at 3:12 AM Yosry Ahmed <yosry@kernel.org> wrote
> > > > Why >16 bytes? Do we need anything extra other than the reverse
> > > > mapping? Also why do we need a double lookup?
> > >
> > > You will have to store at least the following info: memcg (2 bytes),
> > > shadow (8 bytes), count (at least 1 bytes), and revert mapping (8
> > > bytes, since you have to address a full virtual swap space). And some
> > > type info is also needed. Part of them can be shrinked but still,
> > > scientifically, merging two layers into one is considered a kind of
> > > optimization.
> > >
> > > You need lookup the virtual layer, then the lower layer for many
> > > decision making, is was discussed before to introduce more cache bit
> > > or things like that and I think that is getting over complex, reminds
> > > me of the slot cache or HAS_CACHE thing...:
> > > https://lore.kernel.org/linux-mm/CAMgjq7DJrtE-jARik849kCufd0qNnZQs7C8fcyzVOKE14-O+Dw@mail.gmail.com/
> >
> > I think that's where the disconnect is. You are considering these two
> > separate layers, each with its own metadata. The metadata should only
> > live in one place.
> >
> > If we only have swap tables in the virtual swap layer (with the
> > metadata), backends do not have to carry the metadata. In this case,
> > backends should only have a reverse mapping (if needed), and some
> > internal data structure (e.g. bitmaps) to track usage.
>
> Ah, you are right. This is currently an intermediate state, that
> problem might be gone if we unified everything.
What do you mean here?
>
> > This is difficult to achieve if the virtual swap layer is optional,
> > because then the metadata can live in different places. This is why I
>
> But that's not difficult to achieve at all with an optional layer, and
> actually will be achieved naturally without any design change with the
> RFC I posted. Swap count / cgroup / shadow all stay in the top layer,
> lower layer is "reverse map" only (the undone part though, it will
> require to move the cluster cache from global to device level, which
> is also required for YoungJun's tier or any functional tiering to
> work, we may run into more and more detail issue like this).
>
> Might even be easier that way, it's pretty close to the unified states I think.
I feel like you're moving towards the other direction, no? Seems like
you are unifying swap metadata, which is good (vswap will also want to
do this), but the problem is, the lower layer will have to allocate
memory for these metadata too...
Say vswap is optional and runtime enabled. How do you structure a
physical swap device's metadata? Some of the slots might be directly
mapped to PTEs, some might back vswap slots. These two cases require a
two completely different set of metadata: the former needs reference
count, swap cache, swap cgroup etc., whereas the latter only needs
reverse mapping...
I don't think we should mix vswap and non vswap slots in the same
type/address space.
^ permalink raw reply
* [PATCH] cgroup: Defer css percpu_ref kill on rmdir until cgroup is depopulated
From: Tejun Heo @ 2026-05-01 2:29 UTC (permalink / raw)
To: Martin Pitt
Cc: regressions, cgroups, lizefan.x, hannes,
Sebastian Andrzej Siewior, linux-kernel, Tejun Heo
In-Reply-To: <afHNg2VX2jy9bW7y@piware.de>
A chain of commits going back to v7.0 reworked rmdir to satisfy the
controller invariant that a subsystem's ->css_offline() must not run while
tasks are still doing kernel-side work in the cgroup.
[1] d245698d727a ("cgroup: Defer task cgroup unlink until after the task is done switching out")
[2] a72f73c4dd9b ("cgroup: Don't expose dead tasks in cgroup")
[3] 1b164b876c36 ("cgroup: Wait for dying tasks to leave on rmdir")
[4] 4c56a8ac6869 ("cgroup: Fix cgroup_drain_dying() testing the wrong condition")
[5] 13e786b64bd3 ("cgroup: Increment nr_dying_subsys_* from rmdir context")
[1] moved task cset unlink from do_exit() to finish_task_switch() so a
task's cset link drops only after the task has fully stopped scheduling.
That made tasks past exit_signals() linger on cset->tasks until their final
context switch, which led to a series of problems as what userspace expected
to see after rmdir diverged from what the kernel needs to wait for. [2]-[5]
tried to bridge that divergence: [2] filtered the exiting tasks from
cgroup.procs; [3] had rmdir(2) sleep in TASK_UNINTERRUPTIBLE for them; [4]
fixed the wait's condition; [5] made nr_dying_subsys_* visible
synchronously.
The cgroup_drain_dying() wait in [3] turned out to be a dead end. When the
rmdir caller is also the reaper of a zombie that pins a pidns teardown (e.g.
host PID 1 systemd reaping orphan pids that were re-parented to it during
the same teardown), rmdir blocks in TASK_UNINTERRUPTIBLE waiting for those
pids to free, the pids can't free because PID 1 is the reaper and it's stuck
in rmdir, and the system A-A deadlocks. No internal lock ordering breaks
this; the wait itself is the bug.
The css killing side that drove the original reorder, however, can be made
cleanly asynchronous: ->css_offline() is already async, run from
css_killed_work_fn() driven by percpu_ref_kill_and_confirm(). The fix is to
make that chain start only after all tasks have left the cgroup. rmdir's
user-visible side then returns as soon as cgroup.procs and friends are
empty, while ->css_offline() still runs only after the cgroup is fully
drained.
Verified by the original reproducer (pidns teardown + zombie reaper, runs
under vng) which hangs vanilla and succeeds here, and by per-commit
deterministic repros for [2], [3], [4], [5] with a boot parameter that
widens the post-exit_signals() window so each state is reliably reachable.
Some stress tests on top of that.
This seems like the right approach and I don't see problems with it. The
changes are somewhat invasive but not excessively so, so backporting to
-stable should be okay. If something does turn out to be wrong, the fallback
is to revert the entire chain ([1]-[5]) and rework in the development branch
instead.
Fixes: 1b164b876c36 ("cgroup: Wait for dying tasks to leave on rmdir")
Cc: stable@vger.kernel.org # v7.0+
Reported-by: Martin Pitt <martin@piware.de>
Link: https://lore.kernel.org/all/afHNg2VX2jy9bW7y@piware.de/
Link: https://lore.kernel.org/all/35e0670adb4abeab13da2c321582af9f@kernel.org/
Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
Hello Martin,
Could you give this a try? It defers the percpu_ref kill chain on
rmdir until the cgroup is fully drained, which removes the
TASK_UNINTERRUPTIBLE wait that deadlocked against PID 1 reaping. The
patch description has the details.
Thanks.
include/linux/cgroup-defs.h | 4 +-
kernel/cgroup/cgroup.c | 241 ++++++++++++++++--------------------
2 files changed, 110 insertions(+), 135 deletions(-)
diff --git a/include/linux/cgroup-defs.h b/include/linux/cgroup-defs.h
index f42563739d2e..50a784da7a81 100644
--- a/include/linux/cgroup-defs.h
+++ b/include/linux/cgroup-defs.h
@@ -611,8 +611,8 @@ struct cgroup {
/* used to wait for offlining of csses */
wait_queue_head_t offline_waitq;
- /* used by cgroup_rmdir() to wait for dying tasks to leave */
- wait_queue_head_t dying_populated_waitq;
+ /* defers killing csses after removal until cgroup is depopulated */
+ struct work_struct finish_destroy_work;
/* used to schedule release agent */
struct work_struct release_agent_work;
diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c
index c928dea9dea6..5634bac9cb9c 100644
--- a/kernel/cgroup/cgroup.c
+++ b/kernel/cgroup/cgroup.c
@@ -264,10 +264,10 @@ static void cgroup_finalize_control(struct cgroup *cgrp, int ret);
static void css_task_iter_skip(struct css_task_iter *it,
struct task_struct *task);
static int cgroup_destroy_locked(struct cgroup *cgrp);
+static void cgroup_finish_destroy(struct cgroup *cgrp);
static struct cgroup_subsys_state *css_create(struct cgroup *cgrp,
struct cgroup_subsys *ss);
static void css_release(struct percpu_ref *ref);
-static void kill_css(struct cgroup_subsys_state *css);
static int cgroup_addrm_files(struct cgroup_subsys_state *css,
struct cgroup *cgrp, struct cftype cfts[],
bool is_add);
@@ -797,6 +797,10 @@ static void cgroup_update_populated(struct cgroup *cgrp, bool populated)
if (was_populated == cgroup_is_populated(cgrp))
break;
+ /* subtree just emptied below an offlined cgrp; fire deferred destroy */
+ if (was_populated && !css_is_online(&cgrp->self))
+ queue_work(cgroup_offline_wq, &cgrp->finish_destroy_work);
+
cgroup1_check_for_release(cgrp);
TRACE_CGROUP_PATH(notify_populated, cgrp,
cgroup_is_populated(cgrp));
@@ -2039,6 +2043,15 @@ static int cgroup_reconfigure(struct fs_context *fc)
return 0;
}
+static void cgroup_finish_destroy_work_fn(struct work_struct *work)
+{
+ struct cgroup *cgrp = container_of(work, struct cgroup, finish_destroy_work);
+
+ cgroup_lock();
+ cgroup_finish_destroy(cgrp);
+ cgroup_unlock();
+}
+
static void init_cgroup_housekeeping(struct cgroup *cgrp)
{
struct cgroup_subsys *ss;
@@ -2065,7 +2078,7 @@ static void init_cgroup_housekeeping(struct cgroup *cgrp)
#endif
init_waitqueue_head(&cgrp->offline_waitq);
- init_waitqueue_head(&cgrp->dying_populated_waitq);
+ INIT_WORK(&cgrp->finish_destroy_work, cgroup_finish_destroy_work_fn);
INIT_WORK(&cgrp->release_agent_work, cgroup1_release_agent);
}
@@ -3375,7 +3388,8 @@ static void cgroup_apply_control_disable(struct cgroup *cgrp)
if (css->parent &&
!(cgroup_ss_mask(dsct) & (1 << ss->id))) {
- kill_css(css);
+ kill_css_sync(css);
+ kill_css_finish(css);
} else if (!css_visible(css)) {
css_clear_dir(css);
if (ss->css_reset)
@@ -5514,7 +5528,7 @@ static struct cftype cgroup_psi_files[] = {
* css destruction is four-stage process.
*
* 1. Destruction starts. Killing of the percpu_ref is initiated.
- * Implemented in kill_css().
+ * Implemented in kill_css_finish().
*
* 2. When the percpu_ref is confirmed to be visible as killed on all CPUs
* and thus css_tryget_online() is guaranteed to fail, the css can be
@@ -5993,7 +6007,7 @@ int cgroup_mkdir(struct kernfs_node *parent_kn, const char *name, umode_t mode)
/*
* This is called when the refcnt of a css is confirmed to be killed.
* css_tryget_online() is now guaranteed to fail. Tell the subsystem to
- * initiate destruction and put the css ref from kill_css().
+ * initiate destruction and put the css ref from kill_css_finish().
*/
static void css_killed_work_fn(struct work_struct *work)
{
@@ -6025,15 +6039,12 @@ static void css_killed_ref_fn(struct percpu_ref *ref)
}
/**
- * kill_css - destroy a css
- * @css: css to destroy
+ * kill_css_sync - synchronous half of css teardown
+ * @css: css being killed
*
- * This function initiates destruction of @css by removing cgroup interface
- * files and putting its base reference. ->css_offline() will be invoked
- * asynchronously once css_tryget_online() is guaranteed to fail and when
- * the reference count reaches zero, @css will be released.
+ * See cgroup_destroy_locked().
*/
-static void kill_css(struct cgroup_subsys_state *css)
+static void kill_css_sync(struct cgroup_subsys_state *css)
{
struct cgroup_subsys *ss = css->ss;
@@ -6056,24 +6067,6 @@ static void kill_css(struct cgroup_subsys_state *css)
*/
css_clear_dir(css);
- /*
- * Killing would put the base ref, but we need to keep it alive
- * until after ->css_offline().
- */
- css_get(css);
-
- /*
- * cgroup core guarantees that, by the time ->css_offline() is
- * invoked, no new css reference will be given out via
- * css_tryget_online(). We can't simply call percpu_ref_kill() and
- * proceed to offlining css's because percpu_ref_kill() doesn't
- * guarantee that the ref is seen as killed on all CPUs on return.
- *
- * Use percpu_ref_kill_and_confirm() to get notifications as each
- * css is confirmed to be seen as killed on all CPUs.
- */
- percpu_ref_kill_and_confirm(&css->refcnt, css_killed_ref_fn);
-
css->cgroup->nr_dying_subsys[ss->id]++;
/*
* Parent css and cgroup cannot be freed until after the freeing
@@ -6086,44 +6079,88 @@ static void kill_css(struct cgroup_subsys_state *css)
}
/**
- * cgroup_destroy_locked - the first stage of cgroup destruction
+ * kill_css_finish - deferred half of css teardown
+ * @css: css being killed
+ *
+ * See cgroup_destroy_locked().
+ */
+static void kill_css_finish(struct cgroup_subsys_state *css)
+{
+ lockdep_assert_held(&cgroup_mutex);
+
+ /*
+ * Skip on re-entry: cgroup_apply_control_disable() may have killed @css
+ * earlier. cgroup_destroy_locked() can still walk it because
+ * offline_css() (which NULLs cgrp->subsys[ssid]) runs async.
+ */
+ if (percpu_ref_is_dying(&css->refcnt))
+ return;
+
+ /*
+ * Killing would put the base ref, but we need to keep it alive until
+ * after ->css_offline().
+ */
+ css_get(css);
+
+ /*
+ * cgroup core guarantees that, by the time ->css_offline() is invoked,
+ * no new css reference will be given out via css_tryget_online(). We
+ * can't simply call percpu_ref_kill() and proceed to offlining css's
+ * because percpu_ref_kill() doesn't guarantee that the ref is seen as
+ * killed on all CPUs on return.
+ *
+ * Use percpu_ref_kill_and_confirm() to get notifications as each css is
+ * confirmed to be seen as killed on all CPUs.
+ */
+ percpu_ref_kill_and_confirm(&css->refcnt, css_killed_ref_fn);
+}
+
+/**
+ * cgroup_destroy_locked - destroy @cgrp (called on rmdir)
* @cgrp: cgroup to be destroyed
*
- * css's make use of percpu refcnts whose killing latency shouldn't be
- * exposed to userland and are RCU protected. Also, cgroup core needs to
- * guarantee that css_tryget_online() won't succeed by the time
- * ->css_offline() is invoked. To satisfy all the requirements,
- * destruction is implemented in the following two steps.
- *
- * s1. Verify @cgrp can be destroyed and mark it dying. Remove all
- * userland visible parts and start killing the percpu refcnts of
- * css's. Set up so that the next stage will be kicked off once all
- * the percpu refcnts are confirmed to be killed.
- *
- * s2. Invoke ->css_offline(), mark the cgroup dead and proceed with the
- * rest of destruction. Once all cgroup references are gone, the
- * cgroup is RCU-freed.
- *
- * This function implements s1. After this step, @cgrp is gone as far as
- * the userland is concerned and a new cgroup with the same name may be
- * created. As cgroup doesn't care about the names internally, this
- * doesn't cause any problem.
+ * Tear down @cgrp on behalf of rmdir. Constraints:
+ *
+ * - Userspace: rmdir must succeed when cgroup.procs and friends are empty.
+ *
+ * - Kernel: subsystem ->css_offline() must not run while any task in @cgrp's
+ * subtree is still doing kernel work. A task hidden from cgroup.procs (past
+ * exit_signals() with signal->live cleared) can still schedule, allocate, and
+ * consume resources until its final context switch. Dying descendants in the
+ * subtree can host such tasks too.
+ *
+ * - Kernel: css_tryget_online() must fail by the time ->css_offline() runs.
+ *
+ * The destruction runs in three parts:
+ *
+ * - This function: synchronous user-visible state teardown plus kill_css_sync()
+ * on each subsystem css.
+ *
+ * - cgroup_finish_destroy(): kicks the percpu_ref kill via kill_css_finish() on
+ * each subsystem css. Fires once @cgrp's subtree is fully drained, either
+ * inline here or from cgroup_update_populated().
+ *
+ * - The percpu_ref kill chain: css_killed_ref_fn -> css_killed_work_fn ->
+ * ->css_offline() -> release/free.
+ *
+ * Return 0 on success, -EBUSY if a userspace-visible task or an online child
+ * remains.
*/
static int cgroup_destroy_locked(struct cgroup *cgrp)
- __releases(&cgroup_mutex) __acquires(&cgroup_mutex)
{
struct cgroup *tcgrp, *parent = cgroup_parent(cgrp);
struct cgroup_subsys_state *css;
struct cgrp_cset_link *link;
+ struct css_task_iter it;
+ struct task_struct *task;
int ssid, ret;
lockdep_assert_held(&cgroup_mutex);
- /*
- * Only migration can raise populated from zero and we're already
- * holding cgroup_mutex.
- */
- if (cgroup_is_populated(cgrp))
+ css_task_iter_start(&cgrp->self, 0, &it);
+ task = css_task_iter_next(&it);
+ css_task_iter_end(&it);
+ if (task)
return -EBUSY;
/*
@@ -6147,9 +6184,8 @@ static int cgroup_destroy_locked(struct cgroup *cgrp)
link->cset->dead = true;
spin_unlock_irq(&css_set_lock);
- /* initiate massacre of all css's */
for_each_css(css, ssid, cgrp)
- kill_css(css);
+ kill_css_sync(css);
/* clear and remove @cgrp dir, @cgrp has an extra ref on its kn */
css_clear_dir(&cgrp->self);
@@ -6180,79 +6216,27 @@ static int cgroup_destroy_locked(struct cgroup *cgrp)
/* put the base reference */
percpu_ref_kill(&cgrp->self.refcnt);
+ if (!cgroup_is_populated(cgrp))
+ cgroup_finish_destroy(cgrp);
+
return 0;
};
/**
- * cgroup_drain_dying - wait for dying tasks to leave before rmdir
- * @cgrp: the cgroup being removed
- *
- * cgroup.procs and cgroup.threads use css_task_iter which filters out
- * PF_EXITING tasks so that userspace doesn't see tasks that have already been
- * reaped via waitpid(). However, cgroup_has_tasks() - which tests whether the
- * cgroup has non-empty css_sets - is only updated when dying tasks pass through
- * cgroup_task_dead() in finish_task_switch(). This creates a window where
- * cgroup.procs reads empty but cgroup_has_tasks() is still true, making rmdir
- * fail with -EBUSY from cgroup_destroy_locked() even though userspace sees no
- * tasks.
+ * cgroup_finish_destroy - deferred half of @cgrp destruction
+ * @cgrp: cgroup whose subtree just became empty
*
- * This function aligns cgroup_has_tasks() with what userspace can observe. If
- * cgroup_has_tasks() but the task iterator sees nothing (all remaining tasks are
- * PF_EXITING), we wait for cgroup_task_dead() to finish processing them. As the
- * window between PF_EXITING and cgroup_task_dead() is short, the wait is brief.
- *
- * This function only concerns itself with this cgroup's own dying tasks.
- * Whether the cgroup has children is cgroup_destroy_locked()'s problem.
- *
- * Each cgroup_task_dead() kicks the waitqueue via cset->cgrp_links, and we
- * retry the full check from scratch.
- *
- * Must be called with cgroup_mutex held.
+ * See cgroup_destroy_locked() for the rationale.
*/
-static int cgroup_drain_dying(struct cgroup *cgrp)
- __releases(&cgroup_mutex) __acquires(&cgroup_mutex)
+static void cgroup_finish_destroy(struct cgroup *cgrp)
{
- struct css_task_iter it;
- struct task_struct *task;
- DEFINE_WAIT(wait);
+ struct cgroup_subsys_state *css;
+ int ssid;
lockdep_assert_held(&cgroup_mutex);
-retry:
- if (!cgroup_has_tasks(cgrp))
- return 0;
-
- /* Same iterator as cgroup.threads - if any task is visible, it's busy */
- css_task_iter_start(&cgrp->self, 0, &it);
- task = css_task_iter_next(&it);
- css_task_iter_end(&it);
-
- if (task)
- return -EBUSY;
- /*
- * All remaining tasks are PF_EXITING and will pass through
- * cgroup_task_dead() shortly. Wait for a kick and retry.
- *
- * cgroup_has_tasks() can't transition from false to true while we're
- * holding cgroup_mutex, but the true to false transition happens
- * under css_set_lock (via cgroup_task_dead()). We must retest and
- * prepare_to_wait() under css_set_lock. Otherwise, the transition
- * can happen between our first test and prepare_to_wait(), and we
- * sleep with no one to wake us.
- */
- spin_lock_irq(&css_set_lock);
- if (!cgroup_has_tasks(cgrp)) {
- spin_unlock_irq(&css_set_lock);
- return 0;
- }
- prepare_to_wait(&cgrp->dying_populated_waitq, &wait,
- TASK_UNINTERRUPTIBLE);
- spin_unlock_irq(&css_set_lock);
- mutex_unlock(&cgroup_mutex);
- schedule();
- finish_wait(&cgrp->dying_populated_waitq, &wait);
- mutex_lock(&cgroup_mutex);
- goto retry;
+ for_each_css(css, ssid, cgrp)
+ kill_css_finish(css);
}
int cgroup_rmdir(struct kernfs_node *kn)
@@ -6264,12 +6248,9 @@ int cgroup_rmdir(struct kernfs_node *kn)
if (!cgrp)
return 0;
- ret = cgroup_drain_dying(cgrp);
- if (!ret) {
- ret = cgroup_destroy_locked(cgrp);
- if (!ret)
- TRACE_CGROUP_PATH(rmdir, cgrp);
- }
+ ret = cgroup_destroy_locked(cgrp);
+ if (!ret)
+ TRACE_CGROUP_PATH(rmdir, cgrp);
cgroup_kn_unlock(kn);
return ret;
@@ -7029,7 +7010,6 @@ void cgroup_task_exit(struct task_struct *tsk)
static void do_cgroup_task_dead(struct task_struct *tsk)
{
- struct cgrp_cset_link *link;
struct css_set *cset;
unsigned long flags;
@@ -7043,11 +7023,6 @@ static void do_cgroup_task_dead(struct task_struct *tsk)
if (thread_group_leader(tsk) && atomic_read(&tsk->signal->live))
list_add_tail(&tsk->cg_list, &cset->dying_tasks);
- /* kick cgroup_drain_dying() waiters, see cgroup_rmdir() */
- list_for_each_entry(link, &cset->cgrp_links, cgrp_link)
- if (waitqueue_active(&link->cgrp->dying_populated_waitq))
- wake_up(&link->cgrp->dying_populated_waitq);
-
if (dl_task(tsk))
dec_dl_tasks_cs(tsk);
--
2.54.0
^ permalink raw reply related
* [PATCH v2 2/2] selftests/cgroup: include slab in test_percpu_basic memory check
From: Li Wang @ 2026-05-01 2:20 UTC (permalink / raw)
To: akpm, hannes, mhocko, roman.gushchin, shakeel.butt, muchun.song,
tj, mkoutny, shuah
Cc: cgroups, linux-mm, linux-kselftest, linux-kernel, Waiman Long,
Christoph Lameter, Shakeel Butt, Vlastimil Babka
In-Reply-To: <20260501022058.18024-1-li.wang@linux.dev>
test_percpu_basic() currently compares memory.current against only
memory.stat:percpu after creating 1000 child cgroups.
Observed failure:
#./test_kmem
ok 1 test_kmem_basic
ok 2 test_kmem_memcg_deletion
ok 3 test_kmem_proc_kpagecgroup
ok 4 test_kmem_kernel_stacks
ok 5 test_kmem_dead_cgroups
memory.current 11530240
percpu 8440000
not ok 6 test_percpu_basic
That assumption is too strict: child cgroup creation also allocates
slab-backed metadata, so memory.current is expected to be larger than
percpu alone. One visible path is:
cgroup_mkdir()
cgroup_create()
cgroup_addrm_file()
cgroup_add_file()
__kernfs_create_file()
__kernfs_new_node()
kmem_cache_zalloc()
These kernfs allocations are charged as slab and show up in
memory.stat:slab.
Update the check to compare memory.current against (percpu + slab)
within MAX_VMSTAT_ERROR, and print slab/delta in the failure message to
improve diagnostics.
Signed-off-by: Li Wang <li.wang@linux.dev>
Cc: Waiman Long <longman@redhat.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Shakeel Butt <shakeelb@google.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Vlastimil Babka <vbabka@suse.cz>
Reviewed-by: Waiman Long <longman@redhat.com>
---
tools/testing/selftests/cgroup/test_kmem.c | 11 ++++++-----
1 file changed, 6 insertions(+), 5 deletions(-)
diff --git a/tools/testing/selftests/cgroup/test_kmem.c b/tools/testing/selftests/cgroup/test_kmem.c
index 249d7911306..4b579969889 100644
--- a/tools/testing/selftests/cgroup/test_kmem.c
+++ b/tools/testing/selftests/cgroup/test_kmem.c
@@ -353,7 +353,7 @@ static int test_percpu_basic(const char *root)
{
int ret = KSFT_FAIL;
char *parent, *child;
- long current, percpu;
+ long current, percpu, slab;
int i;
parent = cg_name(root, "percpu_basic_test");
@@ -379,13 +379,14 @@ static int test_percpu_basic(const char *root)
current = cg_read_long(parent, "memory.current");
percpu = cg_read_key_long(parent, "memory.stat", "percpu ");
+ slab = cg_read_key_long(parent, "memory.stat", "slab ");
- if (current > 0 && percpu > 0 && labs(current - percpu) <
- MAX_VMSTAT_ERROR)
+ if (current > 0 && percpu > 0 && slab >= 0 &&
+ labs(current - (percpu + slab)) < MAX_VMSTAT_ERROR)
ret = KSFT_PASS;
else
- printf("memory.current %ld\npercpu %ld\n",
- current, percpu);
+ printf("memory.current %ld\npercpu %ld\nslab %ld\ndelta %ld\n",
+ current, percpu, slab, current - (percpu + slab));
cleanup_children:
for (i = 0; i < 1000; i++) {
--
2.54.0
^ permalink raw reply related
* [PATCH v2 0/2] selftests/cgroup: Fix false positive failures in test_percpu_basic
From: Li Wang @ 2026-05-01 2:20 UTC (permalink / raw)
To: akpm, hannes, mhocko, roman.gushchin, shakeel.butt, muchun.song,
tj, mkoutny, shuah
Cc: cgroups, linux-mm, linux-kselftest, linux-kernel
This patch series addresses two separate issues that cause false
positive failures in the test_percpu_basic test within the cgroup
kmem selftests.
The first issue stems from a hardcoded assumption about the system
page size, which breaks the test on architectures with larger page
sizes.
The second issue is an overly strict memory check that fails to
account for the slab metadata allocated during cgroup creation.
v2:
* Replace my email with li.wang@linux.dev
* Adding Waiman's Reviewed-by tag
Li Wang (2):
selftests/cgroup: Fix hardcoded page size in test_percpu_basic
selftests/cgroup: include slab in test_percpu_basic memory check
tools/testing/selftests/cgroup/test_kmem.c | 13 +++++++------
1 file changed, 7 insertions(+), 6 deletions(-)
--
2.54.0
^ permalink raw reply
* [PATCH v2 1/2] selftests/cgroup: Fix hardcoded page size in test_percpu_basic
From: Li Wang @ 2026-05-01 2:20 UTC (permalink / raw)
To: akpm, hannes, mhocko, roman.gushchin, shakeel.butt, muchun.song,
tj, mkoutny, shuah
Cc: cgroups, linux-mm, linux-kselftest, linux-kernel, Waiman Long,
Christoph Lameter, Shakeel Butt, Vlastimil Babka
In-Reply-To: <20260501022058.18024-1-li.wang@linux.dev>
MAX_VMSTAT_ERROR uses a hardcoded page size of 4096, which assumes
4K pages. This causes test_percpu_basic to fail on systems where
the kernel is configured with a larger page size, such as aarch64
systems using 16K or 64K pages, where the maximum permissible
discrepancy between memory.current and percpu charges is
proportionally larger.
Replace the hardcoded 4096 with sysconf(_SC_PAGESIZE) to correctly
derive the page size at runtime regardless of the underlying
architecture or kernel configuration.
Signed-off-by: Li Wang <li.wang@linux.dev>
Cc: Waiman Long <longman@redhat.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Shakeel Butt <shakeelb@google.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Vlastimil Babka <vbabka@suse.cz>
Acked-by: Waiman Long <longman@redhat.com>
---
tools/testing/selftests/cgroup/test_kmem.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/testing/selftests/cgroup/test_kmem.c b/tools/testing/selftests/cgroup/test_kmem.c
index eeabd34bf08..249d7911306 100644
--- a/tools/testing/selftests/cgroup/test_kmem.c
+++ b/tools/testing/selftests/cgroup/test_kmem.c
@@ -24,7 +24,7 @@
* the maximum discrepancy between charge and vmstat entries is number
* of cpus multiplied by 64 pages.
*/
-#define MAX_VMSTAT_ERROR (4096 * 64 * get_nprocs())
+#define MAX_VMSTAT_ERROR (sysconf(_SC_PAGESIZE) * 64 * get_nprocs())
#define KMEM_DEAD_WAIT_RETRIES 80
--
2.54.0
^ permalink raw reply related
* Re: (subset) [PATCH v3 00/14] Remove redundant rcu_read_lock/unlock() in spin_lock
From: Jeff Johnson @ 2026-04-30 21:30 UTC (permalink / raw)
To: tj, tony.luck, jani.nikula, ap420073, jv, freude, bcrl, trondmy,
longman, kees, pengdonglin
Cc: bigeasy, hdanton, paulmck, linux-kernel, linux-rt-devel,
linux-nfs, linux-aio, linux-fsdevel, linux-security-module,
netdev, intel-gfx, linux-wireless, linux-acpi, linux-s390,
cgroups
In-Reply-To: <20250916044735.2316171-1-dolinux.peng@gmail.com>
On Tue, 16 Sep 2025 12:47:21 +0800, pengdonglin wrote:
> Since commit a8bb74acd8efe ("rcu: Consolidate RCU-sched update-side function definitions")
> there is no difference between rcu_read_lock(), rcu_read_lock_bh() and
> rcu_read_lock_sched() in terms of RCU read section and the relevant grace
> period. That means that spin_lock(), which implies rcu_read_lock_sched(),
> also implies rcu_read_lock().
>
> There is no need no explicitly start a RCU read section if one has already
> been started implicitly by spin_lock().
>
> [...]
Applied, thanks!
[14/14] wifi: ath9k: Remove redundant rcu_read_lock/unlock() in spin_lock
commit: c4f518736472c8cfbf1d304e01c631babd2bbf34
Best regards,
--
Jeff Johnson <jeff.johnson@oss.qualcomm.com>
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox