* Re: [PATCH] net: rose: use pskb_may_pull() in CLEAR_REQUEST length check
From: Andrew Lunn @ 2026-04-21 2:50 UTC (permalink / raw)
To: Ashutosh Desai
Cc: netdev, linux-hams, davem, edumazet, kuba, pabeni, horms,
linux-kernel
In-Reply-To: <20260421022712.690822-1-ashutoshdesai993@gmail.com>
On Tue, Apr 21, 2026 at 02:27:12AM +0000, Ashutosh Desai wrote:
> On Mon, Apr 20, 2026 at 15:04:30 +0200, Andrew Lunn wrote:
> > Did you review all the other comparisons on skb->len in rose?
> > Do these need the same fix? Are there other places non linear buffers
> > should be considered?
>
> Reviewed all rose files. Thanks for pointing those out - yes, the two
> spots in rose_route.c and rose_loopback.c have the same issue and need
> the same fix.
>
> While reviewing I also noticed rose_link.c: rose_link_rx_restart()
> accesses skb->data[3] in the ROSE_RESTART_REQUEST case and
> skb->data[3]/skb->data+4 in the ROSE_DIAGNOSTIC case, with only a
> ROSE_MIN_LEN (3 bytes) guard upstream in the caller. Same linearity
> concern.
>
> Is it recommended to send a v2 covering all three files?
Too late, Jakub just posted patches deleted it all. See the netdev
mailing list.
Andrew
^ permalink raw reply
* [PATCH v2 0/2] Bluetooth: ISO: Fix KCSAN data-races on iso_pi(sk)
From: SeungJu Cheon @ 2026-04-21 2:51 UTC (permalink / raw)
To: luiz.dentz, marcel
Cc: linux-bluetooth, netdev, linux-kernel, me, skhan,
linux-kernel-mentees, SeungJu Cheon
Found while auditing iso_pi(sk) field accesses after a KCSAN report.
Patch 1/2 is the reported race on iso_pi(sk)->dst in iso_sock_connect();
patch 2/2 covers related races on other iso_pi(sk) fields accessed in
iso_connect_{bis,cis}() and iso_connect_ind() that were found by
inspection during the same audit.
Changes in v2:
- Patch 1/2: Use sa->iso_bdaddr directly instead of caching the
bacmp() result in a local variable, as suggested by Luiz [1].
This avoids reading from iso_pi(sk) entirely for the broadcast
check.
- Patch 2/2: No changes.
v1: https://lore.kernel.org/linux-bluetooth/20260418053239.128190-1-suunj1331@gmail.com/
[1] https://lore.kernel.org/linux-bluetooth/CABBYNZLBoU3byfK_G+=sTkBx3wNwEh2X6_7dG4+4LFtrc3Skpw@mail.gmail.com/
SeungJu Cheon (2):
Bluetooth: ISO: Fix data-race on dst in iso_sock_connect()
Bluetooth: ISO: Fix data-race on iso_pi(sk) in socket and HCI event
paths
net/bluetooth/iso.c | 56 +++++++++++++++++++++++++--------------------
1 file changed, 31 insertions(+), 25 deletions(-)
--
2.52.0
^ permalink raw reply
* [PATCH v2 1/2] Bluetooth: ISO: Fix data-race on dst in iso_sock_connect()
From: SeungJu Cheon @ 2026-04-21 2:51 UTC (permalink / raw)
To: luiz.dentz, marcel
Cc: linux-bluetooth, netdev, linux-kernel, me, skhan,
linux-kernel-mentees, SeungJu Cheon
In-Reply-To: <20260421025122.55781-1-suunj1331@gmail.com>
iso_sock_connect() copies the destination address into
iso_pi(sk)->dst under lock_sock, then releases the lock and reads
it back with bacmp() to decide between the CIS and BIS connect
paths:
lock_sock(sk);
bacpy(&iso_pi(sk)->dst, &sa->iso_bdaddr);
iso_pi(sk)->dst_type = sa->iso_bdaddr_type;
release_sock(sk);
if (bacmp(&iso_pi(sk)->dst, BDADDR_ANY)) // <- no lock held
This read after release_sock() races with any concurrent write to
iso_pi(sk)->dst on the same socket.
Fix by reading the destination address directly from the local
sockaddr argument (sa->iso_bdaddr) instead of iso_pi(sk)->dst.
Since sa is a function-local argument, reading it requires no
locking and avoids the race.
This patch addresses only the bacmp() race in iso_sock_connect();
other unprotected iso_pi(sk) accesses are fixed separately in the
next patch.
KCSAN report:
BUG: KCSAN: data-race in memcmp+0x39/0xb0
race at unknown origin, with read to 0xffff8f96ea66dde3 of 1 bytes by task 549 on cpu 1:
memcmp+0x39/0xb0
iso_sock_connect+0x275/0xb40
__sys_connect_file+0xbd/0xe0
__sys_connect+0xe0/0x110
__x64_sys_connect+0x40/0x50
x64_sys_call+0xcad/0x1c60
do_syscall_64+0x133/0x590
entry_SYSCALL_64_after_hwframe+0x77/0x7f
value changed: 0x00 -> 0xee
Reported by Kernel Concurrency Sanitizer on:
CPU: 1 UID: 0 PID: 549 Comm: iso_race_combin Not tainted 7.0.0-08391-g1d51b370a0f8 #40 PREEMPT(lazy)
Fixes: ccf74f2390d6 ("Bluetooth: Add BTPROTO_ISO socket type")
Signed-off-by: SeungJu Cheon <suunj1331@gmail.com>
---
net/bluetooth/iso.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/net/bluetooth/iso.c b/net/bluetooth/iso.c
index be145e2736b7..290a1b9a9daa 100644
--- a/net/bluetooth/iso.c
+++ b/net/bluetooth/iso.c
@@ -1193,7 +1193,7 @@ static int iso_sock_connect(struct socket *sock, struct sockaddr_unsized *addr,
release_sock(sk);
- if (bacmp(&iso_pi(sk)->dst, BDADDR_ANY))
+ if (bacmp(&sa->iso_bdaddr, BDADDR_ANY))
err = iso_connect_cis(sk);
else
err = iso_connect_bis(sk);
--
2.52.0
^ permalink raw reply related
* [PATCH v2 2/2] Bluetooth: ISO: Fix data-race on iso_pi(sk) in socket and HCI event paths
From: SeungJu Cheon @ 2026-04-21 2:51 UTC (permalink / raw)
To: luiz.dentz, marcel
Cc: linux-bluetooth, netdev, linux-kernel, me, skhan,
linux-kernel-mentees, SeungJu Cheon
In-Reply-To: <20260421025122.55781-1-suunj1331@gmail.com>
Several iso_pi(sk) fields (qos, qos_user_set, bc_sid, base, base_len,
sync_handle, bc_num_bis) are written under lock_sock in
iso_sock_setsockopt() and iso_sock_bind(), but read and written under
hci_dev_lock only in two other paths:
- iso_connect_bis() / iso_connect_cis(), invoked from connect(2),
read qos/base/bc_sid and reset qos to default_qos on the
qos_user_set validation failure -- all without lock_sock.
- iso_connect_ind(), invoked from hci_rx_work, writes sync_handle,
bc_sid, qos.bcast.encryption, bc_num_bis, base and base_len on
PA_SYNC_ESTABLISHED / PAST_RECEIVED / BIG_INFO_ADV_REPORT /
PER_ADV_REPORT events. The BIG_INFO handler additionally passes
&iso_pi(sk)->qos together with sync_handle / bc_num_bis / bc_bis
to hci_conn_big_create_sync() while setsockopt may be mutating
them.
Acquire lock_sock around the affected accesses in both paths.
The locking order hci_dev_lock -> lock_sock matches the existing
iso_conn_big_sync() precedent, whose comment documents the same
requirement for hci_conn_big_create_sync(). The HCI connect/bind
helpers do not wait for command completion -- they enqueue work via
hci_cmd_sync_queue{,_once}() / hci_le_create_cis_pending() and
return -- so the added hold time is comparable to iso_conn_big_sync().
KCSAN report:
BUG: KCSAN: data-race in iso_connect_cis / iso_sock_setsockopt
read to 0xffffa3ae8ce3cdc8 of 1 bytes by task 335 on cpu 0:
iso_connect_cis+0x49f/0xa20
iso_sock_connect+0x60e/0xb40
__sys_connect_file+0xbd/0xe0
__sys_connect+0xe0/0x110
__x64_sys_connect+0x40/0x50
x64_sys_call+0xcad/0x1c60
do_syscall_64+0x133/0x590
entry_SYSCALL_64_after_hwframe+0x77/0x7f
write to 0xffffa3ae8ce3cdc8 of 60 bytes by task 334 on cpu 1:
iso_sock_setsockopt+0x69a/0x930
do_sock_setsockopt+0xc3/0x170
__sys_setsockopt+0xd1/0x130
__x64_sys_setsockopt+0x64/0x80
x64_sys_call+0x1547/0x1c60
do_syscall_64+0x133/0x590
entry_SYSCALL_64_after_hwframe+0x77/0x7f
Reported by Kernel Concurrency Sanitizer on:
CPU: 1 UID: 0 PID: 334 Comm: iso_setup_race Not tainted 7.0.0-10949-g8541d8f725c6 #44 PREEMPT(lazy)
The iso_connect_ind() races were found by inspection.
Fixes: ccf74f2390d6 ("Bluetooth: Add BTPROTO_ISO socket type")
Signed-off-by: SeungJu Cheon <suunj1331@gmail.com>
---
net/bluetooth/iso.c | 54 +++++++++++++++++++++++++--------------------
1 file changed, 30 insertions(+), 24 deletions(-)
diff --git a/net/bluetooth/iso.c b/net/bluetooth/iso.c
index 290a1b9a9daa..7cb2864fe872 100644
--- a/net/bluetooth/iso.c
+++ b/net/bluetooth/iso.c
@@ -347,6 +347,7 @@ static int iso_connect_bis(struct sock *sk)
return -EHOSTUNREACH;
hci_dev_lock(hdev);
+ lock_sock(sk);
if (!bis_capable(hdev)) {
err = -EOPNOTSUPP;
@@ -399,13 +400,9 @@ static int iso_connect_bis(struct sock *sk)
goto unlock;
}
- lock_sock(sk);
-
err = iso_chan_add(conn, sk, NULL);
- if (err) {
- release_sock(sk);
+ if (err)
goto unlock;
- }
/* Update source addr of the socket */
bacpy(&iso_pi(sk)->src, &hcon->src);
@@ -421,9 +418,8 @@ static int iso_connect_bis(struct sock *sk)
iso_sock_set_timer(sk, READ_ONCE(sk->sk_sndtimeo));
}
- release_sock(sk);
-
unlock:
+ release_sock(sk);
hci_dev_unlock(hdev);
hci_dev_put(hdev);
return err;
@@ -444,6 +440,7 @@ static int iso_connect_cis(struct sock *sk)
return -EHOSTUNREACH;
hci_dev_lock(hdev);
+ lock_sock(sk);
if (!cis_central_capable(hdev)) {
err = -EOPNOTSUPP;
@@ -498,13 +495,9 @@ static int iso_connect_cis(struct sock *sk)
goto unlock;
}
- lock_sock(sk);
-
err = iso_chan_add(conn, sk, NULL);
- if (err) {
- release_sock(sk);
+ if (err)
goto unlock;
- }
/* Update source addr of the socket */
bacpy(&iso_pi(sk)->src, &hcon->src);
@@ -520,9 +513,8 @@ static int iso_connect_cis(struct sock *sk)
iso_sock_set_timer(sk, READ_ONCE(sk->sk_sndtimeo));
}
- release_sock(sk);
-
unlock:
+ release_sock(sk);
hci_dev_unlock(hdev);
hci_dev_put(hdev);
return err;
@@ -2256,8 +2248,10 @@ int iso_connect_ind(struct hci_dev *hdev, bdaddr_t *bdaddr, __u8 *flags)
sk = iso_get_sock(hdev, &hdev->bdaddr, bdaddr, BT_LISTEN,
iso_match_sid, ev1);
if (sk && !ev1->status) {
+ lock_sock(sk);
iso_pi(sk)->sync_handle = le16_to_cpu(ev1->handle);
iso_pi(sk)->bc_sid = ev1->sid;
+ release_sock(sk);
}
goto done;
@@ -2268,8 +2262,10 @@ int iso_connect_ind(struct hci_dev *hdev, bdaddr_t *bdaddr, __u8 *flags)
sk = iso_get_sock(hdev, &hdev->bdaddr, bdaddr, BT_LISTEN,
iso_match_sid_past, ev1a);
if (sk && !ev1a->status) {
+ lock_sock(sk);
iso_pi(sk)->sync_handle = le16_to_cpu(ev1a->sync_handle);
iso_pi(sk)->bc_sid = ev1a->sid;
+ release_sock(sk);
}
goto done;
@@ -2296,27 +2292,35 @@ int iso_connect_ind(struct hci_dev *hdev, bdaddr_t *bdaddr, __u8 *flags)
ev2);
if (sk) {
- int err;
- struct hci_conn *hcon = iso_pi(sk)->conn->hcon;
+ int err = 0;
+ bool big_sync;
+ struct hci_conn *hcon;
+ lock_sock(sk);
+
+ hcon = iso_pi(sk)->conn->hcon;
iso_pi(sk)->qos.bcast.encryption = ev2->encryption;
if (ev2->num_bis < iso_pi(sk)->bc_num_bis)
iso_pi(sk)->bc_num_bis = ev2->num_bis;
- if (!test_bit(BT_SK_DEFER_SETUP, &bt_sk(sk)->flags) &&
- !test_and_set_bit(BT_SK_BIG_SYNC, &iso_pi(sk)->flags)) {
+ big_sync = !test_bit(BT_SK_DEFER_SETUP, &bt_sk(sk)->flags) &&
+ !test_and_set_bit(BT_SK_BIG_SYNC, &iso_pi(sk)->flags);
+
+ if (big_sync)
err = hci_conn_big_create_sync(hdev, hcon,
&iso_pi(sk)->qos,
iso_pi(sk)->sync_handle,
iso_pi(sk)->bc_num_bis,
iso_pi(sk)->bc_bis);
- if (err) {
- bt_dev_err(hdev, "hci_le_big_create_sync: %d",
- err);
- sock_put(sk);
- sk = NULL;
- }
+
+ release_sock(sk);
+
+ if (big_sync && err) {
+ bt_dev_err(hdev, "hci_le_big_create_sync: %d",
+ err);
+ sock_put(sk);
+ sk = NULL;
}
}
@@ -2370,8 +2374,10 @@ int iso_connect_ind(struct hci_dev *hdev, bdaddr_t *bdaddr, __u8 *flags)
if (!base || base_len > BASE_MAX_LENGTH)
goto done;
+ lock_sock(sk);
memcpy(iso_pi(sk)->base, base, base_len);
iso_pi(sk)->base_len = base_len;
+ release_sock(sk);
} else {
/* This is a PA data fragment. Keep pa_data_len set to 0
* until all data has been reassembled.
--
2.52.0
^ permalink raw reply related
* [PATCH net v3] hv_sock: Report EOF instead of -EIO for FIN
From: Dexuan Cui @ 2026-04-21 2:59 UTC (permalink / raw)
To: kys, haiyangz, wei.liu, decui, longli, sgarzare, davem, edumazet,
kuba, pabeni, horms, niuxuewei.nxw, linux-hyperv, virtualization,
netdev, linux-kernel
Cc: stable, Ben Hillis, Mitchell Levy
Commit f0c5827d07cb unluckily causes a regression for the FIN packet,
and the final read syscall gets an error rather than 0.
Ideally, we would want to fix hvs_channel_readable_payload() so that it
could return 0 in the FIN scenario, but it's not good for the hv_sock
driver to use the VMBus ringbuffer's cached priv_read_index, which is
internal data in the VMBus driver.
Fix the regression in hv_sock by returning 0 rather than -EIO.
In case we see a malformed/short packet, we still return -EIO.
Fixes: f0c5827d07cb ("hv_sock: Return the readable bytes in hvs_stream_has_data()")
Cc: stable@vger.kernel.org
Reported-by: Ben Hillis <Ben.Hillis@microsoft.com>
Reported-by: Mitchell Levy <levymitchell0@gmail.com>
Signed-off-by: Dexuan Cui <decui@microsoft.com>
---
Changes since v1:
Removed the local variable 'need_refill' to make the code more
readable. Stefano, thanks!
No other change.
Changes since v2:
Added code to test the flag SEND_SHUTDOWN. Copilot, thanks!
Updated the comment and the commit messages accordingly.
net/vmw_vsock/hyperv_transport.c | 29 +++++++++++++++++++++++++----
1 file changed, 25 insertions(+), 4 deletions(-)
diff --git a/net/vmw_vsock/hyperv_transport.c b/net/vmw_vsock/hyperv_transport.c
index 069386a74557..da150de10f0d 100644
--- a/net/vmw_vsock/hyperv_transport.c
+++ b/net/vmw_vsock/hyperv_transport.c
@@ -694,7 +694,6 @@ static ssize_t hvs_stream_enqueue(struct vsock_sock *vsk, struct msghdr *msg,
static s64 hvs_stream_has_data(struct vsock_sock *vsk)
{
struct hvsock *hvs = vsk->trans;
- bool need_refill;
s64 ret;
if (hvs->recv_data_len > 0)
@@ -702,9 +701,31 @@ static s64 hvs_stream_has_data(struct vsock_sock *vsk)
switch (hvs_channel_readable_payload(hvs->chan)) {
case 1:
- need_refill = !hvs->recv_desc;
- if (!need_refill)
- return -EIO;
+ if (hvs->recv_desc) {
+ /* Here hvs->recv_data_len is 0, so hvs->recv_desc must
+ * be NULL unless it points to the 0-byte-payload FIN
+ * packet or a malformed/short packet: see
+ * hvs_update_recv_data().
+ *
+ * If hvs->recv_desc points to the FIN packet, here all
+ * the payload has been dequeued and the peer_shutdown
+ * flag is set, but hvs_channel_readable_payload() still
+ * returns 1, because the VMBus ringbuffer's read_index
+ * is not updated for the FIN packet:
+ * hvs_stream_dequeue() -> hv_pkt_iter_next() updates
+ * the cached priv_read_index but has no opportunity to
+ * update the read_index in hv_pkt_iter_close() as
+ * hvs_stream_has_data() returns 0 for the FIN packet,
+ * so it won't get dequeued.
+ *
+ * In case hvs->recv_desc points to a malformed/short
+ * packet, return -EIO.
+ */
+ if (hvs->vsk->peer_shutdown & SEND_SHUTDOWN)
+ return 0;
+ else
+ return -EIO;
+ }
hvs->recv_desc = hv_pkt_iter_first(hvs->chan);
if (!hvs->recv_desc)
--
2.49.0
^ permalink raw reply related
* [PATCH-next 00/23] cgroup/cpuset: Enable runtime update of nohz_full and managed_irq CPUs
From: Waiman Long @ 2026-04-21 3:03 UTC (permalink / raw)
To: Tejun Heo, Johannes Weiner, Michal Koutný, Jonathan Corbet,
Shuah Khan, Catalin Marinas, Will Deacon, K. Y. Srinivasan,
Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li, Guenter Roeck,
Frederic Weisbecker, Paul E. McKenney, Neeraj Upadhyay,
Joel Fernandes, Josh Triplett, Boqun Feng, Uladzislau Rezki,
Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
Anna-Maria Behnsen, Ingo Molnar, Thomas Gleixner, Chen Ridong,
Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
Ben Segall, Mel Gorman, Valentin Schneider, K Prateek Nayak,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Simon Horman
Cc: cgroups, linux-doc, linux-kernel, linux-arm-kernel, linux-hyperv,
linux-hwmon, rcu, netdev, linux-kselftest, Costa Shulyupin,
Qiliang Yuan, Waiman Long
The "isolcpus=domain" CPU list implied by the HK_TYPE_DOMAIN housekeeping
cpumask is being dynamically updated whenever the set of cpuset isolated
partitions change. This patch series extends the isolated partition code
to make dynamic changes to the "nohz_full" and "isolcpus=managed_irq"
CPU lists. These CPU lists correspoond to equivalent changes in the
HK_TYPE_KERNEL_NOISE and HK_TYPE_MANAGED_IRQ housekeeping cpumasks.
To facilitate the changing of these CPU lists, the CPU hotplug code
which is doing a lot of heavy lifting is now being used. For changing
the "nohz_full" and HK_TYPE_KERNEL_NOISE cpumasks, the affected CPUs
are torn down into offline mode first. Then the housekeeping cpumask
and the corresponding cpumasks in the RCU, tick and watchdog subsystems
are modified. After that, the affected CPUs are brought online again.
It is slightly different with the "managed_irq" HK_TYPE_MANAGED_IRQ
cpumask, the cpumask is updated first and the affected CPUs are torn
down and brought up to move the managed interrupts away from the newly
isolated CPUs.
Using CPU hotplug does have its drawback when multiple isolated
partitions are being managed in a system. The CPU offline process uses
the stop_machine mechanism to stop all the CPUs except the one being torn
down. This will cause latency spike for isolated CPUs in other isolated
partitions. It is not a problem if only one isolated partition is
needed. This is an issue we need to address in the near future.
Patches 1-7 enables runtime updates to the nohz_full/managed_irq
cpumasks in sched/isolation, tick, RCU and watchdog subsystems.
Patches 8-17 modifies various subsystems that need access to the
HK_TYPE_MANAGED_IRQ cpumask and HK_TYPE_KERNEL_NOISE cpumask and its
aliases. Like the runtime modifiable HK_TYPE_DOMAIN cpumask, RCU is used
to protect access to cpumasks to avoid potential UAF problem. Patch 17
updates the housekeeping_dereference_check() to print a WARNING when
lockdep is enabled if those housekeeping cpumasks are used without the
proper lock protection.
Patch 18 introduces a new cpuhp_offline_cb() API that enables the
shutting down of a given list of CPUs, run a callback function and then
brought those CPUs up again while disallowing any concurrent CPU hotplug
activity.
Patches 19-23 updates the cpuset code, selftest and documentation to
allow change in isolated partition configuration to be reflected in
the housekeeping and other cpumasks dynamically.
As there is a slight overhead in enabling dynamic update to the nohz_full
cpumask, this new nohz_full and managed_irq runtime update feature
has to be explicitly opted in by adding a nohz_full kernel command
line parameter with or without a CPU list to indicate a desire to use
this feature. It is also because a number of subsystems have explicit
check of nohz_full at boot time to adjust their behavior which may not
be easy to modify after boot.
Waiman Long (23):
sched/isolation: Add HK_TYPE_KERNEL_NOISE_BOOT &
HK_TYPE_MANAGED_IRQ_BOOT
sched/isolation: Enhance housekeeping_update() to support updating
more than one HK cpumask
tick/nohz: Make nohz_full parameter optional
tick/nohz: Allow runtime changes in full dynticks CPUs
tick: Pass timer tick job to an online HK CPU in tick_cpu_dying()
rcu/nocbs: Allow runtime changes in RCU NOCBS cpumask
watchdog: Sync up with runtime change of isolated CPUs
arm64: topology: Use RCU to protect access to HK_TYPE_TICK cpumask
workqueue: Use RCU to protect access of HK_TYPE_TIMER cpumask
cpu: Use RCU to protect access of HK_TYPE_TIMER cpumask
hrtimer: Use RCU to protect access of HK_TYPE_TIMER cpumask
net: Use boot time housekeeping cpumask settings for now
sched/core: Use RCU to protect access of HK_TYPE_KERNEL_NOISE cpumask
hwmon/coretemp: Use RCU to protect access of HK_TYPE_MISC cpumask
Drivers: hv: Use RCU to protect access of HK_TYPE_MANAGED_IRQ cpumask
genirq/cpuhotplug: Use RCU to protect access of HK_TYPE_MANAGED_IRQ
cpumask
sched/isolation: Extend housekeeping_dereference_check() to cover
changes in nohz_full or manged_irqs cpumasks
cpu/hotplug: Add a new cpuhp_offline_cb() API
cgroup/cpuset: Improve check for calling housekeeping_update()
cgroup/cpuset: Enable runtime update of
HK_TYPE_{KERNEL_NOISE,MANAGED_IRQ} cpumasks
cgroup/cpuset: Limit the side effect of using CPU hotplug on isolated
partition
cgroup/cpuset: Prevent offline_disabled CPUs from being used in
isolated partition
cgroup/cpuset: Documentation and kselftest updates
Documentation/admin-guide/cgroup-v2.rst | 35 ++-
.../admin-guide/kernel-parameters.txt | 15 +-
arch/arm64/kernel/topology.c | 17 +-
drivers/hv/channel_mgmt.c | 15 +-
drivers/hv/vmbus_drv.c | 7 +-
drivers/hwmon/coretemp.c | 6 +-
include/linux/context_tracking.h | 8 +-
include/linux/cpuhplock.h | 9 +
include/linux/nmi.h | 2 +
include/linux/rcupdate.h | 2 +
include/linux/sched/isolation.h | 18 +-
include/linux/tick.h | 2 +
kernel/cgroup/cpuset-internal.h | 1 +
kernel/cgroup/cpuset.c | 292 +++++++++++++++++-
kernel/context_tracking.c | 19 +-
kernel/cpu.c | 72 +++++
kernel/irq/cpuhotplug.c | 1 +
kernel/irq/manage.c | 1 +
kernel/rcu/tree_nocb.h | 24 +-
kernel/sched/core.c | 4 +-
kernel/sched/isolation.c | 135 +++++---
kernel/time/hrtimer.c | 4 +-
kernel/time/tick-common.c | 16 +-
kernel/time/tick-sched.c | 48 ++-
kernel/watchdog.c | 24 ++
kernel/workqueue.c | 6 +-
net/core/net-sysfs.c | 2 +-
.../selftests/cgroup/test_cpuset_prs.sh | 70 ++++-
28 files changed, 747 insertions(+), 108 deletions(-)
--
2.53.0
^ permalink raw reply
* [PATCH 01/23] sched/isolation: Add HK_TYPE_KERNEL_NOISE_BOOT & HK_TYPE_MANAGED_IRQ_BOOT
From: Waiman Long @ 2026-04-21 3:03 UTC (permalink / raw)
To: Tejun Heo, Johannes Weiner, Michal Koutný, Jonathan Corbet,
Shuah Khan, Catalin Marinas, Will Deacon, K. Y. Srinivasan,
Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li, Guenter Roeck,
Frederic Weisbecker, Paul E. McKenney, Neeraj Upadhyay,
Joel Fernandes, Josh Triplett, Boqun Feng, Uladzislau Rezki,
Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
Anna-Maria Behnsen, Ingo Molnar, Thomas Gleixner, Chen Ridong,
Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
Ben Segall, Mel Gorman, Valentin Schneider, K Prateek Nayak,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Simon Horman
Cc: cgroups, linux-doc, linux-kernel, linux-arm-kernel, linux-hyperv,
linux-hwmon, rcu, netdev, linux-kselftest, Costa Shulyupin,
Qiliang Yuan, Waiman Long
In-Reply-To: <20260421030351.281436-1-longman@redhat.com>
Since commit 4fca0e550d50 ("sched/isolation: Save boot defined
domain flags"), HK_TYPE_DOMAIN_BOOT was added to record the boot
time "isolcpus{=domain}" setting. As we are going to make the
HK_TYPE_MANAGED_IRQ and HK_TYPE_KERNEL_NOISE housekeeping cpumasks
runtime modifiable, we need some additional cpumasks to record the boot
time settings to make sure that those housekeeping cpumasks will always
be a subset of their boot time equivalents.
Introduce the new HK_TYPE_KERNEL_NOISE_BOOT and HK_TYPE_MANAGED_IRQ_BOOT
housekeeping types to do that.
Signed-off-by: Waiman Long <longman@redhat.com>
---
include/linux/sched/isolation.h | 16 ++++++++++++++--
kernel/sched/isolation.c | 16 +++++++++-------
2 files changed, 23 insertions(+), 9 deletions(-)
diff --git a/include/linux/sched/isolation.h b/include/linux/sched/isolation.h
index dc3975ff1b2e..d1707f121e20 100644
--- a/include/linux/sched/isolation.h
+++ b/include/linux/sched/isolation.h
@@ -14,10 +14,22 @@ enum hk_type {
* is always a subset of HK_TYPE_DOMAIN_BOOT.
*/
HK_TYPE_DOMAIN,
- /* Inverse of boot-time isolcpus=managed_irq argument */
- HK_TYPE_MANAGED_IRQ,
+
/* Inverse of boot-time nohz_full= or isolcpus=nohz arguments */
+ HK_TYPE_KERNEL_NOISE_BOOT,
+ /*
+ * A subset of HK_TYPE_KERNEL_NOISE_BOOT as it may excludes some
+ * additional isolated CPUs at run time.
+ */
HK_TYPE_KERNEL_NOISE,
+
+ /* Inverse of boot-time isolcpus=managed_irq argument */
+ HK_TYPE_MANAGED_IRQ_BOOT,
+ /*
+ * A subset of HK_TYPE_MANAGED_IRQ_BOOT as it may excludes some
+ * additional isolated CPUs at run time.
+ */
+ HK_TYPE_MANAGED_IRQ,
HK_TYPE_MAX,
/*
diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c
index a947d75b43f1..9ec9ae510dc7 100644
--- a/kernel/sched/isolation.c
+++ b/kernel/sched/isolation.c
@@ -12,10 +12,12 @@
#include "sched.h"
enum hk_flags {
- HK_FLAG_DOMAIN_BOOT = BIT(HK_TYPE_DOMAIN_BOOT),
- HK_FLAG_DOMAIN = BIT(HK_TYPE_DOMAIN),
- HK_FLAG_MANAGED_IRQ = BIT(HK_TYPE_MANAGED_IRQ),
- HK_FLAG_KERNEL_NOISE = BIT(HK_TYPE_KERNEL_NOISE),
+ HK_FLAG_DOMAIN_BOOT = BIT(HK_TYPE_DOMAIN_BOOT),
+ HK_FLAG_DOMAIN = BIT(HK_TYPE_DOMAIN),
+ HK_FLAG_KERNEL_NOISE_BOOT = BIT(HK_TYPE_KERNEL_NOISE_BOOT),
+ HK_FLAG_KERNEL_NOISE = BIT(HK_TYPE_KERNEL_NOISE),
+ HK_FLAG_MANAGED_IRQ_BOOT = BIT(HK_TYPE_MANAGED_IRQ_BOOT),
+ HK_FLAG_MANAGED_IRQ = BIT(HK_TYPE_MANAGED_IRQ),
};
DEFINE_STATIC_KEY_FALSE(housekeeping_overridden);
@@ -315,7 +317,7 @@ static int __init housekeeping_nohz_full_setup(char *str)
{
unsigned long flags;
- flags = HK_FLAG_KERNEL_NOISE;
+ flags = HK_FLAG_KERNEL_NOISE | HK_FLAG_KERNEL_NOISE_BOOT;
return housekeeping_setup(str, flags);
}
@@ -334,7 +336,7 @@ static int __init housekeeping_isolcpus_setup(char *str)
*/
if (!strncmp(str, "nohz,", 5)) {
str += 5;
- flags |= HK_FLAG_KERNEL_NOISE;
+ flags |= HK_FLAG_KERNEL_NOISE | HK_FLAG_KERNEL_NOISE_BOOT;
continue;
}
@@ -346,7 +348,7 @@ static int __init housekeeping_isolcpus_setup(char *str)
if (!strncmp(str, "managed_irq,", 12)) {
str += 12;
- flags |= HK_FLAG_MANAGED_IRQ;
+ flags |= HK_FLAG_MANAGED_IRQ | HK_FLAG_MANAGED_IRQ_BOOT;
continue;
}
--
2.53.0
^ permalink raw reply related
* [PATCH 02/23] sched/isolation: Enhance housekeeping_update() to support updating more than one HK cpumask
From: Waiman Long @ 2026-04-21 3:03 UTC (permalink / raw)
To: Tejun Heo, Johannes Weiner, Michal Koutný, Jonathan Corbet,
Shuah Khan, Catalin Marinas, Will Deacon, K. Y. Srinivasan,
Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li, Guenter Roeck,
Frederic Weisbecker, Paul E. McKenney, Neeraj Upadhyay,
Joel Fernandes, Josh Triplett, Boqun Feng, Uladzislau Rezki,
Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
Anna-Maria Behnsen, Ingo Molnar, Thomas Gleixner, Chen Ridong,
Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
Ben Segall, Mel Gorman, Valentin Schneider, K Prateek Nayak,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Simon Horman
Cc: cgroups, linux-doc, linux-kernel, linux-arm-kernel, linux-hyperv,
linux-hwmon, rcu, netdev, linux-kselftest, Costa Shulyupin,
Qiliang Yuan, Waiman Long
In-Reply-To: <20260421030351.281436-1-longman@redhat.com>
The housekeeping_update() function currently allows update to the
HK_TYPE_DOMAIN cpumask only. As we are going to enable dynamic
modification of the other housekeeping cpumasks, we need to extend
it to support passing in the information about the HK cpumask(s) to
be updated. In cases where some HK cpumasks happen to be the same,
it will be more efficient to update multiple HK cpumasks in one single
call instead of calling it multiple times. Extend housekeeping_update()
to support that as well.
Also add the restriction that passed in isolated cpumask parameter
of housekeeping_update() must include all the CPUs isolated at boot
time. This is currently the case for cpuset anyway.
Signed-off-by: Waiman Long <longman@redhat.com>
---
include/linux/sched/isolation.h | 2 +-
kernel/cgroup/cpuset.c | 2 +-
kernel/sched/isolation.c | 99 +++++++++++++++++++++++----------
3 files changed, 71 insertions(+), 32 deletions(-)
diff --git a/include/linux/sched/isolation.h b/include/linux/sched/isolation.h
index d1707f121e20..a17f16e0156e 100644
--- a/include/linux/sched/isolation.h
+++ b/include/linux/sched/isolation.h
@@ -51,7 +51,7 @@ extern const struct cpumask *housekeeping_cpumask(enum hk_type type);
extern bool housekeeping_enabled(enum hk_type type);
extern void housekeeping_affine(struct task_struct *t, enum hk_type type);
extern bool housekeeping_test_cpu(int cpu, enum hk_type type);
-extern int housekeeping_update(struct cpumask *isol_mask);
+extern int housekeeping_update(struct cpumask *isol_mask, unsigned long flags);
extern void __init housekeeping_init(void);
#else
diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index 1335e437098e..a4eccb0ec0d1 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -1354,7 +1354,7 @@ static void cpuset_update_sd_hk_unlock(void)
*/
mutex_unlock(&cpuset_mutex);
cpus_read_unlock();
- WARN_ON_ONCE(housekeeping_update(isolated_hk_cpus));
+ WARN_ON_ONCE(housekeeping_update(isolated_hk_cpus, BIT(HK_TYPE_DOMAIN)));
mutex_unlock(&cpuset_top_mutex);
} else {
cpuset_full_unlock();
diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c
index 9ec9ae510dc7..965d6f8fe344 100644
--- a/kernel/sched/isolation.c
+++ b/kernel/sched/isolation.c
@@ -120,48 +120,87 @@ bool housekeeping_test_cpu(int cpu, enum hk_type type)
}
EXPORT_SYMBOL_GPL(housekeeping_test_cpu);
-int housekeeping_update(struct cpumask *isol_mask)
-{
- struct cpumask *trial, *old = NULL;
- int err;
+/* HK type processing table */
+static struct {
+ int type;
+ int boot_type;
+} hk_types[] = {
+ { HK_TYPE_DOMAIN, HK_TYPE_DOMAIN_BOOT },
+ { HK_TYPE_MANAGED_IRQ, HK_TYPE_MANAGED_IRQ_BOOT },
+ { HK_TYPE_KERNEL_NOISE, HK_TYPE_KERNEL_NOISE_BOOT }
+};
- trial = kmalloc(cpumask_size(), GFP_KERNEL);
- if (!trial)
- return -ENOMEM;
+#define HK_TYPE_CNT ARRAY_SIZE(hk_types)
- cpumask_andnot(trial, housekeeping_cpumask(HK_TYPE_DOMAIN_BOOT), isol_mask);
- if (!cpumask_intersects(trial, cpu_online_mask)) {
- kfree(trial);
- return -EINVAL;
+int housekeeping_update(struct cpumask *isol_mask, unsigned long flags)
+{
+ struct cpumask *trial[HK_TYPE_CNT];
+ int i, err = 0;
+
+ for (i = 0; i < HK_TYPE_CNT; i++) {
+ int type = hk_types[i].type;
+ int boot = hk_types[i].boot_type;
+
+ trial[i] = NULL;
+ if (flags & BIT(type)) {
+ trial[i] = kmalloc(cpumask_size(), GFP_KERNEL);
+ if (!trial[i]) {
+ err = -ENOMEM;
+ goto out;
+ }
+ /*
+ * The new HK cpumask must be a subset of its boot
+ * cpumask.
+ */
+ cpumask_andnot(trial[i], cpu_possible_mask, isol_mask);
+ if (!cpumask_intersects(trial[i], cpu_online_mask) ||
+ !cpumask_subset(trial[i], housekeeping_cpumask(boot))) {
+ i++;
+ err = -EINVAL;
+ goto out;
+ }
+ }
}
if (!housekeeping.flags)
static_branch_enable(&housekeeping_overridden);
- if (housekeeping.flags & HK_FLAG_DOMAIN)
- old = housekeeping_cpumask_dereference(HK_TYPE_DOMAIN);
- else
- WRITE_ONCE(housekeeping.flags, housekeeping.flags | HK_FLAG_DOMAIN);
- rcu_assign_pointer(housekeeping.cpumasks[HK_TYPE_DOMAIN], trial);
-
- synchronize_rcu();
-
- pci_probe_flush_workqueue();
- mem_cgroup_flush_workqueue();
- vmstat_flush_workqueue();
+ for (i = 0; i < HK_TYPE_CNT; i++) {
+ int type = hk_types[i].type;
+ struct cpumask *old;
- err = workqueue_unbound_housekeeping_update(housekeeping_cpumask(HK_TYPE_DOMAIN));
- WARN_ON_ONCE(err < 0);
+ if (!trial[i])
+ continue;
+ old = NULL;
+ if (housekeeping.flags & BIT(type))
+ old = housekeeping_cpumask_dereference(type);
+ rcu_assign_pointer(housekeeping.cpumasks[type], trial[i]);
+ trial[i] = old;
+ }
- err = tmigr_isolated_exclude_cpumask(isol_mask);
- WARN_ON_ONCE(err < 0);
+ if ((housekeeping.flags & flags) != flags)
+ WRITE_ONCE(housekeeping.flags, housekeeping.flags | flags);
- err = kthreads_update_housekeeping();
- WARN_ON_ONCE(err < 0);
+ synchronize_rcu();
- kfree(old);
+ if (flags & HK_FLAG_DOMAIN) {
+ /*
+ * HK_TYPE_DOMAIN specific callbacks
+ */
+ pci_probe_flush_workqueue();
+ mem_cgroup_flush_workqueue();
+ vmstat_flush_workqueue();
+
+ WARN_ON_ONCE(workqueue_unbound_housekeeping_update(
+ housekeeping_cpumask(HK_TYPE_DOMAIN)) < 0);
+ WARN_ON_ONCE(tmigr_isolated_exclude_cpumask(isol_mask) < 0);
+ WARN_ON_ONCE(kthreads_update_housekeeping() < 0);
+ }
- return 0;
+out:
+ while (--i >= 0)
+ kfree(trial[i]);
+ return err;
}
void __init housekeeping_init(void)
--
2.53.0
^ permalink raw reply related
* [PATCH 03/23] tick/nohz: Make nohz_full parameter optional
From: Waiman Long @ 2026-04-21 3:03 UTC (permalink / raw)
To: Tejun Heo, Johannes Weiner, Michal Koutný, Jonathan Corbet,
Shuah Khan, Catalin Marinas, Will Deacon, K. Y. Srinivasan,
Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li, Guenter Roeck,
Frederic Weisbecker, Paul E. McKenney, Neeraj Upadhyay,
Joel Fernandes, Josh Triplett, Boqun Feng, Uladzislau Rezki,
Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
Anna-Maria Behnsen, Ingo Molnar, Thomas Gleixner, Chen Ridong,
Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
Ben Segall, Mel Gorman, Valentin Schneider, K Prateek Nayak,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Simon Horman
Cc: cgroups, linux-doc, linux-kernel, linux-arm-kernel, linux-hyperv,
linux-hwmon, rcu, netdev, linux-kselftest, Costa Shulyupin,
Qiliang Yuan, Waiman Long
In-Reply-To: <20260421030351.281436-1-longman@redhat.com>
To provide nohz_full tick support, there is a set of tick dependency
masks that need to be evaluated on every IRQ and context switch.
Switching on nohz_full tick support at runtime will be problematic
as some of the tick dependency masks may not be properly set causing
problem down the road.
Allow nohz_full boot option to be specified without any
parameter to force enable nohz_full tick support without any
CPU in the tick_nohz_full_mask yet. The context_tracking_key and
tick_nohz_full_running flag will be enabled in this case to make
tick_nohz_full_enabled() return true.
There is still a small performance overhead by force enable nohz_full
this way. So it should only be used if there is a chance that some
CPUs may become isolated later via the cpuset isolated partition
functionality and better CPU isolation closed to nohz_full is desired.
Signed-off-by: Waiman Long <longman@redhat.com>
---
Documentation/admin-guide/kernel-parameters.txt | 15 +++++++++------
include/linux/context_tracking.h | 7 ++++++-
kernel/context_tracking.c | 4 +++-
kernel/rcu/tree_nocb.h | 2 +-
kernel/sched/isolation.c | 13 ++++++++++++-
kernel/time/tick-sched.c | 11 +++++++++--
6 files changed, 40 insertions(+), 12 deletions(-)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 95f97ce487a4..f0eedaebe9d6 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -4550,13 +4550,16 @@ Kernel parameters
Valid arguments: on, off
Default: on
- nohz_full= [KNL,BOOT,SMP,ISOL]
- The argument is a cpu list, as described above.
+ nohz_full[=cpu-list]
+ [KNL,BOOT,SMP,ISOL]
In kernels built with CONFIG_NO_HZ_FULL=y, set
- the specified list of CPUs whose tick will be stopped
- whenever possible. The boot CPU will be forced outside
- the range to maintain the timekeeping. Any CPUs
- in this list will have their RCU callbacks offloaded,
+ the specified list of CPUs whose tick will be
+ stopped whenever possible. If the argument is
+ not specified, nohz_full will be forced enabled
+ without any CPU in the nohz_full list yet.
+ The boot CPU will be forced outside the range
+ to maintain the timekeeping. Any CPUs in this
+ list will have their RCU callbacks offloaded,
just as if they had also been called out in the
rcu_nocbs= boot parameter.
diff --git a/include/linux/context_tracking.h b/include/linux/context_tracking.h
index af9fe87a0922..a3fea7f9fef6 100644
--- a/include/linux/context_tracking.h
+++ b/include/linux/context_tracking.h
@@ -9,8 +9,13 @@
#include <asm/ptrace.h>
-
#ifdef CONFIG_CONTEXT_TRACKING_USER
+/*
+ * Pass CONTEXT_TRACKING_FORCE_ENABLE to ct_cpu_track_user() to force enable
+ * user context tracking.
+ */
+#define CONTEXT_TRACKING_FORCE_ENABLE (-1)
+
extern void ct_cpu_track_user(int cpu);
/* Called with interrupts disabled. */
diff --git a/kernel/context_tracking.c b/kernel/context_tracking.c
index a743e7ffa6c0..925999de1a28 100644
--- a/kernel/context_tracking.c
+++ b/kernel/context_tracking.c
@@ -678,7 +678,9 @@ void __init ct_cpu_track_user(int cpu)
{
static __initdata bool initialized = false;
- if (!per_cpu(context_tracking.active, cpu)) {
+ if (cpu == CONTEXT_TRACKING_FORCE_ENABLE) {
+ static_branch_inc(&context_tracking_key);
+ } else if (!per_cpu(context_tracking.active, cpu)) {
per_cpu(context_tracking.active, cpu) = true;
static_branch_inc(&context_tracking_key);
}
diff --git a/kernel/rcu/tree_nocb.h b/kernel/rcu/tree_nocb.h
index b3337c7231cc..2d06dcb61f37 100644
--- a/kernel/rcu/tree_nocb.h
+++ b/kernel/rcu/tree_nocb.h
@@ -1267,7 +1267,7 @@ void __init rcu_init_nohz(void)
struct shrinker * __maybe_unused lazy_rcu_shrinker;
#if defined(CONFIG_NO_HZ_FULL)
- if (tick_nohz_full_running && !cpumask_empty(tick_nohz_full_mask))
+ if (tick_nohz_full_running)
cpumask = tick_nohz_full_mask;
#endif
diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c
index 965d6f8fe344..c233d55a1e95 100644
--- a/kernel/sched/isolation.c
+++ b/kernel/sched/isolation.c
@@ -268,6 +268,7 @@ static int __init housekeeping_setup(char *str, unsigned long flags)
}
alloc_bootmem_cpumask_var(&non_housekeeping_mask);
+
if (cpulist_parse(str, non_housekeeping_mask) < 0) {
pr_warn("Housekeeping: nohz_full= or isolcpus= incorrect CPU range\n");
goto free_non_housekeeping_mask;
@@ -277,6 +278,13 @@ static int __init housekeeping_setup(char *str, unsigned long flags)
cpumask_andnot(housekeeping_staging,
cpu_possible_mask, non_housekeeping_mask);
+ /*
+ * Allow "nohz_full" without parameter to force enable nohz_full
+ * at boot time without any CPUs in the nohz_full list yet.
+ */
+ if ((flags & HK_FLAG_KERNEL_NOISE) && !*str)
+ goto setup_housekeeping_staging;
+
first_cpu = cpumask_first_and(cpu_present_mask, housekeeping_staging);
if (first_cpu >= nr_cpu_ids || first_cpu >= setup_max_cpus) {
__cpumask_set_cpu(smp_processor_id(), housekeeping_staging);
@@ -290,6 +298,7 @@ static int __init housekeeping_setup(char *str, unsigned long flags)
if (cpumask_empty(non_housekeeping_mask))
goto free_housekeeping_staging;
+setup_housekeeping_staging:
if (!housekeeping.flags) {
/* First setup call ("nohz_full=" or "isolcpus=") */
enum hk_type type;
@@ -357,10 +366,12 @@ static int __init housekeeping_nohz_full_setup(char *str)
unsigned long flags;
flags = HK_FLAG_KERNEL_NOISE | HK_FLAG_KERNEL_NOISE_BOOT;
+ if (*str == '=')
+ str++;
return housekeeping_setup(str, flags);
}
-__setup("nohz_full=", housekeeping_nohz_full_setup);
+__setup("nohz_full", housekeeping_nohz_full_setup);
static int __init housekeeping_isolcpus_setup(char *str)
{
diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index 9e5264458414..ed877b2c9040 100644
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -676,8 +676,15 @@ void __init tick_nohz_init(void)
}
}
- for_each_cpu(cpu, tick_nohz_full_mask)
- ct_cpu_track_user(cpu);
+ /*
+ * Force enable context_tracking_key if tick_nohz_full_mask empty
+ */
+ if (cpumask_empty(tick_nohz_full_mask)) {
+ ct_cpu_track_user(CONTEXT_TRACKING_FORCE_ENABLE);
+ } else {
+ for_each_cpu(cpu, tick_nohz_full_mask)
+ ct_cpu_track_user(cpu);
+ }
ret = cpuhp_setup_state_nocalls(CPUHP_AP_ONLINE_DYN,
"kernel/nohz:predown", NULL,
--
2.53.0
^ permalink raw reply related
* [PATCH 04/23] tick/nohz: Allow runtime changes in full dynticks CPUs
From: Waiman Long @ 2026-04-21 3:03 UTC (permalink / raw)
To: Tejun Heo, Johannes Weiner, Michal Koutný, Jonathan Corbet,
Shuah Khan, Catalin Marinas, Will Deacon, K. Y. Srinivasan,
Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li, Guenter Roeck,
Frederic Weisbecker, Paul E. McKenney, Neeraj Upadhyay,
Joel Fernandes, Josh Triplett, Boqun Feng, Uladzislau Rezki,
Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
Anna-Maria Behnsen, Ingo Molnar, Thomas Gleixner, Chen Ridong,
Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
Ben Segall, Mel Gorman, Valentin Schneider, K Prateek Nayak,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Simon Horman
Cc: cgroups, linux-doc, linux-kernel, linux-arm-kernel, linux-hyperv,
linux-hwmon, rcu, netdev, linux-kselftest, Costa Shulyupin,
Qiliang Yuan, Waiman Long
In-Reply-To: <20260421030351.281436-1-longman@redhat.com>
Full dynticks can only be enabled if "nohz_full" boot option has been
been specified with or without parameter. Any change in the list of
nohz_full CPUs have to be reflected in tick_nohz_full_mask. Introduce
a new tick_nohz_full_update_cpus() helper that can be called to update
the tick_nohz_full_mask at run time. The housekeeping_update() function
is modified to call the new helper when the HK_TYPE_KERNEL_NOSIE cpumask
is going to be changed.
We also need to enable CPU context tracking for those CPUs that
are in tick_nohz_full_mask. So remove __init from tick_nohz_init()
and ct_cpu_track_user() so that they be called later when an isolated
cpuset partition is being created. The __ro_after_init attribute is
taken away from context_tracking_key as well.
Also add a new ct_cpu_untrack_user() function to reverse the action of
ct_cpu_track_user() in case we need to disable the nohz_full mode of
a CPU.
With nohz_full enabled, the boot CPU (typically CPU 0) will be the
tick CPU which cannot be shut down easily. So the boot CPU should not
be used in an isolated cpuset partition.
With runtime modification of nohz_full CPUs, tick_do_timer_cpu can become
TICK_DO_TIMER_NONE. So remove the two TICK_DO_TIMER_NONE WARN_ON_ONCE()
checks in tick-sched.c to avoid unnecessary warnings.
Signed-off-by: Waiman Long <longman@redhat.com>
---
include/linux/context_tracking.h | 1 +
include/linux/tick.h | 2 ++
kernel/context_tracking.c | 15 ++++++++++---
kernel/sched/isolation.c | 3 +++
kernel/time/tick-sched.c | 37 ++++++++++++++++++++++++++------
5 files changed, 48 insertions(+), 10 deletions(-)
diff --git a/include/linux/context_tracking.h b/include/linux/context_tracking.h
index a3fea7f9fef6..1a6b816f1ad6 100644
--- a/include/linux/context_tracking.h
+++ b/include/linux/context_tracking.h
@@ -17,6 +17,7 @@
#define CONTEXT_TRACKING_FORCE_ENABLE (-1)
extern void ct_cpu_track_user(int cpu);
+extern void ct_cpu_untrack_user(int cpu);
/* Called with interrupts disabled. */
extern void __ct_user_enter(enum ctx_state state);
diff --git a/include/linux/tick.h b/include/linux/tick.h
index 738007d6f577..05586f14461c 100644
--- a/include/linux/tick.h
+++ b/include/linux/tick.h
@@ -274,6 +274,7 @@ static inline void tick_dep_clear_signal(struct signal_struct *signal,
extern void tick_nohz_full_kick_cpu(int cpu);
extern void __tick_nohz_task_switch(void);
extern void __init tick_nohz_full_setup(cpumask_var_t cpumask);
+extern void tick_nohz_full_update_cpus(struct cpumask *cpumask);
#else
static inline bool tick_nohz_full_enabled(void) { return false; }
static inline bool tick_nohz_full_cpu(int cpu) { return false; }
@@ -299,6 +300,7 @@ static inline void tick_dep_clear_signal(struct signal_struct *signal,
static inline void tick_nohz_full_kick_cpu(int cpu) { }
static inline void __tick_nohz_task_switch(void) { }
static inline void tick_nohz_full_setup(cpumask_var_t cpumask) { }
+static inline void tick_nohz_full_update_cpus(struct cpumask *cpumask) { }
#endif
static inline void tick_nohz_task_switch(void)
diff --git a/kernel/context_tracking.c b/kernel/context_tracking.c
index 925999de1a28..394e432630a3 100644
--- a/kernel/context_tracking.c
+++ b/kernel/context_tracking.c
@@ -411,7 +411,7 @@ static __always_inline void ct_kernel_enter(bool user, int offset) { }
#define CREATE_TRACE_POINTS
#include <trace/events/context_tracking.h>
-DEFINE_STATIC_KEY_FALSE_RO(context_tracking_key);
+DEFINE_STATIC_KEY_FALSE(context_tracking_key);
EXPORT_SYMBOL_GPL(context_tracking_key);
static noinstr bool context_tracking_recursion_enter(void)
@@ -674,9 +674,9 @@ void user_exit_callable(void)
}
NOKPROBE_SYMBOL(user_exit_callable);
-void __init ct_cpu_track_user(int cpu)
+void ct_cpu_track_user(int cpu)
{
- static __initdata bool initialized = false;
+ static bool initialized;
if (cpu == CONTEXT_TRACKING_FORCE_ENABLE) {
static_branch_inc(&context_tracking_key);
@@ -700,6 +700,15 @@ void __init ct_cpu_track_user(int cpu)
initialized = true;
}
+void ct_cpu_untrack_user(int cpu)
+{
+ if (!per_cpu(context_tracking.active, cpu))
+ return;
+
+ per_cpu(context_tracking.active, cpu) = false;
+ static_branch_dec(&context_tracking_key);
+}
+
#ifdef CONFIG_CONTEXT_TRACKING_USER_FORCE
void __init context_tracking_init(void)
{
diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c
index c233d55a1e95..48b155e0b290 100644
--- a/kernel/sched/isolation.c
+++ b/kernel/sched/isolation.c
@@ -181,6 +181,9 @@ int housekeeping_update(struct cpumask *isol_mask, unsigned long flags)
if ((housekeeping.flags & flags) != flags)
WRITE_ONCE(housekeeping.flags, housekeeping.flags | flags);
+ if (flags & HK_FLAG_KERNEL_NOISE)
+ tick_nohz_full_update_cpus(isol_mask);
+
synchronize_rcu();
if (flags & HK_FLAG_DOMAIN) {
diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index ed877b2c9040..7baa757ca45f 100644
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -241,9 +241,6 @@ static void tick_sched_do_timer(struct tick_sched *ts, ktime_t now)
tick_cpu = READ_ONCE(tick_do_timer_cpu);
if (IS_ENABLED(CONFIG_NO_HZ_COMMON) && unlikely(tick_cpu == TICK_DO_TIMER_NONE)) {
-#ifdef CONFIG_NO_HZ_FULL
- WARN_ON_ONCE(tick_nohz_full_running);
-#endif
WRITE_ONCE(tick_do_timer_cpu, cpu);
tick_cpu = cpu;
}
@@ -629,6 +626,36 @@ void __init tick_nohz_full_setup(cpumask_var_t cpumask)
tick_nohz_full_running = true;
}
+/* Get the new set of run-time nohz CPU list & update accordingly */
+void tick_nohz_full_update_cpus(struct cpumask *cpumask)
+{
+ int cpu;
+
+ if (!tick_nohz_full_running) {
+ pr_warn_once("Full dynticks cannot be enabled without the nohz_full kernel boot parameter!\n");
+ return;
+ }
+
+ /*
+ * To properly enable/disable nohz_full dynticks for the affected CPUs,
+ * the new nohz_full CPUs have to be copied to tick_nohz_full_mask and
+ * ct_cpu_track_user/ct_cpu_untrack_user() will have to be called
+ * for those CPUs that have their states changed. Those CPUs should be
+ * in an offline state.
+ */
+ for_each_cpu_andnot(cpu, cpumask, tick_nohz_full_mask) {
+ WARN_ON_ONCE(cpu_online(cpu));
+ ct_cpu_track_user(cpu);
+ cpumask_set_cpu(cpu, tick_nohz_full_mask);
+ }
+
+ for_each_cpu_andnot(cpu, tick_nohz_full_mask, cpumask) {
+ WARN_ON_ONCE(cpu_online(cpu));
+ ct_cpu_untrack_user(cpu);
+ cpumask_clear_cpu(cpu, tick_nohz_full_mask);
+ }
+}
+
bool tick_nohz_cpu_hotpluggable(unsigned int cpu)
{
/*
@@ -1238,10 +1265,6 @@ static bool can_stop_idle_tick(int cpu, struct tick_sched *ts)
*/
if (tick_cpu == cpu)
return false;
-
- /* Should not happen for nohz-full */
- if (WARN_ON_ONCE(tick_cpu == TICK_DO_TIMER_NONE))
- return false;
}
return true;
--
2.53.0
^ permalink raw reply related
* [PATCH 05/23] tick: Pass timer tick job to an online HK CPU in tick_cpu_dying()
From: Waiman Long @ 2026-04-21 3:03 UTC (permalink / raw)
To: Tejun Heo, Johannes Weiner, Michal Koutný, Jonathan Corbet,
Shuah Khan, Catalin Marinas, Will Deacon, K. Y. Srinivasan,
Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li, Guenter Roeck,
Frederic Weisbecker, Paul E. McKenney, Neeraj Upadhyay,
Joel Fernandes, Josh Triplett, Boqun Feng, Uladzislau Rezki,
Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
Anna-Maria Behnsen, Ingo Molnar, Thomas Gleixner, Chen Ridong,
Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
Ben Segall, Mel Gorman, Valentin Schneider, K Prateek Nayak,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Simon Horman
Cc: cgroups, linux-doc, linux-kernel, linux-arm-kernel, linux-hyperv,
linux-hwmon, rcu, netdev, linux-kselftest, Costa Shulyupin,
Qiliang Yuan, Waiman Long
In-Reply-To: <20260421030351.281436-1-longman@redhat.com>
In tick_cpu_dying(), if the dying CPU is the current timekeeper,
it has to pass the job over to another CPU. The current code passes
it to another online CPU. However, that CPU may not be a timer tick
housekeeping CPU. If that happens, another CPU will have to manually
take it over again later. Avoid this unnecessary work by directly
assigning an online housekeeping CPU.
Use READ_ONCE/WRITE_ONCE() to access tick_do_timer_cpu in case the
non-HK CPUs may not be in stop machine in the future.
Signed-off-by: Waiman Long <longman@redhat.com>
---
kernel/time/tick-common.c | 16 ++++++++++++----
1 file changed, 12 insertions(+), 4 deletions(-)
diff --git a/kernel/time/tick-common.c b/kernel/time/tick-common.c
index d305d8521896..4834a1b9044b 100644
--- a/kernel/time/tick-common.c
+++ b/kernel/time/tick-common.c
@@ -17,6 +17,7 @@
#include <linux/profile.h>
#include <linux/sched.h>
#include <linux/module.h>
+#include <linux/sched/isolation.h>
#include <trace/events/power.h>
#include <asm/irq_regs.h>
@@ -394,12 +395,19 @@ int tick_cpu_dying(unsigned int dying_cpu)
{
/*
* If the current CPU is the timekeeper, it's the only one that can
- * safely hand over its duty. Also all online CPUs are in stop
- * machine, guaranteed not to be idle, therefore there is no
+ * safely hand over its duty. Also all online housekeeping CPUs are
+ * in stop machine, guaranteed not to be idle, therefore there is no
* concurrency and it's safe to pick any online successor.
*/
- if (tick_do_timer_cpu == dying_cpu)
- tick_do_timer_cpu = cpumask_first(cpu_online_mask);
+ if (READ_ONCE(tick_do_timer_cpu) == dying_cpu) {
+ unsigned int new_cpu;
+
+ guard(rcu)();
+ new_cpu = cpumask_first_and(cpu_online_mask, housekeeping_cpumask(HK_TYPE_TICK));
+ if (WARN_ON_ONCE(new_cpu >= nr_cpu_ids))
+ new_cpu = cpumask_first(cpu_online_mask);
+ WRITE_ONCE(tick_do_timer_cpu, new_cpu);
+ }
/* Make sure the CPU won't try to retake the timekeeping duty */
tick_sched_timer_dying(dying_cpu);
--
2.53.0
^ permalink raw reply related
* [PATCH 06/23] rcu/nocbs: Allow runtime changes in RCU NOCBS cpumask
From: Waiman Long @ 2026-04-21 3:03 UTC (permalink / raw)
To: Tejun Heo, Johannes Weiner, Michal Koutný, Jonathan Corbet,
Shuah Khan, Catalin Marinas, Will Deacon, K. Y. Srinivasan,
Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li, Guenter Roeck,
Frederic Weisbecker, Paul E. McKenney, Neeraj Upadhyay,
Joel Fernandes, Josh Triplett, Boqun Feng, Uladzislau Rezki,
Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
Anna-Maria Behnsen, Ingo Molnar, Thomas Gleixner, Chen Ridong,
Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
Ben Segall, Mel Gorman, Valentin Schneider, K Prateek Nayak,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Simon Horman
Cc: cgroups, linux-doc, linux-kernel, linux-arm-kernel, linux-hyperv,
linux-hwmon, rcu, netdev, linux-kselftest, Costa Shulyupin,
Qiliang Yuan, Waiman Long
In-Reply-To: <20260421030351.281436-1-longman@redhat.com>
We can make use of the rcu_nocb_cpu_offload()/rcu_nocb_cpu_deoffload()
APIs to enable RCU NO-CB CPU offloading of newly isolated CPUs and
deoffloading of de-isolated CPUs.
Add a new rcu_nocb_update_cpus() helper to do that and call it directly
from housekeeping_update() when the HK_TYPE_KERNEL_NOISE cpumask is
being changed.
This dynamic RCU NO-CB CPU offloading feature can only used if either
"rcs_nocbs" or the "nohz_full" boot command parameters are used with or
without parameter so that the proper RCU NO-CB resources are properly
initialized at boot time.
Signed-off-by: Waiman Long <longman@redhat.com>
---
include/linux/rcupdate.h | 2 ++
kernel/rcu/tree_nocb.h | 22 ++++++++++++++++++++++
kernel/sched/isolation.c | 4 +++-
3 files changed, 27 insertions(+), 1 deletion(-)
diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h
index 04f3f86a4145..987e3d1d413e 100644
--- a/include/linux/rcupdate.h
+++ b/include/linux/rcupdate.h
@@ -150,6 +150,7 @@ void rcu_init_nohz(void);
int rcu_nocb_cpu_offload(int cpu);
int rcu_nocb_cpu_deoffload(int cpu);
void rcu_nocb_flush_deferred_wakeup(void);
+void rcu_nocb_update_cpus(struct cpumask *cpumask);
#define RCU_NOCB_LOCKDEP_WARN(c, s) RCU_LOCKDEP_WARN(c, s)
@@ -159,6 +160,7 @@ static inline void rcu_init_nohz(void) { }
static inline int rcu_nocb_cpu_offload(int cpu) { return -EINVAL; }
static inline int rcu_nocb_cpu_deoffload(int cpu) { return 0; }
static inline void rcu_nocb_flush_deferred_wakeup(void) { }
+static inline void rcu_nocb_update_cpus(struct cpumask *cpumask) { }
#define RCU_NOCB_LOCKDEP_WARN(c, s)
diff --git a/kernel/rcu/tree_nocb.h b/kernel/rcu/tree_nocb.h
index 2d06dcb61f37..b2daba1e5cb9 100644
--- a/kernel/rcu/tree_nocb.h
+++ b/kernel/rcu/tree_nocb.h
@@ -1173,6 +1173,28 @@ int rcu_nocb_cpu_offload(int cpu)
}
EXPORT_SYMBOL_GPL(rcu_nocb_cpu_offload);
+void rcu_nocb_update_cpus(struct cpumask *cpumask)
+{
+ int cpu, ret;
+
+ if (!rcu_state.nocb_is_setup) {
+ pr_warn_once("Dynamic RCU NOCB cannot be enabled without nohz_full/rcu_nocbs kernel boot parameter!\n");
+ return;
+ }
+
+ for_each_cpu_andnot(cpu, cpumask, rcu_nocb_mask) {
+ ret = rcu_nocb_cpu_offload(cpu);
+ if (WARN_ON_ONCE(ret))
+ return;
+ }
+
+ for_each_cpu_andnot(cpu, rcu_nocb_mask, cpumask) {
+ ret = rcu_nocb_cpu_deoffload(cpu);
+ if (WARN_ON_ONCE(ret))
+ return;
+ }
+}
+
#ifdef CONFIG_RCU_LAZY
static unsigned long
lazy_rcu_shrink_count(struct shrinker *shrink, struct shrink_control *sc)
diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c
index 48b155e0b290..b5635484ec69 100644
--- a/kernel/sched/isolation.c
+++ b/kernel/sched/isolation.c
@@ -181,8 +181,10 @@ int housekeeping_update(struct cpumask *isol_mask, unsigned long flags)
if ((housekeeping.flags & flags) != flags)
WRITE_ONCE(housekeeping.flags, housekeeping.flags | flags);
- if (flags & HK_FLAG_KERNEL_NOISE)
+ if (flags & HK_FLAG_KERNEL_NOISE) {
tick_nohz_full_update_cpus(isol_mask);
+ rcu_nocb_update_cpus(isol_mask);
+ }
synchronize_rcu();
--
2.53.0
^ permalink raw reply related
* [PATCH 07/23] watchdog: Sync up with runtime change of isolated CPUs
From: Waiman Long @ 2026-04-21 3:03 UTC (permalink / raw)
To: Tejun Heo, Johannes Weiner, Michal Koutný, Jonathan Corbet,
Shuah Khan, Catalin Marinas, Will Deacon, K. Y. Srinivasan,
Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li, Guenter Roeck,
Frederic Weisbecker, Paul E. McKenney, Neeraj Upadhyay,
Joel Fernandes, Josh Triplett, Boqun Feng, Uladzislau Rezki,
Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
Anna-Maria Behnsen, Ingo Molnar, Thomas Gleixner, Chen Ridong,
Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
Ben Segall, Mel Gorman, Valentin Schneider, K Prateek Nayak,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Simon Horman
Cc: cgroups, linux-doc, linux-kernel, linux-arm-kernel, linux-hyperv,
linux-hwmon, rcu, netdev, linux-kselftest, Costa Shulyupin,
Qiliang Yuan, Waiman Long
In-Reply-To: <20260421030351.281436-1-longman@redhat.com>
At bootup, watchdog will exclude nohz_full CPUs specified at boot time.
As we are now enabling runtime changes to nohz_full CPUs, the list of
CPUs with watchdog timer running should be updated to exclude the
current set of isolated CPUs.
Add a new watchdog_cpumask_update() helper to be invoked
by housekeeping_update() when the HK_TYPE_KERNEL_NOISE
(HK_TYPE_TIMER) cpumask is being updated to update watchdog_cpumask and
watchdog_allowed_mask for soft lockup detector. The cpumask updates will
be done when the affected CPUs are in the offline state. When those
CPUs are brought up later, the new cpumask will be used to determine
if any hard/soft watchdog should be enabled again.
Signed-off-by: Waiman Long <longman@redhat.com>
---
include/linux/nmi.h | 2 ++
kernel/sched/isolation.c | 1 +
kernel/watchdog.c | 24 ++++++++++++++++++++++++
3 files changed, 27 insertions(+)
diff --git a/include/linux/nmi.h b/include/linux/nmi.h
index bc1162895f35..5bf941d2b168 100644
--- a/include/linux/nmi.h
+++ b/include/linux/nmi.h
@@ -17,6 +17,7 @@
void lockup_detector_init(void);
void lockup_detector_retry_init(void);
void lockup_detector_soft_poweroff(void);
+void watchdog_cpumask_update(struct cpumask *mask);
extern int watchdog_user_enabled;
extern int watchdog_thresh;
@@ -37,6 +38,7 @@ extern int sysctl_hardlockup_all_cpu_backtrace;
static inline void lockup_detector_init(void) { }
static inline void lockup_detector_retry_init(void) { }
static inline void lockup_detector_soft_poweroff(void) { }
+static inline void watchdog_cpumask_update(struct cpumask *mask) { }
#endif /* !CONFIG_LOCKUP_DETECTOR */
#ifdef CONFIG_SOFTLOCKUP_DETECTOR
diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c
index b5635484ec69..1f3f1c83dd12 100644
--- a/kernel/sched/isolation.c
+++ b/kernel/sched/isolation.c
@@ -184,6 +184,7 @@ int housekeeping_update(struct cpumask *isol_mask, unsigned long flags)
if (flags & HK_FLAG_KERNEL_NOISE) {
tick_nohz_full_update_cpus(isol_mask);
rcu_nocb_update_cpus(isol_mask);
+ watchdog_cpumask_update(isol_mask);
}
synchronize_rcu();
diff --git a/kernel/watchdog.c b/kernel/watchdog.c
index 87dd5e0f6968..498c1463b843 100644
--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -1071,6 +1071,30 @@ static inline void lockup_detector_setup(void)
}
#endif /* !CONFIG_SOFTLOCKUP_DETECTOR */
+/**
+ * watchdog_cpumask_update - update watchdog_cpumask & watchdog_allowed_mask
+ * @isol_mask: cpumask of isolated CPUs
+ *
+ * Update watchdog_cpumask and watchdog_allowed_mask to be inverse of the
+ * given isolated cpumask to disable watchdog activities on isolated CPUs.
+ * It should be called with the affected CPUs in offline state which will be
+ * brought up online later.
+ *
+ * Any changes made in watchdog_cpumask by users via the sysctl parameter will
+ * be overridden. However, proc_watchdog_update() isn't called. So change will
+ * only happens on CPUs that will brought up later on to minimize changes to
+ * the existing watchdog configuration.
+ */
+void watchdog_cpumask_update(struct cpumask *isol_mask)
+{
+ mutex_lock(&watchdog_mutex);
+ cpumask_andnot(&watchdog_cpumask, cpu_possible_mask, isol_mask);
+#ifdef CONFIG_SOFTLOCKUP_DETECTOR
+ cpumask_copy(&watchdog_allowed_mask, &watchdog_cpumask);
+#endif
+ mutex_unlock(&watchdog_mutex);
+}
+
/**
* lockup_detector_soft_poweroff - Interface to stop lockup detector(s)
*
--
2.53.0
^ permalink raw reply related
* [PATCH 08/23] arm64: topology: Use RCU to protect access to HK_TYPE_TICK cpumask
From: Waiman Long @ 2026-04-21 3:03 UTC (permalink / raw)
To: Tejun Heo, Johannes Weiner, Michal Koutný, Jonathan Corbet,
Shuah Khan, Catalin Marinas, Will Deacon, K. Y. Srinivasan,
Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li, Guenter Roeck,
Frederic Weisbecker, Paul E. McKenney, Neeraj Upadhyay,
Joel Fernandes, Josh Triplett, Boqun Feng, Uladzislau Rezki,
Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
Anna-Maria Behnsen, Ingo Molnar, Thomas Gleixner, Chen Ridong,
Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
Ben Segall, Mel Gorman, Valentin Schneider, K Prateek Nayak,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Simon Horman
Cc: cgroups, linux-doc, linux-kernel, linux-arm-kernel, linux-hyperv,
linux-hwmon, rcu, netdev, linux-kselftest, Costa Shulyupin,
Qiliang Yuan, Waiman Long
In-Reply-To: <20260421030351.281436-1-longman@redhat.com>
As the HK_TYPE_TICK cpumask is going to be changeable at run time, we
need to use RCU to protect access to the cpumask to prevent it from
going away in the middle of the operation.
Signed-off-by: Waiman Long <longman@redhat.com>
---
arch/arm64/kernel/topology.c | 17 ++++++++++++++---
1 file changed, 14 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c
index b32f13358fbb..48f150801689 100644
--- a/arch/arm64/kernel/topology.c
+++ b/arch/arm64/kernel/topology.c
@@ -173,6 +173,7 @@ void arch_cpu_idle_enter(void)
if (!amu_fie_cpu_supported(cpu))
return;
+ guard(rcu)();
/* Kick in AMU update but only if one has not happened already */
if (housekeeping_cpu(cpu, HK_TYPE_TICK) &&
time_is_before_jiffies(per_cpu(cpu_amu_samples.last_scale_update, cpu)))
@@ -187,11 +188,16 @@ int arch_freq_get_on_cpu(int cpu)
unsigned int start_cpu = cpu;
unsigned long last_update;
unsigned int freq = 0;
+ bool hk_cpu;
u64 scale;
if (!amu_fie_cpu_supported(cpu) || !arch_scale_freq_ref(cpu))
return -EOPNOTSUPP;
+ scoped_guard(rcu) {
+ hk_cpu = housekeeping_cpu(cpu, HK_TYPE_TICK);
+ }
+
while (1) {
amu_sample = per_cpu_ptr(&cpu_amu_samples, cpu);
@@ -204,16 +210,21 @@ int arch_freq_get_on_cpu(int cpu)
* (and thus freq scale), if available, for given policy: this boils
* down to identifying an active cpu within the same freq domain, if any.
*/
- if (!housekeeping_cpu(cpu, HK_TYPE_TICK) ||
+ if (!hk_cpu ||
time_is_before_jiffies(last_update + msecs_to_jiffies(AMU_SAMPLE_EXP_MS))) {
struct cpufreq_policy *policy = cpufreq_cpu_get(cpu);
+ bool hk_intersects;
int ref_cpu;
if (!policy)
return -EINVAL;
- if (!cpumask_intersects(policy->related_cpus,
- housekeeping_cpumask(HK_TYPE_TICK))) {
+ scoped_guard(rcu) {
+ hk_intersects = cpumask_intersects(policy->related_cpus,
+ housekeeping_cpumask(HK_TYPE_TICK));
+ }
+
+ if (!hk_intersects) {
cpufreq_cpu_put(policy);
return -EOPNOTSUPP;
}
--
2.53.0
^ permalink raw reply related
* [PATCH 09/23] workqueue: Use RCU to protect access of HK_TYPE_TIMER cpumask
From: Waiman Long @ 2026-04-21 3:03 UTC (permalink / raw)
To: Tejun Heo, Johannes Weiner, Michal Koutný, Jonathan Corbet,
Shuah Khan, Catalin Marinas, Will Deacon, K. Y. Srinivasan,
Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li, Guenter Roeck,
Frederic Weisbecker, Paul E. McKenney, Neeraj Upadhyay,
Joel Fernandes, Josh Triplett, Boqun Feng, Uladzislau Rezki,
Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
Anna-Maria Behnsen, Ingo Molnar, Thomas Gleixner, Chen Ridong,
Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
Ben Segall, Mel Gorman, Valentin Schneider, K Prateek Nayak,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Simon Horman
Cc: cgroups, linux-doc, linux-kernel, linux-arm-kernel, linux-hyperv,
linux-hwmon, rcu, netdev, linux-kselftest, Costa Shulyupin,
Qiliang Yuan, Waiman Long
In-Reply-To: <20260421030351.281436-1-longman@redhat.com>
As HK_TYPE_TIMER cpumask is going to be changeable at run time, use
RCU to protect access to the cpumask.
Signed-off-by: Waiman Long <longman@redhat.com>
---
kernel/workqueue.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/kernel/workqueue.c b/kernel/workqueue.c
index 08b1c786b463..2dab3872281a 100644
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -2557,8 +2557,10 @@ static void __queue_delayed_work(int cpu, struct workqueue_struct *wq,
if (housekeeping_enabled(HK_TYPE_TIMER)) {
/* If the current cpu is a housekeeping cpu, use it. */
cpu = smp_processor_id();
- if (!housekeeping_test_cpu(cpu, HK_TYPE_TIMER))
- cpu = housekeeping_any_cpu(HK_TYPE_TIMER);
+ scoped_guard(rcu) {
+ if (!housekeeping_test_cpu(cpu, HK_TYPE_TIMER))
+ cpu = housekeeping_any_cpu(HK_TYPE_TIMER);
+ }
add_timer_on(timer, cpu);
} else {
if (likely(cpu == WORK_CPU_UNBOUND))
--
2.53.0
^ permalink raw reply related
* [PATCH 10/23] cpu: Use RCU to protect access of HK_TYPE_TIMER cpumask
From: Waiman Long @ 2026-04-21 3:03 UTC (permalink / raw)
To: Tejun Heo, Johannes Weiner, Michal Koutný, Jonathan Corbet,
Shuah Khan, Catalin Marinas, Will Deacon, K. Y. Srinivasan,
Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li, Guenter Roeck,
Frederic Weisbecker, Paul E. McKenney, Neeraj Upadhyay,
Joel Fernandes, Josh Triplett, Boqun Feng, Uladzislau Rezki,
Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
Anna-Maria Behnsen, Ingo Molnar, Thomas Gleixner, Chen Ridong,
Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
Ben Segall, Mel Gorman, Valentin Schneider, K Prateek Nayak,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Simon Horman
Cc: cgroups, linux-doc, linux-kernel, linux-arm-kernel, linux-hyperv,
linux-hwmon, rcu, netdev, linux-kselftest, Costa Shulyupin,
Qiliang Yuan, Waiman Long
In-Reply-To: <20260421030351.281436-1-longman@redhat.com>
As HK_TYPE_TIMER cpumask is going to be changeable at run time, use
RCU to protect access to the cpumask.
Signed-off-by: Waiman Long <longman@redhat.com>
---
kernel/cpu.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/kernel/cpu.c b/kernel/cpu.c
index bc4f7a9ba64e..0d02b5d7a7ba 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -1890,6 +1890,8 @@ int freeze_secondary_cpus(int primary)
cpu_maps_update_begin();
if (primary == -1) {
primary = cpumask_first(cpu_online_mask);
+
+ guard(rcu)();
if (!housekeeping_cpu(primary, HK_TYPE_TIMER))
primary = housekeeping_any_cpu(HK_TYPE_TIMER);
} else {
--
2.53.0
^ permalink raw reply related
* [PATCH 11/23] hrtimer: Use RCU to protect access of HK_TYPE_TIMER cpumask
From: Waiman Long @ 2026-04-21 3:03 UTC (permalink / raw)
To: Tejun Heo, Johannes Weiner, Michal Koutný, Jonathan Corbet,
Shuah Khan, Catalin Marinas, Will Deacon, K. Y. Srinivasan,
Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li, Guenter Roeck,
Frederic Weisbecker, Paul E. McKenney, Neeraj Upadhyay,
Joel Fernandes, Josh Triplett, Boqun Feng, Uladzislau Rezki,
Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
Anna-Maria Behnsen, Ingo Molnar, Thomas Gleixner, Chen Ridong,
Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
Ben Segall, Mel Gorman, Valentin Schneider, K Prateek Nayak,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Simon Horman
Cc: cgroups, linux-doc, linux-kernel, linux-arm-kernel, linux-hyperv,
linux-hwmon, rcu, netdev, linux-kselftest, Costa Shulyupin,
Qiliang Yuan, Waiman Long
In-Reply-To: <20260421030351.281436-1-longman@redhat.com>
As HK_TYPE_TIMER cpumask is going to be changeable at run time, use
RCU to protect access to the cpumask.
The access of HK_TYPE_TIMER cpumask within hrtimers_cpu_dying() is
protected as interrupt is disabled and all the other CPUs are stopped
when this function is invoked as part of the CPU tear down process.
Signed-off-by: Waiman Long <longman@redhat.com>
---
kernel/time/hrtimer.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index 000fb6ba7d74..85495400a193 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -242,8 +242,10 @@ static bool hrtimer_suitable_target(struct hrtimer *timer, struct hrtimer_clock_
static inline struct hrtimer_cpu_base *get_target_base(struct hrtimer_cpu_base *base, bool pinned)
{
if (!hrtimer_base_is_online(base)) {
- int cpu = cpumask_any_and(cpu_online_mask, housekeeping_cpumask(HK_TYPE_TIMER));
+ int cpu;
+ guard(rcu)();
+ cpu = cpumask_any_and(cpu_online_mask, housekeeping_cpumask(HK_TYPE_TIMER));
return &per_cpu(hrtimer_bases, cpu);
}
--
2.53.0
^ permalink raw reply related
* [PATCH 12/23] net: Use boot time housekeeping cpumask settings for now
From: Waiman Long @ 2026-04-21 3:03 UTC (permalink / raw)
To: Tejun Heo, Johannes Weiner, Michal Koutný, Jonathan Corbet,
Shuah Khan, Catalin Marinas, Will Deacon, K. Y. Srinivasan,
Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li, Guenter Roeck,
Frederic Weisbecker, Paul E. McKenney, Neeraj Upadhyay,
Joel Fernandes, Josh Triplett, Boqun Feng, Uladzislau Rezki,
Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
Anna-Maria Behnsen, Ingo Molnar, Thomas Gleixner, Chen Ridong,
Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
Ben Segall, Mel Gorman, Valentin Schneider, K Prateek Nayak,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Simon Horman
Cc: cgroups, linux-doc, linux-kernel, linux-arm-kernel, linux-hyperv,
linux-hwmon, rcu, netdev, linux-kselftest, Costa Shulyupin,
Qiliang Yuan, Waiman Long
In-Reply-To: <20260421030351.281436-1-longman@redhat.com>
Checking of the housekeeping cpumasks was originally added to the
store_rps_map() method in commit 07bbecb34106 ("net: Restrict receive
packets queuing to housekeeping CPUs"). This store method ensures
that only CPUs in housekeeping cpumask can be set in the "rps_cpus"
of the individual network devices. Later, commit 662ff1aac854 ("net:
Keep ignoring isolated cpuset change") replaced HK_TYPE_DOMAIN by
HK_TYPE_DOMAIN_BOOT as HK_TYPE_DOMAIN cpumask was going to be made
changeable at run time.
Now we are going to make the nohz_full housekeeping cpumask to be
changeable at run time too. Ideally we should update the housekeeping
cpumasks changes should be propagated to the RPS cpumasks, but we
need more thoughts to properly synchronize them. Ignore the run time
housekeeping cpumask changes and only use the boot time "isolcpus="
and "nohz_full=" values for now.
Signed-off-by: Waiman Long <longman@redhat.com>
---
net/core/net-sysfs.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/net/core/net-sysfs.c b/net/core/net-sysfs.c
index e430645748a7..9e666cc3b34c 100644
--- a/net/core/net-sysfs.c
+++ b/net/core/net-sysfs.c
@@ -1023,7 +1023,7 @@ int rps_cpumask_housekeeping(struct cpumask *mask)
{
if (!cpumask_empty(mask)) {
cpumask_and(mask, mask, housekeeping_cpumask(HK_TYPE_DOMAIN_BOOT));
- cpumask_and(mask, mask, housekeeping_cpumask(HK_TYPE_WQ));
+ cpumask_and(mask, mask, housekeeping_cpumask(HK_TYPE_KERNEL_NOISE_BOOT));
if (cpumask_empty(mask))
return -EINVAL;
}
--
2.53.0
^ permalink raw reply related
* [PATCH 13/23] sched/core: Use RCU to protect access of HK_TYPE_KERNEL_NOISE cpumask
From: Waiman Long @ 2026-04-21 3:03 UTC (permalink / raw)
To: Tejun Heo, Johannes Weiner, Michal Koutný, Jonathan Corbet,
Shuah Khan, Catalin Marinas, Will Deacon, K. Y. Srinivasan,
Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li, Guenter Roeck,
Frederic Weisbecker, Paul E. McKenney, Neeraj Upadhyay,
Joel Fernandes, Josh Triplett, Boqun Feng, Uladzislau Rezki,
Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
Anna-Maria Behnsen, Ingo Molnar, Thomas Gleixner, Chen Ridong,
Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
Ben Segall, Mel Gorman, Valentin Schneider, K Prateek Nayak,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Simon Horman
Cc: cgroups, linux-doc, linux-kernel, linux-arm-kernel, linux-hyperv,
linux-hwmon, rcu, netdev, linux-kselftest, Costa Shulyupin,
Qiliang Yuan, Waiman Long
In-Reply-To: <20260421030351.281436-1-longman@redhat.com>
As HK_TYPE_KERNEL_NOISE is going to be changeable at run time, use
RCU to protect access to the cpumask when needed. Some access of the
HK_TYPE_KERNEL_NOISE cpumask is done inside the tick code that has
interrupt disabled which is a rcu_read_lock() critical section. Anyway,
housekeeping_cpumask() will warn if they are used in invalid context.
Signed-off-by: Waiman Long <longman@redhat.com>
---
kernel/sched/core.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 8952f5764517..6ae00c23d8a7 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -1261,6 +1261,8 @@ int get_nohz_timer_target(void)
struct sched_domain *sd;
const struct cpumask *hk_mask;
+ guard(rcu)();
+
if (housekeeping_cpu(cpu, HK_TYPE_KERNEL_NOISE)) {
if (!idle_cpu(cpu))
return cpu;
@@ -1269,8 +1271,6 @@ int get_nohz_timer_target(void)
hk_mask = housekeeping_cpumask(HK_TYPE_KERNEL_NOISE);
- guard(rcu)();
-
for_each_domain(cpu, sd) {
for_each_cpu_and(i, sched_domain_span(sd), hk_mask) {
if (cpu == i)
--
2.53.0
^ permalink raw reply related
* [PATCH 14/23] hwmon/coretemp: Use RCU to protect access of HK_TYPE_MISC cpumask
From: Waiman Long @ 2026-04-21 3:03 UTC (permalink / raw)
To: Tejun Heo, Johannes Weiner, Michal Koutný, Jonathan Corbet,
Shuah Khan, Catalin Marinas, Will Deacon, K. Y. Srinivasan,
Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li, Guenter Roeck,
Frederic Weisbecker, Paul E. McKenney, Neeraj Upadhyay,
Joel Fernandes, Josh Triplett, Boqun Feng, Uladzislau Rezki,
Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
Anna-Maria Behnsen, Ingo Molnar, Thomas Gleixner, Chen Ridong,
Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
Ben Segall, Mel Gorman, Valentin Schneider, K Prateek Nayak,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Simon Horman
Cc: cgroups, linux-doc, linux-kernel, linux-arm-kernel, linux-hyperv,
linux-hwmon, rcu, netdev, linux-kselftest, Costa Shulyupin,
Qiliang Yuan, Waiman Long
In-Reply-To: <20260421030351.281436-1-longman@redhat.com>
As HK_TYPE_MISC cpumask is going to be changeable at run time, use RCU
to protect access to the cpumask.
Signed-off-by: Waiman Long <longman@redhat.com>
---
drivers/hwmon/coretemp.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/drivers/hwmon/coretemp.c b/drivers/hwmon/coretemp.c
index 6a0d94711ead..0284ea6a3b79 100644
--- a/drivers/hwmon/coretemp.c
+++ b/drivers/hwmon/coretemp.c
@@ -563,8 +563,10 @@ static int create_core_data(struct platform_device *pdev, unsigned int cpu,
u32 eax, edx;
int err;
- if (!housekeeping_cpu(cpu, HK_TYPE_MISC))
- return 0;
+ scoped_guard(rcu) {
+ if (!housekeeping_cpu(cpu, HK_TYPE_MISC))
+ return 0;
+ }
tdata = init_temp_data(pdata, cpu, pkg_flag);
if (!tdata)
--
2.53.0
^ permalink raw reply related
* [PATCH 15/23] Drivers: hv: Use RCU to protect access of HK_TYPE_MANAGED_IRQ cpumask
From: Waiman Long @ 2026-04-21 3:03 UTC (permalink / raw)
To: Tejun Heo, Johannes Weiner, Michal Koutný, Jonathan Corbet,
Shuah Khan, Catalin Marinas, Will Deacon, K. Y. Srinivasan,
Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li, Guenter Roeck,
Frederic Weisbecker, Paul E. McKenney, Neeraj Upadhyay,
Joel Fernandes, Josh Triplett, Boqun Feng, Uladzislau Rezki,
Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
Anna-Maria Behnsen, Ingo Molnar, Thomas Gleixner, Chen Ridong,
Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
Ben Segall, Mel Gorman, Valentin Schneider, K Prateek Nayak,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Simon Horman
Cc: cgroups, linux-doc, linux-kernel, linux-arm-kernel, linux-hyperv,
linux-hwmon, rcu, netdev, linux-kselftest, Costa Shulyupin,
Qiliang Yuan, Waiman Long
In-Reply-To: <20260421030351.281436-1-longman@redhat.com>
As HK_TYPE_MANAGED_IRQ cpumask is going to be changeable at run time,
use RCU to protect access to the cpumask. The memory allocation
alloc_cpumask_var() call is done before taking the rcu_read_lock()
as this call can be sleepable.
Signed-off-by: Waiman Long <longman@redhat.com>
---
drivers/hv/channel_mgmt.c | 15 +++++++++------
drivers/hv/vmbus_drv.c | 7 +++++--
2 files changed, 14 insertions(+), 8 deletions(-)
diff --git a/drivers/hv/channel_mgmt.c b/drivers/hv/channel_mgmt.c
index 84eb0a6a0b54..44441dafed90 100644
--- a/drivers/hv/channel_mgmt.c
+++ b/drivers/hv/channel_mgmt.c
@@ -752,13 +752,16 @@ static void init_vp_index(struct vmbus_channel *channel)
u32 i, ncpu = num_online_cpus();
cpumask_var_t available_mask;
struct cpumask *allocated_mask;
- const struct cpumask *hk_mask = housekeeping_cpumask(HK_TYPE_MANAGED_IRQ);
+ const struct cpumask *hk_mask;
u32 target_cpu;
int numa_node;
+ bool alloc_ok;
- if (!perf_chn ||
- !alloc_cpumask_var(&available_mask, GFP_KERNEL) ||
- cpumask_empty(hk_mask)) {
+ alloc_ok = alloc_cpumask_var(&available_mask, GFP_KERNEL);
+ guard(rcu)();
+ hk_mask = housekeeping_cpumask(HK_TYPE_MANAGED_IRQ);
+
+ if (!perf_chn || !alloc_ok || cpumask_empty(hk_mask)) {
/*
* If the channel is not a performance critical
* channel, bind it to VMBUS_CONNECT_CPU.
@@ -770,7 +773,7 @@ static void init_vp_index(struct vmbus_channel *channel)
channel->target_cpu = VMBUS_CONNECT_CPU;
if (perf_chn)
hv_set_allocated_cpu(VMBUS_CONNECT_CPU);
- return;
+ goto out_free;
}
for (i = 1; i <= ncpu + 1; i++) {
@@ -808,7 +811,7 @@ static void init_vp_index(struct vmbus_channel *channel)
}
channel->target_cpu = target_cpu;
-
+out_free:
free_cpumask_var(available_mask);
}
diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
index 3faa74e49a6b..60c7a5ac15c0 100644
--- a/drivers/hv/vmbus_drv.c
+++ b/drivers/hv/vmbus_drv.c
@@ -1763,8 +1763,11 @@ int vmbus_channel_set_cpu(struct vmbus_channel *channel, u32 target_cpu)
if (target_cpu >= nr_cpumask_bits)
return -EINVAL;
- if (!cpumask_test_cpu(target_cpu, housekeeping_cpumask(HK_TYPE_MANAGED_IRQ)))
- return -EINVAL;
+ scoped_guard(rcu) {
+ if (!cpumask_test_cpu(target_cpu,
+ housekeeping_cpumask(HK_TYPE_MANAGED_IRQ)))
+ return -EINVAL;
+ }
if (!cpu_online(target_cpu))
return -EINVAL;
--
2.53.0
^ permalink raw reply related
* [PATCH 16/23] genirq/cpuhotplug: Use RCU to protect access of HK_TYPE_MANAGED_IRQ cpumask
From: Waiman Long @ 2026-04-21 3:03 UTC (permalink / raw)
To: Tejun Heo, Johannes Weiner, Michal Koutný, Jonathan Corbet,
Shuah Khan, Catalin Marinas, Will Deacon, K. Y. Srinivasan,
Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li, Guenter Roeck,
Frederic Weisbecker, Paul E. McKenney, Neeraj Upadhyay,
Joel Fernandes, Josh Triplett, Boqun Feng, Uladzislau Rezki,
Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
Anna-Maria Behnsen, Ingo Molnar, Thomas Gleixner, Chen Ridong,
Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
Ben Segall, Mel Gorman, Valentin Schneider, K Prateek Nayak,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Simon Horman
Cc: cgroups, linux-doc, linux-kernel, linux-arm-kernel, linux-hyperv,
linux-hwmon, rcu, netdev, linux-kselftest, Costa Shulyupin,
Qiliang Yuan, Waiman Long
In-Reply-To: <20260421030351.281436-1-longman@redhat.com>
As HK_TYPE_MANAGED_IRQ cpumask is going to be changeable at run time,
use RCU to protect access to the cpumask.
To enable the new HK_TYPE_MANAGED_IRQ cpumask to take effect, the
following steps can be done.
1) Update the HK_TYPE_MANAGED_IRQ cpumask to take out the newly isolated
CPUs and add back the de-isolated CPUs.
2) Tear down the affected CPUs to cause irq_migrate_all_off_this_cpu()
to be called on the affected CPUs to migrate the irqs to other
HK_TYPE_MANAGED_IRQ housekeeping CPUs.
3) Bring up the previously offline CPUs to invoke
irq_affinity_online_cpu() to allow the newly de-isolated CPUs to
be used for managed irqs.
Signed-off-by: Waiman Long <longman@redhat.com>
---
kernel/irq/cpuhotplug.c | 1 +
kernel/irq/manage.c | 1 +
2 files changed, 2 insertions(+)
diff --git a/kernel/irq/cpuhotplug.c b/kernel/irq/cpuhotplug.c
index cd5689e383b0..86437c78f1f2 100644
--- a/kernel/irq/cpuhotplug.c
+++ b/kernel/irq/cpuhotplug.c
@@ -196,6 +196,7 @@ static bool hk_should_isolate(struct irq_data *data, unsigned int cpu)
if (!housekeeping_enabled(HK_TYPE_MANAGED_IRQ))
return false;
+ guard(rcu)();
hk_mask = housekeeping_cpumask(HK_TYPE_MANAGED_IRQ);
if (cpumask_subset(irq_data_get_effective_affinity_mask(data), hk_mask))
return false;
diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c
index 2e8072437826..8270c4de260b 100644
--- a/kernel/irq/manage.c
+++ b/kernel/irq/manage.c
@@ -263,6 +263,7 @@ int irq_do_set_affinity(struct irq_data *data, const struct cpumask *mask, bool
housekeeping_enabled(HK_TYPE_MANAGED_IRQ)) {
const struct cpumask *hk_mask;
+ guard(rcu)();
hk_mask = housekeeping_cpumask(HK_TYPE_MANAGED_IRQ);
cpumask_and(tmp_mask, mask, hk_mask);
--
2.53.0
^ permalink raw reply related
* [PATCH 17/23] sched/isolation: Extend housekeeping_dereference_check() to cover changes in nohz_full or manged_irqs cpumasks
From: Waiman Long @ 2026-04-21 3:03 UTC (permalink / raw)
To: Tejun Heo, Johannes Weiner, Michal Koutný, Jonathan Corbet,
Shuah Khan, Catalin Marinas, Will Deacon, K. Y. Srinivasan,
Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li, Guenter Roeck,
Frederic Weisbecker, Paul E. McKenney, Neeraj Upadhyay,
Joel Fernandes, Josh Triplett, Boqun Feng, Uladzislau Rezki,
Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
Anna-Maria Behnsen, Ingo Molnar, Thomas Gleixner, Chen Ridong,
Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
Ben Segall, Mel Gorman, Valentin Schneider, K Prateek Nayak,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Simon Horman
Cc: cgroups, linux-doc, linux-kernel, linux-arm-kernel, linux-hyperv,
linux-hwmon, rcu, netdev, linux-kselftest, Costa Shulyupin,
Qiliang Yuan, Waiman Long
In-Reply-To: <20260421030351.281436-1-longman@redhat.com>
As we are going to make HK_TYPE_KERNEL_NOISE and
HK_TYPE_MANAGED_IRQ housekeeping cpumasks run time changeable, extend
housekeeping_dereference_check() to cover changes to those cpumasks
as well.
Signed-off-by: Waiman Long <longman@redhat.com>
---
kernel/sched/isolation.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c
index 1f3f1c83dd12..1647e7b08bac 100644
--- a/kernel/sched/isolation.c
+++ b/kernel/sched/isolation.c
@@ -38,7 +38,8 @@ EXPORT_SYMBOL_GPL(housekeeping_enabled);
static bool housekeeping_dereference_check(enum hk_type type)
{
- if (IS_ENABLED(CONFIG_LOCKDEP) && type == HK_TYPE_DOMAIN) {
+ if (IS_ENABLED(CONFIG_LOCKDEP) &&
+ (BIT(type) & (HK_FLAG_DOMAIN | HK_FLAG_KERNEL_NOISE | HK_FLAG_MANAGED_IRQ))) {
/* Cpuset isn't even writable yet? */
if (system_state <= SYSTEM_SCHEDULING)
return true;
--
2.53.0
^ permalink raw reply related
* [PATCH 18/23] cpu/hotplug: Add a new cpuhp_offline_cb() API
From: Waiman Long @ 2026-04-21 3:03 UTC (permalink / raw)
To: Tejun Heo, Johannes Weiner, Michal Koutný, Jonathan Corbet,
Shuah Khan, Catalin Marinas, Will Deacon, K. Y. Srinivasan,
Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li, Guenter Roeck,
Frederic Weisbecker, Paul E. McKenney, Neeraj Upadhyay,
Joel Fernandes, Josh Triplett, Boqun Feng, Uladzislau Rezki,
Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
Anna-Maria Behnsen, Ingo Molnar, Thomas Gleixner, Chen Ridong,
Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
Ben Segall, Mel Gorman, Valentin Schneider, K Prateek Nayak,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Simon Horman
Cc: cgroups, linux-doc, linux-kernel, linux-arm-kernel, linux-hyperv,
linux-hwmon, rcu, netdev, linux-kselftest, Costa Shulyupin,
Qiliang Yuan, Waiman Long
In-Reply-To: <20260421030351.281436-1-longman@redhat.com>
Add a new cpuhp_offline_cb() API that allows us to offline a set of
CPUs one-by-one, run the given callback function and then bring those
CPUs back online again while inhibiting any concurrent CPU hotplug
operations from happening.
This new API can be used to enable runtime adjustment of nohz_full and
isolcpus boot command line options. A new cpuhp_offline_cb_mode flag
is also added to signal that the system is in this offline callback
transient state so that some hotplug operations can be optimized out
if we choose to.
Signed-off-by: Waiman Long <longman@redhat.com>
---
include/linux/cpuhplock.h | 9 +++++
kernel/cpu.c | 70 +++++++++++++++++++++++++++++++++++++++
2 files changed, 79 insertions(+)
diff --git a/include/linux/cpuhplock.h b/include/linux/cpuhplock.h
index 286b3ab92e15..37637baa32eb 100644
--- a/include/linux/cpuhplock.h
+++ b/include/linux/cpuhplock.h
@@ -9,7 +9,9 @@
#include <linux/cleanup.h>
#include <linux/errno.h>
+#include <linux/cpumask_types.h>
+typedef int (*cpuhp_cb_t)(void *arg);
struct device;
extern int lockdep_is_cpus_held(void);
@@ -29,6 +31,8 @@ void clear_tasks_mm_cpumask(int cpu);
int remove_cpu(unsigned int cpu);
int cpu_device_down(struct device *dev);
void smp_shutdown_nonboot_cpus(unsigned int primary_cpu);
+int cpuhp_offline_cb(struct cpumask *mask, cpuhp_cb_t func, void *arg);
+extern bool cpuhp_offline_cb_mode;
#else /* CONFIG_HOTPLUG_CPU */
@@ -43,6 +47,11 @@ static inline void cpu_hotplug_disable(void) { }
static inline void cpu_hotplug_enable(void) { }
static inline int remove_cpu(unsigned int cpu) { return -EPERM; }
static inline void smp_shutdown_nonboot_cpus(unsigned int primary_cpu) { }
+static inline int cpuhp_offline_cb(struct cpumask *mask, cpuhp_cb_t func, void *arg)
+{
+ return -EPERM;
+}
+#define cpuhp_offline_cb_mode false
#endif /* !CONFIG_HOTPLUG_CPU */
DEFINE_LOCK_GUARD_0(cpus_read_lock, cpus_read_lock(), cpus_read_unlock())
diff --git a/kernel/cpu.c b/kernel/cpu.c
index 0d02b5d7a7ba..9b32f742cd1d 100644
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -1520,6 +1520,76 @@ int remove_cpu(unsigned int cpu)
}
EXPORT_SYMBOL_GPL(remove_cpu);
+bool cpuhp_offline_cb_mode;
+
+/**
+ * cpuhp_offline_cb - offline CPUs, invoke callback function & online CPUs afterward
+ * @mask: A mask of CPUs to be taken offline and then online
+ * @func: A callback function to be invoked while the given CPUs are offline
+ * @arg: Argument to be passed back to the callback function
+ *
+ * Return: 0 if successful, an error code otherwise
+ */
+int cpuhp_offline_cb(struct cpumask *mask, cpuhp_cb_t func, void *arg)
+{
+ int off_cpu, on_cpu, ret, ret2 = 0;
+
+ if (WARN_ON_ONCE(cpumask_empty(mask) ||
+ !cpumask_subset(mask, cpu_online_mask)))
+ return -EINVAL;
+
+ pr_debug("%s: begin (CPU list = %*pbl)\n", __func__, cpumask_pr_args(mask));
+ lock_device_hotplug();
+ cpuhp_offline_cb_mode = true;
+ /*
+ * If all offline operations succeed, off_cpu should become nr_cpu_ids.
+ */
+ for_each_cpu(off_cpu, mask) {
+ ret = device_offline(get_cpu_device(off_cpu));
+ if (unlikely(ret))
+ break;
+ }
+ if (!ret)
+ ret = func(arg);
+
+ /* Bring previously offline CPUs back online */
+ for_each_cpu(on_cpu, mask) {
+ int retries = 0;
+
+ if (on_cpu == off_cpu)
+ break;
+
+retry:
+ ret2 = device_online(get_cpu_device(on_cpu));
+
+ /*
+ * With the unlikely event that CPU hotplug is disabled while
+ * this operation is in progress, we will need to wait a bit
+ * for hotplug to hopefully be re-enabled again. If not, print
+ * a warning and return the error.
+ *
+ * cpu_hotplug_disabled is supposed to be accessed while
+ * holding the cpu_add_remove_lock mutex. So we need to
+ * use the data_race() macro to access it here.
+ */
+ while ((ret2 == -EBUSY) && data_race(cpu_hotplug_disabled) &&
+ (++retries <= 5)) {
+ msleep(20);
+ if (!data_race(cpu_hotplug_disabled))
+ goto retry;
+ }
+ if (ret2) {
+ pr_warn("%s: Failed to bring CPU %d back online!\n",
+ __func__, on_cpu);
+ break;
+ }
+ }
+ cpuhp_offline_cb_mode = false;
+ unlock_device_hotplug();
+ pr_debug("%s: end\n", __func__);
+ return ret ? ret : ret2;
+}
+
void smp_shutdown_nonboot_cpus(unsigned int primary_cpu)
{
unsigned int cpu;
--
2.53.0
^ permalink raw reply related
* [PATCH 19/23] cgroup/cpuset: Improve check for calling housekeeping_update()
From: Waiman Long @ 2026-04-21 3:03 UTC (permalink / raw)
To: Tejun Heo, Johannes Weiner, Michal Koutný, Jonathan Corbet,
Shuah Khan, Catalin Marinas, Will Deacon, K. Y. Srinivasan,
Haiyang Zhang, Wei Liu, Dexuan Cui, Long Li, Guenter Roeck,
Frederic Weisbecker, Paul E. McKenney, Neeraj Upadhyay,
Joel Fernandes, Josh Triplett, Boqun Feng, Uladzislau Rezki,
Steven Rostedt, Mathieu Desnoyers, Lai Jiangshan, Zqiang,
Anna-Maria Behnsen, Ingo Molnar, Thomas Gleixner, Chen Ridong,
Peter Zijlstra, Juri Lelli, Vincent Guittot, Dietmar Eggemann,
Ben Segall, Mel Gorman, Valentin Schneider, K Prateek Nayak,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Simon Horman
Cc: cgroups, linux-doc, linux-kernel, linux-arm-kernel, linux-hyperv,
linux-hwmon, rcu, netdev, linux-kselftest, Costa Shulyupin,
Qiliang Yuan, Waiman Long
In-Reply-To: <20260421030351.281436-1-longman@redhat.com>
By making sure that isolated_hk_cpus matches isolated_cpus at boot time,
we can more accurately determine if calling housekeeping_update()
is needed by comparing if the two cpumasks are equal. The
update_housekeeping flag still have a use in cpuset_handle_hotplug()
to determine if a work function should be queued to invoke
cpuset_update_sd_hk_unlock() as it is not supposed to look at
isolated_hk_cpus without holding cpuset_top_mutex.
Signed-off-by: Waiman Long <longman@redhat.com>
---
kernel/cgroup/cpuset.c | 36 ++++++++++++++++++++----------------
1 file changed, 20 insertions(+), 16 deletions(-)
diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c
index a4eccb0ec0d1..1b0c50b46a49 100644
--- a/kernel/cgroup/cpuset.c
+++ b/kernel/cgroup/cpuset.c
@@ -1339,26 +1339,29 @@ static void cpuset_update_sd_hk_unlock(void)
__releases(&cpuset_mutex)
__releases(&cpuset_top_mutex)
{
+ update_housekeeping = false;
+
/* force_sd_rebuild will be cleared in rebuild_sched_domains_locked() */
if (force_sd_rebuild)
rebuild_sched_domains_locked();
- if (update_housekeeping) {
- update_housekeeping = false;
- cpumask_copy(isolated_hk_cpus, isolated_cpus);
-
- /*
- * housekeeping_update() is now called without holding
- * cpus_read_lock and cpuset_mutex. Only cpuset_top_mutex
- * is still being held for mutual exclusion.
- */
- mutex_unlock(&cpuset_mutex);
- cpus_read_unlock();
- WARN_ON_ONCE(housekeeping_update(isolated_hk_cpus, BIT(HK_TYPE_DOMAIN)));
- mutex_unlock(&cpuset_top_mutex);
- } else {
+ if (cpumask_equal(isolated_hk_cpus, isolated_cpus)) {
+ /* No housekeeping cpumask update needed */
cpuset_full_unlock();
+ return;
}
+
+ cpumask_copy(isolated_hk_cpus, isolated_cpus);
+
+ /*
+ * housekeeping_update() is now called without holding
+ * cpus_read_lock and cpuset_mutex. Only cpuset_top_mutex
+ * is still being held for mutual exclusion.
+ */
+ mutex_unlock(&cpuset_mutex);
+ cpus_read_unlock();
+ WARN_ON_ONCE(housekeeping_update(isolated_hk_cpus, BIT(HK_TYPE_DOMAIN)));
+ mutex_unlock(&cpuset_top_mutex);
}
/*
@@ -3692,10 +3695,11 @@ int __init cpuset_init(void)
BUG_ON(!alloc_cpumask_var(&cpus_attach, GFP_KERNEL));
- if (housekeeping_enabled(HK_TYPE_DOMAIN_BOOT))
+ if (housekeeping_enabled(HK_TYPE_DOMAIN_BOOT)) {
cpumask_andnot(isolated_cpus, cpu_possible_mask,
housekeeping_cpumask(HK_TYPE_DOMAIN_BOOT));
-
+ cpumask_copy(isolated_hk_cpus, isolated_cpus);
+ }
return 0;
}
--
2.53.0
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox