From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.remote-ip=74.125.227.134 ARC-Seal: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; cv=none; t=1795158404; b=h6jZGpd3I1Upe9tX7gMRX901ZeJeQRZwkJRGreuKlro/MIoU4uS6ilpuMqG9LG7h+jAJ 9KndBeU0SxqkU+LPDEYSJKm5dLWFtpjfr4pir1bq6yjkHwC4S/JxrBLliXgeu37GKm20y a80UihIhQ3KjRumLPngxkUa2uqz2GY0ZZw9O+Swsew9mpFE6v+aNHjtidYT5SJucddKbc WkiOeTNbGYwywlX5KwRee/eX7w6nVYDTKFeCUN0jQOEj+wlu0HimpBJZd86CFC/g4rklq 1M3kKObrcy8Hb+RCfVsqpCsrjvQN/gfIf3uaJQtJkSdRgm8hCOZqwBCz4G1qDIHrDtA== ARC-Message-Signature: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; c=relaxed/relaxed; t=1795158404; h=DMARC-Filter:Received:DKIM-Signature:X-Google-DKIM-Signature: X-Gm-Message-State:X-Gm-Gg:X-Received:Received:From:To:Cc:Subject: Date:Message-ID:X-Mailer:MIME-Version:Content-Transfer-Encoding; bh=DNfItCD3Z4BoYtins2/yOU+Coq3wbYX7eM32Pd4EAjg=; b=t8PevVqgn0X7VgoKjMYPA9utJ4hXj5Cgw3HLZNUG+kIsV3d59lsoZ6sXsx765UzRxAR0 eX/zSUfbG+b2l4e95yt+y534FoQG8oCGHKHZ5RnPawhAR7zeAasJeX+52F7lQU8Z2vkkt jOAWMkk1/f5/87xoVovRLfCs7HbTCz2kzn/A+joGhqTxJOUlbL2Dc4xypne3iCKabdWfy sQN6I4/0joBbhP0B9aOMJGyI6Rszr6d/qaemu1MlqElCmXfrwiPVMhFo+cSs92OIobu4Y CWYUCCJvFS2DnfF8TvIL7KTL+L4wBhkDQz+dXvCwj0Gi3d6SOXCFkS1JUBsYA+IP8+A== ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20260707 header.b=yNkqUQDp; arc=none smtp.remote-ip=74.125.227.134 Received: from mail-pj2-f6.google.com (mail-pj2-f6.google.com [74.125.227.134]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F2524606A320 for ; Fri, 20 Nov 2026 07:06:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.2 smtp.subspace.kernel.org F2524606A320 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key, unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20260707 header.b=yNkqUQDp Received: by mail-pj2-f6.google.com with SMTP id 98e67ed59e1d1-3b1c4ce7151so136023a91.0 for ; Thu, 19 Nov 2026 23:06:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20260707; t=1795158403; x=1795763203; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to:content-type; bh=DNfItCD3Z4BoYtins2/yOU+Coq3wbYX7eM32Pd4EAjg=; b=yNkqUQDp/xwCPgqx89ZP4vPdesgz1rlsl4SZQBgBrU605uAVv4guL5/5EcoSMZvcub e4FOS96jDw/7Un0GSykdM2VNZAkzuMS4QWn1VzRou49poveVu8V8kKCQ1/AfL5a8TLH2 MddV0F6c36pKXjKS/TE9Ks9DUJ/IH4EPSMzVPl6VObmkLeYUYCupCbftBf9El7FJ7sHo TYlDqFFvpZrR2Ppeu1Msp8PcwkHiaedBILByqBvUoU9z69rNq10f+S1TRohOIuBD5oZh 3zQR4RLOZ/Eak2hURKqlgs5o/Y4z7a3d3IFOav8YQNgqxRtZwb+Eq3uMdeh9BbR60M04 0nRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1795158403; x=1795763203; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=DNfItCD3Z4BoYtins2/yOU+Coq3wbYX7eM32Pd4EAjg=; b=GYB5K8MUvv6wesMkCwvVNyz9WSdyEsWruD1NXUooNMzMKpLQDA7fx73mbhMQzH4eTQ V0k3tiQvYhzpI239EFOWUeAQ+IhljZase4hlA5SDQeLstrzNQrmU3inbwbvEJVEs/eru 6WlkpVIzG9zbCEp+ugD0f8jNS9OWskJrn658dlVdQ1NrsNgE8SqkIQw3vvE9z5++PMeF Y/40taGzqq9UIiwynAjf2M+AS+7VbSpZtGmRb9/GecXs77bjnMjNX4Rv7d7cNqsKFGxK Vp07ZeTNnpfusGLiTxn+4t5/fbAOKhflQa9wd6fZhNGZewIXVClNHAg4CErmoICtKIS9 N1kw== X-Gm-Message-State: APuU4Z9vInuxgXfTXq796Zh7nHsc7NjVr+skZ+98qCCuLXBMvbPPsJi8 KgvOcHkxHuFRlZ5aJo0tBdVrkkFcZZvHyWwddbksQFLrnUR6cnWCVppEyd3xmi8qyMcKYg== X-Gm-Gg: AZFdTZf1GE7ik0XMkPUT1Yrhc780MARATwLivhey1Ll2jCoKyFrA1wyJmji4tRCgMI1 fKuEeK7QqRrX4oYzm2aqPcBZAAwSt0Grc6n3mVOE/jUXuo9B38l0dKF620ktFdizk50VNXk8Td2 z9Qk9WjclMQnnCSNXSNZKQmks9QD+EU0CRSbwFeqs5p5EWCzYpMlbGmvDQms1kr1WGaawt+xlDg PAJbWWFvKT0JFjMVf0LTnYfmrVeUHWYOZkw1+Yp25YOuGJe+7HuW/PRrXLp25Xd01XRP+6SWM9h IwXH3htedZyqTa+vIUeo9avqdyt39uDaLA1l+ZYQN5YFDXHI2JfkGR7733kCyX4Q2IJdgDN1oq0 /HBSILlatBmV5nccRYEiGjGWP9rrIaAN+M5WeO6nAMeGG5HCKIcBQE0i/ZWOlHmbqXxdcI7SrQQ 28vpzp0f/w87NBXuaOFXcNC3KTjTkY+0bIliSEj2Fc1H+fuYXr9OT3Edcwplf2Ntxc/KplWAItt gjxbjhi/rsvh7tyrURML5f7LpAckm2kaWipde7IJoGFthAhZNpqamSCx5EYllN+DkyevZObn13N sSXReefuCHEjkuU72NYbH28RJ+GO2gwu4p5gY1a6NkhXEQ== X-Received: by 2002:a17:90b:2c84:b0:3ac:a203:e092 with SMTP id 98e67ed59e1d1-3b41c7b6864mr1838751a91.1.1795158403117; Thu, 19 Nov 2026 23:06:43 -0800 (PST) Received: from nixos ([2409:8a28:ae4:7f20:fd8b:76d8:48d:6284]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-3b41ffbab7fsm1016361a91.0.2026.11.19.23.06.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Nov 2026 23:06:42 -0800 (PST) From: Zixin Wang To: linux-doc@vger.kernel.org Cc: linux-rt-devel@lists.linux.dev, bigeasy@linutronix.de, clrkwllms@kernel.org, rostedt@goodmis.org, corbet@lwn.net, linux-kernel@vger.kernel.org, Zixin Wang Subject: [PATCH v2] Documentation: real-time: Add boot parameters and task isolation section Date: Fri, 20 Nov 2026 15:06:20 +0800 Message-ID: <20261120070620.535693-1-w1355457260@gmail.com> X-Mailer: git-send-email 2.54.0 X-Mailing-List: linux-doc@vger.kernel.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Document the boot parameters and runtime tunables that affect the worst-case latency of a real-time system, alongside the existing kernel configuration guidance. Cover CPU isolation, nohz_full, RCU callback offloading, interrupt affinity, idle states, and real-time throttling, with references to the relevant documentation. Link: https://bugzilla.kernel.org/show_bug.cgi?id=220633 Link: https://github.com/elisa-tech/wg-aerospace/issues/222 Signed-off-by: Zixin Wang --- v1 -> v2: - Fix the kernel.sched_rt_runtime_us description: the default runtime is 1000000 us and, without CONFIG_RT_GROUP_SCHED, SCHED_FIFO/RR tasks are no longer throttled through this sysctl. Describe the per-CPU fair deadline server (50 ms per 1000 ms) instead. Reported by Sashiko. - Clarify the isolcpus= flag semantics: only the domain flag removes CPUs from load balancing, and the remaining flags do not affect unrelated kernel threads, timers, or interrupts. Review feedback. .../real-time/kernel-configuration.rst | 141 ++++++++++++++++++ 1 file changed, 141 insertions(+) diff --git a/Documentation/core-api/real-time/kernel-configuration.rst b/Documentation/core-api/real-time/kernel-configuration.rst index d7f08e7b..995674b9 100644 --- a/Documentation/core-api/real-time/kernel-configuration.rst +++ b/Documentation/core-api/real-time/kernel-configuration.rst @@ -216,6 +216,147 @@ just by being enabled. These tracers do incur measurable latency overhead even when tracing is not currently active. +Boot parameters and task isolation +================================== + +Kernel configuration options alone do not determine a real-time system's +worst-case latency. Integrators commonly rely on boot parameters and +runtime tunables as well, to keep kernel housekeeping work away from +latency-sensitive tasks. The parameters below complement the options +described above. Each one is documented in detail in +Documentation/admin-guide/kernel-parameters.rst; the notes here focus on +the aspects that matter for real-time latency. For the full CPU +isolation model and its limitations, see +Documentation/admin-guide/cpu-isolation.rst. + +.. caution:: + + CPU isolation is best effort. Unbound workqueues, kernel threads, + timers, and interrupts may still run on isolated CPUs unless they are + explicitly steered away. Measure the resulting latency with tools + such as ``osnoise``, ``timerlat``, and ``rtla`` rather than assuming + that an isolated CPU is free of kernel noise. See + Documentation/timers/no_hz.rst and Documentation/tools/rtla/index.rst. + +CPU isolation +------------- + +``isolcpus=`` + Format: ``[flag-list,]``. Isolates the listed CPUs from the + disturbances selected by the flag list (default: ``domain``): + + ``domain`` + Remove the CPUs from the scheduler domains. This is the default + when no flag is given. Domain isolation cannot be undone at + runtime; use cpusets when the isolated CPU set must change + dynamically. Isolated CPUs remain usable through CPU affinity or + cpusets. + + ``nohz`` + Stop the tick when a single task runs and offload kernel noise, + including RCU callbacks. This is equivalent to ``nohz_full=``. + The residual 1 Hz tick is offloaded to workqueues, which must be + affined to housekeeping CPUs through the global workqueue + ``cpumask`` (/sys/devices/virtual/workqueue/cpumask); by default + that workqueue may run on any CPU. + + ``managed_irq`` + Prevent managed interrupts from being targeted at isolated CPUs + when their automatically computed affinity mask also contains + housekeeping CPUs. This is best effort and has no effect when the + mask contains isolated CPUs only. See + Documentation/core-api/irq/managed_irq.rst. + + Apart from the best-effort managed IRQ redirection, these flags do + not stop kernel threads, timers, or interrupts that are already + affined to the isolated CPUs. A practical configuration therefore + combines ``isolcpus=`` with ``nohz_full=``, ``rcu_nocbs=``, an + interrupt affinity mask, and a C-state limit. +``nohz_full=`` + Format: ````. Requires ``CONFIG_NO_HZ_FULL=y``. Stops the + scheduler tick on the listed CPUs whenever possible. The boot CPU is + forced outside the range to maintain timekeeping, and the RCU + callbacks of the listed CPUs are offloaded as if they had been + passed to ``rcu_nocbs=``. This parameter takes precedence over + ``CONFIG_RCU_NOCB_CPU_DEFAULT_ALL``. + + Tickless operation can increase the kernel-to-userspace transition + latency, so it is most useful for workloads that execute in + userspace for long periods with few syscalls. Periodic workloads + with short periods may prefer a consistent tick instead. + +RCU callback offloading +----------------------- + +``rcu_nocbs[=cpu-list]`` + In kernels built with ``CONFIG_RCU_NOCB_CPU=y``, move RCU callback + invocation for the listed CPUs out of softirq context and into + per-CPU ``rcuo`` kthreads. This reduces OS jitter on the offloaded + CPUs. If the argument is omitted, no CPU is offloaded at boot, but + the mode can be toggled at runtime through cpusets. ``nohz_full=`` + implies the same offload for the CPUs it lists. + +``rcu_nocb_poll`` + Make the ``rcuo`` kthreads poll for callbacks instead of requiring + the offloaded CPUs to wake them. This improves the real-time + response at the cost of higher power consumption, because the + kthreads wake up periodically to do the polling. + +Interrupt affinity +------------------ + +``irqaffinity=`` + Format: ````. Sets the default affinity mask for + interrupts that do not have an explicit affinity. Managed + interrupts, such as MSI-X queue interrupts, derive their affinity + from the device queue mask instead and cannot be changed through + ``/proc/irq/*``. Combine ``irqaffinity=`` with the ``managed_irq`` + flag of ``isolcpus=`` to keep interrupts away from isolated CPUs + where possible. See Documentation/core-api/irq/managed_irq.rst. + +Idle state limits +----------------- + +``processor.max_cstate=`` + ACPI platforms only. Limits the deepest C-state the processor may + enter. ``processor.max_cstate=1`` caps the processor at C1 and + avoids the entry and exit latency, cache flushing, and clock effects + of deeper states. A value of 9 overrides any DMI blacklist limit. + +``intel_idle.max_cstate=`` + x86 platforms using ``intel_idle`` only. A value between 1 and 9 + specifies the maximum C-state depth. A value of 0 disables + ``intel_idle`` and falls back to ``acpi_idle``; it does not disable + C-states. + +Deeper C-states save power but increase wakeup latency. Choose the +shallowest state that still meets the platform's power and thermal +requirements, and validate the result with the tools listed above. + +Real-time throttling +-------------------- + +``kernel.sched_rt_runtime_us`` + Together with ``kernel.sched_rt_period_us``, this sysctl defines the + total bandwidth available to SCHED_DEADLINE entities. When + ``CONFIG_RT_GROUP_SCHED`` is enabled, it also limits the bandwidth of + the real-time scheduling groups. The default runtime equals the + period (1000000 us), so no bandwidth is subtracted from deadline + tasks by default. + + Without ``CONFIG_RT_GROUP_SCHED``, SCHED_FIFO and SCHED_RR tasks are + no longer throttled through this interface. Instead, a per-CPU fair + deadline server keeps non-real-time tasks running by providing + 50 ms of runtime per 1000 ms period. That reservation is not + disabled by setting this sysctl to -1; the server parameters are + exposed under /sys/kernel/debug/sched/fair_server/cpuX/. + + A value of -1 removes the total deadline bandwidth limit, that is, + runtime equals period. Do this only after validating the workload, + because runaway real-time tasks can then starve the rest of the + system. See Documentation/scheduler/sched-rt-group.rst for RT group + scheduling details. + Kernel Debug Options ==================== -- 2.54.0