[PATCH] Documentation: printk: warn about lockups from excessive use

[PATCH] Documentation: printk: warn about lockups from excessive use

[hujinfei]

From: hujinfei <hujinfei3@huawei.com>

Add a section 'Avoiding lockups from excessive printk() use' to
printk-basics.rst, explaining the risk of calling printk() in hot paths
with slow consoles and suggesting alternatives like ratelimited printing,
tracepoints, nbcon, and log level filtering.

Signed-off-by: hujinfei <hujinfei3@huawei.com>
---
 Documentation/core-api/printk-basics.rst | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/Documentation/core-api/printk-basics.rst b/Documentation/core-api/printk-basics.rst
index 2dde24ca7..a9da8c336 100644
--- a/Documentation/core-api/printk-basics.rst
+++ b/Documentation/core-api/printk-basics.rst
@@ -103,6 +103,28 @@ For debugging purposes there are also two conditionally-compiled macros:
 pr_debug() and pr_devel(), which are compiled-out unless ``DEBUG`` (or
 also ``CONFIG_DYNAMIC_DEBUG`` in the case of pr_debug()) is defined.
 
+Avoiding lockups from excessive printk() use
+============================================
+
+Do not use ``printk()`` in hot paths such as interrupt handlers, timer callbacks,
+or high-frequency network receive routines. When a slow console (e.g., ``console=ttyS0``)
+is active, ``printk()`` may synchronously acquire ``console_sem`` and block while
+flushing messages, potentially disabling interrupts long enough to trigger hard or
+soft lockup detectors.
+
+To avoid this:
+
+- Avoid ``printk()`` in hot paths and interrupt contexts.
+- Use rate-limited variants (e.g., pr_xxx_ratelimited()) or one-time macros (e.g., pr_*_once()).
+- Assign lower log levels (e.g., ``KERN_DEBUG``) to non-essential messages and filter
+  console output via ``console_loglevel``.
+- Use consoles that implement the non-blocking ``nbcon`` API (indicated by ``CON_NBCON``),
+  which offload message printing to a dedicated kernel thread outside emergency contexts.
+  Note that asynchronous printing increases the risk of message loss during crashes;
+  increasing the kernel log buffer size may help retain more messages.
+
+Temporary debugging may use ``trace_printk()``, but it must not appear in mainline
+code. See the section about ``trace_printk()`` in Documentation/trace/debugging.rst.
 
 Function reference
 ==================
-- 
2.33.0

Re: [PATCH] Documentation: printk: warn about lockups from excessive use

[Randy Dunlap]

Hi,

On 3/4/26 6:40 AM, hujinfei wrote:
> From: hujinfei <hujinfei3@huawei.com>
> 
> Add a section 'Avoiding lockups from excessive printk() use' to
> printk-basics.rst, explaining the risk of calling printk() in hot paths
> with slow consoles and suggesting alternatives like ratelimited printing,
> tracepoints, nbcon, and log level filtering.
> 
> Signed-off-by: hujinfei <hujinfei3@huawei.com>
> ---
>  Documentation/core-api/printk-basics.rst | 22 ++++++++++++++++++++++
>  1 file changed, 22 insertions(+)
> 

Please limit lines to maximum of 80 characters each. The rest of the
file is already like that.

> diff --git a/Documentation/core-api/printk-basics.rst b/Documentation/core-api/printk-basics.rst
> index 2dde24ca7..a9da8c336 100644
> --- a/Documentation/core-api/printk-basics.rst
> +++ b/Documentation/core-api/printk-basics.rst
> @@ -103,6 +103,28 @@ For debugging purposes there are also two conditionally-compiled macros:
>  pr_debug() and pr_devel(), which are compiled-out unless ``DEBUG`` (or
>  also ``CONFIG_DYNAMIC_DEBUG`` in the case of pr_debug()) is defined.
>  
> +Avoiding lockups from excessive printk() use
> +============================================
> +
> +Do not use ``printk()`` in hot paths such as interrupt handlers, timer callbacks,
> +or high-frequency network receive routines. When a slow console (e.g., ``console=ttyS0``)
> +is active, ``printk()`` may synchronously acquire ``console_sem`` and block while
> +flushing messages, potentially disabling interrupts long enough to trigger hard or
> +soft lockup detectors.
> +
> +To avoid this:
> +
> +- Avoid ``printk()`` in hot paths and interrupt contexts.
> +- Use rate-limited variants (e.g., pr_xxx_ratelimited()) or one-time macros (e.g., pr_*_once()).

Since you use pr_xxx_ratelimited(), how about doing similar to that
with pr_xxx_once()?
I only care because the '*' mucks up the html output. :(

> +- Assign lower log levels (e.g., ``KERN_DEBUG``) to non-essential messages and filter
> +  console output via ``console_loglevel``.
> +- Use consoles that implement the non-blocking ``nbcon`` API (indicated by ``CON_NBCON``),
> +  which offload message printing to a dedicated kernel thread outside emergency contexts.
> +  Note that asynchronous printing increases the risk of message loss during crashes;
> +  increasing the kernel log buffer size may help retain more messages.
> +
> +Temporary debugging may use ``trace_printk()``, but it must not appear in mainline
> +code. See the section about ``trace_printk()`` in Documentation/trace/debugging.rst.
>  
>  Function reference
>  ==================

Tested-by: Randy Dunlap <rdunlap@infradead.org>

thanks.
-- 
~Randy

Re: [PATCH] Documentation: printk: warn about lockups from excessive use

[Steven Rostedt]

On Wed,  4 Mar 2026 22:40:32 +0800
hujinfei <3288824963@qq.com> wrote:

> From: hujinfei <hujinfei3@huawei.com>
> 
> Add a section 'Avoiding lockups from excessive printk() use' to
> printk-basics.rst, explaining the risk of calling printk() in hot paths
> with slow consoles and suggesting alternatives like ratelimited printing,
> tracepoints, nbcon, and log level filtering.
> 
> Signed-off-by: hujinfei <hujinfei3@huawei.com>
> ---
>  Documentation/core-api/printk-basics.rst | 22 ++++++++++++++++++++++
>  1 file changed, 22 insertions(+)
> 
> diff --git a/Documentation/core-api/printk-basics.rst b/Documentation/core-api/printk-basics.rst
> index 2dde24ca7..a9da8c336 100644
> --- a/Documentation/core-api/printk-basics.rst
> +++ b/Documentation/core-api/printk-basics.rst
> @@ -103,6 +103,28 @@ For debugging purposes there are also two conditionally-compiled macros:
>  pr_debug() and pr_devel(), which are compiled-out unless ``DEBUG`` (or
>  also ``CONFIG_DYNAMIC_DEBUG`` in the case of pr_debug()) is defined.
>  
> +Avoiding lockups from excessive printk() use
> +============================================
> +
> +Do not use ``printk()`` in hot paths such as interrupt handlers, timer callbacks,
> +or high-frequency network receive routines. When a slow console (e.g., ``console=ttyS0``)
> +is active, ``printk()`` may synchronously acquire ``console_sem`` and block while
> +flushing messages, potentially disabling interrupts long enough to trigger hard or
> +soft lockup detectors.
> +
> +To avoid this:
> +
> +- Avoid ``printk()`` in hot paths and interrupt contexts.
> +- Use rate-limited variants (e.g., pr_xxx_ratelimited()) or one-time macros (e.g., pr_*_once()).
> +- Assign lower log levels (e.g., ``KERN_DEBUG``) to non-essential messages and filter
> +  console output via ``console_loglevel``.
> +- Use consoles that implement the non-blocking ``nbcon`` API (indicated by ``CON_NBCON``),
> +  which offload message printing to a dedicated kernel thread outside emergency contexts.
> +  Note that asynchronous printing increases the risk of message loss during crashes;
> +  increasing the kernel log buffer size may help retain more messages.
> +
> +Temporary debugging may use ``trace_printk()``, but it must not appear in mainline
> +code. See the section about ``trace_printk()`` in Documentation/trace/debugging.rst.

You could also add:

  If more permanent output is needed in a hot path, trace events can be
  used. See Documentation/trace/events.rst and
  samples/trace_events/trace-events-sample.[ch]

-- Steve


>  
>  Function reference
>  ==================

Re: [PATCH] Documentation: printk: warn about lockups from excessive use

[John Ogness]

On 2026-03-04, hujinfei <3288824963@qq.com> wrote:
> From: hujinfei <hujinfei3@huawei.com>
>
> Add a section 'Avoiding lockups from excessive printk() use' to
> printk-basics.rst, explaining the risk of calling printk() in hot paths
> with slow consoles and suggesting alternatives like ratelimited printing,
> tracepoints, nbcon, and log level filtering.
>
> Signed-off-by: hujinfei <hujinfei3@huawei.com>
> ---
>  Documentation/core-api/printk-basics.rst | 22 ++++++++++++++++++++++
>  1 file changed, 22 insertions(+)
>
> diff --git a/Documentation/core-api/printk-basics.rst b/Documentation/core-api/printk-basics.rst
> index 2dde24ca7..a9da8c336 100644
> --- a/Documentation/core-api/printk-basics.rst
> +++ b/Documentation/core-api/printk-basics.rst
> @@ -103,6 +103,28 @@ For debugging purposes there are also two conditionally-compiled macros:
>  pr_debug() and pr_devel(), which are compiled-out unless ``DEBUG`` (or
>  also ``CONFIG_DYNAMIC_DEBUG`` in the case of pr_debug()) is defined.
>  
> +Avoiding lockups from excessive printk() use
> +============================================
> +
> +Do not use ``printk()`` in hot paths such as interrupt handlers, timer callbacks,
> +or high-frequency network receive routines. When a slow console (e.g., ``console=ttyS0``)
> +is active, ``printk()`` may synchronously acquire ``console_sem`` and block while
> +flushing messages, potentially disabling interrupts long enough to trigger hard or
> +soft lockup detectors.

I hesitate supporting the addition of this documetation because it is
only relevant for legacy consoles and !PREEMPT_RT. Perhaps we could make
that point clearer. Once all console drivers have been updated to nbcon,
we can then remove this documentation.

> +
> +To avoid this:
> +
> +- Avoid ``printk()`` in hot paths and interrupt contexts.
> +- Use rate-limited variants (e.g., pr_xxx_ratelimited()) or one-time macros (e.g., pr_*_once()).
> +- Assign lower log levels (e.g., ``KERN_DEBUG``) to non-essential messages and filter
> +  console output via ``console_loglevel``.
> +- Use consoles that implement the non-blocking ``nbcon`` API (indicated by ``CON_NBCON``),
> +  which offload message printing to a dedicated kernel thread outside emergency contexts.

If the first paragraph made it clear that legacy consoles are the
problem, the point about using nbcon would not be necessary.

> +  Note that asynchronous printing increases the risk of message loss during crashes;
> +  increasing the kernel log buffer size may help retain more messages.

This last sentence is misleading. Calling printk() immediately logs the
message to the lockless ringbuffer. Upon panic, the buffer is
flushed. The nbcon consoles also transition to atomic printing. The only
messages that might get lost due to mass printk() calling are older
messages. But this problem is not specific to nbcon consoles.

There is also printk_deferred(), which immediately logs to the
ringbuffer and defers the console printing. It is a workaround for
legacy consoles. For nbcon consoles there is no difference between
printk() and printk_deferrred().

And since this is supposed to be general tips for developers, perhaps we
could mention that porting a legacy console driver to nbcon is also a
solution (and, in fact, is the preferred solution).

John Ogness

[PATCH] Documentation: printk: warn about lockups from excessive use

[hujinfei]

From: hujinfei <hujinfei3@huawei.com>

Add a section 'Avoiding lockups from excessive printk() use' to
printk-basics.rst, explaining the risk of calling printk() in hot paths
with slow consoles and suggesting alternatives like ratelimited printing,
tracepoints, nbcon, and log level filtering.

Signed-off-by: hujinfei <hujinfei3@huawei.com>
---
 Documentation/core-api/printk-basics.rst | 29 ++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

diff --git a/Documentation/core-api/printk-basics.rst b/Documentation/core-api/printk-basics.rst
index 2dde24ca7..f6ca1bc55 100644
--- a/Documentation/core-api/printk-basics.rst
+++ b/Documentation/core-api/printk-basics.rst
@@ -103,6 +103,35 @@ For debugging purposes there are also two conditionally-compiled macros:
 pr_debug() and pr_devel(), which are compiled-out unless ``DEBUG`` (or
 also ``CONFIG_DYNAMIC_DEBUG`` in the case of pr_debug()) is defined.
 
+Avoiding lockups from excessive printk() use
+============================================
+
+Do not use ``printk()`` in hot paths such as interrupt handlers, timer
+callbacks, or high-frequency network receive routines. When a slow console
+(e.g., ``console=ttyS0``) is active, ``printk()`` may synchronously acquire
+``console_sem`` and block while flushing messages, potentially disabling
+interrupts long enough to trigger hard or soft lockup detectors.
+
+To avoid this:
+
+- Avoid ``printk()`` in hot paths and interrupt contexts.
+- Use rate-limited variants (e.g., ``pr_xxx_ratelimited()``) or one-time macros
+  (e.g., ``pr_xxx_once()``).
+- Assign lower log levels (e.g., ``KERN_DEBUG``) to non-essential messages and
+  filter console output via ``console_loglevel``.
+- Use consoles that implement the non-blocking ``nbcon`` API (indicated by
+  ``CON_NBCON``), which offload message printing to a dedicated kernel thread
+  outside emergency contexts. Note that asynchronous printing increases the risk
+  of message loss during crashes; increasing the kernel log buffer size may help
+  retain more messages.
+
+Temporary debugging may use ``trace_printk()``, but it must not appear in
+mainline code. See the section about ``trace_printk()`` in
+``Documentation/trace/debugging.rst``.
+
+If more permanent output is needed in a hot path, trace events can be used. See
+``Documentation/trace/events.rst`` and
+``samples/trace_events/trace-events-sample.[ch]``.
 
 Function reference
 ==================
-- 
2.33.0

Re: [PATCH] Documentation: printk: warn about lockups from excessive use

[Randy Dunlap]

On 3/4/26 10:41 PM, hujinfei wrote:
> From: hujinfei <hujinfei3@huawei.com>
> 
> Add a section 'Avoiding lockups from excessive printk() use' to
> printk-basics.rst, explaining the risk of calling printk() in hot paths
> with slow consoles and suggesting alternatives like ratelimited printing,
> tracepoints, nbcon, and log level filtering.
> 
> Signed-off-by: hujinfei <hujinfei3@huawei.com>
> ---
>  Documentation/core-api/printk-basics.rst | 29 ++++++++++++++++++++++++
>  1 file changed, 29 insertions(+)

Tested-by: Randy Dunlap <rdunlap@infradead.org>
Acked-by: Randy Dunlap <rdunlap@infradead.org>

thanks.
-- 
~Randy

Re: [PATCH] Documentation: printk: warn about lockups from excessive use

[Petr Mladek]

On Thu 2026-03-05 14:41:40, hujinfei wrote:
> From: hujinfei <hujinfei3@huawei.com>
> 
> Add a section 'Avoiding lockups from excessive printk() use' to
> printk-basics.rst, explaining the risk of calling printk() in hot paths
> with slow consoles and suggesting alternatives like ratelimited printing,
> tracepoints, nbcon, and log level filtering.
> 
> Signed-off-by: hujinfei <hujinfei3@huawei.com>

Looks good to me.

Reviewed-by: Petr Mladek <pmladek@suse.com>

Best Regards,
Petr

PS: I am going to wait few more days for a potential feedback and
    will push it to printk/linux.git if nobody complains.