public inbox for linux-kernel@vger.kernel.org
 help / color / mirror / Atom feed
From: Petr Mladek <pmladek@suse.com>
To: John Ogness <john.ogness@linutronix.de>
Cc: Rik van Riel <riel@surriel.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Omar Sandoval <osandov@meta.com>,
	linux-kernel@vger.kernel.org,
	Steven Rostedt <rostedt@goodmis.org>,
	Sergey Senozhatsky <senozhatsky@chromium.org>,
	kernel-team <kernel-team@meta.com>
Subject: Re: [RFC PATCH] nmi,printk: fix ABBA deadlock between nmi_backtrace and dump_stack_lvl
Date: Wed, 24 Jul 2024 17:08:25 +0200	[thread overview]
Message-ID: <ZqEY6ZIB7XThgKW3@pathway.suse.cz> (raw)
In-Reply-To: <871q3ix2zp.fsf@jogness.linutronix.de>

On Wed 2024-07-24 16:51:46, John Ogness wrote:
> On 2024-07-24, Petr Mladek <pmladek@suse.com> wrote:
> > My quess is that it looked like:
> >
> > CPU A				CPU B
> >
> > 				printk()
> > 				  console_try_lock_spinning()
> > 				  console_unlock()
> > 				    console_emit_next_record()
> > 				      console_lock_spinning_enable();
> > 					con->write()
> > 					  spin_lock(port->lock);
> >
> > printk_cpu_sync_get()
> >   printk()
> >     console_try_lock_spinning()
> >       # spinning and wating for CPU B
> >
> > 				NMI:
> >
> > 				  printk_cpu_sync_get()
> > 				    # waiting for CPU A
> >
> > => DEADLOCK
> >
> >
> > The deadlock is caused under/by printk_cpu_sync_get() but only because
> > console_try_lock_spinning() is blocked. It is not a true "try_lock"
> > operation which should never get blocked.
> >
> > => The above patch should solve the problem as well. It will cause
> >    that console_try_lock_spinning() would fail immediately on CPU A.
> >
> > Note that port->lock can't cause any deadlock in this scenario.
> > console_try_lock_spinning() will always fail on CPU A until
> > the NMI gets handled on CPU B.
> >
> > By other words, printk_cpu_sync_get() will behave as a tail lock
> > on CPU A because of the failing trylock.
> 
> But only in _this_ scenario. The port lock could be taken by CPU B for
> non-console-printing reasons. Then you still have deadlock, due to
> spinning on the port lock.

I see. I agree that deferring printk on that CPU [0] is the right solution.

> [0] https://lore.kernel.org/lkml/87plrcqyii.fsf@jogness.linutronix.de

Best Regards,
Petr

  reply	other threads:[~2024-07-24 15:08 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-07-16  3:20 [RFC PATCH] nmi,printk: fix ABBA deadlock between nmi_backtrace and dump_stack_lvl Rik van Riel
2024-07-17  7:16 ` John Ogness
2024-07-17 13:47   ` Rik van Riel
2024-07-18  7:25     ` John Ogness
2024-07-18 13:38       ` Rik van Riel
2024-07-18 14:09         ` John Ogness
2024-07-18 15:23           ` Rik van Riel
2024-07-24 12:56           ` Petr Mladek
2024-07-24 14:45             ` John Ogness
2024-07-24 15:08               ` Petr Mladek [this message]
2024-09-13 17:25                 ` Rik van Riel
2024-09-16 14:33                   ` Petr Mladek
2024-07-24 16:55             ` Rik van Riel
2024-07-24 12:45   ` Petr Mladek

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=ZqEY6ZIB7XThgKW3@pathway.suse.cz \
    --to=pmladek@suse.com \
    --cc=akpm@linux-foundation.org \
    --cc=john.ogness@linutronix.de \
    --cc=kernel-team@meta.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=osandov@meta.com \
    --cc=riel@surriel.com \
    --cc=rostedt@goodmis.org \
    --cc=senozhatsky@chromium.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox