[RFC] [PATCH] printk: Don't emit console_cpu_notify() for CPU_DYING

[RFC] [PATCH] printk: Don't emit console_cpu_notify() for CPU_DYING

[Yong Zhang]

When running -rt, I got some warning when doing cpu hotplug,
for details please check:
http://marc.info/?l=linux-kernel&m=132143661616297&w=2

But seems the problem also exists in mainline, a typical
scenario is:

        CPU A                           CPU B

doing printk with console_sem held
                                take_cpu_down();
                                  cpu_notify(CPU_DYING);
                                    console_cpu_notify();
                                      console_lock();
                                        down(&console_sem);
                                          *BANG*
  up(&console_sem);


Signed-off-by: Yong Zhang <yong.zhang0@gmail.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: James Morris <jmorris@namei.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Kevin Cernekee <cernekee@gmail.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Steven Rostedt <rostedt@goodmis.org>
---
 kernel/printk.c |    1 -
 1 files changed, 0 insertions(+), 1 deletions(-)

diff --git a/kernel/printk.c b/kernel/printk.c
index 535bc3b..ba0393e 100644
--- a/kernel/printk.c
+++ b/kernel/printk.c
@@ -1223,7 +1223,6 @@ static int __cpuinit console_cpu_notify(struct notifier_block *self,
 	switch (action) {
 	case CPU_ONLINE:
 	case CPU_DEAD:
-	case CPU_DYING:
 	case CPU_DOWN_FAILED:
 	case CPU_UP_CANCELED:
 		console_lock();
-- 
1.7.5.4

Re: [RFC] [PATCH] printk: Don't emit console_cpu_notify() for CPU_DYING

[Andrew Morton]

On Fri, 18 Nov 2011 10:19:24 +0800
Yong Zhang <yong.zhang0@gmail.com> wrote:

> When running -rt, I got some warning when doing cpu hotplug,
> for details please check:
> http://marc.info/?l=linux-kernel&m=132143661616297&w=2
> 
> But seems the problem also exists in mainline, a typical
> scenario is:
> 
>         CPU A                           CPU B
> 
> doing printk with console_sem held
>                                 take_cpu_down();
>                                   cpu_notify(CPU_DYING);
>                                     console_cpu_notify();
>                                       console_lock();
>                                         down(&console_sem);
>                                           *BANG*
>   up(&console_sem);

This doesn't describe a deadlock.  For a deadlock to occur, something
must prevent CPU A from proceeding to the up().

Are you able to more fully describe this?  I'm suspecting that CPU A
got shut down by CPU B?  If so, shutting down a CPU while it holds
locks seems rather bad.  Or perhaps it's something else...

Re: [RFC] [PATCH] printk: Don't emit console_cpu_notify() for CPU_DYING

[Yong Zhang]

On Fri, Nov 18, 2011 at 11:01:00AM -0800, Andrew Morton wrote:
> On Fri, 18 Nov 2011 10:19:24 +0800
> Yong Zhang <yong.zhang0@gmail.com> wrote:
> 
> > When running -rt, I got some warning when doing cpu hotplug,
> > for details please check:
> > http://marc.info/?l=linux-kernel&m=132143661616297&w=2
> > 
> > But seems the problem also exists in mainline, a typical
> > scenario is:
> > 
> >         CPU A                           CPU B
> > 
> > doing printk with console_sem held
> >                                 take_cpu_down();
> >                                   cpu_notify(CPU_DYING);
> >                                     console_cpu_notify();
> >                                       console_lock();
> >                                         down(&console_sem);
> >                                           *BANG*
> >   up(&console_sem);
> 
> This doesn't describe a deadlock.  For a deadlock to occur, something
> must prevent CPU A from proceeding to the up().

Hmm, after revisting the code, seems console_unlock() can't be
preempted when helding console_sem. IOW, stop_machine() can't
run if someone loops in printing something.

So the problem only affect -rt, sorry for the noise.

Thanks,
Yong

> 
> Are you able to more fully describe this?  I'm suspecting that CPU A
> got shut down by CPU B?  If so, shutting down a CPU while it holds
> locks seems rather bad.  Or perhaps it's something else...