All of lore.kernel.org
 help / color / mirror / Atom feed
From: Alex Elder <aelder@sgi.com>
To: Dave Chinner <david@fromorbit.com>
Cc: xfs@oss.sgi.com
Subject: Re: [PATCH 01/12] xfs: prevent NMI timeouts in cmn_err
Date: Tue, 11 Jan 2011 22:13:23 -0600	[thread overview]
Message-ID: <1294805603.3115.127.camel@doink> (raw)
In-Reply-To: <1294792553-8378-2-git-send-email-david@fromorbit.com>

On Wed, 2011-01-12 at 11:35 +1100, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
> 
> We currently have a global error message buffer in cmn_err that is
> protected by a spin lock that disables interrupts.  Recently there
> have been reports of NMI timeouts occurring when the console is
> being flooded by SCSI error reports due to cmn_err() getting stuck
> trying to print to the console while holding this lock (i.e. with
> interrupts disabled). The NMI watchdog is seeing this CPU as
> non-responding and so is triggering a panic.  While the trigger for
> the reported case is SCSI errors, pretty much anything that spams
> the kernel log could cause this to occur.
> 
> Realistically the only reason that we have the intemediate message
> buffer is to prepend the correct kernel log level prefix to the log
> message. The only reason we have the lock is to protect the global
> message buffer and the only reason the message buffer is global is
> to keep it off the stack. Hence if we can avoid needing a global
> message buffer we avoid needing the lock, and we can do this with a
> small amount of cleanup and some preprocessor tricks:
> 
> 	1. clean up xfs_cmn_err() panic mask functionality to avoid
> 	   needing debug code in xfs_cmn_err()
> 	2. remove the couple of "!" message prefixes that still exist that
> 	   the existing cmn_err() code steps over.
> 	3. redefine CE_* levels directly to KERN_*
> 	4. redefine cmn_err() and friends to use printk() directly
> 	   via variable argument length macros.
> 
> By doing this, we can completely remove the cmn_err() code and the
> lock that is causing the problems, and rely solely on printk()
> serialisation to ensure that we don't get garbled messages.
> 
> A series of followup patches is really needed to clean up all the
> cmn_err() calls and related messages properly, but that results in a
> series that is not easily back portable to enterprise kernels. Hence
> this initial fix is only to address the direct problem in the lowest
> impact way possible.

I had two trivial remarks but, well, what you have is just fine...

Reviewed-by: Alex Elder <aelder@sgi.com>

> Signed-off-by: Dave Chinner <dchinner@redhat.com>
> ---
>  fs/xfs/linux-2.6/xfs_sysctl.c |   23 ++++++++-
>  fs/xfs/support/debug.c        |  109 +++++++++++++++++++----------------------
>  fs/xfs/support/debug.h        |   25 ++++++---
>  

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

  reply	other threads:[~2011-01-12  4:13 UTC|newest]

Thread overview: 27+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-01-12  0:35 [RFC, PATCH 0/12] xfs: rework error logging infrastructure Dave Chinner
2011-01-12  0:35 ` [PATCH 01/12] xfs: prevent NMI timeouts in cmn_err Dave Chinner
2011-01-12  4:13   ` Alex Elder [this message]
2011-01-12 12:10   ` Christoph Hellwig
2011-01-12 12:19     ` Dave Chinner
2011-01-12 15:56       ` XFS master branch update Alex Elder
2011-01-12  0:35 ` [PATCH 02/12] xfs: introduce new logging API Dave Chinner
2011-01-12  0:35 ` [PATCH 03/12] xfs: Convert linux-2.6/ files to new logging interface Dave Chinner
2011-01-12  0:35 ` [PATCH 04/12] xfs: Convert xlog_warn " Dave Chinner
2011-01-12  4:13   ` Alex Elder
2011-01-12  0:35 ` [PATCH 05/12] xfs: convert xfs_cmn_err to xfs_alert_tag Dave Chinner
2011-01-12  0:35 ` [PATCH 06/12] xfs: kill xfs_fs_repair_cmn_err() macro Dave Chinner
2011-01-12  0:35 ` [PATCH 07/12] xfs: kill xfs_fs_mount_cmn_err() macro Dave Chinner
2011-01-18 13:00   ` Christoph Hellwig
2011-01-18 21:46     ` Dave Chinner
2011-01-19 11:39       ` Christoph Hellwig
2011-01-12  0:35 ` [PATCH 08/12] xfs: convert xfs_fs_cmn_err to new error logging API Dave Chinner
2011-01-12  0:35 ` [PATCH 09/12] xfs: rename xfs_cmn_err_fsblock_zero() Dave Chinner
2011-01-12  0:35 ` [PATCH 10/12] xfs: convert the quota debug prints to new API Dave Chinner
2011-01-12  0:35 ` [PATCH 11/12] xfs: Convert remaining cmn_err() callers " Dave Chinner
2011-01-12  0:35 ` [PATCH 12/12] xfs: kill support/debug.[ch] Dave Chinner
2011-01-12  4:48   ` Alex Elder
2011-01-18 13:02   ` Christoph Hellwig
2011-01-12  4:16 ` [RFC, PATCH 0/12] xfs: rework error logging infrastructure Alex Elder
2011-01-12  7:43   ` Dave Chinner
2011-01-18 13:03 ` Christoph Hellwig
2011-01-18 21:48   ` Dave Chinner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1294805603.3115.127.camel@doink \
    --to=aelder@sgi.com \
    --cc=david@fromorbit.com \
    --cc=xfs@oss.sgi.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.