All of lore.kernel.org
 help / color / mirror / Atom feed
From: Eric Sandeen <sandeen@sandeen.net>
To: Brian Foster <bfoster@redhat.com>
Cc: "Ben Myers" <bpm@sgi.com>, 符永涛 <yongtaofu@gmail.com>,
	"xfs@oss.sgi.com" <xfs@oss.sgi.com>
Subject: Re: need help how to debug xfs crash issue xfs_iunlink_remove: xfs_inotobp() returned error 22
Date: Thu, 11 Apr 2013 20:07:51 -0500	[thread overview]
Message-ID: <51675E67.7030600@sandeen.net> (raw)
In-Reply-To: <516746AC.3090808@redhat.com>

On 4/11/13 6:26 PM, Brian Foster wrote:
> On 04/11/2013 03:11 PM, 符永涛 wrote:
>> It happens tonight again on one of our servers, how to debug the root
>> cause? Thank you.
>>
> 
> Hi,
> 
> I've attached a system tap script (stap -v xfs.stp) that should
> hopefully print out a bit more data should the issue happen again. Do
> you have a small enough number of nodes (or predictable enough pattern)
> that you could run this on the nodes that tend to fail and collect the
> output?
> 
> Also, could you collect an xfs_metadump of the filesystem in question
> and make it available for download and analysis somewhere? I believe the
> ideal approach is to mount/umount the filesystem first to replay the log
> before collecting a metadump, but somebody could correct me on that (to
> be safe, you could collect multiple dumps: pre-mount and post-mount).

Dave suggested yesterday that this would be best: metadump right
after unmounting post-failure, then mount/umount & generate another metadump.

-Eric

> Could you also describe your workload a little bit? Thanks.
> 
> Brian
> 
>> Apr 12 02:32:10 cqdx kernel: XFS (sdb): xfs_iunlink_remove:
>> xfs_inotobp() returned error 22.
>> Apr 12 02:32:10 cqdx kernel: XFS (sdb): xfs_inactive: xfs_ifree returned
>> error 22
>> Apr 12 02:32:10 cqdx kernel: XFS (sdb): xfs_do_force_shutdown(0x1)
>> called from line 1184 of file fs/xfs/xfs_vnodeops.c.  Return address =
>> 0xffffffffa02ee20a
>> Apr 12 02:32:10 cqdx kernel: XFS (sdb): I/O Error Detected. Shutting
>> down filesystem
>> Apr 12 02:32:10 cqdx kernel: XFS (sdb): Please umount the filesystem and
>> rectify the problem(s)
>> Apr 12 02:32:19 cqdx kernel: XFS (sdb): xfs_log_force: error 5 returned.
>> Apr 12 02:32:49 cqdx kernel: XFS (sdb): xfs_log_force: error 5 returned.
>> Apr 12 02:33:19 cqdx kernel: XFS (sdb): xfs_log_force: error 5 returned.
>> Apr 12 02:33:49 cqdx kernel: XFS (sdb): xfs_log_force: error 5 returned.
>>

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

  parent reply	other threads:[~2013-04-12  1:07 UTC|newest]

Thread overview: 60+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-04-09 12:53 need help how to debug xfs crash issue xfs_iunlink_remove: xfs_inotobp() returned error 22 符永涛
2013-04-09 13:03 ` 符永涛
2013-04-09 13:05   ` 符永涛
2013-04-09 14:52     ` Ben Myers
2013-04-09 15:00       ` 符永涛
2013-04-09 15:07       ` 符永涛
2013-04-09 15:10         ` 符永涛
2013-04-10 10:10           ` Emmanuel Florac
2013-04-10 12:52             ` Dave Chinner
2013-04-10 13:52             ` 符永涛
2013-04-11 19:11               ` 符永涛
2013-04-11 19:55                 ` 符永涛
2013-04-11 23:26                 ` Brian Foster
2013-04-12  0:45                   ` 符永涛
2013-04-12 12:50                     ` Brian Foster
2013-04-12 13:42                       ` 符永涛
2013-04-12 13:48                         ` 符永涛
2013-04-12 13:51                           ` 符永涛
2013-04-12 13:59                             ` 符永涛
2013-04-12  1:07                   ` Eric Sandeen [this message]
2013-04-12  1:36                     ` 符永涛
2013-04-12  1:38                       ` 符永涛
2013-04-12  6:15                     ` 符永涛
2013-04-12  4:32                   ` 符永涛
2013-04-12  5:16                     ` Eric Sandeen
2013-04-12  5:40                       ` 符永涛
2013-04-12  6:00                         ` 符永涛
2013-04-12 12:11                           ` Brian Foster
2013-04-12  7:44                         ` 符永涛
2013-04-12  8:32                           ` 符永涛
2013-04-12 12:41                             ` Brian Foster
2013-04-12 14:48                               ` 符永涛
2013-04-15  2:08                                 ` 符永涛
2013-04-15  5:04                                   ` 符永涛
2013-04-15 12:54                                     ` 符永涛
2013-04-15 13:33                                       ` 符永涛
2013-04-15 13:36                                         ` 符永涛
2013-04-15 13:45                                           ` 符永涛
2013-04-15 13:57                                             ` Eric Sandeen
2013-04-15 14:21                                               ` 符永涛
2013-04-15 15:24                                                 ` 符永涛
2013-04-15 19:34                                                 ` Eric Sandeen
2013-04-15 14:13                                       ` Brian Foster
2013-04-12  5:23                     ` 符永涛
2013-04-09 22:16   ` Michael L. Semon
2013-04-09 22:18     ` Eric Sandeen
2013-04-09 22:48       ` Ben Myers
2013-04-09 23:30         ` Dave Chinner
2013-04-09 15:06 ` Eric Sandeen
2013-04-09 15:18   ` 符永涛
2013-04-09 15:23     ` Eric Sandeen
2013-04-09 15:25       ` 符永涛
2013-04-09 15:23     ` 符永涛
2013-04-09 15:44       ` Eric Sandeen
2013-04-09 15:48         ` 符永涛
2013-04-09 15:49           ` 符永涛
2013-04-09 15:58           ` Brian Foster
2013-04-09 17:10     ` Eric Sandeen
2013-04-10  5:34       ` 符永涛
2013-04-10  5:36         ` 符永涛

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=51675E67.7030600@sandeen.net \
    --to=sandeen@sandeen.net \
    --cc=bfoster@redhat.com \
    --cc=bpm@sgi.com \
    --cc=xfs@oss.sgi.com \
    --cc=yongtaofu@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.