From: Dave Chinner <david@fromorbit.com>
To: L Ox <lox8096@gmail.com>
Cc: xfs@oss.sgi.com
Subject: Re: 88TB filesystem going off-line without warning
Date: Thu, 4 Apr 2013 15:35:15 +1100 [thread overview]
Message-ID: <20130404043515.GC12011@dastard> (raw)
In-Reply-To: <CA+dre-c0K=AWRNRPgNVWANNkMW++FXEJvQzSe=68_JXyJMeNxQ@mail.gmail.com>
On Tue, Apr 02, 2013 at 11:44:15AM -0700, L Ox wrote:
> Hi,
>
> We have a new Linux/XFS deployment (about a month old) and randomly without
> warning the XFS filesystem will go off-line. We are running Scientific
> Linux release 5.9 with the latest updates.
>
> # uname -a
> Linux node24 2.6.18-348.3.1.el5 #1 SMP Mon Mar 11 15:43:13 EDT 2013 x86_64
> x86_64 x86_64 GNU/Linux
>
> # cat /etc/redhat-release
> Scientific Linux release 5.9 (Boron)
>
> Here are the errors we see in /var/log/messages after the initial off-line
> event:
>
> -- snip --
>
> Apr 2 07:50:28 node24 kernel: xfs_iunlink_remove: xfs_inotobp() returned
> an error 22 on dm-6. Returning error.
> Apr 2 07:50:28 node24 kernel: xfs_inactive: xfs_ifree() returned an error
> = 22 on dm-6
#define EINVAL 22 /* Invalid argument */
That tends to imply a corrupt inode number in the unlinked list
chain.
> Here are the messages after I umount/xfs_repair/mount the filesystem:
What did xfs_repair detect/fix?
> # xfs_repair /dev/mapper/vol_d24-root
> Phase 1 - find and verify superblock...
.....
> Phase 6 - check inode connectivity...
> - resetting contents of realtime bitmap and summary inodes
> - traversing filesystem ...
> - traversal finished ...
> - moving disconnected inodes to lost+found ...
> disconnected inode 202102936036, moving to lost+found
> disconnected inode 215350040250, moving to lost+found
> disconnected inode 215350208634, moving to lost+found
> disconnected inode 271016406074, moving to lost+found
Some inodes that had been unlinked from the directory structure
but not freed. They were probably on an unlinked inode list that
couldn't be walked.
> Any ideas?
If the problem is a one off, there isn't anything that can be done.
If you can reproduce it, try to narrow it down to the simplest case
you can...
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
prev parent reply other threads:[~2013-04-04 4:35 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-04-02 18:44 88TB filesystem going off-line without warning L Ox
2013-04-03 19:40 ` Emmanuel Florac
2013-04-04 4:35 ` Dave Chinner [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20130404043515.GC12011@dastard \
--to=david@fromorbit.com \
--cc=lox8096@gmail.com \
--cc=xfs@oss.sgi.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox