From: Dave Chinner <david@fromorbit.com>
To: Volker <mail@blafoo.org>
Cc: xfs@oss.sgi.com
Subject: Re: OOM on quotacheck (again?)
Date: Wed, 3 Oct 2012 08:15:25 +1000 [thread overview]
Message-ID: <20121002221525.GU23520@dastard> (raw)
In-Reply-To: <506B5357.6060609@blafoo.org>
On Tue, Oct 02, 2012 at 10:49:27PM +0200, Volker wrote:
> Hi,
>
> >> If its of any interest, i can supply the stack-traces.
> >
> > Yes, it is of interest, can you post everything you found out about
> > the problem? (dmesg, stack traces, repair output, etc).
>
> Everything posted here is from a single server and its chronologically
> top to bottom. Without having checked each and every stacktrace, it
> looked quite similar on the other servers.
>
> http://pastebin.com/PXquE4sM
So you had a hang on 2.6.37 to do with dquot reclaim, you rebooted
the server into what I think is a 3.6 kernel.
Log recovery failed with "bad clientid 0x0", so no superblock
problem. It does tend to indicate that 2.6.37 wrote bad data to the
log, though. If you reboot into 2.6.37, does log recovery run
successfully? i.e. does the failure only occur on 2.6.37 -> 3.6
with a dirty log?
You then ran xfs_repair -P -L, which threw lots of metadata
away and moved lots of stuff to lost+found.
You them mounted the filesystem on the same kernel (has
xfs_trans_read_buf_map() in the trace, hence the 3.6 version), and
it appears to be hung waiting for IO to complete on a dquot buffer.
That tends to indicate that maybe there's a problem with IO
completion somewhere below the XFS layer.
And if there's a problem below XFS w.r.t. IO compeltion, that also
makes me wonder if the log recovery problem isn't also caused by
something below XFS...
What mount options are you using on the 2.6.37 kernel?
> Sidenote:
> The xfs_repair would not finish without supplying -P, otherwise the
> repair hang in phase 6 (might be related to this bug:
> http://oss.sgi.com/archives/xfs-masters/2011-01/msg00009.html)
If you are upgrading your kernel, you should also upgrade your
xfsprogs installation as well.
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
next prev parent reply other threads:[~2012-10-02 22:14 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-09-19 14:12 OOM on quotacheck (again?) blafoo
2012-09-19 20:59 ` Dave Chinner
2012-09-20 9:32 ` Volker
2012-09-24 13:21 ` Dave Chinner
2012-09-24 14:47 ` Volker
2012-10-02 16:29 ` Volker
2012-10-02 20:09 ` Dave Chinner
2012-10-02 20:49 ` Volker
2012-10-02 22:15 ` Dave Chinner [this message]
2012-10-04 14:19 ` Volker
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20121002221525.GU23520@dastard \
--to=david@fromorbit.com \
--cc=mail@blafoo.org \
--cc=xfs@oss.sgi.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox