From: Michael Meier <michael.meier@fau.de>
To: xfs@oss.sgi.com
Subject: XFS hangs with XFS: possible memory allocation deadlock in kmem_alloc
Date: Sat, 07 Mar 2015 08:51:50 +0100 [thread overview]
Message-ID: <54FAAE16.6090505@fau.de> (raw)
We've recently upgraded the OS on one of our servers, and since then
have been experiencing frequent stalls of the XFS filesystem on it.
Other filesystems on the machine seem to still respond fine while XFS
hangs. The stalls sometimes last for around 30 minutes, during which all
attempts to access that filesystem hang completely - after that, the
filesystem suddenly responds instantly again, as if there had never been
any problem. The dmesg is full of these messages while it stalls:
XFS: possible memory allocation deadlock in kmem_alloc (mode:0x8250)
These also occour from time to time without the filesystem stalling (or
at least it's not noticeable) - the messages appear about once in two
hours, the stalls about once a day.
Google did point me to some reports of these messages occouring at the
end of 2013, but the kernels in question should all have had the fixes
proposed back then - although one message back then suggested there were
more places where this problem could occour that were not fixed yet.
Kernels used were:
- Ubuntu 3.13.0-44 - shows stalls, according to
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1382333 has the fix
- Ubuntu 3.16.0-31 - shows stalls
- Ubuntu 3.2.0-various - no stalls in more than 1 year
We can actually still boot the machine with the 3.2.0 kernel, and it
will run absolutely fine, but as that kernel will not be supported
forever, I do not consider that a permanent solution.
The machine should not be low on memory, the disk array far from its
limits, and the I/O-load is mostly reads with very little writes, as
this is a public FTP server.
I have tried to collect some information, available at
https://grid.rrze.uni-erlangen.de/~unrz191/syslog-with-xfs-hangs.log
Regards,
--
Michael Meier, Zentrale Systeme
Friedrich-Alexander-Universitaet Erlangen-Nuernberg
Regionales Rechenzentrum Erlangen
Martensstrasse 1, 91058 Erlangen, Germany
Tel.: +49 9131 85-28973, Fax: +49 9131 302941
michael.meier@fau.de
www.rrze.fau.de
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
next reply other threads:[~2015-03-07 7:52 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-03-07 7:51 Michael Meier [this message]
2015-03-07 14:07 ` XFS hangs with XFS: possible memory allocation deadlock in kmem_alloc Brian Foster
2015-03-07 19:14 ` Michael Meier
2015-03-09 11:52 ` Dave Chinner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=54FAAE16.6090505@fau.de \
--to=michael.meier@fau.de \
--cc=xfs@oss.sgi.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox