From: Dave Chinner <david@fromorbit.com>
To: Christian Kujau <lists@nerdbynature.de>
Cc: LKML <linux-kernel@vger.kernel.org>, xfs@oss.sgi.com
Subject: Re: 2.6.39-rc4+: oom-killer busy killing tasks
Date: Wed, 27 Apr 2011 20:28:24 +1000 [thread overview]
Message-ID: <20110427102824.GI12436@dastard> (raw)
In-Reply-To: <alpine.DEB.2.01.1104270042510.18728@trent.utfs.org>
On Wed, Apr 27, 2011 at 12:46:51AM -0700, Christian Kujau wrote:
> On Wed, 27 Apr 2011 at 12:26, Dave Chinner wrote:
> > What this shows is that VFS inode cache memory usage increases until
> > about the 550 sample mark before the VM starts to reclaim it with
> > extreme prejudice. At that point, I'd expect the XFS inode cache to
> > then shrink, and it doesn't. I've got no idea why the either the
>
> Do you remember any XFS changes past 2.6.38 that could be related to
> something like this?
There's plenty of changes that coul dbe the cause - we've changed
the inode reclaim to run in the background out of a workqueue as
well as via the shrinker, so it could even be workqueue starvation
causing the the problem...
hmmmm. Speaking of which - have you changed any of the XFS tunables
in /proc/sys/fs/xfs/ on your machine (specifically
xfssyncd_centisecs)?
> Bisecting is pretty slow on this machine. Could I somehow try to run
> 2.6.39-rc4 but w/o the XFS changes merged after 2.6.38? (Does someone know
> how to do this via git?)
Not easy because there are tree-wide changes that need to be
preserved (e.g. block layer plugging changes) while others around it
would need to be reverted....
> > Can you check if there are any blocked tasks nearing OOM (i.e. "echo
> > w > /proc/sysrq-trigger") so we can see if XFS inode reclaim is
> > stuck somewhere?
>
> Will do, tomorrow.
>
> Should I open a regression bug, so we don't loose track of this thing?
Whatever you want.
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
WARNING: multiple messages have this Message-ID (diff)
From: Dave Chinner <david@fromorbit.com>
To: Christian Kujau <lists@nerdbynature.de>
Cc: LKML <linux-kernel@vger.kernel.org>, xfs@oss.sgi.com
Subject: Re: 2.6.39-rc4+: oom-killer busy killing tasks
Date: Wed, 27 Apr 2011 20:28:24 +1000 [thread overview]
Message-ID: <20110427102824.GI12436@dastard> (raw)
In-Reply-To: <alpine.DEB.2.01.1104270042510.18728@trent.utfs.org>
On Wed, Apr 27, 2011 at 12:46:51AM -0700, Christian Kujau wrote:
> On Wed, 27 Apr 2011 at 12:26, Dave Chinner wrote:
> > What this shows is that VFS inode cache memory usage increases until
> > about the 550 sample mark before the VM starts to reclaim it with
> > extreme prejudice. At that point, I'd expect the XFS inode cache to
> > then shrink, and it doesn't. I've got no idea why the either the
>
> Do you remember any XFS changes past 2.6.38 that could be related to
> something like this?
There's plenty of changes that coul dbe the cause - we've changed
the inode reclaim to run in the background out of a workqueue as
well as via the shrinker, so it could even be workqueue starvation
causing the the problem...
hmmmm. Speaking of which - have you changed any of the XFS tunables
in /proc/sys/fs/xfs/ on your machine (specifically
xfssyncd_centisecs)?
> Bisecting is pretty slow on this machine. Could I somehow try to run
> 2.6.39-rc4 but w/o the XFS changes merged after 2.6.38? (Does someone know
> how to do this via git?)
Not easy because there are tree-wide changes that need to be
preserved (e.g. block layer plugging changes) while others around it
would need to be reverted....
> > Can you check if there are any blocked tasks nearing OOM (i.e. "echo
> > w > /proc/sysrq-trigger") so we can see if XFS inode reclaim is
> > stuck somewhere?
>
> Will do, tomorrow.
>
> Should I open a regression bug, so we don't loose track of this thing?
Whatever you want.
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
next prev parent reply other threads:[~2011-04-27 10:24 UTC|newest]
Thread overview: 84+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-04-22 1:57 2.6.39-rc4+: oom-killer busy killing tasks Christian Kujau
2011-04-22 1:57 ` Christian Kujau
2011-04-22 2:58 ` Minchan Kim
2011-04-22 2:58 ` Minchan Kim
2011-04-22 3:03 ` Christian Kujau
2011-04-22 3:03 ` Christian Kujau
2011-04-22 17:41 ` Christian Kujau
2011-04-22 17:41 ` Christian Kujau
2011-04-22 18:46 ` Christian Kujau
2011-04-22 18:46 ` Christian Kujau
2011-04-22 22:47 ` Minchan Kim
2011-04-22 22:47 ` Minchan Kim
2011-04-24 23:46 ` Dave Chinner
2011-04-24 23:46 ` Dave Chinner
2011-04-25 5:51 ` Christian Kujau
2011-04-25 5:51 ` Christian Kujau
2011-04-25 7:19 ` Christian Kujau
2011-04-25 7:19 ` Christian Kujau
2011-04-26 15:14 ` Christian Kujau
2011-04-26 15:14 ` Christian Kujau
2011-04-27 2:26 ` Dave Chinner
2011-04-27 2:26 ` Dave Chinner
2011-04-27 7:46 ` Christian Kujau
2011-04-27 7:46 ` Christian Kujau
2011-04-27 10:28 ` Dave Chinner [this message]
2011-04-27 10:28 ` Dave Chinner
2011-04-27 23:16 ` Minchan Kim
2011-04-27 23:16 ` Minchan Kim
2011-04-27 23:56 ` Dave Chinner
2011-04-27 23:56 ` Dave Chinner
2011-04-28 17:30 ` Christian Kujau
2011-04-28 17:30 ` Christian Kujau
2011-04-28 23:37 ` Dave Chinner
2011-04-28 23:37 ` Dave Chinner
2011-04-29 17:32 ` Christian Kujau
2011-04-29 17:32 ` Christian Kujau
2011-04-29 19:58 ` Christian Kujau
2011-04-29 19:58 ` Christian Kujau
2011-04-29 20:17 ` Markus Trippelsdorf
2011-04-29 20:17 ` Markus Trippelsdorf
2011-04-29 20:20 ` Christian Kujau
2011-04-29 20:20 ` Christian Kujau
2011-04-29 20:21 ` Markus Trippelsdorf
2011-04-29 20:21 ` Markus Trippelsdorf
2011-04-30 0:17 ` Christian Kujau
2011-04-30 0:17 ` Christian Kujau
2011-05-01 8:01 ` Dave Chinner
2011-05-01 8:01 ` Dave Chinner
2011-05-02 4:59 ` Christian Kujau
2011-05-02 4:59 ` Christian Kujau
2011-05-02 12:19 ` Dave Chinner
2011-05-02 12:19 ` Dave Chinner
2011-05-02 19:59 ` Christian Kujau
2011-05-02 19:59 ` Christian Kujau
2011-05-03 0:51 ` Dave Chinner
2011-05-03 0:51 ` Dave Chinner
2011-05-03 4:04 ` Christian Kujau
2011-05-03 4:04 ` Christian Kujau
2011-05-03 6:36 ` Dave Chinner
2011-05-03 6:36 ` Dave Chinner
2011-05-03 20:53 ` Christian Kujau
2011-05-03 20:53 ` Christian Kujau
2011-05-04 0:46 ` Christian Kujau
2011-05-04 0:46 ` Christian Kujau
2011-05-04 1:51 ` Christian Kujau
2011-05-04 1:51 ` Christian Kujau
2011-05-04 7:36 ` Dave Chinner
2011-05-04 7:36 ` Dave Chinner
2011-05-04 11:12 ` Dave Chinner
2011-05-04 11:12 ` Dave Chinner
2011-05-04 19:10 ` Christian Kujau
2011-05-04 19:10 ` Christian Kujau
2011-05-04 23:15 ` Dave Chinner
2011-05-04 23:15 ` Dave Chinner
2011-05-05 2:07 ` Christian Kujau
2011-05-05 2:07 ` Christian Kujau
2011-05-02 9:26 ` Christian Kujau
2011-05-02 9:26 ` Christian Kujau
2011-05-02 12:38 ` Dave Chinner
2011-05-02 12:38 ` Dave Chinner
2011-04-25 8:02 ` Christian Kujau
2011-04-25 8:02 ` Christian Kujau
2011-04-25 9:50 ` Christian Kujau
2011-04-25 9:50 ` Christian Kujau
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20110427102824.GI12436@dastard \
--to=david@fromorbit.com \
--cc=linux-kernel@vger.kernel.org \
--cc=lists@nerdbynature.de \
--cc=xfs@oss.sgi.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.