All of lore.kernel.org
 help / color / mirror / Atom feed
From: Andrew Morton <akpm@linux-foundation.org>
To: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: nfs@lists.sourceforge.net,
	linux-pm <linux-pm@lists.linux-foundation.org>,
	lkml <linux-kernel@vger.kernel.org>,
	Trond Myklebust <trond.myklebust@fys.uio.no>
Subject: Re: A unresponsive file system can hang all I/O in the system on linux-2.6.23-rc6 (dirty_thresh problem?)
Date: Fri, 28 Sep 2007 12:16:42 -0700	[thread overview]
Message-ID: <20070928121642.56a380ce.akpm@linux-foundation.org> (raw)
In-Reply-To: <1191005339.18147.89.camel@lappy>

On Fri, 28 Sep 2007 20:48:59 +0200 Peter Zijlstra <a.p.zijlstra@chello.nl> wrote:

> 
> On Fri, 2007-09-28 at 11:49 -0700, Andrew Morton wrote:
> 
> > Do you know where the stalls are occurring?  throttle_vm_writeout(), or via
> > direct calls to congestion_wait() from page_alloc.c and vmscan.c?  (running
> > sysrq-w five or ten times will probably be enough to determine this)
> 
> would it make sense to instrument congestion_wait() callsites with
> vmstats?

Better than nothing, but it isn't a great fit: we'd need one vmstat counter
per congestion_wait() callsite, and it's all rather specific to the
kernel-of-the-day.

taskstats delay accounting isn't useful either - it will aggregate all the
schedule() callsites.

profile=sleep is just about ideal for this, isn't it?  I suspect that most
people don't know it's there, or forgot about it.

It could be that profile=sleep just tells us "you're spending a lot of time
in io_schedule()" or congestion_wait(), so perhaps we need to teach it to
go for walk up the stack somehow.

But lockdep knows how to do that already so perhaps we (ie: you ;)) can
bolt sleep instrumentation onto lockdep as we (ie you ;)) did with the
lockstat stuff?

(Searches for the lockstat documentation)

Did we forget to do that?

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

WARNING: multiple messages have this Message-ID (diff)
From: Andrew Morton <akpm@linux-foundation.org>
To: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Trond Myklebust <trond.myklebust@fys.uio.no>,
	Chakri n <chakriin5@gmail.com>,
	linux-pm <linux-pm@lists.linux-foundation.org>,
	lkml <linux-kernel@vger.kernel.org>,
	nfs@lists.sourceforge.net
Subject: Re: A unresponsive file system can hang all I/O in the system on linux-2.6.23-rc6 (dirty_thresh problem?)
Date: Fri, 28 Sep 2007 12:16:42 -0700	[thread overview]
Message-ID: <20070928121642.56a380ce.akpm@linux-foundation.org> (raw)
In-Reply-To: <1191005339.18147.89.camel@lappy>

On Fri, 28 Sep 2007 20:48:59 +0200 Peter Zijlstra <a.p.zijlstra@chello.nl> wrote:

> 
> On Fri, 2007-09-28 at 11:49 -0700, Andrew Morton wrote:
> 
> > Do you know where the stalls are occurring?  throttle_vm_writeout(), or via
> > direct calls to congestion_wait() from page_alloc.c and vmscan.c?  (running
> > sysrq-w five or ten times will probably be enough to determine this)
> 
> would it make sense to instrument congestion_wait() callsites with
> vmstats?

Better than nothing, but it isn't a great fit: we'd need one vmstat counter
per congestion_wait() callsite, and it's all rather specific to the
kernel-of-the-day.

taskstats delay accounting isn't useful either - it will aggregate all the
schedule() callsites.

profile=sleep is just about ideal for this, isn't it?  I suspect that most
people don't know it's there, or forgot about it.

It could be that profile=sleep just tells us "you're spending a lot of time
in io_schedule()" or congestion_wait(), so perhaps we need to teach it to
go for walk up the stack somehow.

But lockdep knows how to do that already so perhaps we (ie: you ;)) can
bolt sleep instrumentation onto lockdep as we (ie you ;)) did with the
lockstat stuff?

(Searches for the lockstat documentation)

Did we forget to do that?

  reply	other threads:[~2007-09-28 19:17 UTC|newest]

Thread overview: 106+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2007-09-28  6:32 A unresponsive file system can hang all I/O in the system on linux-2.6.23-rc6 (dirty_thresh problem?) Chakri n
2007-09-28  6:50 ` Andrew Morton
2007-09-28  6:59   ` Peter Zijlstra
2007-09-28  6:59   ` Peter Zijlstra
2007-09-28  8:27     ` Chakri n
2007-09-28  8:27       ` Chakri n
2007-09-28  8:40       ` Peter Zijlstra
2007-09-28  9:01         ` Chakri n
2007-09-28  9:01         ` Chakri n
2007-09-28  9:01           ` Chakri n
2007-09-28  9:12           ` Peter Zijlstra
2007-09-28  9:12           ` Peter Zijlstra
2007-09-28  9:20             ` Chakri n
2007-09-28  9:20               ` Chakri n
2007-09-28  9:23               ` Peter Zijlstra
2007-09-28  9:23                 ` Peter Zijlstra
2007-09-28 10:36                 ` Chakri n
2007-09-28 10:36                   ` Chakri n
2007-09-28 10:36                 ` Chakri n
2007-09-28  9:23               ` Peter Zijlstra
2007-09-28  9:20             ` Chakri n
2007-09-28  8:40       ` Peter Zijlstra
2007-09-28  8:27     ` Chakri n
2007-09-28 13:28   ` Jonathan Corbet
2007-09-28 13:28     ` Jonathan Corbet
2007-09-28 13:35     ` Peter Zijlstra
2007-09-28 13:35     ` Peter Zijlstra
2007-09-28 16:45       ` Alan Stern
2007-09-28 16:45       ` [linux-pm] " Alan Stern
2007-09-28 16:45         ` Alan Stern
2007-09-29  1:27       ` Daniel Phillips
2007-09-29  1:27         ` Daniel Phillips
2007-09-29  1:27       ` Daniel Phillips
2007-09-28 18:04     ` Andrew Morton
2007-09-28 18:04       ` Andrew Morton
2007-09-28 18:04       ` Andrew Morton
2007-09-28 13:28   ` Jonathan Corbet
2007-09-28 17:00   ` Trond Myklebust
2007-09-28 18:49     ` Andrew Morton
2007-09-28 18:49       ` Andrew Morton
2007-09-28 18:48       ` Peter Zijlstra
2007-09-28 18:48       ` Peter Zijlstra
2007-09-28 19:16         ` Andrew Morton [this message]
2007-09-28 19:16           ` Andrew Morton
2007-10-02 13:36           ` Peter Zijlstra
2007-10-02 15:42             ` Randy Dunlap
2007-10-03  9:28               ` [PATCH] lockstat: documentation Peter Zijlstra
2007-10-03  9:35                 ` Ingo Molnar
2007-09-28 19:16         ` A unresponsive file system can hang all I/O in the system on linux-2.6.23-rc6 (dirty_thresh problem?) Andrew Morton
2007-09-28 19:16       ` Trond Myklebust
2007-09-28 19:26         ` Andrew Morton
2007-09-28 19:26           ` Andrew Morton
2007-09-28 19:52           ` Trond Myklebust
2007-09-28 19:52           ` Trond Myklebust
2007-09-28 19:52             ` Trond Myklebust
2007-09-28 20:10             ` Andrew Morton
2007-09-28 20:10               ` Andrew Morton
2007-09-28 20:32               ` Trond Myklebust
2007-09-28 20:32               ` Trond Myklebust
2007-09-28 20:32                 ` Trond Myklebust
2007-09-28 20:43                 ` Andrew Morton
2007-09-28 20:43                   ` Andrew Morton
2007-09-28 21:36                   ` Chakri n
2007-09-28 23:33                     ` Chakri n
2007-09-28 23:33                     ` Chakri n
2007-09-28 23:33                       ` Chakri n
2007-09-28 21:36                   ` Chakri n
2007-09-28 20:10             ` Andrew Morton
2007-09-28 20:24             ` Daniel Phillips
2007-09-28 20:24             ` Daniel Phillips
2007-09-28 19:26         ` Andrew Morton
2007-09-29  1:51         ` KDB? Daniel Phillips
2007-09-29  1:51         ` KDB? Daniel Phillips
2007-09-29  1:51           ` KDB? Daniel Phillips
2007-09-28 19:16       ` A unresponsive file system can hang all I/O in the system on linux-2.6.23-rc6 (dirty_thresh problem?) Trond Myklebust
2007-09-28 18:49     ` Andrew Morton
2007-09-28 17:00   ` Trond Myklebust
2007-09-29  0:46   ` Daniel Phillips
2007-09-29  0:46   ` Daniel Phillips
2007-09-29  0:46     ` Daniel Phillips
2007-09-28  6:50 ` Andrew Morton
2007-09-29 11:04 ` Fengguang Wu
2007-09-29 11:04   ` Fengguang Wu
2007-09-29 11:48     ` Peter Zijlstra
2007-09-29 12:28       ` Fengguang Wu
2007-09-29 12:28         ` Fengguang Wu
2007-09-29 14:43           ` Peter Zijlstra
2007-09-29 14:43           ` Peter Zijlstra
2007-09-29 12:28         ` Fengguang Wu
2007-09-29 11:48     ` Peter Zijlstra
2007-10-01 15:57     ` Chuck Ebbert
2007-10-01 15:57     ` Chuck Ebbert
2007-10-02  2:00       ` [PATCH] writeback: avoid possible balance_dirty_pages() lockup on a light-load bdi Fengguang Wu
2007-10-02  2:00         ` Fengguang Wu
2007-10-02  2:14           ` Andrew Morton
2007-10-02  2:14             ` Andrew Morton
2007-10-02 12:13             ` Fengguang Wu
2007-10-02 12:13               ` Fengguang Wu
2007-10-02 12:13               ` Fengguang Wu
2007-10-02 13:27               ` Fengguang Wu
2007-10-02 13:27                 ` Fengguang Wu
2007-10-02 13:27                 ` Fengguang Wu
2007-10-02 18:35                   ` Chuck Ebbert
2007-10-02 18:35                     ` Chuck Ebbert
2007-10-02  2:00         ` Fengguang Wu
2007-10-03 12:46         ` richard kennedy
2007-10-04  1:50           ` Fengguang Wu
2007-10-04  1:50             ` Fengguang Wu
2007-10-04  1:50             ` Fengguang Wu
2007-10-03 12:46         ` richard kennedy
2007-09-29 11:04   ` A unresponsive file system can hang all I/O in the system on linux-2.6.23-rc6 (dirty_thresh problem?) Fengguang Wu
  -- strict thread matches above, loose matches on Subject: below --
2007-09-28  6:32 Chakri n

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20070928121642.56a380ce.akpm@linux-foundation.org \
    --to=akpm@linux-foundation.org \
    --cc=a.p.zijlstra@chello.nl \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-pm@lists.linux-foundation.org \
    --cc=nfs@lists.sourceforge.net \
    --cc=trond.myklebust@fys.uio.no \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.