All of lore.kernel.org
 help / color / mirror / Atom feed
From: Wu Fengguang <fengguang.wu@intel.com>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: Dave Chinner <david@fromorbit.com>,
	Wu Fengguang <fengguang.wu@intel.com>
Cc: Christoph Hellwig <hch@infradead.org>
Cc: Mel Gorman <mel@csn.ul.ie>
Cc: Chris Mason <chris.mason@oracle.com>
Cc: Jens Axboe <jens.axboe@oracle.com>
Cc: LKML <linux-kernel@vger.kernel.org>
Cc: linux-fsdevel@vger.kernel.org
Cc: linux-mm@kvack.org
Subject: [PATCH 5/6] writeback: try more writeback as long as something was written
Date: Thu, 22 Jul 2010 13:09:33 +0800	[thread overview]
Message-ID: <20100722061823.050523298@intel.com> (raw)
In-Reply-To: 20100722050928.653312535@intel.com

[-- Attachment #1: writeback-background-retry.patch --]
[-- Type: text/plain, Size: 2268 bytes --]

writeback_inodes_wb()/__writeback_inodes_sb() are not agressive in that
they only populate b_io when necessary at entrance time. When the queued
set of inodes are all synced, they just return, possibly with
wbc.nr_to_write > 0.

For kupdate and background writeback, there may be more eligible inodes
sitting in b_dirty when the current set of b_io inodes are completed. So
it is necessary to try another round of writeback as long as we made some
progress in this round. When there are no more eligible inodes, no more
inodes will be enqueued in queue_io(), hence nothing could/will be
synced and we may safely bail.

This will livelock sync when there are heavy dirtiers. However in that case
sync will already be livelocked w/o this patch, as the current livelock
avoidance code is virtually a no-op (for one thing, wb_time should be
set statically at sync start time and be used in move_expired_inodes()).
The sync livelock problem will be addressed in other patches.

Signed-off-by: Wu Fengguang <fengguang.wu@intel.com>
---
 fs/fs-writeback.c |   19 +++++++++++--------
 1 file changed, 11 insertions(+), 8 deletions(-)

--- linux-next.orig/fs/fs-writeback.c	2010-07-22 13:07:51.000000000 +0800
+++ linux-next/fs/fs-writeback.c	2010-07-22 13:07:54.000000000 +0800
@@ -640,20 +640,23 @@ static long wb_writeback(struct bdi_writ
 		wrote += MAX_WRITEBACK_PAGES - wbc.nr_to_write;
 
 		/*
-		 * If we consumed everything, see if we have more
+		 * Did we write something? Try for more
+		 *
+		 * This is needed _before_ the b_more_io test because the
+		 * background writeback moves inodes to b_io and works on
+		 * them in batches (in order to sync old pages first).  The
+		 * completion of the current batch does not necessarily mean
+		 * the overall work is done.
 		 */
-		if (wbc.nr_to_write <= 0)
+		if (wbc.nr_to_write < MAX_WRITEBACK_PAGES)
 			continue;
+
 		/*
-		 * Didn't write everything and we don't have more IO, bail
+		 * Nothing written and no more inodes for IO, bail
 		 */
 		if (list_empty(&wb->b_more_io))
 			break;
-		/*
-		 * Did we write something? Try for more
-		 */
-		if (wbc.nr_to_write < MAX_WRITEBACK_PAGES)
-			continue;
+
 		/*
 		 * Nothing written. Wait for some inode to
 		 * become available for writeback. Otherwise



WARNING: multiple messages have this Message-ID (diff)
From: Wu Fengguang <fengguang.wu@intel.com>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: Dave Chinner <david@fromorbit.com>,
	Wu Fengguang <fengguang.wu@intel.com>
Cc: Christoph Hellwig <hch@infradead.org>
Cc: Mel Gorman <mel@csn.ul.ie>
Cc: Chris Mason <chris.mason@oracle.com>
Cc: Jens Axboe <jens.axboe@oracle.com>
Cc: LKML <linux-kernel@vger.kernel.org>
Cc: linux-fsdevel@vger.kernel.org
Cc: linux-mm@kvack.org
Subject: [PATCH 5/6] writeback: try more writeback as long as something was written
Date: Thu, 22 Jul 2010 13:09:33 +0800	[thread overview]
Message-ID: <20100722061823.050523298@intel.com> (raw)
In-Reply-To: 20100722050928.653312535@intel.com

[-- Attachment #1: writeback-background-retry.patch --]
[-- Type: text/plain, Size: 2493 bytes --]

writeback_inodes_wb()/__writeback_inodes_sb() are not agressive in that
they only populate b_io when necessary at entrance time. When the queued
set of inodes are all synced, they just return, possibly with
wbc.nr_to_write > 0.

For kupdate and background writeback, there may be more eligible inodes
sitting in b_dirty when the current set of b_io inodes are completed. So
it is necessary to try another round of writeback as long as we made some
progress in this round. When there are no more eligible inodes, no more
inodes will be enqueued in queue_io(), hence nothing could/will be
synced and we may safely bail.

This will livelock sync when there are heavy dirtiers. However in that case
sync will already be livelocked w/o this patch, as the current livelock
avoidance code is virtually a no-op (for one thing, wb_time should be
set statically at sync start time and be used in move_expired_inodes()).
The sync livelock problem will be addressed in other patches.

Signed-off-by: Wu Fengguang <fengguang.wu@intel.com>
---
 fs/fs-writeback.c |   19 +++++++++++--------
 1 file changed, 11 insertions(+), 8 deletions(-)

--- linux-next.orig/fs/fs-writeback.c	2010-07-22 13:07:51.000000000 +0800
+++ linux-next/fs/fs-writeback.c	2010-07-22 13:07:54.000000000 +0800
@@ -640,20 +640,23 @@ static long wb_writeback(struct bdi_writ
 		wrote += MAX_WRITEBACK_PAGES - wbc.nr_to_write;
 
 		/*
-		 * If we consumed everything, see if we have more
+		 * Did we write something? Try for more
+		 *
+		 * This is needed _before_ the b_more_io test because the
+		 * background writeback moves inodes to b_io and works on
+		 * them in batches (in order to sync old pages first).  The
+		 * completion of the current batch does not necessarily mean
+		 * the overall work is done.
 		 */
-		if (wbc.nr_to_write <= 0)
+		if (wbc.nr_to_write < MAX_WRITEBACK_PAGES)
 			continue;
+
 		/*
-		 * Didn't write everything and we don't have more IO, bail
+		 * Nothing written and no more inodes for IO, bail
 		 */
 		if (list_empty(&wb->b_more_io))
 			break;
-		/*
-		 * Did we write something? Try for more
-		 */
-		if (wbc.nr_to_write < MAX_WRITEBACK_PAGES)
-			continue;
+
 		/*
 		 * Nothing written. Wait for some inode to
 		 * become available for writeback. Otherwise


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

WARNING: multiple messages have this Message-ID (diff)
From: Wu Fengguang <fengguang.wu@intel.com>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: Dave Chinner <david@fromorbit.com>,
	Wu Fengguang <fengguang.wu@intel.com>,
	Christoph Hellwig <hch@infradead.org>, Mel Gorman <mel@csn.ul.ie>,
	Chris Mason <chris.mason@oracle.com>,
	Jens Axboe <jens.axboe@oracle.com>,
	LKML <linux-kernel@vger.kernel.org>,
	linux-fsdevel@vger.kernel.org, linux-mm@kvack.org
Subject: [PATCH 5/6] writeback: try more writeback as long as something was written
Date: Thu, 22 Jul 2010 13:09:33 +0800	[thread overview]
Message-ID: <20100722061823.050523298@intel.com> (raw)
In-Reply-To: 20100722050928.653312535@intel.com

[-- Attachment #1: writeback-background-retry.patch --]
[-- Type: text/plain, Size: 2493 bytes --]

writeback_inodes_wb()/__writeback_inodes_sb() are not agressive in that
they only populate b_io when necessary at entrance time. When the queued
set of inodes are all synced, they just return, possibly with
wbc.nr_to_write > 0.

For kupdate and background writeback, there may be more eligible inodes
sitting in b_dirty when the current set of b_io inodes are completed. So
it is necessary to try another round of writeback as long as we made some
progress in this round. When there are no more eligible inodes, no more
inodes will be enqueued in queue_io(), hence nothing could/will be
synced and we may safely bail.

This will livelock sync when there are heavy dirtiers. However in that case
sync will already be livelocked w/o this patch, as the current livelock
avoidance code is virtually a no-op (for one thing, wb_time should be
set statically at sync start time and be used in move_expired_inodes()).
The sync livelock problem will be addressed in other patches.

Signed-off-by: Wu Fengguang <fengguang.wu@intel.com>
---
 fs/fs-writeback.c |   19 +++++++++++--------
 1 file changed, 11 insertions(+), 8 deletions(-)

--- linux-next.orig/fs/fs-writeback.c	2010-07-22 13:07:51.000000000 +0800
+++ linux-next/fs/fs-writeback.c	2010-07-22 13:07:54.000000000 +0800
@@ -640,20 +640,23 @@ static long wb_writeback(struct bdi_writ
 		wrote += MAX_WRITEBACK_PAGES - wbc.nr_to_write;
 
 		/*
-		 * If we consumed everything, see if we have more
+		 * Did we write something? Try for more
+		 *
+		 * This is needed _before_ the b_more_io test because the
+		 * background writeback moves inodes to b_io and works on
+		 * them in batches (in order to sync old pages first).  The
+		 * completion of the current batch does not necessarily mean
+		 * the overall work is done.
 		 */
-		if (wbc.nr_to_write <= 0)
+		if (wbc.nr_to_write < MAX_WRITEBACK_PAGES)
 			continue;
+
 		/*
-		 * Didn't write everything and we don't have more IO, bail
+		 * Nothing written and no more inodes for IO, bail
 		 */
 		if (list_empty(&wb->b_more_io))
 			break;
-		/*
-		 * Did we write something? Try for more
-		 */
-		if (wbc.nr_to_write < MAX_WRITEBACK_PAGES)
-			continue;
+
 		/*
 		 * Nothing written. Wait for some inode to
 		 * become available for writeback. Otherwise


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  parent reply	other threads:[~2010-07-22  6:20 UTC|newest]

Thread overview: 130+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2010-07-22  5:09 [PATCH 0/6] [RFC] writeback: try to write older pages first Wu Fengguang
2010-07-22  5:09 ` Wu Fengguang
2010-07-22  5:09 ` Wu Fengguang
2010-07-22  5:09 ` [PATCH 1/6] writeback: pass writeback_control down to move_expired_inodes() Wu Fengguang
2010-07-22  5:09   ` Wu Fengguang
2010-07-22  5:09   ` Wu Fengguang
2010-07-23 18:16   ` Jan Kara
2010-07-23 18:16     ` Jan Kara
2010-07-26 10:44   ` Mel Gorman
2010-07-26 10:44     ` Mel Gorman
2010-08-01 15:23   ` Minchan Kim
2010-08-01 15:23     ` Minchan Kim
2010-07-22  5:09 ` [PATCH 2/6] writeback: the kupdate expire timestamp should be a moving target Wu Fengguang
2010-07-22  5:09   ` Wu Fengguang
2010-07-22  5:09   ` Wu Fengguang
2010-07-23 18:17   ` Jan Kara
2010-07-23 18:17     ` Jan Kara
2010-07-26 10:52   ` Mel Gorman
2010-07-26 10:52     ` Mel Gorman
2010-07-26 11:32     ` Wu Fengguang
2010-07-26 11:32       ` Wu Fengguang
2010-08-01 15:29   ` Minchan Kim
2010-08-01 15:29     ` Minchan Kim
2010-07-22  5:09 ` [PATCH 3/6] writeback: kill writeback_control.more_io Wu Fengguang
2010-07-22  5:09   ` Wu Fengguang
2010-07-22  5:09   ` Wu Fengguang
2010-07-23 18:24   ` Jan Kara
2010-07-23 18:24     ` Jan Kara
2010-07-26 10:53   ` Mel Gorman
2010-07-26 10:53     ` Mel Gorman
2010-08-01 15:34   ` Minchan Kim
2010-08-01 15:34     ` Minchan Kim
2010-08-05 14:50     ` Wu Fengguang
2010-08-05 14:50       ` Wu Fengguang
2010-08-05 14:55       ` Wu Fengguang
2010-08-05 14:55         ` Wu Fengguang
2010-08-05 14:56       ` Minchan Kim
2010-08-05 14:56         ` Minchan Kim
2010-08-05 15:26         ` Wu Fengguang
2010-08-05 15:26           ` Wu Fengguang
2010-07-22  5:09 ` [PATCH 4/6] writeback: sync expired inodes first in background writeback Wu Fengguang
2010-07-22  5:09   ` Wu Fengguang
2010-07-23 18:15   ` Jan Kara
2010-07-23 18:15     ` Jan Kara
2010-07-26 11:51     ` Wu Fengguang
2010-07-26 11:51       ` Wu Fengguang
2010-07-26 12:12       ` Jan Kara
2010-07-26 12:12         ` Jan Kara
2010-07-26 12:29         ` Wu Fengguang
2010-07-26 12:29           ` Wu Fengguang
2010-07-26 10:57   ` Mel Gorman
2010-07-26 10:57     ` Mel Gorman
2010-07-26 12:00     ` Wu Fengguang
2010-07-26 12:00       ` Wu Fengguang
2010-07-26 12:20       ` Jan Kara
2010-07-26 12:20         ` Jan Kara
2010-07-26 12:31         ` Wu Fengguang
2010-07-26 12:31           ` Wu Fengguang
2010-07-26 12:39           ` Jan Kara
2010-07-26 12:39             ` Jan Kara
2010-07-26 12:47             ` Wu Fengguang
2010-07-26 12:47               ` Wu Fengguang
2010-07-26 12:56     ` Wu Fengguang
2010-07-26 12:56       ` Wu Fengguang
2010-07-26 12:59       ` Mel Gorman
2010-07-26 12:59         ` Mel Gorman
2010-07-26 13:11         ` Wu Fengguang
2010-07-26 13:11           ` Wu Fengguang
2010-07-27  9:45           ` Mel Gorman
2010-07-27  9:45             ` Mel Gorman
2010-08-01 15:15           ` Minchan Kim
2010-08-01 15:15             ` Minchan Kim
2010-07-22  5:09 ` Wu Fengguang [this message]
2010-07-22  5:09   ` [PATCH 5/6] writeback: try more writeback as long as something was written Wu Fengguang
2010-07-22  5:09   ` Wu Fengguang
2010-07-23 17:39   ` Jan Kara
2010-07-23 17:39     ` Jan Kara
2010-07-26 12:39     ` Wu Fengguang
2010-07-26 12:39       ` Wu Fengguang
2010-07-26 11:01   ` Mel Gorman
2010-07-26 11:01     ` Mel Gorman
2010-07-26 11:39     ` Wu Fengguang
2010-07-26 11:39       ` Wu Fengguang
2010-07-22  5:09 ` [PATCH 6/6] writeback: introduce writeback_control.inodes_written Wu Fengguang
2010-07-22  5:09   ` Wu Fengguang
2010-07-22  5:09   ` Wu Fengguang
2010-07-26 11:04   ` Mel Gorman
2010-07-26 11:04     ` Mel Gorman
2010-07-23 10:24 ` [PATCH 0/6] [RFC] writeback: try to write older pages first Mel Gorman
2010-07-23 10:24   ` Mel Gorman
2010-07-26  7:18   ` Wu Fengguang
2010-07-26  7:18     ` Wu Fengguang
2010-07-26 10:42     ` Mel Gorman
2010-07-26 10:42       ` Mel Gorman
2010-07-26 10:28 ` Itaru Kitayama
2010-07-26 10:28   ` Itaru Kitayama
2010-07-26 11:47   ` Wu Fengguang
2010-07-26 11:47     ` Wu Fengguang
  -- strict thread matches above, loose matches on Subject: below --
2011-04-19  3:00 [PATCH 0/6] writeback: moving expire targets for background/kupdate works Wu Fengguang
2011-04-19  3:00 ` [PATCH 5/6] writeback: try more writeback as long as something was written Wu Fengguang
2011-04-19  3:00   ` Wu Fengguang
2011-04-19  3:00   ` Wu Fengguang
2011-04-19 10:20   ` Jan Kara
2011-04-19 10:20     ` Jan Kara
2011-04-19 11:16     ` Wu Fengguang
2011-04-19 11:16       ` Wu Fengguang
2011-04-19 21:10       ` Jan Kara
2011-04-19 21:10         ` Jan Kara
2011-04-20  7:50         ` Wu Fengguang
2011-04-20  7:50           ` Wu Fengguang
2011-04-20 15:22           ` Jan Kara
2011-04-20 15:22             ` Jan Kara
2011-04-21  3:33             ` Wu Fengguang
2011-04-21  4:39               ` Christoph Hellwig
2011-04-21  4:39                 ` Christoph Hellwig
2011-04-21  6:05                 ` Wu Fengguang
2011-04-21  6:05                   ` Wu Fengguang
2011-04-21 16:41                   ` Jan Kara
2011-04-21 16:41                     ` Jan Kara
2011-04-22  2:32                     ` Wu Fengguang
2011-04-22  2:32                       ` Wu Fengguang
2011-04-22 21:23                       ` Jan Kara
2011-04-22 21:23                         ` Jan Kara
2011-04-21  7:09               ` Dave Chinner
2011-04-21  7:09                 ` Dave Chinner
2011-04-21  7:14                 ` Christoph Hellwig
2011-04-21  7:14                   ` Christoph Hellwig
2011-04-21  7:52                   ` Dave Chinner
2011-04-21  7:52                     ` Dave Chinner
2011-04-21  8:00                     ` Christoph Hellwig
2011-04-21  8:00                       ` Christoph Hellwig

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20100722061823.050523298@intel.com \
    --to=fengguang.wu@intel.com \
    --cc=akpm@linux-foundation.org \
    --cc=david@fromorbit.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.