All of lore.kernel.org
 help / color / mirror / Atom feed
From: Wu Fengguang <fengguang.wu@intel.com>
To: Mel Gorman <mel@csn.ul.ie>
Cc: Christoph Hellwig <hch@infradead.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-fsdevel@vger.kernel.org" <linux-fsdevel@vger.kernel.org>,
	"linux-mm@kvack.org" <linux-mm@kvack.org>,
	Dave Chinner <david@fromorbit.com>,
	Chris Mason <chris.mason@oracle.com>,
	Nick Piggin <npiggin@suse.de>, Rik van Riel <riel@redhat.com>,
	Johannes Weiner <hannes@cmpxchg.org>,
	KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>,
	KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Andrea Arcangeli <aarcange@redhat.com>,
	Minchan Kim <minchan.kim@gmail.com>
Subject: Re: [PATCH 7/8] writeback: sync old inodes first in background writeback
Date: Thu, 22 Jul 2010 16:52:10 +0800	[thread overview]
Message-ID: <20100722085210.GA26714@localhost> (raw)
In-Reply-To: <20100719144046.GR13117@csn.ul.ie>

> Some insight on how the other writeback changes that are being floated
> around might affect the number of dirty pages reclaim encounters would also
> be helpful.

Here is an interesting related problem about the wait_on_page_writeback() call
inside shrink_page_list():

        http://lkml.org/lkml/2010/4/4/86

The problem is, wait_on_page_writeback() is called too early in the
direct reclaim path, which blocks many random/unrelated processes when
some slow (USB stick) writeback is on the way.

A simple dd can easily create a big range of dirty pages in the LRU
list. Therefore priority can easily go below (DEF_PRIORITY - 2) in a
typical desktop, which triggers the lumpy reclaim mode and hence
wait_on_page_writeback().

I proposed this patch at the time, which was confirmed to solve the problem:

--- linux-next.orig/mm/vmscan.c	2010-06-24 14:32:03.000000000 +0800
+++ linux-next/mm/vmscan.c	2010-07-22 16:12:34.000000000 +0800
@@ -1650,7 +1650,7 @@ static void set_lumpy_reclaim_mode(int p
 	 */
 	if (sc->order > PAGE_ALLOC_COSTLY_ORDER)
 		sc->lumpy_reclaim_mode = 1;
-	else if (sc->order && priority < DEF_PRIORITY - 2)
+	else if (sc->order && priority < DEF_PRIORITY / 2)
 		sc->lumpy_reclaim_mode = 1;
 	else
 		sc->lumpy_reclaim_mode = 0;


However KOSAKI and Minchan raised concerns about raising the bar.
I guess this new patch is more problem oriented and acceptable:

--- linux-next.orig/mm/vmscan.c	2010-07-22 16:36:58.000000000 +0800
+++ linux-next/mm/vmscan.c	2010-07-22 16:39:57.000000000 +0800
@@ -1217,7 +1217,8 @@ static unsigned long shrink_inactive_lis
 			count_vm_events(PGDEACTIVATE, nr_active);
 
 			nr_freed += shrink_page_list(&page_list, sc,
-							PAGEOUT_IO_SYNC);
+					priority < DEF_PRIORITY / 3 ?
+					PAGEOUT_IO_SYNC : PAGEOUT_IO_ASYNC);
 		}
 
 		nr_reclaimed += nr_freed;

Thanks,
Fengguang

WARNING: multiple messages have this Message-ID (diff)
From: Wu Fengguang <fengguang.wu@intel.com>
To: Mel Gorman <mel@csn.ul.ie>
Cc: Christoph Hellwig <hch@infradead.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-fsdevel@vger.kernel.org" <linux-fsdevel@vger.kernel.org>,
	"linux-mm@kvack.org" <linux-mm@kvack.org>,
	Dave Chinner <david@fromorbit.com>,
	Chris Mason <chris.mason@oracle.com>,
	Nick Piggin <npiggin@suse.de>, Rik van Riel <riel@redhat.com>,
	Johannes Weiner <hannes@cmpxchg.org>,
	KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>,
	KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Andrea Arcangeli <aarcange@redhat.com>,
	Minchan Kim <minchan.kim@gmail.com>
Subject: Re: [PATCH 7/8] writeback: sync old inodes first in background writeback
Date: Thu, 22 Jul 2010 16:52:10 +0800	[thread overview]
Message-ID: <20100722085210.GA26714@localhost> (raw)
In-Reply-To: <20100719144046.GR13117@csn.ul.ie>

> Some insight on how the other writeback changes that are being floated
> around might affect the number of dirty pages reclaim encounters would also
> be helpful.

Here is an interesting related problem about the wait_on_page_writeback() call
inside shrink_page_list():

        http://lkml.org/lkml/2010/4/4/86

The problem is, wait_on_page_writeback() is called too early in the
direct reclaim path, which blocks many random/unrelated processes when
some slow (USB stick) writeback is on the way.

A simple dd can easily create a big range of dirty pages in the LRU
list. Therefore priority can easily go below (DEF_PRIORITY - 2) in a
typical desktop, which triggers the lumpy reclaim mode and hence
wait_on_page_writeback().

I proposed this patch at the time, which was confirmed to solve the problem:

--- linux-next.orig/mm/vmscan.c	2010-06-24 14:32:03.000000000 +0800
+++ linux-next/mm/vmscan.c	2010-07-22 16:12:34.000000000 +0800
@@ -1650,7 +1650,7 @@ static void set_lumpy_reclaim_mode(int p
 	 */
 	if (sc->order > PAGE_ALLOC_COSTLY_ORDER)
 		sc->lumpy_reclaim_mode = 1;
-	else if (sc->order && priority < DEF_PRIORITY - 2)
+	else if (sc->order && priority < DEF_PRIORITY / 2)
 		sc->lumpy_reclaim_mode = 1;
 	else
 		sc->lumpy_reclaim_mode = 0;


However KOSAKI and Minchan raised concerns about raising the bar.
I guess this new patch is more problem oriented and acceptable:

--- linux-next.orig/mm/vmscan.c	2010-07-22 16:36:58.000000000 +0800
+++ linux-next/mm/vmscan.c	2010-07-22 16:39:57.000000000 +0800
@@ -1217,7 +1217,8 @@ static unsigned long shrink_inactive_lis
 			count_vm_events(PGDEACTIVATE, nr_active);
 
 			nr_freed += shrink_page_list(&page_list, sc,
-							PAGEOUT_IO_SYNC);
+					priority < DEF_PRIORITY / 3 ?
+					PAGEOUT_IO_SYNC : PAGEOUT_IO_ASYNC);
 		}
 
 		nr_reclaimed += nr_freed;

Thanks,
Fengguang

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  parent reply	other threads:[~2010-07-22  8:52 UTC|newest]

Thread overview: 177+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2010-07-19 13:11 [PATCH 0/8] Reduce writeback from page reclaim context V4 Mel Gorman
2010-07-19 13:11 ` Mel Gorman
2010-07-19 13:11 ` [PATCH 1/8] vmscan: tracing: Roll up of patches currently in mmotm Mel Gorman
2010-07-19 13:11   ` Mel Gorman
2010-07-19 13:11 ` [PATCH 2/8] vmscan: tracing: Update trace event to track if page reclaim IO is for anon or file pages Mel Gorman
2010-07-19 13:11   ` Mel Gorman
2010-07-19 13:24   ` Rik van Riel
2010-07-19 13:24     ` Rik van Riel
2010-07-19 14:15   ` Christoph Hellwig
2010-07-19 14:15     ` Christoph Hellwig
2010-07-19 14:24     ` Mel Gorman
2010-07-19 14:24       ` Mel Gorman
2010-07-19 14:26       ` Christoph Hellwig
2010-07-19 14:26         ` Christoph Hellwig
2010-07-19 13:11 ` [PATCH 3/8] vmscan: tracing: Update post-processing script to distinguish between anon and file IO from page reclaim Mel Gorman
2010-07-19 13:11   ` Mel Gorman
2010-07-19 13:32   ` Rik van Riel
2010-07-19 13:32     ` Rik van Riel
2010-07-19 13:11 ` [PATCH 4/8] vmscan: Do not writeback filesystem pages in direct reclaim Mel Gorman
2010-07-19 13:11   ` Mel Gorman
2010-07-19 14:19   ` Christoph Hellwig
2010-07-19 14:19     ` Christoph Hellwig
2010-07-19 14:26     ` Mel Gorman
2010-07-19 14:26       ` Mel Gorman
2010-07-19 18:25   ` Rik van Riel
2010-07-19 18:25     ` Rik van Riel
2010-07-19 22:14   ` Johannes Weiner
2010-07-19 22:14     ` Johannes Weiner
2010-07-20 13:45     ` Mel Gorman
2010-07-20 13:45       ` Mel Gorman
2010-07-20 22:02       ` Johannes Weiner
2010-07-20 22:02         ` Johannes Weiner
2010-07-21 11:36         ` Johannes Weiner
2010-07-21 11:36           ` Johannes Weiner
2010-07-21 11:52         ` Mel Gorman
2010-07-21 11:52           ` Mel Gorman
2010-07-21 12:01           ` KAMEZAWA Hiroyuki
2010-07-21 12:01             ` KAMEZAWA Hiroyuki
2010-07-21 14:27             ` Mel Gorman
2010-07-21 14:27               ` Mel Gorman
2010-07-21 23:57               ` KAMEZAWA Hiroyuki
2010-07-21 23:57                 ` KAMEZAWA Hiroyuki
2010-07-22  9:19                 ` Mel Gorman
2010-07-22  9:19                   ` Mel Gorman
2010-07-22  9:22                   ` KAMEZAWA Hiroyuki
2010-07-22  9:22                     ` KAMEZAWA Hiroyuki
2010-07-21 13:04           ` Johannes Weiner
2010-07-21 13:04             ` Johannes Weiner
2010-07-21 13:38             ` Mel Gorman
2010-07-21 13:38               ` Mel Gorman
2010-07-21 14:28               ` Johannes Weiner
2010-07-21 14:28                 ` Johannes Weiner
2010-07-21 14:31                 ` Mel Gorman
2010-07-21 14:31                   ` Mel Gorman
2010-07-21 14:39                   ` Johannes Weiner
2010-07-21 14:39                     ` Johannes Weiner
2010-07-21 15:06                     ` Mel Gorman
2010-07-21 15:06                       ` Mel Gorman
2010-07-26  8:29               ` Wu Fengguang
2010-07-26  8:29                 ` Wu Fengguang
2010-07-26  9:12                 ` Mel Gorman
2010-07-26  9:12                   ` Mel Gorman
2010-07-26 11:19                   ` Wu Fengguang
2010-07-26 11:19                     ` Wu Fengguang
2010-07-26 12:53                     ` Mel Gorman
2010-07-26 12:53                       ` Mel Gorman
2010-07-26 13:03                       ` Wu Fengguang
2010-07-26 13:03                         ` Wu Fengguang
2010-07-19 13:11 ` [PATCH 5/8] fs,btrfs: Allow kswapd to writeback pages Mel Gorman
2010-07-19 13:11   ` Mel Gorman
2010-07-19 18:27   ` Rik van Riel
2010-07-19 18:27     ` Rik van Riel
2010-07-19 13:11 ` [PATCH 6/8] fs,xfs: " Mel Gorman
2010-07-19 13:11   ` Mel Gorman
2010-07-19 14:20   ` Christoph Hellwig
2010-07-19 14:20     ` Christoph Hellwig
2010-07-19 14:43     ` Mel Gorman
2010-07-19 14:43       ` Mel Gorman
2010-07-19 13:11 ` [PATCH 7/8] writeback: sync old inodes first in background writeback Mel Gorman
2010-07-19 13:11   ` Mel Gorman
2010-07-19 14:21   ` Christoph Hellwig
2010-07-19 14:21     ` Christoph Hellwig
2010-07-19 14:40     ` Mel Gorman
2010-07-19 14:40       ` Mel Gorman
2010-07-19 14:48       ` Christoph Hellwig
2010-07-19 14:48         ` Christoph Hellwig
2010-07-22  8:52       ` Wu Fengguang [this message]
2010-07-22  8:52         ` Wu Fengguang
2010-07-22  9:02         ` Wu Fengguang
2010-07-22  9:02           ` Wu Fengguang
2010-07-22  9:21         ` Wu Fengguang
2010-07-22  9:21           ` Wu Fengguang
2010-07-22 10:48           ` Mel Gorman
2010-07-22 10:48             ` Mel Gorman
2010-07-23  9:45             ` Wu Fengguang
2010-07-23  9:45               ` Wu Fengguang
2010-07-23 10:57               ` Mel Gorman
2010-07-23 10:57                 ` Mel Gorman
2010-07-23 11:49                 ` Wu Fengguang
2010-07-23 11:49                   ` Wu Fengguang
2010-07-23 12:20                   ` Wu Fengguang
2010-07-23 12:20                     ` Wu Fengguang
2010-07-25 10:43                 ` KOSAKI Motohiro
2010-07-25 10:43                   ` KOSAKI Motohiro
2010-07-25 12:03                   ` Minchan Kim
2010-07-25 12:03                     ` Minchan Kim
2010-07-26  3:27                     ` Wu Fengguang
2010-07-26  3:27                       ` Wu Fengguang
2010-07-26  4:11                       ` Minchan Kim
2010-07-26  4:11                         ` Minchan Kim
2010-07-26  4:37                         ` Wu Fengguang
2010-07-26  4:37                           ` Wu Fengguang
2010-07-26  4:37                           ` Wu Fengguang
2010-07-26 16:30                           ` Minchan Kim
2010-07-26 16:30                             ` Minchan Kim
2010-07-26 16:30                             ` Minchan Kim
2010-07-26 22:48                             ` Wu Fengguang
2010-07-26 22:48                               ` Wu Fengguang
2010-07-26 22:48                               ` Wu Fengguang
2010-07-26  3:08                   ` Wu Fengguang
2010-07-26  3:08                     ` Wu Fengguang
2010-07-26  3:11                     ` Rik van Riel
2010-07-26  3:11                       ` Rik van Riel
2010-07-26  3:17                       ` Wu Fengguang
2010-07-26  3:17                         ` Wu Fengguang
2010-07-22 15:34           ` Minchan Kim
2010-07-22 15:34             ` Minchan Kim
2010-07-23 11:59             ` Wu Fengguang
2010-07-23 11:59               ` Wu Fengguang
2010-07-22  9:42         ` Mel Gorman
2010-07-22  9:42           ` Mel Gorman
2010-07-23  8:33           ` Wu Fengguang
2010-07-23  8:33             ` Wu Fengguang
2010-07-22  1:13     ` Wu Fengguang
2010-07-22  1:13       ` Wu Fengguang
2010-07-19 18:43   ` Rik van Riel
2010-07-19 18:43     ` Rik van Riel
2010-07-19 13:11 ` [PATCH 8/8] vmscan: Kick flusher threads to clean pages when reclaim is encountering dirty pages Mel Gorman
2010-07-19 13:11   ` Mel Gorman
2010-07-19 14:23   ` Christoph Hellwig
2010-07-19 14:23     ` Christoph Hellwig
2010-07-19 14:37     ` Mel Gorman
2010-07-19 14:37       ` Mel Gorman
2010-07-19 22:48       ` Johannes Weiner
2010-07-19 22:48         ` Johannes Weiner
2010-07-20 14:10         ` Mel Gorman
2010-07-20 14:10           ` Mel Gorman
2010-07-20 22:05           ` Johannes Weiner
2010-07-20 22:05             ` Johannes Weiner
2010-07-19 18:59   ` Rik van Riel
2010-07-19 18:59     ` Rik van Riel
2010-07-19 22:26   ` Johannes Weiner
2010-07-19 22:26     ` Johannes Weiner
2010-07-26  7:28   ` Wu Fengguang
2010-07-26  7:28     ` Wu Fengguang
2010-07-26  9:26     ` Mel Gorman
2010-07-26  9:26       ` Mel Gorman
2010-07-26 11:27       ` Wu Fengguang
2010-07-26 11:27         ` Wu Fengguang
2010-07-26 12:57         ` Mel Gorman
2010-07-26 12:57           ` Mel Gorman
2010-07-26 13:10           ` Wu Fengguang
2010-07-26 13:10             ` Wu Fengguang
2010-07-27 13:35             ` Mel Gorman
2010-07-27 13:35               ` Mel Gorman
2010-07-27 14:24               ` Wu Fengguang
2010-07-27 14:24                 ` Wu Fengguang
2010-07-27 14:34                 ` Wu Fengguang
2010-07-27 14:34                   ` Wu Fengguang
2010-07-27 14:40                   ` Mel Gorman
2010-07-27 14:40                     ` Mel Gorman
2010-07-27 14:55                     ` Wu Fengguang
2010-07-27 14:55                       ` Wu Fengguang
2010-07-27 14:38                 ` Mel Gorman
2010-07-27 14:38                   ` Mel Gorman
2010-07-27 15:21                   ` Wu Fengguang
2010-07-27 15:21                     ` Wu Fengguang

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20100722085210.GA26714@localhost \
    --to=fengguang.wu@intel.com \
    --cc=aarcange@redhat.com \
    --cc=akpm@linux-foundation.org \
    --cc=chris.mason@oracle.com \
    --cc=david@fromorbit.com \
    --cc=hannes@cmpxchg.org \
    --cc=hch@infradead.org \
    --cc=kamezawa.hiroyu@jp.fujitsu.com \
    --cc=kosaki.motohiro@jp.fujitsu.com \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mel@csn.ul.ie \
    --cc=minchan.kim@gmail.com \
    --cc=npiggin@suse.de \
    --cc=riel@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.