All of lore.kernel.org
 help / color / mirror / Atom feed
From: Wu Fengguang <fengguang.wu@intel.com>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: Jan Kara <jack@suse.cz>,
	Trond Myklebust <Trond.Myklebust@netapp.com>,
	Wu Fengguang <fengguang.wu@intel.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Dave Chinner <david@fromorbit.com>
Cc: "Theodore Ts'o" <tytso@mit.edu>
Cc: Chris Mason <chris.mason@oracle.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Mel Gorman <mel@csn.ul.ie>
Cc: Rik van Riel <riel@redhat.com>
Cc: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Cc: Greg Thelen <gthelen@google.com>
Cc: Minchan Kim <minchan.kim@gmail.com>
Cc: linux-mm <linux-mm@kvack.org>
Cc: <linux-fsdevel@vger.kernel.org>
Cc: LKML <linux-kernel@vger.kernel.org>
Subject: [PATCH 45/47] nfs: adapt congestion threshold to dirty threshold
Date: Mon, 13 Dec 2010 14:43:34 +0800	[thread overview]
Message-ID: <20101213064842.559030592@intel.com> (raw)
In-Reply-To: 20101213064249.648862451@intel.com

[-- Attachment #1: nfs-congestion-thresh.patch --]
[-- Type: text/plain, Size: 1577 bytes --]

nfs_congestion_kb is to control the max allowed writeback and in-commit
pages. It's not reasonable for them to outnumber dirty and to-commit
pages. So each of them should not take more than 1/4 dirty threshold.

Considering that nfs_init_writepagecache() is called on fresh boot,
at the time dirty_thresh is much higher than the real dirty limit after
lots of user space memory consumptions, use 1/8 instead.

We might update nfs_congestion_kb when global dirty limit is changed
at runtime, but whatever, do it simple first.

CC: Trond Myklebust <Trond.Myklebust@netapp.com>
Signed-off-by: Wu Fengguang <fengguang.wu@intel.com>
---
 fs/nfs/write.c |   13 +++++++++++++
 1 file changed, 13 insertions(+)

--- linux-next.orig/fs/nfs/write.c	2010-12-08 22:44:37.000000000 +0800
+++ linux-next/fs/nfs/write.c	2010-12-08 22:44:37.000000000 +0800
@@ -1698,6 +1698,9 @@ out:
 
 int __init nfs_init_writepagecache(void)
 {
+	unsigned long background_thresh;
+	unsigned long dirty_thresh;
+
 	nfs_wdata_cachep = kmem_cache_create("nfs_write_data",
 					     sizeof(struct nfs_write_data),
 					     0, SLAB_HWCACHE_ALIGN,
@@ -1735,6 +1738,16 @@ int __init nfs_init_writepagecache(void)
 	if (nfs_congestion_kb > 256*1024)
 		nfs_congestion_kb = 256*1024;
 
+	/*
+	 * Limit to 1/8 dirty threshold, so that writeback+in_commit pages
+	 * won't overnumber dirty+to_commit pages.
+	 */
+	global_dirty_limits(&background_thresh, &dirty_thresh);
+	dirty_thresh <<= PAGE_SHIFT - 10;
+
+	if (nfs_congestion_kb > dirty_thresh / 8)
+		nfs_congestion_kb = dirty_thresh / 8;
+
 	return 0;
 }
 



WARNING: multiple messages have this Message-ID (diff)
From: Wu Fengguang <fengguang.wu@intel.com>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: Jan Kara <jack@suse.cz>,
	Trond Myklebust <Trond.Myklebust@netapp.com>,
	Wu Fengguang <fengguang.wu@intel.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Dave Chinner <david@fromorbit.com>
Cc: Theodore Ts'o <tytso@mit.edu>
Cc: Chris Mason <chris.mason@oracle.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Mel Gorman <mel@csn.ul.ie>
Cc: Rik van Riel <riel@redhat.com>
Cc: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Cc: Greg Thelen <gthelen@google.com>
Cc: Minchan Kim <minchan.kim@gmail.com>
Cc: linux-mm <linux-mm@kvack.org>
Cc: <linux-fsdevel@vger.kernel.org>
Cc: LKML <linux-kernel@vger.kernel.org>
Subject: [PATCH 45/47] nfs: adapt congestion threshold to dirty threshold
Date: Mon, 13 Dec 2010 14:43:34 +0800	[thread overview]
Message-ID: <20101213064842.559030592@intel.com> (raw)
In-Reply-To: 20101213064249.648862451@intel.com

[-- Attachment #1: nfs-congestion-thresh.patch --]
[-- Type: text/plain, Size: 1873 bytes --]

nfs_congestion_kb is to control the max allowed writeback and in-commit
pages. It's not reasonable for them to outnumber dirty and to-commit
pages. So each of them should not take more than 1/4 dirty threshold.

Considering that nfs_init_writepagecache() is called on fresh boot,
at the time dirty_thresh is much higher than the real dirty limit after
lots of user space memory consumptions, use 1/8 instead.

We might update nfs_congestion_kb when global dirty limit is changed
at runtime, but whatever, do it simple first.

CC: Trond Myklebust <Trond.Myklebust@netapp.com>
Signed-off-by: Wu Fengguang <fengguang.wu@intel.com>
---
 fs/nfs/write.c |   13 +++++++++++++
 1 file changed, 13 insertions(+)

--- linux-next.orig/fs/nfs/write.c	2010-12-08 22:44:37.000000000 +0800
+++ linux-next/fs/nfs/write.c	2010-12-08 22:44:37.000000000 +0800
@@ -1698,6 +1698,9 @@ out:
 
 int __init nfs_init_writepagecache(void)
 {
+	unsigned long background_thresh;
+	unsigned long dirty_thresh;
+
 	nfs_wdata_cachep = kmem_cache_create("nfs_write_data",
 					     sizeof(struct nfs_write_data),
 					     0, SLAB_HWCACHE_ALIGN,
@@ -1735,6 +1738,16 @@ int __init nfs_init_writepagecache(void)
 	if (nfs_congestion_kb > 256*1024)
 		nfs_congestion_kb = 256*1024;
 
+	/*
+	 * Limit to 1/8 dirty threshold, so that writeback+in_commit pages
+	 * won't overnumber dirty+to_commit pages.
+	 */
+	global_dirty_limits(&background_thresh, &dirty_thresh);
+	dirty_thresh <<= PAGE_SHIFT - 10;
+
+	if (nfs_congestion_kb > dirty_thresh / 8)
+		nfs_congestion_kb = dirty_thresh / 8;
+
 	return 0;
 }
 


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom policy in Canada: sign http://dissolvethecrtc.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

WARNING: multiple messages have this Message-ID (diff)
From: Wu Fengguang <fengguang.wu@intel.com>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: Jan Kara <jack@suse.cz>,
	Trond Myklebust <Trond.Myklebust@netapp.com>,
	Wu Fengguang <fengguang.wu@intel.com>,
	Christoph Hellwig <hch@lst.de>,
	Dave Chinner <david@fromorbit.com>, Theodore Ts'o <tytso@mit.edu>,
	Chris Mason <chris.mason@oracle.com>,
	Peter Zijlstra <a.p.zijlstra@chello.nl>,
	Mel Gorman <mel@csn.ul.ie>, Rik van Riel <riel@redhat.com>,
	KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>,
	Greg Thelen <gthelen@google.com>,
	Minchan Kim <minchan.kim@gmail.com>,
	linux-mm <linux-mm@kvack.org>,
	linux-fsdevel@vger.kernel.org,
	LKML <linux-kernel@vger.kernel.org>
Subject: [PATCH 45/47] nfs: adapt congestion threshold to dirty threshold
Date: Mon, 13 Dec 2010 14:43:34 +0800	[thread overview]
Message-ID: <20101213064842.559030592@intel.com> (raw)
In-Reply-To: 20101213064249.648862451@intel.com

[-- Attachment #1: nfs-congestion-thresh.patch --]
[-- Type: text/plain, Size: 1873 bytes --]

nfs_congestion_kb is to control the max allowed writeback and in-commit
pages. It's not reasonable for them to outnumber dirty and to-commit
pages. So each of them should not take more than 1/4 dirty threshold.

Considering that nfs_init_writepagecache() is called on fresh boot,
at the time dirty_thresh is much higher than the real dirty limit after
lots of user space memory consumptions, use 1/8 instead.

We might update nfs_congestion_kb when global dirty limit is changed
at runtime, but whatever, do it simple first.

CC: Trond Myklebust <Trond.Myklebust@netapp.com>
Signed-off-by: Wu Fengguang <fengguang.wu@intel.com>
---
 fs/nfs/write.c |   13 +++++++++++++
 1 file changed, 13 insertions(+)

--- linux-next.orig/fs/nfs/write.c	2010-12-08 22:44:37.000000000 +0800
+++ linux-next/fs/nfs/write.c	2010-12-08 22:44:37.000000000 +0800
@@ -1698,6 +1698,9 @@ out:
 
 int __init nfs_init_writepagecache(void)
 {
+	unsigned long background_thresh;
+	unsigned long dirty_thresh;
+
 	nfs_wdata_cachep = kmem_cache_create("nfs_write_data",
 					     sizeof(struct nfs_write_data),
 					     0, SLAB_HWCACHE_ALIGN,
@@ -1735,6 +1738,16 @@ int __init nfs_init_writepagecache(void)
 	if (nfs_congestion_kb > 256*1024)
 		nfs_congestion_kb = 256*1024;
 
+	/*
+	 * Limit to 1/8 dirty threshold, so that writeback+in_commit pages
+	 * won't overnumber dirty+to_commit pages.
+	 */
+	global_dirty_limits(&background_thresh, &dirty_thresh);
+	dirty_thresh <<= PAGE_SHIFT - 10;
+
+	if (nfs_congestion_kb > dirty_thresh / 8)
+		nfs_congestion_kb = dirty_thresh / 8;
+
 	return 0;
 }
 


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom policy in Canada: sign http://dissolvethecrtc.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  parent reply	other threads:[~2010-12-13  6:53 UTC|newest]

Thread overview: 143+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2010-12-13  6:42 [PATCH 00/47] IO-less dirty throttling v3 Wu Fengguang
2010-12-13  6:42 ` Wu Fengguang
2010-12-13  6:42 ` Wu Fengguang
2010-12-13  6:42 ` [PATCH 01/47] writeback: enabling gate limit for light dirtied bdi Wu Fengguang
2010-12-13  6:42   ` Wu Fengguang
2010-12-13  6:42   ` Wu Fengguang
2010-12-13  6:42 ` [PATCH 02/47] writeback: safety margin for bdi stat error Wu Fengguang
2010-12-13  6:42   ` Wu Fengguang
2010-12-13  6:42   ` Wu Fengguang
2010-12-13  6:42 ` [PATCH 03/47] writeback: IO-less balance_dirty_pages() Wu Fengguang
2010-12-13  6:42   ` Wu Fengguang
2010-12-13  6:42   ` Wu Fengguang
2010-12-13  6:42 ` [PATCH 04/47] writeback: consolidate variable names in balance_dirty_pages() Wu Fengguang
2010-12-13  6:42   ` Wu Fengguang
2010-12-13  6:42   ` Wu Fengguang
2010-12-13  6:42 ` [PATCH 05/47] writeback: per-task rate limit on balance_dirty_pages() Wu Fengguang
2010-12-13  6:42   ` Wu Fengguang
2010-12-13  6:42   ` Wu Fengguang
2010-12-13  6:42 ` [PATCH 06/47] writeback: prevent duplicate balance_dirty_pages_ratelimited() calls Wu Fengguang
2010-12-13  6:42   ` Wu Fengguang
2010-12-13  6:42   ` Wu Fengguang
2010-12-13  6:42 ` [PATCH 07/47] writeback: account per-bdi accumulated written pages Wu Fengguang
2010-12-13  6:42   ` Wu Fengguang
2010-12-13  6:42   ` Wu Fengguang
2010-12-13  6:42 ` [PATCH 08/47] writeback: bdi write bandwidth estimation Wu Fengguang
2010-12-13  6:42   ` Wu Fengguang
2010-12-13  6:42   ` Wu Fengguang
2010-12-13  6:42 ` [PATCH 09/47] writeback: show bdi write bandwidth in debugfs Wu Fengguang
2010-12-13  6:42   ` Wu Fengguang
2010-12-13  6:42   ` Wu Fengguang
2010-12-13  6:42 ` [PATCH 10/47] writeback: quit throttling when bdi dirty pages dropped low Wu Fengguang
2010-12-13  6:42   ` Wu Fengguang
2010-12-13  6:42   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 11/47] writeback: reduce per-bdi dirty threshold ramp up time Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 12/47] writeback: make reasonable gap between the dirty/background thresholds Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 13/47] writeback: scale down max throttle bandwidth on concurrent dirtiers Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 14/47] writeback: add trace event for balance_dirty_pages() Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 15/47] writeback: make nr_to_write a per-file limit Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 16/47] writeback: make-nr_to_write-a-per-file-limit fix Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 17/47] writeback: do uninterruptible sleep in balance_dirty_pages() Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 18/47] writeback: move BDI_WRITTEN accounting into __bdi_writeout_inc() Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 19/47] writeback: fix increasement of nr_dirtied_pause Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 20/47] writeback: use do_div in bw calculation Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 21/47] writeback: prevent divide error on tiny HZ Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 22/47] writeback: prevent bandwidth calculation overflow Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 23/47] writeback: spinlock protected bdi bandwidth update Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 24/47] writeback: increase pause time on concurrent dirtiers Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 25/47] writeback: make it easier to break from a dirty exceeded bdi Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 26/47] writeback: start background writeback earlier Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 27/47] writeback: user space think time compensation Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 28/47] writeback: bdi base throttle bandwidth Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 29/47] writeback: smoothed bdi dirty pages Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 30/47] writeback: adapt max balance pause time to memory size Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 31/47] writeback: increase min pause time on concurrent dirtiers Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 32/47] writeback: extend balance_dirty_pages() trace event Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 33/47] writeback: trace global dirty page states Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 34/47] writeback: trace writeback_single_inode() Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 35/47] writeback: scale IO chunk size up to device bandwidth Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 36/47] btrfs: dont call balance_dirty_pages_ratelimited() on already dirty pages Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 37/47] btrfs: lower the dirty balacing rate limit Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 38/47] btrfs: wait on too many nr_async_bios Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 39/47] nfs: livelock prevention is now done in VFS Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 40/47] nfs: writeback pages wait queue Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 41/47] nfs: in-commit pages accounting and " Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 42/47] nfs: heuristics to avoid commit Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 43/47] nfs: dont change wbc->nr_to_write in write_inode() Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 44/47] nfs: limit the range of commits Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` Wu Fengguang [this message]
2010-12-13  6:43   ` [PATCH 45/47] nfs: adapt congestion threshold to dirty threshold Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 46/47] nfs: trace nfs_commit_unstable_pages() Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43 ` [PATCH 47/47] nfs: trace nfs_commit_release() Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13  6:43   ` Wu Fengguang
2010-12-13 11:27 ` [PATCH 00/47] IO-less dirty throttling v3 Peter Zijlstra
2010-12-13 11:27   ` Peter Zijlstra
2010-12-13 11:49   ` Wu Fengguang
2010-12-13 11:49     ` Wu Fengguang
2010-12-13 12:38     ` Peter Zijlstra
2010-12-13 12:38       ` Peter Zijlstra

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20101213064842.559030592@intel.com \
    --to=fengguang.wu@intel.com \
    --cc=Trond.Myklebust@netapp.com \
    --cc=akpm@linux-foundation.org \
    --cc=jack@suse.cz \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.