From: Wu Fengguang <fengguang.wu@intel.com>
To: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: "linux-fsdevel@vger.kernel.org" <linux-fsdevel@vger.kernel.org>,
Andrew Morton <akpm@linux-foundation.org>,
Jan Kara <jack@suse.cz>, Christoph Hellwig <hch@lst.de>,
Dave Chinner <david@fromorbit.com>,
Greg Thelen <gthelen@google.com>,
Minchan Kim <minchan.kim@gmail.com>,
Vivek Goyal <vgoyal@redhat.com>,
Andrea Righi <arighi@develer.com>, linux-mm <linux-mm@kvack.org>,
LKML <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH 10/18] writeback: dirty position control - bdi reserve area
Date: Sun, 18 Sep 2011 22:17:05 +0800 [thread overview]
Message-ID: <20110918141705.GB15366@localhost> (raw)
In-Reply-To: <1315822779.26517.23.camel@twins>
[-- Attachment #1: Type: text/plain, Size: 6558 bytes --]
On Mon, Sep 12, 2011 at 06:19:38PM +0800, Peter Zijlstra wrote:
> On Wed, 2011-09-07 at 20:31 +0800, Wu Fengguang wrote:
> > > > + x_intercept = min(write_bw, freerun);
> > > > + if (bdi_dirty < x_intercept) {
> > >
> > > So the point of the freerun point is that we never throttle before it,
> > > so basically all the below shouldn't be needed at all, right?
> >
> > Yes!
> >
> > > > + if (bdi_dirty > x_intercept / 8) {
> > > > + pos_ratio *= x_intercept;
> > > > + do_div(pos_ratio, bdi_dirty);
> > > > + } else
> > > > + pos_ratio *= 8;
> > > > + }
> > > > +
> > > > return pos_ratio;
> > > > }
>
> Does that mean we can remove this whole block?
Right, if the bdi freerun concept is proved to work fine.
Unfortunately I find it mostly yields lower performance than bdi
reserve area. Patch is attached. If you would like me try other
patches, I can easily kick off new tests and redo the comparison.
Here is the nr_written numbers over various JBOD test cases,
the larger, the better:
bdi-reserve bdi-freerun diff case
---------------------------------------------------------------------------------------
38375271 31553807 -17.8% JBOD-10HDD-6G/xfs-100dd-1M-16p-5895M-20
30478879 28631491 -6.1% JBOD-10HDD-6G/xfs-10dd-1M-16p-5895M-20
29735407 28871956 -2.9% JBOD-10HDD-6G/xfs-1dd-1M-16p-5895M-20
30850350 28344165 -8.1% JBOD-10HDD-6G/xfs-2dd-1M-16p-5895M-20
17706200 16174684 -8.6% JBOD-10HDD-thresh=100M/xfs-100dd-1M-16p-5895M-100M
23374918 14376942 -38.5% JBOD-10HDD-thresh=100M/xfs-10dd-1M-16p-5895M-100M
20659278 19640375 -4.9% JBOD-10HDD-thresh=100M/xfs-1dd-1M-16p-5895M-100M
22517497 14552321 -35.4% JBOD-10HDD-thresh=100M/xfs-2dd-1M-16p-5895M-100M
68287850 61078553 -10.6% JBOD-10HDD-thresh=2G/xfs-100dd-1M-16p-5895M-2048M
33835247 32018425 -5.4% JBOD-10HDD-thresh=2G/xfs-10dd-1M-16p-5895M-2048M
30187817 29942083 -0.8% JBOD-10HDD-thresh=2G/xfs-1dd-1M-16p-5895M-2048M
30563144 30204022 -1.2% JBOD-10HDD-thresh=2G/xfs-2dd-1M-16p-5895M-2048M
34476862 34645398 +0.5% JBOD-10HDD-thresh=4G/xfs-10dd-1M-16p-5895M-4096M
30326479 30097263 -0.8% JBOD-10HDD-thresh=4G/xfs-1dd-1M-16p-5895M-4096M
30446767 30339683 -0.4% JBOD-10HDD-thresh=4G/xfs-2dd-1M-16p-5895M-4096M
40793956 45936678 +12.6% JBOD-10HDD-thresh=800M/xfs-100dd-1M-16p-5895M-800M
27481305 24867282 -9.5% JBOD-10HDD-thresh=800M/xfs-10dd-1M-16p-5895M-800M
25651257 22507406 -12.3% JBOD-10HDD-thresh=800M/xfs-1dd-1M-16p-5895M-800M
19849350 21298787 +7.3% JBOD-10HDD-thresh=800M/xfs-2dd-1M-16p-5895M-800M
raw data by "grep":
JBOD-10HDD-6G/xfs-100dd-1M-16p-5895M-20:10-3.1.0-rc4+/vmstat-end:nr_written 38375271
JBOD-10HDD-6G/xfs-10dd-1M-16p-5895M-20:10-3.1.0-rc4+/vmstat-end:nr_written 30478879
JBOD-10HDD-6G/xfs-1dd-1M-16p-5895M-20:10-3.1.0-rc4+/vmstat-end:nr_written 29735407
JBOD-10HDD-6G/xfs-2dd-1M-16p-5895M-20:10-3.1.0-rc4+/vmstat-end:nr_written 30850350
JBOD-10HDD-thresh=100M/xfs-100dd-1M-16p-5895M-100M:10-3.1.0-rc4+/vmstat-end:nr_written 17706200
JBOD-10HDD-thresh=100M/xfs-10dd-1M-16p-5895M-100M:10-3.1.0-rc4+/vmstat-end:nr_written 23374918
JBOD-10HDD-thresh=100M/xfs-1dd-1M-16p-5895M-100M:10-3.1.0-rc4+/vmstat-end:nr_written 20659278
JBOD-10HDD-thresh=100M/xfs-2dd-1M-16p-5895M-100M:10-3.1.0-rc4+/vmstat-end:nr_written 22517497
JBOD-10HDD-thresh=2G/xfs-100dd-1M-16p-5895M-2048M:10-3.1.0-rc4+/vmstat-end:nr_written 68287850
JBOD-10HDD-thresh=2G/xfs-10dd-1M-16p-5895M-2048M:10-3.1.0-rc4+/vmstat-end:nr_written 33835247
JBOD-10HDD-thresh=2G/xfs-1dd-1M-16p-5895M-2048M:10-3.1.0-rc4+/vmstat-end:nr_written 30187817
JBOD-10HDD-thresh=2G/xfs-2dd-1M-16p-5895M-2048M:10-3.1.0-rc4+/vmstat-end:nr_written 30563144
JBOD-10HDD-thresh=4G/xfs-10dd-1M-16p-5895M-4096M:10-3.1.0-rc4+/vmstat-end:nr_written 34476862
JBOD-10HDD-thresh=4G/xfs-1dd-1M-16p-5895M-4096M:10-3.1.0-rc4+/vmstat-end:nr_written 30326479
JBOD-10HDD-thresh=4G/xfs-2dd-1M-16p-5895M-4096M:10-3.1.0-rc4+/vmstat-end:nr_written 30446767
JBOD-10HDD-thresh=800M/xfs-100dd-1M-16p-5895M-800M:10-3.1.0-rc4+/vmstat-end:nr_written 40793956
JBOD-10HDD-thresh=800M/xfs-10dd-1M-16p-5895M-800M:10-3.1.0-rc4+/vmstat-end:nr_written 27481305
JBOD-10HDD-thresh=800M/xfs-1dd-1M-16p-5895M-800M:10-3.1.0-rc4+/vmstat-end:nr_written 25651257
JBOD-10HDD-thresh=800M/xfs-2dd-1M-16p-5895M-800M:10-3.1.0-rc4+/vmstat-end:nr_written 19849350
JBOD-10HDD-6G/xfs-100dd-1M-16p-5895M-20:10-3.1.0-rc4-bdi-freerun+/vmstat-end:nr_written 31553807
JBOD-10HDD-6G/xfs-10dd-1M-16p-5895M-20:10-3.1.0-rc4-bdi-freerun+/vmstat-end:nr_written 28631491
JBOD-10HDD-6G/xfs-1dd-1M-16p-5895M-20:10-3.1.0-rc4-bdi-freerun+/vmstat-end:nr_written 28871956
JBOD-10HDD-6G/xfs-2dd-1M-16p-5895M-20:10-3.1.0-rc4-bdi-freerun+/vmstat-end:nr_written 28344165
JBOD-10HDD-thresh=100M/xfs-100dd-1M-16p-5895M-100M:10-3.1.0-rc4-bdi-freerun+/vmstat-end:nr_written 16174684
JBOD-10HDD-thresh=100M/xfs-10dd-1M-16p-5895M-100M:10-3.1.0-rc4-bdi-freerun+/vmstat-end:nr_written 14376942
JBOD-10HDD-thresh=100M/xfs-1dd-1M-16p-5895M-100M:10-3.1.0-rc4-bdi-freerun+/vmstat-end:nr_written 19640375
JBOD-10HDD-thresh=100M/xfs-2dd-1M-16p-5895M-100M:10-3.1.0-rc4-bdi-freerun+/vmstat-end:nr_written 14552321
JBOD-10HDD-thresh=2G/xfs-100dd-1M-16p-5895M-2048M:10-3.1.0-rc4-bdi-freerun+/vmstat-end:nr_written 61078553
JBOD-10HDD-thresh=2G/xfs-10dd-1M-16p-5895M-2048M:10-3.1.0-rc4-bdi-freerun+/vmstat-end:nr_written 32018425
JBOD-10HDD-thresh=2G/xfs-1dd-1M-16p-5895M-2048M:10-3.1.0-rc4-bdi-freerun+/vmstat-end:nr_written 29942083
JBOD-10HDD-thresh=2G/xfs-2dd-1M-16p-5895M-2048M:10-3.1.0-rc4-bdi-freerun+/vmstat-end:nr_written 30204022
JBOD-10HDD-thresh=4G/xfs-10dd-1M-16p-5895M-4096M:10-3.1.0-rc4-bdi-freerun+/vmstat-end:nr_written 34645398
JBOD-10HDD-thresh=4G/xfs-1dd-1M-16p-5895M-4096M:10-3.1.0-rc4-bdi-freerun+/vmstat-end:nr_written 30097263
JBOD-10HDD-thresh=4G/xfs-2dd-1M-16p-5895M-4096M:10-3.1.0-rc4-bdi-freerun+/vmstat-end:nr_written 30339683
JBOD-10HDD-thresh=800M/xfs-100dd-1M-16p-5895M-800M:10-3.1.0-rc4-bdi-freerun+/vmstat-end:nr_written 45936678
JBOD-10HDD-thresh=800M/xfs-10dd-1M-16p-5895M-800M:10-3.1.0-rc4-bdi-freerun+/vmstat-end:nr_written 24867282
JBOD-10HDD-thresh=800M/xfs-1dd-1M-16p-5895M-800M:10-3.1.0-rc4-bdi-freerun+/vmstat-end:nr_written 22507406
JBOD-10HDD-thresh=800M/xfs-2dd-1M-16p-5895M-800M:10-3.1.0-rc4-bdi-freerun+/vmstat-end:nr_written 21298787
[-- Attachment #2: bdi-freerun --]
[-- Type: text/plain, Size: 1488 bytes --]
Subject:
Date: Wed Sep 14 22:57:43 CST 2011
Signed-off-by: Wu Fengguang <fengguang.wu@intel.com>
---
mm/page-writeback.c | 26 ++++++++------------------
1 file changed, 8 insertions(+), 18 deletions(-)
--- linux-next.orig/mm/page-writeback.c 2011-09-14 22:50:33.000000000 +0800
+++ linux-next/mm/page-writeback.c 2011-09-14 22:58:15.000000000 +0800
@@ -614,22 +614,6 @@ static unsigned long bdi_position_ratio(
} else
pos_ratio /= 4;
- /*
- * bdi reserve area, safeguard against dirty pool underrun and disk idle
- *
- * It may push the desired control point of global dirty pages higher
- * than setpoint. It's not necessary in single-bdi case because a
- * minimal pool of @freerun dirty pages will already be guaranteed.
- */
- x_intercept = min(write_bw, freerun);
- if (bdi_dirty < x_intercept) {
- if (bdi_dirty > x_intercept / 8) {
- pos_ratio *= x_intercept;
- do_div(pos_ratio, bdi_dirty);
- } else
- pos_ratio *= 8;
- }
-
return pos_ratio;
}
@@ -1089,8 +1073,14 @@ static void balance_dirty_pages(struct a
nr_dirty, bdi_thresh, bdi_dirty,
start_time);
- if (unlikely(!dirty_exceeded && bdi_async_underrun(bdi)))
- break;
+ freerun = min(bdi->avg_write_bandwidth + MIN_WRITEBACK_PAGES,
+ global_dirty_limit - nr_dirty) / 8;
+ if (!dirty_exceeded) {
+ if (unlikely(bdi_dirty < freerun))
+ break;
+ if (unlikely(bdi_async_underrun(bdi)))
+ break;
+ }
max_pause = bdi_max_pause(bdi, bdi_dirty);
next prev parent reply other threads:[~2011-09-18 14:17 UTC|newest]
Thread overview: 160+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-09-04 1:53 [PATCH 00/18] IO-less dirty throttling v11 Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-04 1:53 ` [PATCH 01/18] writeback: account per-bdi accumulated dirtied pages Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-04 1:53 ` [PATCH 02/18] writeback: dirty position control Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-05 15:02 ` Peter Zijlstra
2011-09-05 15:02 ` Peter Zijlstra
2011-09-06 2:10 ` Wu Fengguang
2011-09-06 2:10 ` Wu Fengguang
2011-09-05 15:05 ` Peter Zijlstra
2011-09-05 15:05 ` Peter Zijlstra
2011-09-06 2:43 ` Wu Fengguang
2011-09-06 2:43 ` Wu Fengguang
2011-09-06 18:20 ` Vivek Goyal
2011-09-06 18:20 ` Vivek Goyal
2011-09-08 2:53 ` Wu Fengguang
2011-09-08 2:53 ` Wu Fengguang
2011-11-12 5:44 ` Nai Xia
2011-11-12 5:44 ` Nai Xia
2011-09-04 1:53 ` [PATCH 03/18] writeback: dirty rate control Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-29 11:57 ` Wu Fengguang
2011-09-29 11:57 ` Wu Fengguang
2011-09-04 1:53 ` [PATCH 04/18] writeback: stabilize bdi->dirty_ratelimit Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-04 1:53 ` [PATCH 05/18] writeback: per task dirty rate limit Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-06 15:47 ` Peter Zijlstra
2011-09-06 15:47 ` Peter Zijlstra
2011-09-06 23:27 ` Jan Kara
2011-09-06 23:27 ` Jan Kara
2011-09-06 23:34 ` Jan Kara
2011-09-06 23:34 ` Jan Kara
2011-09-07 7:27 ` Peter Zijlstra
2011-09-07 7:27 ` Peter Zijlstra
2011-09-07 1:04 ` Wu Fengguang
2011-09-07 1:04 ` Wu Fengguang
2011-09-07 7:31 ` Peter Zijlstra
2011-09-07 7:31 ` Peter Zijlstra
2011-09-07 11:00 ` Wu Fengguang
2011-09-07 11:00 ` Wu Fengguang
2011-09-04 1:53 ` [PATCH 06/18] writeback: IO-less balance_dirty_pages() Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-06 12:13 ` Peter Zijlstra
2011-09-06 12:13 ` Peter Zijlstra
2011-09-07 2:46 ` Wu Fengguang
2011-09-07 2:46 ` Wu Fengguang
2011-09-04 1:53 ` [PATCH 07/18] writeback: dirty ratelimit - think time compensation Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-04 1:53 ` [PATCH 08/18] writeback: trace dirty_ratelimit Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-04 1:53 ` [PATCH 09/18] writeback: trace balance_dirty_pages Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-04 1:53 ` [PATCH 10/18] writeback: dirty position control - bdi reserve area Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-06 14:09 ` Peter Zijlstra
2011-09-06 14:09 ` Peter Zijlstra
2011-09-07 12:31 ` Wu Fengguang
2011-09-07 12:31 ` Wu Fengguang
2011-09-12 10:19 ` Peter Zijlstra
2011-09-12 10:19 ` Peter Zijlstra
2011-09-18 14:17 ` Wu Fengguang [this message]
2011-09-18 14:37 ` Wu Fengguang
2011-09-18 14:37 ` Wu Fengguang
2011-09-18 14:47 ` Wu Fengguang
2011-09-18 14:47 ` Wu Fengguang
2011-09-28 14:02 ` Wu Fengguang
2011-09-28 14:50 ` Peter Zijlstra
2011-09-28 14:50 ` Peter Zijlstra
2011-09-29 3:32 ` Wu Fengguang
2011-09-29 3:32 ` Wu Fengguang
2011-09-29 8:49 ` Peter Zijlstra
2011-09-29 8:49 ` Peter Zijlstra
2011-09-29 11:05 ` Wu Fengguang
2011-09-29 11:05 ` Wu Fengguang
2011-09-29 12:15 ` Wu Fengguang
2011-09-04 1:53 ` [PATCH 11/18] block: add bdi flag to indicate risk of io queue underrun Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-06 14:22 ` Peter Zijlstra
2011-09-06 14:22 ` Peter Zijlstra
2011-09-07 2:37 ` Wu Fengguang
2011-09-07 2:37 ` Wu Fengguang
2011-09-07 7:31 ` Peter Zijlstra
2011-09-07 7:31 ` Peter Zijlstra
2011-09-04 1:53 ` [PATCH 12/18] writeback: balanced_rate cannot exceed write bandwidth Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-04 1:53 ` [PATCH 13/18] writeback: limit max dirty pause time Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-06 14:52 ` Peter Zijlstra
2011-09-06 14:52 ` Peter Zijlstra
2011-09-07 2:35 ` Wu Fengguang
2011-09-07 2:35 ` Wu Fengguang
2011-09-12 10:22 ` Peter Zijlstra
2011-09-12 10:22 ` Peter Zijlstra
2011-09-18 14:23 ` Wu Fengguang
2011-09-18 14:23 ` Wu Fengguang
2011-09-04 1:53 ` [PATCH 14/18] writeback: control " Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-06 15:51 ` Peter Zijlstra
2011-09-06 15:51 ` Peter Zijlstra
2011-09-07 2:02 ` Wu Fengguang
2011-09-07 2:02 ` Wu Fengguang
2011-09-12 10:28 ` Peter Zijlstra
2011-09-12 10:28 ` Peter Zijlstra
2011-09-04 1:53 ` [PATCH 15/18] writeback: charge leaked page dirties to active tasks Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-06 16:16 ` Peter Zijlstra
2011-09-06 16:16 ` Peter Zijlstra
2011-09-07 9:06 ` Wu Fengguang
2011-09-07 9:06 ` Wu Fengguang
2011-09-07 0:17 ` Jan Kara
2011-09-07 0:17 ` Jan Kara
2011-09-07 9:37 ` Wu Fengguang
2011-09-04 1:53 ` [PATCH 16/18] writeback: fix dirtied pages accounting on sub-page writes Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-04 1:53 ` [PATCH 17/18] writeback: fix dirtied pages accounting on redirty Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-06 16:18 ` Peter Zijlstra
2011-09-06 16:18 ` Peter Zijlstra
2011-09-07 0:22 ` Jan Kara
2011-09-07 0:22 ` Jan Kara
2011-09-07 1:18 ` Wu Fengguang
2011-09-07 6:56 ` Christoph Hellwig
2011-09-07 6:56 ` Christoph Hellwig
2011-09-07 8:19 ` Peter Zijlstra
2011-09-07 8:19 ` Peter Zijlstra
2011-09-07 16:42 ` Jan Kara
2011-09-07 16:42 ` Jan Kara
2011-09-07 16:46 ` Christoph Hellwig
2011-09-07 16:46 ` Christoph Hellwig
2011-09-08 8:51 ` Steven Whitehouse
2011-09-08 8:51 ` Steven Whitehouse
2011-09-04 1:53 ` [PATCH 18/18] btrfs: fix dirtied pages accounting on sub-page writes Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-04 1:53 ` Wu Fengguang
2011-09-07 13:32 ` [PATCH 00/18] IO-less dirty throttling v11 Wu Fengguang
2011-09-07 13:32 ` Wu Fengguang
2011-09-07 19:14 ` Trond Myklebust
2011-09-07 19:14 ` Trond Myklebust
2011-09-28 14:58 ` Christoph Hellwig
2011-09-28 14:58 ` Christoph Hellwig
2011-09-29 4:11 ` Wu Fengguang
2011-09-29 4:11 ` Wu Fengguang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20110918141705.GB15366@localhost \
--to=fengguang.wu@intel.com \
--cc=a.p.zijlstra@chello.nl \
--cc=akpm@linux-foundation.org \
--cc=arighi@develer.com \
--cc=david@fromorbit.com \
--cc=gthelen@google.com \
--cc=hch@lst.de \
--cc=jack@suse.cz \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=minchan.kim@gmail.com \
--cc=vgoyal@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.