* Re: [PATCH 00/11] IO-less dirty throttling v12
[not found] ` <1318248846.14400.21.camel@laptop>
@ 2011-10-10 13:07 ` Wu Fengguang
2011-10-10 13:10 ` [RFC][PATCH 1/2] nfs: writeback pages wait queue Wu Fengguang
2011-10-10 14:28 ` [PATCH 00/11] IO-less dirty throttling v12 Wu Fengguang
0 siblings, 2 replies; 10+ messages in thread
From: Wu Fengguang @ 2011-10-10 13:07 UTC (permalink / raw)
To: Peter Zijlstra
Cc: linux-fsdevel@vger.kernel.org, Andrew Morton, Jan Kara,
Christoph Hellwig, Dave Chinner, Greg Thelen, Minchan Kim,
Vivek Goyal, Andrea Righi, linux-mm, LKML, Trond Myklebust,
linux-nfs
On Mon, Oct 10, 2011 at 08:14:06PM +0800, Peter Zijlstra wrote:
> On Mon, 2011-10-03 at 21:42 +0800, Wu Fengguang wrote:
> > This is the minimal IO-less balance_dirty_pages() changes that are expected to
> > be regression free (well, except for NFS).
>
> I can't seem to get around reviewing these patches in detail, but fwiw
> I'm fine with pushing fwd with this set (plus a possible NFS fix).
>
> I don't see a reason to strip it down even further.
>
> So I guess that's:
>
> Acked-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Thanks :-) In fact you've already reviewed the major parts of the
patchset in great details and helped simplify parts of the algorithm,
which I appreciate a lot.
As for the NFS performance, the dd tests show that adding a writeback
wait queue to limit the number of NFS PG_writeback pages (patches
will follow) is able to gain 48% throughput in itself:
3.1.0-rc8-ioless6+ 3.1.0-rc8-nfs-wq+
------------------------ ------------------------
22.43 +81.8% 40.77 NFS-thresh=100M/nfs-10dd-1M-32p-32768M-100M:10-X
28.21 +52.6% 43.07 NFS-thresh=100M/nfs-1dd-1M-32p-32768M-100M:10-X
29.21 +55.4% 45.39 NFS-thresh=100M/nfs-2dd-1M-32p-32768M-100M:10-X
14.12 +40.4% 19.83 NFS-thresh=10M/nfs-10dd-1M-32p-32768M-10M:10-X
29.44 +11.4% 32.81 NFS-thresh=10M/nfs-1dd-1M-32p-32768M-10M:10-X
9.09 +240.9% 30.97 NFS-thresh=10M/nfs-2dd-1M-32p-32768M-10M:10-X
25.68 +84.6% 47.42 NFS-thresh=1G/nfs-10dd-1M-32p-32768M-1024M:10-X
41.06 +7.6% 44.20 NFS-thresh=1G/nfs-1dd-1M-32p-32768M-1024M:10-X
39.13 +25.9% 49.26 NFS-thresh=1G/nfs-2dd-1M-32p-32768M-1024M:10-X
238.38 +48.4% 353.72 TOTAL
Which will result in 28% overall improvements over the vanilla kernel:
3.1.0-rc4-vanilla+ 3.1.0-rc8-nfs-wq+
------------------------ ------------------------
20.89 +95.2% 40.77 NFS-thresh=100M/nfs-10dd-1M-32p-32768M-100M:10-X
39.43 +9.2% 43.07 NFS-thresh=100M/nfs-1dd-1M-32p-32768M-100M:10-X
26.60 +70.6% 45.39 NFS-thresh=100M/nfs-2dd-1M-32p-32768M-100M:10-X
12.70 +56.1% 19.83 NFS-thresh=10M/nfs-10dd-1M-32p-32768M-10M:10-X
27.41 +19.7% 32.81 NFS-thresh=10M/nfs-1dd-1M-32p-32768M-10M:10-X
26.52 +16.8% 30.97 NFS-thresh=10M/nfs-2dd-1M-32p-32768M-10M:10-X
40.70 +16.5% 47.42 NFS-thresh=1G/nfs-10dd-1M-32p-32768M-1024M:10-X
45.28 -2.4% 44.20 NFS-thresh=1G/nfs-1dd-1M-32p-32768M-1024M:10-X
35.74 +37.8% 49.26 NFS-thresh=1G/nfs-2dd-1M-32p-32768M-1024M:10-X
275.28 +28.5% 353.72 TOTAL
As for the most concerned NFS commits, the wait queue patch increases
the (nr_commits / bytes_written) ratio by +74% for the thresh=1G,10dd
case, +55% for the thresh=100M,10dd case, and mostly ignorable in the
other 1dd, 2dd cases, which looks acceptable.
The other noticeable change of the wait queue is, the RTT time per
write is reduced by 1-2 order(s) in many of the below cases (from
dozens of seconds to hundreds of milliseconds).
Thanks,
Fengguang
---
PS. mountstats numbers
thresh=1GB
==========
1dd
vanilla WRITE: 33108 33108 0 13794766688 4502688 89550800 1826162 91643336
ioless6 WRITE: 104355 104355 0 12824990824 14192280 1677501539 13497260 1691407074
nfs-wq WRITE: 58632 58632 0 13635750848 7973952 148662395 4735943 153535047
vanilla COMMIT: 29 29 0 3248 3712 45210 191022 236235
ioless6 COMMIT: 26 26 0 2912 3328 32875 196848 229725
nfs-wq COMMIT: 35 35 0 3920 4480 1156 223393 224550
2dd
vanilla WRITE: 28681 28681 0 11507024952 3900616 178242698 5849890 184288501
ioless6 WRITE: 151075 151075 0 12192866408 20546200 3195004617 5748708 3200969292
nfs-wq WRITE: 89925 89925 0 15450966104 12229800 212096905 3443883 215849660
vanilla COMMIT: 43 43 0 4816 5504 45252 349816 396792
ioless6 COMMIT: 52 52 0 5824 6656 40798 376099 417068
nfs-wq COMMIT: 66 66 0 7392 8448 10854 490021 502373
10dd
vanilla WRITE: 47281 47281 0 14044390136 6430216 1378503679 11994453 1390582846
ioless6 WRITE: 35972 35972 0 7959317984 4892192 1205239506 7412186 1212670083
nfs-wq WRITE: 49625 49625 0 14819167672 6749000 10704223 4135391 14876589
vanilla COMMIT: 235 235 0 26320 30080 328532 1097793 1426737
ioless6 COMMIT: 128 128 0 14336 16384 73611 388716 462470
nfs-wq COMMIT: 431 432 0 48384 55168 217056 1775499 1993006
thresh=100MB
============
1dd
vanilla WRITE: 28858 28858 0 12427843376 3924688 6384263 2308574 8722669
nfs-wq WRITE: 206620 206620 0 13104059680 28100320 90597897 10245879 101016004
vanilla COMMIT: 250 250 0 28000 32000 27030 229750 256786
nfs-wq COMMIT: 267 267 0 29904 34176 4672 247504 252184
2dd
vanilla WRITE: 32593 32593 0 8382655992 4432648 193667999 3611697 197302564
nfs-wq WRITE: 98662 98662 0 14025467856 13418032 183280630 5381343 188715890
vanilla COMMIT: 272 272 0 30464 34816 24445 295949 320576
nfs-wq COMMIT: 584 584 0 65408 74752 1318 483049 484442
10dd
vanilla WRITE: 32294 32294 0 6651515344 4391984 104926130 8666874 113596871
nfs-wq WRITE: 27571 27571 0 12711521256 3749656 6129491 2248486 8385102
vanilla COMMIT: 825 825 0 92400 105600 82135 739763 822179
nfs-wq COMMIT: 2449 2449 0 274288 313472 6091 2057767 2064555
^ permalink raw reply [flat|nested] 10+ messages in thread
* [RFC][PATCH 1/2] nfs: writeback pages wait queue
2011-10-10 13:07 ` [PATCH 00/11] IO-less dirty throttling v12 Wu Fengguang
@ 2011-10-10 13:10 ` Wu Fengguang
2011-10-10 13:11 ` [RFC][PATCH 2/2] nfs: scale writeback threshold proportional to dirty threshold Wu Fengguang
2011-10-18 8:51 ` [RFC][PATCH 1/2] nfs: writeback pages wait queue Wu Fengguang
2011-10-10 14:28 ` [PATCH 00/11] IO-less dirty throttling v12 Wu Fengguang
1 sibling, 2 replies; 10+ messages in thread
From: Wu Fengguang @ 2011-10-10 13:10 UTC (permalink / raw)
To: Trond Myklebust, linux-nfs
Cc: Peter Zijlstra, linux-fsdevel@vger.kernel.org, Andrew Morton,
Jan Kara, Christoph Hellwig, Dave Chinner, Greg Thelen,
Minchan Kim, Vivek Goyal, Andrea Righi, linux-mm, LKML
[known bug: this patch will block sync(1) in schecule() if dirty
threshold is set to as low as 1MB.]
The generic writeback routines are departing from congestion_wait()
in preference of get_request_wait(), aka. waiting on the block queues.
Introduce the missing writeback wait queue for NFS, otherwise its
writeback pages will grow out of control, exhausting all PG_dirty pages.
CC: Jens Axboe <axboe@kernel.dk>
CC: Chris Mason <chris.mason@oracle.com>
CC: Peter Zijlstra <a.p.zijlstra@chello.nl>
CC: Trond Myklebust <Trond.Myklebust@netapp.com>
Signed-off-by: Wu Fengguang <fengguang.wu@intel.com>
---
fs/nfs/client.c | 2
fs/nfs/write.c | 89 +++++++++++++++++++++++++++++++-----
include/linux/nfs_fs_sb.h | 1
3 files changed, 81 insertions(+), 11 deletions(-)
--- linux-next.orig/fs/nfs/write.c 2011-09-29 20:23:44.000000000 +0800
+++ linux-next/fs/nfs/write.c 2011-10-05 10:43:28.000000000 +0800
@@ -189,11 +189,64 @@ static int wb_priority(struct writeback_
* NFS congestion control
*/
+#define NFS_WAIT_PAGES (1024L >> (PAGE_SHIFT - 10))
int nfs_congestion_kb;
-#define NFS_CONGESTION_ON_THRESH (nfs_congestion_kb >> (PAGE_SHIFT-10))
-#define NFS_CONGESTION_OFF_THRESH \
- (NFS_CONGESTION_ON_THRESH - (NFS_CONGESTION_ON_THRESH >> 2))
+/*
+ * SYNC requests will block on (2*limit) and wakeup on (2*limit-NFS_WAIT_PAGES)
+ * ASYNC requests will block on (limit) and wakeup on (limit - NFS_WAIT_PAGES)
+ * In this way SYNC writes will never be blocked by ASYNC ones.
+ */
+
+static void nfs_set_congested(long nr, struct backing_dev_info *bdi)
+{
+ long limit = nfs_congestion_kb >> (PAGE_SHIFT - 10);
+
+ if (nr > limit && !test_bit(BDI_async_congested, &bdi->state))
+ set_bdi_congested(bdi, BLK_RW_ASYNC);
+ else if (nr > 2 * limit && !test_bit(BDI_sync_congested, &bdi->state))
+ set_bdi_congested(bdi, BLK_RW_SYNC);
+}
+
+static void nfs_wait_congested(int is_sync,
+ struct backing_dev_info *bdi,
+ wait_queue_head_t *wqh)
+{
+ int waitbit = is_sync ? BDI_sync_congested : BDI_async_congested;
+ DEFINE_WAIT(wait);
+
+ if (!test_bit(waitbit, &bdi->state))
+ return;
+
+ for (;;) {
+ prepare_to_wait(&wqh[is_sync], &wait, TASK_UNINTERRUPTIBLE);
+ if (!test_bit(waitbit, &bdi->state))
+ break;
+
+ io_schedule();
+ }
+ finish_wait(&wqh[is_sync], &wait);
+}
+
+static void nfs_wakeup_congested(long nr,
+ struct backing_dev_info *bdi,
+ wait_queue_head_t *wqh)
+{
+ long limit = nfs_congestion_kb >> (PAGE_SHIFT - 10);
+
+ if (nr < 2 * limit - min(limit / 8, NFS_WAIT_PAGES)) {
+ if (test_bit(BDI_sync_congested, &bdi->state))
+ clear_bdi_congested(bdi, BLK_RW_SYNC);
+ if (waitqueue_active(&wqh[BLK_RW_SYNC]))
+ wake_up(&wqh[BLK_RW_SYNC]);
+ }
+ if (nr < limit - min(limit / 8, NFS_WAIT_PAGES)) {
+ if (test_bit(BDI_async_congested, &bdi->state))
+ clear_bdi_congested(bdi, BLK_RW_ASYNC);
+ if (waitqueue_active(&wqh[BLK_RW_ASYNC]))
+ wake_up(&wqh[BLK_RW_ASYNC]);
+ }
+}
static int nfs_set_page_writeback(struct page *page)
{
@@ -204,11 +257,8 @@ static int nfs_set_page_writeback(struct
struct nfs_server *nfss = NFS_SERVER(inode);
page_cache_get(page);
- if (atomic_long_inc_return(&nfss->writeback) >
- NFS_CONGESTION_ON_THRESH) {
- set_bdi_congested(&nfss->backing_dev_info,
- BLK_RW_ASYNC);
- }
+ nfs_set_congested(atomic_long_inc_return(&nfss->writeback),
+ &nfss->backing_dev_info);
}
return ret;
}
@@ -220,8 +270,10 @@ static void nfs_end_page_writeback(struc
end_page_writeback(page);
page_cache_release(page);
- if (atomic_long_dec_return(&nfss->writeback) < NFS_CONGESTION_OFF_THRESH)
- clear_bdi_congested(&nfss->backing_dev_info, BLK_RW_ASYNC);
+
+ nfs_wakeup_congested(atomic_long_dec_return(&nfss->writeback),
+ &nfss->backing_dev_info,
+ nfss->writeback_wait);
}
static struct nfs_page *nfs_find_and_lock_request(struct page *page, bool nonblock)
@@ -322,19 +374,34 @@ static int nfs_writepage_locked(struct p
int nfs_writepage(struct page *page, struct writeback_control *wbc)
{
+ struct inode *inode = page->mapping->host;
+ struct nfs_server *nfss = NFS_SERVER(inode);
int ret;
ret = nfs_writepage_locked(page, wbc);
unlock_page(page);
+
+ nfs_wait_congested(wbc->sync_mode == WB_SYNC_ALL,
+ &nfss->backing_dev_info,
+ nfss->writeback_wait);
+
return ret;
}
-static int nfs_writepages_callback(struct page *page, struct writeback_control *wbc, void *data)
+static int nfs_writepages_callback(struct page *page,
+ struct writeback_control *wbc, void *data)
{
+ struct inode *inode = page->mapping->host;
+ struct nfs_server *nfss = NFS_SERVER(inode);
int ret;
ret = nfs_do_writepage(page, wbc, data);
unlock_page(page);
+
+ nfs_wait_congested(wbc->sync_mode == WB_SYNC_ALL,
+ &nfss->backing_dev_info,
+ nfss->writeback_wait);
+
return ret;
}
--- linux-next.orig/include/linux/nfs_fs_sb.h 2011-09-02 09:02:07.000000000 +0800
+++ linux-next/include/linux/nfs_fs_sb.h 2011-10-04 20:26:03.000000000 +0800
@@ -102,6 +102,7 @@ struct nfs_server {
struct nfs_iostats __percpu *io_stats; /* I/O statistics */
struct backing_dev_info backing_dev_info;
atomic_long_t writeback; /* number of writeback pages */
+ wait_queue_head_t writeback_wait[2];
int flags; /* various flags */
unsigned int caps; /* server capabilities */
unsigned int rsize; /* read size */
--- linux-next.orig/fs/nfs/client.c 2011-08-22 13:59:52.000000000 +0800
+++ linux-next/fs/nfs/client.c 2011-10-04 20:26:03.000000000 +0800
@@ -1066,6 +1066,8 @@ static struct nfs_server *nfs_alloc_serv
INIT_LIST_HEAD(&server->layouts);
atomic_set(&server->active, 0);
+ init_waitqueue_head(&server->writeback_wait[BLK_RW_SYNC]);
+ init_waitqueue_head(&server->writeback_wait[BLK_RW_ASYNC]);
server->io_stats = nfs_alloc_iostats();
if (!server->io_stats) {
^ permalink raw reply [flat|nested] 10+ messages in thread
* [RFC][PATCH 2/2] nfs: scale writeback threshold proportional to dirty threshold
2011-10-10 13:10 ` [RFC][PATCH 1/2] nfs: writeback pages wait queue Wu Fengguang
@ 2011-10-10 13:11 ` Wu Fengguang
2011-10-18 8:53 ` Wu Fengguang
2011-10-18 8:51 ` [RFC][PATCH 1/2] nfs: writeback pages wait queue Wu Fengguang
1 sibling, 1 reply; 10+ messages in thread
From: Wu Fengguang @ 2011-10-10 13:11 UTC (permalink / raw)
To: Trond Myklebust, linux-nfs
Cc: Peter Zijlstra, linux-fsdevel@vger.kernel.org, Andrew Morton,
Jan Kara, Christoph Hellwig, Dave Chinner, Greg Thelen,
Minchan Kim, Vivek Goyal, Andrea Righi, linux-mm, LKML
nfs_congestion_kb is to control the max allowed writeback and in-commit
pages. It's not reasonable for them to outnumber dirty and to-commit
pages. So each of them should not take more than 1/4 dirty threshold.
Considering that nfs_init_writepagecache() is called on fresh boot,
at the time dirty_thresh is much higher than the real dirty limit after
lots of user space memory consumptions, use 1/8 instead.
We might update nfs_congestion_kb when global dirty limit is changed
at runtime, but whatever, do it simple first.
CC: Trond Myklebust <Trond.Myklebust@netapp.com>
Signed-off-by: Wu Fengguang <fengguang.wu@intel.com>
---
fs/nfs/write.c | 52 ++++++++++++++++++++++++++++--------------
mm/page-writeback.c | 6 ++++
2 files changed, 41 insertions(+), 17 deletions(-)
--- linux-next.orig/fs/nfs/write.c 2011-10-09 21:36:22.000000000 +0800
+++ linux-next/fs/nfs/write.c 2011-10-10 21:05:07.000000000 +0800
@@ -1775,61 +1775,79 @@ int nfs_migrate_page(struct address_spac
set_page_private(newpage, (unsigned long)req);
ClearPagePrivate(page);
set_page_private(page, 0);
spin_unlock(&mapping->host->i_lock);
page_cache_release(page);
out_unlock:
nfs_clear_page_tag_locked(req);
out:
return ret;
}
#endif
-int __init nfs_init_writepagecache(void)
+void nfs_update_congestion_thresh(void)
{
- nfs_wdata_cachep = kmem_cache_create("nfs_write_data",
- sizeof(struct nfs_write_data),
- 0, SLAB_HWCACHE_ALIGN,
- NULL);
- if (nfs_wdata_cachep == NULL)
- return -ENOMEM;
-
- nfs_wdata_mempool = mempool_create_slab_pool(MIN_POOL_WRITE,
- nfs_wdata_cachep);
- if (nfs_wdata_mempool == NULL)
- return -ENOMEM;
-
- nfs_commit_mempool = mempool_create_slab_pool(MIN_POOL_COMMIT,
- nfs_wdata_cachep);
- if (nfs_commit_mempool == NULL)
- return -ENOMEM;
+ unsigned long background_thresh;
+ unsigned long dirty_thresh;
/*
* NFS congestion size, scale with available memory.
*
* 64MB: 8192k
* 128MB: 11585k
* 256MB: 16384k
* 512MB: 23170k
* 1GB: 32768k
* 2GB: 46340k
* 4GB: 65536k
* 8GB: 92681k
* 16GB: 131072k
*
* This allows larger machines to have larger/more transfers.
* Limit the default to 256M
*/
nfs_congestion_kb = (16*int_sqrt(totalram_pages)) << (PAGE_SHIFT-10);
if (nfs_congestion_kb > 256*1024)
nfs_congestion_kb = 256*1024;
+ /*
+ * Limit to 1/8 dirty threshold, so that writeback+in_commit pages
+ * won't overnumber dirty+to_commit pages.
+ */
+ global_dirty_limits(&background_thresh, &dirty_thresh);
+ dirty_thresh <<= PAGE_SHIFT - 10;
+
+ if (nfs_congestion_kb > dirty_thresh / 8)
+ nfs_congestion_kb = dirty_thresh / 8;
+}
+
+int __init nfs_init_writepagecache(void)
+{
+ nfs_wdata_cachep = kmem_cache_create("nfs_write_data",
+ sizeof(struct nfs_write_data),
+ 0, SLAB_HWCACHE_ALIGN,
+ NULL);
+ if (nfs_wdata_cachep == NULL)
+ return -ENOMEM;
+
+ nfs_wdata_mempool = mempool_create_slab_pool(MIN_POOL_WRITE,
+ nfs_wdata_cachep);
+ if (nfs_wdata_mempool == NULL)
+ return -ENOMEM;
+
+ nfs_commit_mempool = mempool_create_slab_pool(MIN_POOL_COMMIT,
+ nfs_wdata_cachep);
+ if (nfs_commit_mempool == NULL)
+ return -ENOMEM;
+
+ nfs_update_congestion_thresh();
+
return 0;
}
void nfs_destroy_writepagecache(void)
{
mempool_destroy(nfs_commit_mempool);
mempool_destroy(nfs_wdata_mempool);
kmem_cache_destroy(nfs_wdata_cachep);
}
--- linux-next.orig/mm/page-writeback.c 2011-10-09 21:36:06.000000000 +0800
+++ linux-next/mm/page-writeback.c 2011-10-10 21:05:07.000000000 +0800
@@ -138,34 +138,39 @@ static struct prop_descriptor vm_dirties
static int calc_period_shift(void)
{
unsigned long dirty_total;
if (vm_dirty_bytes)
dirty_total = vm_dirty_bytes / PAGE_SIZE;
else
dirty_total = (vm_dirty_ratio * determine_dirtyable_memory()) /
100;
return 2 + ilog2(dirty_total - 1);
}
+void __weak nfs_update_congestion_thresh(void)
+{
+}
+
/*
* update the period when the dirty threshold changes.
*/
static void update_completion_period(void)
{
int shift = calc_period_shift();
prop_change_shift(&vm_completions, shift);
prop_change_shift(&vm_dirties, shift);
writeback_set_ratelimit();
+ nfs_update_congestion_thresh();
}
int dirty_background_ratio_handler(struct ctl_table *table, int write,
void __user *buffer, size_t *lenp,
loff_t *ppos)
{
int ret;
ret = proc_dointvec_minmax(table, write, buffer, lenp, ppos);
if (ret == 0 && write)
dirty_background_bytes = 0;
return ret;
@@ -438,24 +443,25 @@ unsigned long bdi_dirty_limit(struct bac
bdi_writeout_fraction(bdi, &numerator, &denominator);
bdi_dirty = (dirty * (100 - bdi_min_ratio)) / 100;
bdi_dirty *= numerator;
do_div(bdi_dirty, denominator);
bdi_dirty += (dirty * bdi->min_ratio) / 100;
if (bdi_dirty > (dirty * bdi->max_ratio) / 100)
bdi_dirty = dirty * bdi->max_ratio / 100;
return bdi_dirty;
}
+EXPORT_SYMBOL_GPL(global_dirty_limits);
/*
* Dirty position control.
*
* (o) global/bdi setpoints
*
* We want the dirty pages be balanced around the global/bdi setpoints.
* When the number of dirty pages is higher/lower than the setpoint, the
* dirty position control ratio (and hence task dirty ratelimit) will be
* decreased/increased to bring the dirty pages back to the setpoint.
*
* pos_ratio = 1 << RATELIMIT_CALC_SHIFT
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 00/11] IO-less dirty throttling v12
2011-10-10 13:07 ` [PATCH 00/11] IO-less dirty throttling v12 Wu Fengguang
2011-10-10 13:10 ` [RFC][PATCH 1/2] nfs: writeback pages wait queue Wu Fengguang
@ 2011-10-10 14:28 ` Wu Fengguang
2011-10-17 3:03 ` Wu Fengguang
1 sibling, 1 reply; 10+ messages in thread
From: Wu Fengguang @ 2011-10-10 14:28 UTC (permalink / raw)
To: Trond Myklebust, linux-nfs
Cc: Peter Zijlstra, linux-fsdevel@vger.kernel.org, Andrew Morton,
Jan Kara, Christoph Hellwig, Dave Chinner, Greg Thelen,
Minchan Kim, Vivek Goyal, Andrea Righi, linux-mm, LKML
Hi Trond,
> As for the NFS performance, the dd tests show that adding a writeback
> wait queue to limit the number of NFS PG_writeback pages (patches
> will follow) is able to gain 48% throughput in itself:
>
> 3.1.0-rc8-ioless6+ 3.1.0-rc8-nfs-wq+
> ------------------------ ------------------------
> 22.43 +81.8% 40.77 NFS-thresh=100M/nfs-10dd-1M-32p-32768M-100M:10-X
> 28.21 +52.6% 43.07 NFS-thresh=100M/nfs-1dd-1M-32p-32768M-100M:10-X
> 29.21 +55.4% 45.39 NFS-thresh=100M/nfs-2dd-1M-32p-32768M-100M:10-X
> 14.12 +40.4% 19.83 NFS-thresh=10M/nfs-10dd-1M-32p-32768M-10M:10-X
> 29.44 +11.4% 32.81 NFS-thresh=10M/nfs-1dd-1M-32p-32768M-10M:10-X
> 9.09 +240.9% 30.97 NFS-thresh=10M/nfs-2dd-1M-32p-32768M-10M:10-X
> 25.68 +84.6% 47.42 NFS-thresh=1G/nfs-10dd-1M-32p-32768M-1024M:10-X
> 41.06 +7.6% 44.20 NFS-thresh=1G/nfs-1dd-1M-32p-32768M-1024M:10-X
> 39.13 +25.9% 49.26 NFS-thresh=1G/nfs-2dd-1M-32p-32768M-1024M:10-X
> 238.38 +48.4% 353.72 TOTAL
>
> Which will result in 28% overall improvements over the vanilla kernel:
>
> 3.1.0-rc4-vanilla+ 3.1.0-rc8-nfs-wq+
> ------------------------ ------------------------
> 20.89 +95.2% 40.77 NFS-thresh=100M/nfs-10dd-1M-32p-32768M-100M:10-X
> 39.43 +9.2% 43.07 NFS-thresh=100M/nfs-1dd-1M-32p-32768M-100M:10-X
> 26.60 +70.6% 45.39 NFS-thresh=100M/nfs-2dd-1M-32p-32768M-100M:10-X
> 12.70 +56.1% 19.83 NFS-thresh=10M/nfs-10dd-1M-32p-32768M-10M:10-X
> 27.41 +19.7% 32.81 NFS-thresh=10M/nfs-1dd-1M-32p-32768M-10M:10-X
> 26.52 +16.8% 30.97 NFS-thresh=10M/nfs-2dd-1M-32p-32768M-10M:10-X
> 40.70 +16.5% 47.42 NFS-thresh=1G/nfs-10dd-1M-32p-32768M-1024M:10-X
> 45.28 -2.4% 44.20 NFS-thresh=1G/nfs-1dd-1M-32p-32768M-1024M:10-X
> 35.74 +37.8% 49.26 NFS-thresh=1G/nfs-2dd-1M-32p-32768M-1024M:10-X
> 275.28 +28.5% 353.72 TOTAL
>
> As for the most concerned NFS commits, the wait queue patch increases
> the (nr_commits / bytes_written) ratio by +74% for the thresh=1G,10dd
> case, +55% for the thresh=100M,10dd case, and mostly ignorable in the
> other 1dd, 2dd cases, which looks acceptable.
>
> The other noticeable change of the wait queue is, the RTT time per
Sorry it's not RTT, but mainly the local queue time of the WRITE RPCs.
> write is reduced by 1-2 order(s) in many of the below cases (from
> dozens of seconds to hundreds of milliseconds).
I also measured the stddev of the network bandwidths, and find more
smooth network transfers in general with the wait queue, which is
expected.
thresh=1G
vanilla ioless6 nfs-wq
1dd 83088173.728 53468627.578 53627922.011
2dd 52398918.208 43733074.167 53531381.177
10dd 67792638.857 44734947.283 39681731.234
However the major difference should still be that the writeback wait
queue can significantly reduce the local queue time for the WRITE RPCs.
The wait queue patch looks reasonable in that it keeps the pages in
PG_dirty state rather than to prematurely put them to PG_writeback
state only to queue them up for dozens of seconds before xmit.
It should be safe because that's exactly the old proved behavior
before the per-bdi writeback patches introduced in 2.6.32. The 2nd
patch on proportional nfs_congestion_kb is a new change, though.
Thanks,
Fengguang
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 00/11] IO-less dirty throttling v12
2011-10-10 14:28 ` [PATCH 00/11] IO-less dirty throttling v12 Wu Fengguang
@ 2011-10-17 3:03 ` Wu Fengguang
0 siblings, 0 replies; 10+ messages in thread
From: Wu Fengguang @ 2011-10-17 3:03 UTC (permalink / raw)
To: Trond Myklebust, linux-nfs
Cc: Peter Zijlstra, linux-fsdevel@vger.kernel.org, Andrew Morton,
Jan Kara, Christoph Hellwig, Dave Chinner, Greg Thelen,
Minchan Kim, Vivek Goyal, Andrea Righi, linux-mm, LKML
[-- Attachment #1: Type: text/plain, Size: 13557 bytes --]
Hi Trond,
I enhanced the script to compare the write_bw as well as the NFS
write/commit stats.
3.1.0-rc4-vanilla+ 3.1.0-rc8-nfs-wq+
------------------------ ------------------------
(MB/s) 275.28 +28.5% 353.72 TOTAL write_bw
5649.00 +192.3% 16510.00 TOTAL nfs_nr_commits (*)
261987.00 +205.1% 799451.00 TOTAL nfs_nr_writes (*)
(MB) 866.52 -18.1% 709.85 TOTAL nfs_commit_size (**)
2.94 -44.8% 1.62 TOTAL nfs_write_size
(ms) 47814.05 -84.0% 7631.57 TOTAL nfs_write_queue_time
1405.05 -53.6% 652.59 TOTAL nfs_write_rtt_time
49237.94 -83.2% 8292.74 TOTAL nfs_write_execute_time
4320.98 -83.2% 726.27 TOTAL nfs_commit_queue_time
22943.13 -8.6% 20963.46 TOTAL nfs_commit_rtt_time
27307.42 -20.5% 21714.12 TOTAL nfs_commit_execute_time
(*) The x3 nfs_nr_writes and nfs_nr_commits numbers should be taken
with a salt because the total written bytes are increased at the
same time.
(**) The TOTAL nfs_commit_size mainly reflects the thresh=1G cases
because the numbers in the 10M/100M cases are very small
comparing to the 1G cases (as shown in the below case by case
values). Ditto for the *_time values. However the thresh=1G
cases should be most close to the typical NFS client setup, so
the values are still mostly representative numbers.
Below are the detailed case by case views. The script is attached,
which shows how exactly the numbers are calculated from mountstats.
Thanks,
Fengguang
---
3.1.0-rc4-vanilla+ 3.1.0-rc8-nfs-wq+
------------------------ ------------------------
20.89 +95.2% 40.77 NFS-thresh=100M/nfs-10dd-1M-32p-32768M-100M:10-X
39.43 +9.2% 43.07 NFS-thresh=100M/nfs-1dd-1M-32p-32768M-100M:10-X
26.60 +70.6% 45.39 NFS-thresh=100M/nfs-2dd-1M-32p-32768M-100M:10-X
12.70 +56.1% 19.83 NFS-thresh=10M/nfs-10dd-1M-32p-32768M-10M:10-X
27.41 +19.7% 32.81 NFS-thresh=10M/nfs-1dd-1M-32p-32768M-10M:10-X
26.52 +16.8% 30.97 NFS-thresh=10M/nfs-2dd-1M-32p-32768M-10M:10-X
40.70 +16.5% 47.42 NFS-thresh=1G/nfs-10dd-1M-32p-32768M-1024M:10-X
45.28 -2.4% 44.20 NFS-thresh=1G/nfs-1dd-1M-32p-32768M-1024M:10-X
35.74 +37.8% 49.26 NFS-thresh=1G/nfs-2dd-1M-32p-32768M-1024M:10-X
275.28 +28.5% 353.72 TOTAL write_bw
3.1.0-rc4-vanilla+ 3.1.0-rc8-nfs-wq+
------------------------ ------------------------
825.00 +196.8% 2449.00 NFS-thresh=100M/nfs-10dd-1M-32p-32768M-100M:10-X
250.00 +6.8% 267.00 NFS-thresh=100M/nfs-1dd-1M-32p-32768M-100M:10-X
272.00 +114.7% 584.00 NFS-thresh=100M/nfs-2dd-1M-32p-32768M-100M:10-X
1477.00 +350.8% 6658.00 NFS-thresh=10M/nfs-10dd-1M-32p-32768M-10M:10-X
997.00 +115.8% 2152.00 NFS-thresh=10M/nfs-1dd-1M-32p-32768M-10M:10-X
1521.00 +154.3% 3868.00 NFS-thresh=10M/nfs-2dd-1M-32p-32768M-10M:10-X
235.00 +83.4% 431.00 NFS-thresh=1G/nfs-10dd-1M-32p-32768M-1024M:10-X
29.00 +20.7% 35.00 NFS-thresh=1G/nfs-1dd-1M-32p-32768M-1024M:10-X
43.00 +53.5% 66.00 NFS-thresh=1G/nfs-2dd-1M-32p-32768M-1024M:10-X
5649.00 +192.3% 16510.00 TOTAL nfs_nr_commits
3.1.0-rc4-vanilla+ 3.1.0-rc8-nfs-wq+
------------------------ ------------------------
32294.00 -14.6% 27571.00 NFS-thresh=100M/nfs-10dd-1M-32p-32768M-100M:10-X
28858.00 +616.0% 206620.00 NFS-thresh=100M/nfs-1dd-1M-32p-32768M-100M:10-X
32593.00 +202.7% 98662.00 NFS-thresh=100M/nfs-2dd-1M-32p-32768M-100M:10-X
18937.00 +111.7% 40085.00 NFS-thresh=10M/nfs-10dd-1M-32p-32768M-10M:10-X
18762.00 +660.5% 142691.00 NFS-thresh=10M/nfs-1dd-1M-32p-32768M-10M:10-X
21473.00 +298.8% 85640.00 NFS-thresh=10M/nfs-2dd-1M-32p-32768M-10M:10-X
47281.00 +5.0% 49625.00 NFS-thresh=1G/nfs-10dd-1M-32p-32768M-1024M:10-X
33108.00 +77.1% 58632.00 NFS-thresh=1G/nfs-1dd-1M-32p-32768M-1024M:10-X
28681.00 +213.5% 89925.00 NFS-thresh=1G/nfs-2dd-1M-32p-32768M-1024M:10-X
261987.00 +205.1% 799451.00 TOTAL nfs_nr_writes
3.1.0-rc4-vanilla+ 3.1.0-rc8-nfs-wq+
------------------------ ------------------------
7.69 -35.6% 4.95 NFS-thresh=100M/nfs-10dd-1M-32p-32768M-100M:10-X
47.41 -1.3% 46.81 NFS-thresh=100M/nfs-1dd-1M-32p-32768M-100M:10-X
29.39 -22.1% 22.90 NFS-thresh=100M/nfs-2dd-1M-32p-32768M-100M:10-X
2.73 -68.3% 0.87 NFS-thresh=10M/nfs-10dd-1M-32p-32768M-10M:10-X
8.24 -46.7% 4.39 NFS-thresh=10M/nfs-1dd-1M-32p-32768M-10M:10-X
5.21 -55.1% 2.34 NFS-thresh=10M/nfs-2dd-1M-32p-32768M-10M:10-X
56.99 -42.5% 32.79 NFS-thresh=1G/nfs-10dd-1M-32p-32768M-1024M:10-X
453.65 -18.1% 371.54 NFS-thresh=1G/nfs-1dd-1M-32p-32768M-1024M:10-X
255.21 -12.5% 223.26 NFS-thresh=1G/nfs-2dd-1M-32p-32768M-1024M:10-X
866.52 -18.1% 709.85 TOTAL nfs_commit_size
3.1.0-rc4-vanilla+ 3.1.0-rc8-nfs-wq+
------------------------ ------------------------
0.20 +123.8% 0.44 NFS-thresh=100M/nfs-10dd-1M-32p-32768M-100M:10-X
0.41 -85.3% 0.06 NFS-thresh=100M/nfs-1dd-1M-32p-32768M-100M:10-X
0.25 -44.7% 0.14 NFS-thresh=100M/nfs-2dd-1M-32p-32768M-100M:10-X
0.21 -32.4% 0.14 NFS-thresh=10M/nfs-10dd-1M-32p-32768M-10M:10-X
0.44 -84.9% 0.07 NFS-thresh=10M/nfs-1dd-1M-32p-32768M-10M:10-X
0.37 -71.4% 0.11 NFS-thresh=10M/nfs-2dd-1M-32p-32768M-10M:10-X
0.28 +0.5% 0.28 NFS-thresh=1G/nfs-10dd-1M-32p-32768M-1024M:10-X
0.40 -44.2% 0.22 NFS-thresh=1G/nfs-1dd-1M-32p-32768M-1024M:10-X
0.38 -57.2% 0.16 NFS-thresh=1G/nfs-2dd-1M-32p-32768M-1024M:10-X
2.94 -44.8% 1.62 TOTAL nfs_write_size
3.1.0-rc4-vanilla+ 3.1.0-rc8-nfs-wq+
------------------------ ------------------------
3249.09 -93.2% 222.32 NFS-thresh=100M/nfs-10dd-1M-32p-32768M-100M:10-X
221.23 +98.2% 438.48 NFS-thresh=100M/nfs-1dd-1M-32p-32768M-100M:10-X
5942.01 -68.7% 1857.66 NFS-thresh=100M/nfs-2dd-1M-32p-32768M-100M:10-X
285.75 -99.9% 0.38 NFS-thresh=10M/nfs-10dd-1M-32p-32768M-10M:10-X
6.21 -95.4% 0.28 NFS-thresh=10M/nfs-1dd-1M-32p-32768M-10M:10-X
34.73 -92.4% 2.63 NFS-thresh=10M/nfs-2dd-1M-32p-32768M-10M:10-X
29155.55 -99.3% 215.70 NFS-thresh=1G/nfs-10dd-1M-32p-32768M-1024M:10-X
2704.81 -6.3% 2535.52 NFS-thresh=1G/nfs-1dd-1M-32p-32768M-1024M:10-X
6214.66 -62.0% 2358.60 NFS-thresh=1G/nfs-2dd-1M-32p-32768M-1024M:10-X
47814.05 -84.0% 7631.57 TOTAL nfs_write_queue_time
3.1.0-rc4-vanilla+ 3.1.0-rc8-nfs-wq+
------------------------ ------------------------
268.37 -69.6% 81.55 NFS-thresh=100M/nfs-10dd-1M-32p-32768M-100M:10-X
80.00 -38.0% 49.59 NFS-thresh=100M/nfs-1dd-1M-32p-32768M-100M:10-X
110.81 -50.8% 54.54 NFS-thresh=100M/nfs-2dd-1M-32p-32768M-100M:10-X
295.72 -41.7% 172.52 NFS-thresh=10M/nfs-10dd-1M-32p-32768M-10M:10-X
36.64 -31.6% 25.05 NFS-thresh=10M/nfs-1dd-1M-32p-32768M-10M:10-X
100.70 -33.5% 66.93 NFS-thresh=10M/nfs-2dd-1M-32p-32768M-10M:10-X
253.68 -67.2% 83.33 NFS-thresh=1G/nfs-10dd-1M-32p-32768M-1024M:10-X
55.16 +46.4% 80.77 NFS-thresh=1G/nfs-1dd-1M-32p-32768M-1024M:10-X
203.96 -81.2% 38.30 NFS-thresh=1G/nfs-2dd-1M-32p-32768M-1024M:10-X
1405.05 -53.6% 652.59 TOTAL nfs_write_rtt_time
3.1.0-rc4-vanilla+ 3.1.0-rc8-nfs-wq+
------------------------ ------------------------
3517.58 -91.4% 304.13 NFS-thresh=100M/nfs-10dd-1M-32p-32768M-100M:10-X
302.26 +61.7% 488.90 NFS-thresh=100M/nfs-1dd-1M-32p-32768M-100M:10-X
6053.53 -68.4% 1912.75 NFS-thresh=100M/nfs-2dd-1M-32p-32768M-100M:10-X
581.52 -70.3% 173.00 NFS-thresh=10M/nfs-10dd-1M-32p-32768M-10M:10-X
42.99 -40.7% 25.47 NFS-thresh=10M/nfs-1dd-1M-32p-32768M-10M:10-X
135.56 -48.5% 69.75 NFS-thresh=10M/nfs-2dd-1M-32p-32768M-10M:10-X
29411.03 -99.0% 299.78 NFS-thresh=1G/nfs-10dd-1M-32p-32768M-1024M:10-X
2768.01 -5.4% 2618.62 NFS-thresh=1G/nfs-1dd-1M-32p-32768M-1024M:10-X
6425.46 -62.6% 2400.33 NFS-thresh=1G/nfs-2dd-1M-32p-32768M-1024M:10-X
49237.94 -83.2% 8292.74 TOTAL nfs_write_execute_time
3.1.0-rc4-vanilla+ 3.1.0-rc8-nfs-wq+
------------------------ ------------------------
99.56 -97.5% 2.49 NFS-thresh=100M/nfs-10dd-1M-32p-32768M-100M:10-X
108.12 -83.8% 17.50 NFS-thresh=100M/nfs-1dd-1M-32p-32768M-100M:10-X
89.87 -97.5% 2.26 NFS-thresh=100M/nfs-2dd-1M-32p-32768M-100M:10-X
9.00 -90.5% 0.85 NFS-thresh=10M/nfs-10dd-1M-32p-32768M-10M:10-X
2.41 -58.2% 1.01 NFS-thresh=10M/nfs-1dd-1M-32p-32768M-10M:10-X
2.68 -59.9% 1.07 NFS-thresh=10M/nfs-2dd-1M-32p-32768M-10M:10-X
1398.01 -64.0% 503.61 NFS-thresh=1G/nfs-10dd-1M-32p-32768M-1024M:10-X
1558.97 -97.9% 33.03 NFS-thresh=1G/nfs-1dd-1M-32p-32768M-1024M:10-X
1052.37 -84.4% 164.45 NFS-thresh=1G/nfs-2dd-1M-32p-32768M-1024M:10-X
4320.98 -83.2% 726.27 TOTAL nfs_commit_queue_time
3.1.0-rc4-vanilla+ 3.1.0-rc8-nfs-wq+
------------------------ ------------------------
896.68 -6.3% 840.25 NFS-thresh=100M/nfs-10dd-1M-32p-32768M-100M:10-X
919.00 +0.9% 926.98 NFS-thresh=100M/nfs-1dd-1M-32p-32768M-100M:10-X
1088.05 -24.0% 827.14 NFS-thresh=100M/nfs-2dd-1M-32p-32768M-100M:10-X
266.54 -22.7% 206.09 NFS-thresh=10M/nfs-10dd-1M-32p-32768M-10M:10-X
191.28 -41.3% 112.32 NFS-thresh=10M/nfs-1dd-1M-32p-32768M-10M:10-X
187.90 -34.0% 123.98 NFS-thresh=10M/nfs-2dd-1M-32p-32768M-10M:10-X
4671.46 -11.8% 4119.49 NFS-thresh=1G/nfs-10dd-1M-32p-32768M-1024M:10-X
6586.97 -3.1% 6382.66 NFS-thresh=1G/nfs-1dd-1M-32p-32768M-1024M:10-X
8135.26 -8.7% 7424.56 NFS-thresh=1G/nfs-2dd-1M-32p-32768M-1024M:10-X
22943.13 -8.6% 20963.46 TOTAL nfs_commit_rtt_time
3.1.0-rc4-vanilla+ 3.1.0-rc8-nfs-wq+
------------------------ ------------------------
996.58 -15.4% 843.02 NFS-thresh=100M/nfs-10dd-1M-32p-32768M-100M:10-X
1027.14 -8.0% 944.51 NFS-thresh=100M/nfs-1dd-1M-32p-32768M-100M:10-X
1178.59 -29.6% 829.52 NFS-thresh=100M/nfs-2dd-1M-32p-32768M-100M:10-X
275.75 -24.9% 207.03 NFS-thresh=10M/nfs-10dd-1M-32p-32768M-10M:10-X
193.71 -41.5% 113.35 NFS-thresh=10M/nfs-1dd-1M-32p-32768M-10M:10-X
190.67 -34.4% 125.12 NFS-thresh=10M/nfs-2dd-1M-32p-32768M-10M:10-X
6071.22 -23.8% 4624.14 NFS-thresh=1G/nfs-10dd-1M-32p-32768M-1024M:10-X
8146.03 -21.2% 6415.71 NFS-thresh=1G/nfs-1dd-1M-32p-32768M-1024M:10-X
9227.72 -17.5% 7611.71 NFS-thresh=1G/nfs-2dd-1M-32p-32768M-1024M:10-X
27307.42 -20.5% 21714.12 TOTAL nfs_commit_execute_time
[-- Attachment #2: compare.rb --]
[-- Type: application/x-ruby, Size: 6250 bytes --]
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [RFC][PATCH 1/2] nfs: writeback pages wait queue
2011-10-10 13:10 ` [RFC][PATCH 1/2] nfs: writeback pages wait queue Wu Fengguang
2011-10-10 13:11 ` [RFC][PATCH 2/2] nfs: scale writeback threshold proportional to dirty threshold Wu Fengguang
@ 2011-10-18 8:51 ` Wu Fengguang
2011-10-20 3:59 ` Wu Fengguang
1 sibling, 1 reply; 10+ messages in thread
From: Wu Fengguang @ 2011-10-18 8:51 UTC (permalink / raw)
To: Trond Myklebust, linux-nfs
Cc: Peter Zijlstra, linux-fsdevel@vger.kernel.org, Andrew Morton,
Jan Kara, Christoph Hellwig, Dave Chinner, Greg Thelen,
Minchan Kim, Vivek Goyal, Andrea Righi, linux-mm, LKML
On Mon, Oct 10, 2011 at 09:10:51PM +0800, Wu Fengguang wrote:
> [known bug: this patch will block sync(1) in schecule() if dirty
> threshold is set to as low as 1MB.]
The root cause of the deadlock is found to be, the flusher
generated enough PG_writeback pages, and got blocked just before it's
able to assemble one complete NFS WRITE RPC. So the PG_writeback pages
never manage to reach the NFS server!
Feng kindly offers a fix that converts the per-page throttling to the
more coarse grained per-write_pages throttling, which is found to
further increase the performance as well as commit size. Bingo!
3.1.0-rc8-vanilla+ 3.1.0-rc8-nfs-wq4+
------------------------ ------------------------
354.65 +45.4% 515.48 TOTAL write_bw
10498.00 +91.7% 20120.00 TOTAL nfs_nr_commits
233013.00 +99.9% 465751.00 TOTAL nfs_nr_writes
895.47 +3.1% 923.62 TOTAL nfs_commit_size
5.71 -14.5% 4.88 TOTAL nfs_write_size
108269.33 -84.3% 17003.69 TOTAL nfs_write_queue_time
1836.03 -34.4% 1204.27 TOTAL nfs_write_rtt_time
110144.96 -83.5% 18220.96 TOTAL nfs_write_execute_time
2902.62 -88.6% 332.20 TOTAL nfs_commit_queue_time
16282.75 -23.3% 12490.87 TOTAL nfs_commit_rtt_time
19234.16 -33.3% 12833.00 TOTAL nfs_commit_execute_time
3.1.0-rc8-vanilla+ 3.1.0-rc8-nfs-wq4+
------------------------ ------------------------
21.85 +97.9% 43.23 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
51.38 +42.6% 73.26 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
28.81 +145.3% 70.68 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
13.74 +57.1% 21.59 NFS-thresh=10M/nfs-10dd-4k-32p-32768M-10M:10-X
29.11 -0.3% 29.02 NFS-thresh=10M/nfs-1dd-4k-32p-32768M-10M:10-X
16.68 +90.5% 31.78 NFS-thresh=10M/nfs-2dd-4k-32p-32768M-10M:10-X
48.88 +41.2% 69.01 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
57.85 +32.7% 76.74 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
47.13 +63.1% 76.87 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
9.82 -33.0% 6.58 NFS-thresh=1M/nfs-10dd-4k-32p-32768M-1M:10-X
13.72 -18.1% 11.24 NFS-thresh=1M/nfs-1dd-4k-32p-32768M-1M:10-X
15.68 -65.0% 5.48 NFS-thresh=1M/nfs-2dd-4k-32p-32768M-1M:10-X
354.65 +45.4% 515.48 TOTAL write_bw
3.1.0-rc8-vanilla+ 3.1.0-rc8-nfs-wq4+
------------------------ ------------------------
834.00 +224.2% 2704.00 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
311.00 +144.1% 759.00 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
282.00 +253.5% 997.00 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
1387.00 +334.2% 6023.00 NFS-thresh=10M/nfs-10dd-4k-32p-32768M-10M:10-X
1081.00 +280.3% 4111.00 NFS-thresh=10M/nfs-1dd-4k-32p-32768M-10M:10-X
930.00 +368.0% 4352.00 NFS-thresh=10M/nfs-2dd-4k-32p-32768M-10M:10-X
254.00 +108.7% 530.00 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
38.00 +55.3% 59.00 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
54.00 +96.3% 106.00 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
1321.00 -74.9% 332.00 NFS-thresh=1M/nfs-10dd-4k-32p-32768M-1M:10-X
1932.00 -99.1% 17.00 NFS-thresh=1M/nfs-1dd-4k-32p-32768M-1M:10-X
2074.00 -93.7% 130.00 NFS-thresh=1M/nfs-2dd-4k-32p-32768M-1M:10-X
10498.00 +91.7% 20120.00 TOTAL nfs_nr_commits
3.1.0-rc8-vanilla+ 3.1.0-rc8-nfs-wq4+
------------------------ ------------------------
28359.00 -39.2% 17230.00 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
22241.00 +550.6% 144695.00 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
24969.00 +27.8% 31900.00 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
21722.00 +38.2% 30030.00 NFS-thresh=10M/nfs-10dd-4k-32p-32768M-10M:10-X
11015.00 +28.2% 14117.00 NFS-thresh=10M/nfs-1dd-4k-32p-32768M-10M:10-X
17012.00 +217.7% 54039.00 NFS-thresh=10M/nfs-2dd-4k-32p-32768M-10M:10-X
25616.00 +3.1% 26403.00 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
24761.00 +177.5% 68702.00 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
29235.00 +37.1% 40089.00 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
12929.00 +21.6% 15720.00 NFS-thresh=1M/nfs-10dd-4k-32p-32768M-1M:10-X
7683.00 +24.2% 9542.00 NFS-thresh=1M/nfs-1dd-4k-32p-32768M-1M:10-X
7471.00 +77.8% 13284.00 NFS-thresh=1M/nfs-2dd-4k-32p-32768M-1M:10-X
233013.00 +99.9% 465751.00 TOTAL nfs_nr_writes
3.1.0-rc8-vanilla+ 3.1.0-rc8-nfs-wq4+
------------------------ ------------------------
7.84 -38.6% 4.81 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
49.58 -41.6% 28.94 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
30.58 -30.5% 21.27 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
2.99 -63.9% 1.08 NFS-thresh=10M/nfs-10dd-4k-32p-32768M-10M:10-X
8.06 -73.8% 2.12 NFS-thresh=10M/nfs-1dd-4k-32p-32768M-10M:10-X
5.33 -58.9% 2.19 NFS-thresh=10M/nfs-2dd-4k-32p-32768M-10M:10-X
57.68 -32.1% 39.15 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
465.15 -16.5% 388.43 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
261.60 -16.4% 218.80 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
2.25 +163.5% 5.93 NFS-thresh=1M/nfs-10dd-4k-32p-32768M-1M:10-X
2.13 +9221.2% 198.29 NFS-thresh=1M/nfs-1dd-4k-32p-32768M-1M:10-X
2.27 +455.3% 12.61 NFS-thresh=1M/nfs-2dd-4k-32p-32768M-1M:10-X
895.47 +3.1% 923.62 TOTAL nfs_commit_size
3.1.0-rc8-vanilla+ 3.1.0-rc8-nfs-wq4+
------------------------ ------------------------
0.23 +227.7% 0.76 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
0.69 -78.1% 0.15 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
0.35 +92.4% 0.66 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
0.19 +13.4% 0.22 NFS-thresh=10M/nfs-10dd-4k-32p-32768M-10M:10-X
0.79 -22.2% 0.62 NFS-thresh=10M/nfs-1dd-4k-32p-32768M-10M:10-X
0.29 -39.5% 0.18 NFS-thresh=10M/nfs-2dd-4k-32p-32768M-10M:10-X
0.57 +37.4% 0.79 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
0.71 -53.3% 0.33 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
0.48 +19.7% 0.58 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
0.23 -45.5% 0.13 NFS-thresh=1M/nfs-10dd-4k-32p-32768M-1M:10-X
0.53 -34.0% 0.35 NFS-thresh=1M/nfs-1dd-4k-32p-32768M-1M:10-X
0.63 -80.4% 0.12 NFS-thresh=1M/nfs-2dd-4k-32p-32768M-1M:10-X
5.71 -14.5% 4.88 TOTAL nfs_write_size
3.1.0-rc8-vanilla+ 3.1.0-rc8-nfs-wq4+
------------------------ ------------------------
6544.25 -95.1% 321.04 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
1064.82 +11.2% 1184.16 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
22801.48 -86.3% 3113.39 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
1083.47 -99.8% 2.56 NFS-thresh=10M/nfs-10dd-4k-32p-32768M-10M:10-X
3.82 -55.8% 1.69 NFS-thresh=10M/nfs-1dd-4k-32p-32768M-10M:10-X
2840.08 -99.3% 20.09 NFS-thresh=10M/nfs-2dd-4k-32p-32768M-10M:10-X
20227.73 -96.6% 683.65 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
2346.04 +274.0% 8774.87 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
50812.68 -94.3% 2901.88 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
417.03 -99.9% 0.25 NFS-thresh=1M/nfs-10dd-4k-32p-32768M-1M:10-X
1.70 -97.9% 0.04 NFS-thresh=1M/nfs-1dd-4k-32p-32768M-1M:10-X
126.23 -99.9% 0.08 NFS-thresh=1M/nfs-2dd-4k-32p-32768M-1M:10-X
108269.33 -84.3% 17003.69 TOTAL nfs_write_queue_time
3.1.0-rc8-vanilla+ 3.1.0-rc8-nfs-wq4+
------------------------ ------------------------
276.99 -41.1% 163.20 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
106.71 -67.0% 35.21 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
76.32 +13.4% 86.53 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
335.96 -41.8% 195.49 NFS-thresh=10M/nfs-10dd-4k-32p-32768M-10M:10-X
33.67 +70.7% 57.48 NFS-thresh=10M/nfs-1dd-4k-32p-32768M-10M:10-X
159.03 -46.0% 85.80 NFS-thresh=10M/nfs-2dd-4k-32p-32768M-10M:10-X
340.25 -67.6% 110.23 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
47.88 +12.7% 53.96 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
118.13 -53.8% 54.62 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
223.24 -53.0% 104.83 NFS-thresh=1M/nfs-10dd-4k-32p-32768M-1M:10-X
58.89 +12.8% 66.43 NFS-thresh=1M/nfs-1dd-4k-32p-32768M-1M:10-X
58.97 +223.0% 190.49 NFS-thresh=1M/nfs-2dd-4k-32p-32768M-1M:10-X
1836.03 -34.4% 1204.27 TOTAL nfs_write_rtt_time
3.1.0-rc8-vanilla+ 3.1.0-rc8-nfs-wq4+
------------------------ ------------------------
6821.43 -92.9% 484.70 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
1173.98 +4.0% 1220.80 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
22878.44 -86.0% 3201.00 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
1419.50 -86.0% 198.20 NFS-thresh=10M/nfs-10dd-4k-32p-32768M-10M:10-X
37.72 +57.1% 59.27 NFS-thresh=10M/nfs-1dd-4k-32p-32768M-10M:10-X
2999.22 -96.5% 106.41 NFS-thresh=10M/nfs-2dd-4k-32p-32768M-10M:10-X
20570.86 -96.1% 795.46 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
2416.09 +265.6% 8832.81 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
50941.27 -94.2% 2960.10 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
640.32 -83.6% 105.13 NFS-thresh=1M/nfs-10dd-4k-32p-32768M-1M:10-X
60.78 +9.4% 66.49 NFS-thresh=1M/nfs-1dd-4k-32p-32768M-1M:10-X
185.35 +2.8% 190.59 NFS-thresh=1M/nfs-2dd-4k-32p-32768M-1M:10-X
110144.96 -83.5% 18220.96 TOTAL nfs_write_execute_time
3.1.0-rc8-vanilla+ 3.1.0-rc8-nfs-wq4+
------------------------ ------------------------
54.75 -89.4% 5.82 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
88.26 -98.7% 1.12 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
38.41 -92.1% 3.05 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
7.59 -91.0% 0.68 NFS-thresh=10M/nfs-10dd-4k-32p-32768M-10M:10-X
0.42 -93.3% 0.03 NFS-thresh=10M/nfs-1dd-4k-32p-32768M-10M:10-X
2.57 -75.1% 0.64 NFS-thresh=10M/nfs-2dd-4k-32p-32768M-10M:10-X
784.08 -93.8% 48.69 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
1338.39 -81.4% 248.51 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
586.69 -96.0% 23.32 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
1.27 -84.3% 0.20 NFS-thresh=1M/nfs-10dd-4k-32p-32768M-1M:10-X
0.02 +147.1% 0.06 NFS-thresh=1M/nfs-1dd-4k-32p-32768M-1M:10-X
0.16 -41.3% 0.09 NFS-thresh=1M/nfs-2dd-4k-32p-32768M-1M:10-X
2902.62 -88.6% 332.20 TOTAL nfs_commit_queue_time
3.1.0-rc8-vanilla+ 3.1.0-rc8-nfs-wq4+
------------------------ ------------------------
702.80 +8.2% 760.66 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
538.99 -35.8% 346.08 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
704.42 -37.1% 443.00 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
228.96 -18.4% 186.78 NFS-thresh=10M/nfs-10dd-4k-32p-32768M-10M:10-X
155.88 -54.6% 70.75 NFS-thresh=10M/nfs-1dd-4k-32p-32768M-10M:10-X
169.51 -28.9% 120.53 NFS-thresh=10M/nfs-2dd-4k-32p-32768M-10M:10-X
3791.44 -11.4% 3361.05 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
4229.79 -17.8% 3476.80 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
5534.04 -35.4% 3574.73 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
96.34 -31.4% 66.11 NFS-thresh=1M/nfs-10dd-4k-32p-32768M-1M:10-X
60.95 -35.5% 39.29 NFS-thresh=1M/nfs-1dd-4k-32p-32768M-1M:10-X
69.64 -35.3% 45.08 NFS-thresh=1M/nfs-2dd-4k-32p-32768M-1M:10-X
16282.75 -23.3% 12490.87 TOTAL nfs_commit_rtt_time
3.1.0-rc8-vanilla+ 3.1.0-rc8-nfs-wq4+
------------------------ ------------------------
757.92 +1.2% 766.73 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
627.36 -44.6% 347.25 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
743.59 -40.0% 446.39 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
236.73 -20.8% 187.57 NFS-thresh=10M/nfs-10dd-4k-32p-32768M-10M:10-X
156.31 -54.7% 70.79 NFS-thresh=10M/nfs-1dd-4k-32p-32768M-10M:10-X
172.16 -29.6% 121.26 NFS-thresh=10M/nfs-2dd-4k-32p-32768M-10M:10-X
4579.56 -25.5% 3411.34 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
5568.53 -33.1% 3725.49 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
6163.54 -41.5% 3605.27 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
97.67 -32.1% 66.35 NFS-thresh=1M/nfs-10dd-4k-32p-32768M-1M:10-X
60.99 -35.5% 39.35 NFS-thresh=1M/nfs-1dd-4k-32p-32768M-1M:10-X
69.82 -35.3% 45.20 NFS-thresh=1M/nfs-2dd-4k-32p-32768M-1M:10-X
19234.16 -33.3% 12833.00 TOTAL nfs_commit_execute_time
---
Subject: nfs: fix a deadlock in nfs writeback path
Date: Tue Oct 18 16:49:19 CST 2011
From: "Tang, Feng" <feng.tang@intel.com>
In a corner case where nfs_congestion_kb is set very small, there
will be a deadlock happens in nfs_writepages():
err = write_cache_pages(mapping, wbc, nfs_writepages_callback, &pgio);
/* this func may be congested when calling nfs_writepages_callback, before
* the real pageio req is really issued, thus get blocked for ever */
nfs_pageio_complete(&pgio);
So move the nfs_wait_congested() after nfs_pageio_complet(&pgio) will fix
the issue, which also is more efficient in calling nfs_wait_congested()
per inode instead of per dirty page of that inode.
Suggested-by: Wu Fengguang <fengguang.wu@intel.com>
Signed-off-by: Feng Tang <feng.tang@intel.com>
Signed-off-by: Wu Fengguang <fengguang.wu@intel.com>
---
fs/nfs/write.c | 11 +++++------
1 file changed, 5 insertions(+), 6 deletions(-)
--- linux-next.orig/fs/nfs/write.c 2011-10-17 16:07:24.000000000 +0800
+++ linux-next/fs/nfs/write.c 2011-10-18 16:49:17.000000000 +0800
@@ -391,23 +391,18 @@ int nfs_writepage(struct page *page, str
static int nfs_writepages_callback(struct page *page,
struct writeback_control *wbc, void *data)
{
- struct inode *inode = page->mapping->host;
- struct nfs_server *nfss = NFS_SERVER(inode);
int ret;
ret = nfs_do_writepage(page, wbc, data);
unlock_page(page);
- nfs_wait_congested(wbc->sync_mode == WB_SYNC_ALL,
- &nfss->backing_dev_info,
- nfss->writeback_wait);
-
return ret;
}
int nfs_writepages(struct address_space *mapping, struct writeback_control *wbc)
{
struct inode *inode = mapping->host;
+ struct nfs_server *nfss = NFS_SERVER(inode);
unsigned long *bitlock = &NFS_I(inode)->flags;
struct nfs_pageio_descriptor pgio;
int err;
@@ -424,6 +419,10 @@ int nfs_writepages(struct address_space
err = write_cache_pages(mapping, wbc, nfs_writepages_callback, &pgio);
nfs_pageio_complete(&pgio);
+ nfs_wait_congested(wbc->sync_mode == WB_SYNC_ALL,
+ &nfss->backing_dev_info,
+ nfss->writeback_wait);
+
clear_bit_unlock(NFS_INO_FLUSHING, bitlock);
smp_mb__after_clear_bit();
wake_up_bit(bitlock, NFS_INO_FLUSHING);
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [RFC][PATCH 2/2] nfs: scale writeback threshold proportional to dirty threshold
2011-10-10 13:11 ` [RFC][PATCH 2/2] nfs: scale writeback threshold proportional to dirty threshold Wu Fengguang
@ 2011-10-18 8:53 ` Wu Fengguang
2011-10-18 8:59 ` Wu Fengguang
0 siblings, 1 reply; 10+ messages in thread
From: Wu Fengguang @ 2011-10-18 8:53 UTC (permalink / raw)
To: Trond Myklebust, linux-nfs
Cc: Peter Zijlstra, linux-fsdevel@vger.kernel.org, Andrew Morton,
Jan Kara, Christoph Hellwig, Dave Chinner, Greg Thelen,
Minchan Kim, Vivek Goyal, Andrea Righi, linux-mm, LKML
An update from Feng:
Subject: nfs: fix a bug about adjusting nfs_congestion_kb
Date: Tue Oct 18 12:47:58 CST 2011
From: "Tang, Feng" <feng.tang@intel.com>
The VM dirty_thresh may be set to very small(even 0) by wired user, in
such case, nfs_congestion_kb may be adjusted to 0, will cause the normal
NFS write function get congested and deaklocked. So let's set the bottom
line of nfs_congestion_kb to 128kb.
Signed-off-by: Feng Tang <feng.tang@intel.com>
Signed-off-by: Wu Fengguang <fengguang.wu@intel.com>
---
fs/nfs/write.c | 1 +
1 file changed, 1 insertion(+)
--- linux-next.orig/fs/nfs/write.c 2011-10-17 16:07:40.000000000 +0800
+++ linux-next/fs/nfs/write.c 2011-10-18 12:47:46.000000000 +0800
@@ -1814,6 +1814,7 @@ void nfs_update_congestion_thresh(void)
*/
global_dirty_limits(&background_thresh, &dirty_thresh);
dirty_thresh <<= PAGE_SHIFT - 10;
+ dirty_thresh += 1024;
if (nfs_congestion_kb > dirty_thresh / 8)
nfs_congestion_kb = dirty_thresh / 8;
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [RFC][PATCH 2/2] nfs: scale writeback threshold proportional to dirty threshold
2011-10-18 8:53 ` Wu Fengguang
@ 2011-10-18 8:59 ` Wu Fengguang
2011-10-20 2:49 ` Wu Fengguang
0 siblings, 1 reply; 10+ messages in thread
From: Wu Fengguang @ 2011-10-18 8:59 UTC (permalink / raw)
To: Trond Myklebust, linux-nfs
Cc: Peter Zijlstra, linux-fsdevel@vger.kernel.org, Andrew Morton,
Jan Kara, Christoph Hellwig, Dave Chinner, Greg Thelen,
Minchan Kim, Vivek Goyal, Andrea Righi, linux-mm, LKML, Tang Feng
> @@ -1814,6 +1814,7 @@ void nfs_update_congestion_thresh(void)
> */
> global_dirty_limits(&background_thresh, &dirty_thresh);
> dirty_thresh <<= PAGE_SHIFT - 10;
> + dirty_thresh += 1024;
>
> if (nfs_congestion_kb > dirty_thresh / 8)
> nfs_congestion_kb = dirty_thresh / 8;
Here are the new results with ioless + this fix + fix to patch 1.
3.1.0-rc8-vanilla+ 3.1.0-rc8-nfs-wq4+
------------------------ ------------------------
354.65 +45.4% 515.48 TOTAL write_bw
10498.00 +91.7% 20120.00 TOTAL nfs_nr_commits
233013.00 +99.9% 465751.00 TOTAL nfs_nr_writes
895.47 +3.1% 923.62 TOTAL nfs_commit_size
5.71 -14.5% 4.88 TOTAL nfs_write_size
108269.33 -84.3% 17003.69 TOTAL nfs_write_queue_time
1836.03 -34.4% 1204.27 TOTAL nfs_write_rtt_time
110144.96 -83.5% 18220.96 TOTAL nfs_write_execute_time
2902.62 -88.6% 332.20 TOTAL nfs_commit_queue_time
16282.75 -23.3% 12490.87 TOTAL nfs_commit_rtt_time
19234.16 -33.3% 12833.00 TOTAL nfs_commit_execute_time
before/after this single fix:
(performance is mostly unchanged except for the thresh=1M cases)
3.1.0-rc8-nfs-wq3+ 3.1.0-rc8-nfs-wq4+
------------------------ ------------------------
41.67 +3.8% 43.23 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
77.49 -5.5% 73.26 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
69.31 +2.0% 70.68 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
21.99 -1.8% 21.59 NFS-thresh=10M/nfs-10dd-4k-32p-32768M-10M:10-X
29.03 -0.0% 29.02 NFS-thresh=10M/nfs-1dd-4k-32p-32768M-10M:10-X
32.51 -2.2% 31.78 NFS-thresh=10M/nfs-2dd-4k-32p-32768M-10M:10-X
69.78 -1.1% 69.01 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
73.39 +4.6% 76.74 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
80.84 -4.9% 76.87 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
5.78 +13.7% 6.58 NFS-thresh=1M/nfs-10dd-4k-32p-32768M-1M:10-X
11.10 +1.3% 11.24 NFS-thresh=1M/nfs-1dd-4k-32p-32768M-1M:10-X
7.60 -27.9% 5.48 NFS-thresh=1M/nfs-2dd-4k-32p-32768M-1M:10-X
520.49 -1.0% 515.48 TOTAL write_bw
More detailed comparison to vanilla kernel:
3.1.0-rc8-vanilla+ 3.1.0-rc8-nfs-wq4+
------------------------ ------------------------
21.85 +97.9% 43.23 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
51.38 +42.6% 73.26 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
28.81 +145.3% 70.68 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
13.74 +57.1% 21.59 NFS-thresh=10M/nfs-10dd-4k-32p-32768M-10M:10-X
29.11 -0.3% 29.02 NFS-thresh=10M/nfs-1dd-4k-32p-32768M-10M:10-X
16.68 +90.5% 31.78 NFS-thresh=10M/nfs-2dd-4k-32p-32768M-10M:10-X
48.88 +41.2% 69.01 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
57.85 +32.7% 76.74 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
47.13 +63.1% 76.87 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
9.82 -33.0% 6.58 NFS-thresh=1M/nfs-10dd-4k-32p-32768M-1M:10-X
13.72 -18.1% 11.24 NFS-thresh=1M/nfs-1dd-4k-32p-32768M-1M:10-X
15.68 -65.0% 5.48 NFS-thresh=1M/nfs-2dd-4k-32p-32768M-1M:10-X
354.65 +45.4% 515.48 TOTAL write_bw
3.1.0-rc8-vanilla+ 3.1.0-rc8-nfs-wq4+
------------------------ ------------------------
834.00 +224.2% 2704.00 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
311.00 +144.1% 759.00 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
282.00 +253.5% 997.00 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
1387.00 +334.2% 6023.00 NFS-thresh=10M/nfs-10dd-4k-32p-32768M-10M:10-X
1081.00 +280.3% 4111.00 NFS-thresh=10M/nfs-1dd-4k-32p-32768M-10M:10-X
930.00 +368.0% 4352.00 NFS-thresh=10M/nfs-2dd-4k-32p-32768M-10M:10-X
254.00 +108.7% 530.00 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
38.00 +55.3% 59.00 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
54.00 +96.3% 106.00 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
1321.00 -74.9% 332.00 NFS-thresh=1M/nfs-10dd-4k-32p-32768M-1M:10-X
1932.00 -99.1% 17.00 NFS-thresh=1M/nfs-1dd-4k-32p-32768M-1M:10-X
2074.00 -93.7% 130.00 NFS-thresh=1M/nfs-2dd-4k-32p-32768M-1M:10-X
10498.00 +91.7% 20120.00 TOTAL nfs_nr_commits
3.1.0-rc8-vanilla+ 3.1.0-rc8-nfs-wq4+
------------------------ ------------------------
28359.00 -39.2% 17230.00 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
22241.00 +550.6% 144695.00 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
24969.00 +27.8% 31900.00 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
21722.00 +38.2% 30030.00 NFS-thresh=10M/nfs-10dd-4k-32p-32768M-10M:10-X
11015.00 +28.2% 14117.00 NFS-thresh=10M/nfs-1dd-4k-32p-32768M-10M:10-X
17012.00 +217.7% 54039.00 NFS-thresh=10M/nfs-2dd-4k-32p-32768M-10M:10-X
25616.00 +3.1% 26403.00 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
24761.00 +177.5% 68702.00 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
29235.00 +37.1% 40089.00 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
12929.00 +21.6% 15720.00 NFS-thresh=1M/nfs-10dd-4k-32p-32768M-1M:10-X
7683.00 +24.2% 9542.00 NFS-thresh=1M/nfs-1dd-4k-32p-32768M-1M:10-X
7471.00 +77.8% 13284.00 NFS-thresh=1M/nfs-2dd-4k-32p-32768M-1M:10-X
233013.00 +99.9% 465751.00 TOTAL nfs_nr_writes
3.1.0-rc8-vanilla+ 3.1.0-rc8-nfs-wq4+
------------------------ ------------------------
7.84 -38.6% 4.81 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
49.58 -41.6% 28.94 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
30.58 -30.5% 21.27 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
2.99 -63.9% 1.08 NFS-thresh=10M/nfs-10dd-4k-32p-32768M-10M:10-X
8.06 -73.8% 2.12 NFS-thresh=10M/nfs-1dd-4k-32p-32768M-10M:10-X
5.33 -58.9% 2.19 NFS-thresh=10M/nfs-2dd-4k-32p-32768M-10M:10-X
57.68 -32.1% 39.15 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
465.15 -16.5% 388.43 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
261.60 -16.4% 218.80 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
2.25 +163.5% 5.93 NFS-thresh=1M/nfs-10dd-4k-32p-32768M-1M:10-X
2.13 +9221.2% 198.29 NFS-thresh=1M/nfs-1dd-4k-32p-32768M-1M:10-X
2.27 +455.3% 12.61 NFS-thresh=1M/nfs-2dd-4k-32p-32768M-1M:10-X
895.47 +3.1% 923.62 TOTAL nfs_commit_size
3.1.0-rc8-vanilla+ 3.1.0-rc8-nfs-wq4+
------------------------ ------------------------
0.23 +227.7% 0.76 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
0.69 -78.1% 0.15 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
0.35 +92.4% 0.66 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
0.19 +13.4% 0.22 NFS-thresh=10M/nfs-10dd-4k-32p-32768M-10M:10-X
0.79 -22.2% 0.62 NFS-thresh=10M/nfs-1dd-4k-32p-32768M-10M:10-X
0.29 -39.5% 0.18 NFS-thresh=10M/nfs-2dd-4k-32p-32768M-10M:10-X
0.57 +37.4% 0.79 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
0.71 -53.3% 0.33 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
0.48 +19.7% 0.58 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
0.23 -45.5% 0.13 NFS-thresh=1M/nfs-10dd-4k-32p-32768M-1M:10-X
0.53 -34.0% 0.35 NFS-thresh=1M/nfs-1dd-4k-32p-32768M-1M:10-X
0.63 -80.4% 0.12 NFS-thresh=1M/nfs-2dd-4k-32p-32768M-1M:10-X
5.71 -14.5% 4.88 TOTAL nfs_write_size
3.1.0-rc8-vanilla+ 3.1.0-rc8-nfs-wq4+
------------------------ ------------------------
6544.25 -95.1% 321.04 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
1064.82 +11.2% 1184.16 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
22801.48 -86.3% 3113.39 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
1083.47 -99.8% 2.56 NFS-thresh=10M/nfs-10dd-4k-32p-32768M-10M:10-X
3.82 -55.8% 1.69 NFS-thresh=10M/nfs-1dd-4k-32p-32768M-10M:10-X
2840.08 -99.3% 20.09 NFS-thresh=10M/nfs-2dd-4k-32p-32768M-10M:10-X
20227.73 -96.6% 683.65 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
2346.04 +274.0% 8774.87 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
50812.68 -94.3% 2901.88 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
417.03 -99.9% 0.25 NFS-thresh=1M/nfs-10dd-4k-32p-32768M-1M:10-X
1.70 -97.9% 0.04 NFS-thresh=1M/nfs-1dd-4k-32p-32768M-1M:10-X
126.23 -99.9% 0.08 NFS-thresh=1M/nfs-2dd-4k-32p-32768M-1M:10-X
108269.33 -84.3% 17003.69 TOTAL nfs_write_queue_time
3.1.0-rc8-vanilla+ 3.1.0-rc8-nfs-wq4+
------------------------ ------------------------
276.99 -41.1% 163.20 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
106.71 -67.0% 35.21 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
76.32 +13.4% 86.53 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
335.96 -41.8% 195.49 NFS-thresh=10M/nfs-10dd-4k-32p-32768M-10M:10-X
33.67 +70.7% 57.48 NFS-thresh=10M/nfs-1dd-4k-32p-32768M-10M:10-X
159.03 -46.0% 85.80 NFS-thresh=10M/nfs-2dd-4k-32p-32768M-10M:10-X
340.25 -67.6% 110.23 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
47.88 +12.7% 53.96 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
118.13 -53.8% 54.62 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
223.24 -53.0% 104.83 NFS-thresh=1M/nfs-10dd-4k-32p-32768M-1M:10-X
58.89 +12.8% 66.43 NFS-thresh=1M/nfs-1dd-4k-32p-32768M-1M:10-X
58.97 +223.0% 190.49 NFS-thresh=1M/nfs-2dd-4k-32p-32768M-1M:10-X
1836.03 -34.4% 1204.27 TOTAL nfs_write_rtt_time
3.1.0-rc8-vanilla+ 3.1.0-rc8-nfs-wq4+
------------------------ ------------------------
6821.43 -92.9% 484.70 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
1173.98 +4.0% 1220.80 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
22878.44 -86.0% 3201.00 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
1419.50 -86.0% 198.20 NFS-thresh=10M/nfs-10dd-4k-32p-32768M-10M:10-X
37.72 +57.1% 59.27 NFS-thresh=10M/nfs-1dd-4k-32p-32768M-10M:10-X
2999.22 -96.5% 106.41 NFS-thresh=10M/nfs-2dd-4k-32p-32768M-10M:10-X
20570.86 -96.1% 795.46 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
2416.09 +265.6% 8832.81 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
50941.27 -94.2% 2960.10 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
640.32 -83.6% 105.13 NFS-thresh=1M/nfs-10dd-4k-32p-32768M-1M:10-X
60.78 +9.4% 66.49 NFS-thresh=1M/nfs-1dd-4k-32p-32768M-1M:10-X
185.35 +2.8% 190.59 NFS-thresh=1M/nfs-2dd-4k-32p-32768M-1M:10-X
110144.96 -83.5% 18220.96 TOTAL nfs_write_execute_time
3.1.0-rc8-vanilla+ 3.1.0-rc8-nfs-wq4+
------------------------ ------------------------
54.75 -89.4% 5.82 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
88.26 -98.7% 1.12 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
38.41 -92.1% 3.05 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
7.59 -91.0% 0.68 NFS-thresh=10M/nfs-10dd-4k-32p-32768M-10M:10-X
0.42 -93.3% 0.03 NFS-thresh=10M/nfs-1dd-4k-32p-32768M-10M:10-X
2.57 -75.1% 0.64 NFS-thresh=10M/nfs-2dd-4k-32p-32768M-10M:10-X
784.08 -93.8% 48.69 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
1338.39 -81.4% 248.51 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
586.69 -96.0% 23.32 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
1.27 -84.3% 0.20 NFS-thresh=1M/nfs-10dd-4k-32p-32768M-1M:10-X
0.02 +147.1% 0.06 NFS-thresh=1M/nfs-1dd-4k-32p-32768M-1M:10-X
0.16 -41.3% 0.09 NFS-thresh=1M/nfs-2dd-4k-32p-32768M-1M:10-X
2902.62 -88.6% 332.20 TOTAL nfs_commit_queue_time
3.1.0-rc8-vanilla+ 3.1.0-rc8-nfs-wq4+
------------------------ ------------------------
702.80 +8.2% 760.66 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
538.99 -35.8% 346.08 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
704.42 -37.1% 443.00 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
228.96 -18.4% 186.78 NFS-thresh=10M/nfs-10dd-4k-32p-32768M-10M:10-X
155.88 -54.6% 70.75 NFS-thresh=10M/nfs-1dd-4k-32p-32768M-10M:10-X
169.51 -28.9% 120.53 NFS-thresh=10M/nfs-2dd-4k-32p-32768M-10M:10-X
3791.44 -11.4% 3361.05 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
4229.79 -17.8% 3476.80 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
5534.04 -35.4% 3574.73 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
96.34 -31.4% 66.11 NFS-thresh=1M/nfs-10dd-4k-32p-32768M-1M:10-X
60.95 -35.5% 39.29 NFS-thresh=1M/nfs-1dd-4k-32p-32768M-1M:10-X
69.64 -35.3% 45.08 NFS-thresh=1M/nfs-2dd-4k-32p-32768M-1M:10-X
16282.75 -23.3% 12490.87 TOTAL nfs_commit_rtt_time
3.1.0-rc8-vanilla+ 3.1.0-rc8-nfs-wq4+
------------------------ ------------------------
757.92 +1.2% 766.73 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
627.36 -44.6% 347.25 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
743.59 -40.0% 446.39 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
236.73 -20.8% 187.57 NFS-thresh=10M/nfs-10dd-4k-32p-32768M-10M:10-X
156.31 -54.7% 70.79 NFS-thresh=10M/nfs-1dd-4k-32p-32768M-10M:10-X
172.16 -29.6% 121.26 NFS-thresh=10M/nfs-2dd-4k-32p-32768M-10M:10-X
4579.56 -25.5% 3411.34 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
5568.53 -33.1% 3725.49 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
6163.54 -41.5% 3605.27 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
97.67 -32.1% 66.35 NFS-thresh=1M/nfs-10dd-4k-32p-32768M-1M:10-X
60.99 -35.5% 39.35 NFS-thresh=1M/nfs-1dd-4k-32p-32768M-1M:10-X
69.82 -35.3% 45.20 NFS-thresh=1M/nfs-2dd-4k-32p-32768M-1M:10-X
19234.16 -33.3% 12833.00 TOTAL nfs_commit_execute_time
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [RFC][PATCH 2/2] nfs: scale writeback threshold proportional to dirty threshold
2011-10-18 8:59 ` Wu Fengguang
@ 2011-10-20 2:49 ` Wu Fengguang
0 siblings, 0 replies; 10+ messages in thread
From: Wu Fengguang @ 2011-10-20 2:49 UTC (permalink / raw)
To: Trond Myklebust, linux-nfs
Cc: Peter Zijlstra, linux-fsdevel@vger.kernel.org, Andrew Morton,
Jan Kara, Christoph Hellwig, Dave Chinner, Greg Thelen,
Minchan Kim, Vivek Goyal, Andrea Righi, linux-mm, LKML, Tang Feng
> > if (nfs_congestion_kb > dirty_thresh / 8)
> > nfs_congestion_kb = dirty_thresh / 8;
To confirm whether that's a good threshold, I tried double it by
using "/ 4". It results in -8.4% overall throughput regression. So
I'll stick with the current "/ 8".
Thanks,
Fengguang
---
3.1.0-rc9-ioless-full-next-20111014+ 3.1.0-rc9-ioless-full-nfs-thresh-4-next-20111014+
------------------------ ------------------------
472.17 -8.4% 432.32 TOTAL write_bw
15776.00 -27.3% 11466.00 TOTAL nfs_nr_commits
401731.00 +75.5% 705206.00 TOTAL nfs_nr_writes
918.32 +22.4% 1124.07 TOTAL nfs_commit_size
4.97 -25.3% 3.71 TOTAL nfs_write_size
14098.08 -30.6% 9788.99 TOTAL nfs_write_queue_time
1314.33 +55.3% 2041.51 TOTAL nfs_write_rtt_time
15438.68 -23.2% 11851.79 TOTAL nfs_write_execute_time
177.75 +73.1% 307.68 TOTAL nfs_commit_queue_time
15026.97 -3.6% 14491.26 TOTAL nfs_commit_rtt_time
15227.06 -2.7% 14809.49 TOTAL nfs_commit_execute_time
3.1.0-rc9-ioless-full-next-20111014+ 3.1.0-rc9-ioless-full-nfs-thresh-4-next-20111014+
------------------------ ------------------------
44.49 -7.7% 41.08 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
62.35 -1.7% 61.28 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
67.51 -10.0% 60.76 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
22.90 -30.0% 16.03 NFS-thresh=10M/nfs-10dd-4k-32p-32768M-10M:10-X
29.91 -16.9% 24.87 NFS-thresh=10M/nfs-1dd-4k-32p-32768M-10M:10-X
26.51 -20.0% 21.20 NFS-thresh=10M/nfs-2dd-4k-32p-32768M-10M:10-X
68.96 -1.2% 68.13 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
60.08 -17.9% 49.30 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
70.55 +4.1% 73.44 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
5.63 -16.9% 4.68 NFS-thresh=1M/nfs-10dd-4k-32p-32768M-1M:10-X
7.91 -2.4% 7.71 NFS-thresh=1M/nfs-1dd-4k-32p-32768M-1M:10-X
5.38 -28.7% 3.84 NFS-thresh=1M/nfs-2dd-4k-32p-32768M-1M:10-X
472.17 -8.4% 432.32 TOTAL write_bw
3.1.0-rc9-ioless-full-next-20111014+ 3.1.0-rc9-ioless-full-nfs-thresh-4-next-20111014+
------------------------ ------------------------
2565.00 -11.6% 2267.00 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
384.00 +1.8% 391.00 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
866.00 -9.9% 780.00 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
5801.00 -41.0% 3423.00 NFS-thresh=10M/nfs-10dd-4k-32p-32768M-10M:10-X
2092.00 -27.6% 1515.00 NFS-thresh=10M/nfs-1dd-4k-32p-32768M-10M:10-X
3231.00 -24.2% 2450.00 NFS-thresh=10M/nfs-2dd-4k-32p-32768M-10M:10-X
408.00 -2.9% 396.00 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
46.00 -4.3% 44.00 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
86.00 +9.3% 94.00 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
138.00 -51.4% 67.00 NFS-thresh=1M/nfs-10dd-4k-32p-32768M-1M:10-X
119.00 -91.6% 10.00 NFS-thresh=1M/nfs-1dd-4k-32p-32768M-1M:10-X
40.00 -27.5% 29.00 NFS-thresh=1M/nfs-2dd-4k-32p-32768M-1M:10-X
15776.00 -27.3% 11466.00 TOTAL nfs_nr_commits
3.1.0-rc9-ioless-full-next-20111014+ 3.1.0-rc9-ioless-full-nfs-thresh-4-next-20111014+
------------------------ ------------------------
17896.00 +30.5% 23348.00 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
62623.00 +371.7% 295418.00 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
37830.00 +37.4% 51991.00 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
27685.00 +13.1% 31312.00 NFS-thresh=10M/nfs-10dd-4k-32p-32768M-10M:10-X
51531.00 +7.4% 55330.00 NFS-thresh=10M/nfs-1dd-4k-32p-32768M-10M:10-X
84310.00 +14.8% 96814.00 NFS-thresh=10M/nfs-2dd-4k-32p-32768M-10M:10-X
21208.00 +0.8% 21375.00 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
26724.00 +5.1% 28092.00 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
31308.00 +67.6% 52461.00 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
12866.00 +13.9% 14649.00 NFS-thresh=1M/nfs-10dd-4k-32p-32768M-1M:10-X
12907.00 +9.2% 14088.00 NFS-thresh=1M/nfs-1dd-4k-32p-32768M-1M:10-X
14843.00 +37.0% 20328.00 NFS-thresh=1M/nfs-2dd-4k-32p-32768M-1M:10-X
401731.00 +75.5% 705206.00 TOTAL nfs_nr_writes
3.1.0-rc9-ioless-full-next-20111014+ 3.1.0-rc9-ioless-full-nfs-thresh-4-next-20111014+
------------------------ ------------------------
5.21 +5.2% 5.48 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
48.66 -3.4% 47.01 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
23.39 -0.1% 23.36 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
1.19 +18.7% 1.41 NFS-thresh=10M/nfs-10dd-4k-32p-32768M-10M:10-X
4.28 +15.1% 4.92 NFS-thresh=10M/nfs-1dd-4k-32p-32768M-10M:10-X
2.46 +5.5% 2.60 NFS-thresh=10M/nfs-2dd-4k-32p-32768M-10M:10-X
50.71 +1.7% 51.57 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
464.69 -0.5% 462.22 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
245.31 -4.8% 233.60 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
12.22 +71.4% 20.94 NFS-thresh=1M/nfs-10dd-4k-32p-32768M-1M:10-X
19.89 +1063.9% 231.46 NFS-thresh=1M/nfs-1dd-4k-32p-32768M-1M:10-X
40.33 -2.0% 39.52 NFS-thresh=1M/nfs-2dd-4k-32p-32768M-1M:10-X
918.32 +22.4% 1124.07 TOTAL nfs_commit_size
3.1.0-rc9-ioless-full-next-20111014+ 3.1.0-rc9-ioless-full-nfs-thresh-4-next-20111014+
------------------------ ------------------------
0.75 -28.7% 0.53 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
0.30 -79.1% 0.06 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
0.54 -34.6% 0.35 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
0.25 -38.1% 0.15 NFS-thresh=10M/nfs-10dd-4k-32p-32768M-10M:10-X
0.17 -22.4% 0.13 NFS-thresh=10M/nfs-1dd-4k-32p-32768M-10M:10-X
0.09 -30.3% 0.07 NFS-thresh=10M/nfs-2dd-4k-32p-32768M-10M:10-X
0.98 -2.1% 0.96 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
0.80 -9.5% 0.72 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
0.67 -37.9% 0.42 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
0.13 -26.9% 0.10 NFS-thresh=1M/nfs-10dd-4k-32p-32768M-1M:10-X
0.18 -10.4% 0.16 NFS-thresh=1M/nfs-1dd-4k-32p-32768M-1M:10-X
0.11 -48.1% 0.06 NFS-thresh=1M/nfs-2dd-4k-32p-32768M-1M:10-X
4.97 -25.3% 3.71 TOTAL nfs_write_size
3.1.0-rc9-ioless-full-next-20111014+ 3.1.0-rc9-ioless-full-nfs-thresh-4-next-20111014+
------------------------ ------------------------
383.37 -32.7% 258.17 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
5483.72 -67.9% 1759.47 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
3757.03 -36.1% 2399.96 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
2.61 +9692.1% 255.20 NFS-thresh=10M/nfs-10dd-4k-32p-32768M-10M:10-X
145.09 +277.8% 548.13 NFS-thresh=10M/nfs-1dd-4k-32p-32768M-10M:10-X
57.93 +584.8% 396.71 NFS-thresh=10M/nfs-2dd-4k-32p-32768M-10M:10-X
498.56 -1.1% 492.84 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
2475.43 +15.0% 2847.51 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
1293.33 -35.9% 829.56 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
0.20 +188.5% 0.59 NFS-thresh=1M/nfs-10dd-4k-32p-32768M-1M:10-X
0.56 -11.5% 0.49 NFS-thresh=1M/nfs-1dd-4k-32p-32768M-1M:10-X
0.25 +39.0% 0.35 NFS-thresh=1M/nfs-2dd-4k-32p-32768M-1M:10-X
14098.08 -30.6% 9788.99 TOTAL nfs_write_queue_time
3.1.0-rc9-ioless-full-next-20111014+ 3.1.0-rc9-ioless-full-nfs-thresh-4-next-20111014+
------------------------ ------------------------
156.19 +153.8% 396.37 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
47.14 -60.0% 18.86 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
74.76 +59.7% 119.40 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
176.05 +77.5% 312.45 NFS-thresh=10M/nfs-10dd-4k-32p-32768M-10M:10-X
88.28 +32.3% 116.82 NFS-thresh=10M/nfs-1dd-4k-32p-32768M-10M:10-X
107.48 +12.3% 120.71 NFS-thresh=10M/nfs-2dd-4k-32p-32768M-10M:10-X
129.32 -14.8% 110.23 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
85.62 +36.9% 117.24 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
58.52 -20.8% 46.34 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
118.53 +115.7% 255.64 NFS-thresh=1M/nfs-10dd-4k-32p-32768M-1M:10-X
112.27 +40.8% 158.12 NFS-thresh=1M/nfs-1dd-4k-32p-32768M-1M:10-X
160.16 +68.2% 269.33 NFS-thresh=1M/nfs-2dd-4k-32p-32768M-1M:10-X
1314.33 +55.3% 2041.51 TOTAL nfs_write_rtt_time
3.1.0-rc9-ioless-full-next-20111014+ 3.1.0-rc9-ioless-full-nfs-thresh-4-next-20111014+
------------------------ ------------------------
540.09 +21.2% 654.86 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
5533.64 -67.8% 1779.65 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
3833.18 -34.2% 2521.16 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
178.79 +217.6% 567.74 NFS-thresh=10M/nfs-10dd-4k-32p-32768M-10M:10-X
233.78 +184.6% 665.23 NFS-thresh=10M/nfs-1dd-4k-32p-32768M-10M:10-X
165.69 +212.4% 517.63 NFS-thresh=10M/nfs-2dd-4k-32p-32768M-10M:10-X
630.13 -4.0% 605.10 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
2574.27 +15.6% 2976.37 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
1357.04 -35.2% 879.46 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
118.76 +115.8% 256.26 NFS-thresh=1M/nfs-10dd-4k-32p-32768M-1M:10-X
112.86 +40.6% 158.64 NFS-thresh=1M/nfs-1dd-4k-32p-32768M-1M:10-X
160.43 +68.1% 269.70 NFS-thresh=1M/nfs-2dd-4k-32p-32768M-1M:10-X
15438.68 -23.2% 11851.79 TOTAL nfs_write_execute_time
3.1.0-rc9-ioless-full-next-20111014+ 3.1.0-rc9-ioless-full-nfs-thresh-4-next-20111014+
------------------------ ------------------------
4.15 -29.5% 2.92 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
5.79 +550.6% 37.64 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
5.30 +161.4% 13.86 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
0.63 +23.4% 0.78 NFS-thresh=10M/nfs-10dd-4k-32p-32768M-10M:10-X
0.77 +52.3% 1.17 NFS-thresh=10M/nfs-1dd-4k-32p-32768M-10M:10-X
0.54 +100.0% 1.08 NFS-thresh=10M/nfs-2dd-4k-32p-32768M-10M:10-X
66.40 +44.8% 96.17 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
25.80 +41.4% 36.48 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
67.99 +72.2% 117.05 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
0.19 +66.4% 0.31 NFS-thresh=1M/nfs-10dd-4k-32p-32768M-1M:10-X
0.10 0.00 NFS-thresh=1M/nfs-1dd-4k-32p-32768M-1M:10-X
0.10 +106.9% 0.21 NFS-thresh=1M/nfs-2dd-4k-32p-32768M-1M:10-X
177.75 +73.1% 307.68 TOTAL nfs_commit_queue_time
3.1.0-rc9-ioless-full-next-20111014+ 3.1.0-rc9-ioless-full-nfs-thresh-4-next-20111014+
------------------------ ------------------------
786.25 -7.3% 729.04 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
523.35 +4.8% 548.62 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
487.58 +4.9% 511.71 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
181.60 +22.2% 221.92 NFS-thresh=10M/nfs-10dd-4k-32p-32768M-10M:10-X
110.03 +13.2% 124.60 NFS-thresh=10M/nfs-1dd-4k-32p-32768M-10M:10-X
128.24 +11.0% 142.35 NFS-thresh=10M/nfs-2dd-4k-32p-32768M-10M:10-X
4352.76 -5.2% 4126.10 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
4022.83 -2.6% 3917.34 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
4282.52 -6.6% 3999.64 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
62.09 +56.1% 96.94 NFS-thresh=1M/nfs-10dd-4k-32p-32768M-1M:10-X
39.20 -22.5% 30.40 NFS-thresh=1M/nfs-1dd-4k-32p-32768M-1M:10-X
50.52 -15.7% 42.59 NFS-thresh=1M/nfs-2dd-4k-32p-32768M-1M:10-X
15026.97 -3.6% 14491.26 TOTAL nfs_commit_rtt_time
3.1.0-rc9-ioless-full-next-20111014+ 3.1.0-rc9-ioless-full-nfs-thresh-4-next-20111014+
------------------------ ------------------------
790.68 -7.4% 732.23 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
529.17 +10.8% 586.28 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
493.32 +6.6% 525.82 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
182.35 +22.2% 222.82 NFS-thresh=10M/nfs-10dd-4k-32p-32768M-10M:10-X
110.82 +13.5% 125.80 NFS-thresh=10M/nfs-1dd-4k-32p-32768M-10M:10-X
128.80 +11.4% 143.46 NFS-thresh=10M/nfs-2dd-4k-32p-32768M-10M:10-X
4420.89 -4.4% 4224.24 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
4048.67 -2.3% 3953.91 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
4370.05 -5.6% 4124.34 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
62.33 +56.1% 97.28 NFS-thresh=1M/nfs-10dd-4k-32p-32768M-1M:10-X
39.33 -22.4% 30.50 NFS-thresh=1M/nfs-1dd-4k-32p-32768M-1M:10-X
50.65 -15.5% 42.79 NFS-thresh=1M/nfs-2dd-4k-32p-32768M-1M:10-X
15227.06 -2.7% 14809.49 TOTAL nfs_commit_execute_time
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [RFC][PATCH 1/2] nfs: writeback pages wait queue
2011-10-18 8:51 ` [RFC][PATCH 1/2] nfs: writeback pages wait queue Wu Fengguang
@ 2011-10-20 3:59 ` Wu Fengguang
0 siblings, 0 replies; 10+ messages in thread
From: Wu Fengguang @ 2011-10-20 3:59 UTC (permalink / raw)
To: Trond Myklebust, linux-nfs
Cc: Peter Zijlstra, linux-fsdevel@vger.kernel.org, Andrew Morton,
Jan Kara, Christoph Hellwig, Dave Chinner, Greg Thelen,
Minchan Kim, Vivek Goyal, Andrea Righi, linux-mm, LKML
> @@ -424,6 +419,10 @@ int nfs_writepages(struct address_space
> err = write_cache_pages(mapping, wbc, nfs_writepages_callback, &pgio);
> nfs_pageio_complete(&pgio);
>
> + nfs_wait_congested(wbc->sync_mode == WB_SYNC_ALL,
> + &nfss->backing_dev_info,
> + nfss->writeback_wait);
> +
> clear_bit_unlock(NFS_INO_FLUSHING, bitlock);
> smp_mb__after_clear_bit();
> wake_up_bit(bitlock, NFS_INO_FLUSHING);
The "wakeup NFS_INO_FLUSHING after congestion wait" logic looks
strange, so I tried moving the nfs_wait_congested() _after_
wake_up_bit()...and got write_bw regressions.
OK, not knowing what's going on underneath, I'll just stick to the current form.
3.1.0-rc9-ioless-full-nfs-ino-flushing-next-20111014+ 3.1.0-rc9-ioless-full-nfs-wakeup-wait-next-20111014+
------------------------ ------------------------
417.26 -5.5% 394.24 TOTAL write_bw
5179.00 -12.6% 4529.00 TOTAL nfs_nr_commits
340466.00 -37.2% 213939.00 TOTAL nfs_nr_writes
722.54 +17.6% 849.42 TOTAL nfs_commit_size
3.75 +6.3% 3.99 TOTAL nfs_write_size
15477.38 -14.5% 13235.34 TOTAL nfs_write_queue_time
517.54 +13.0% 585.00 TOTAL nfs_write_rtt_time
16011.09 -13.5% 13848.09 TOTAL nfs_write_execute_time
714.65 -43.4% 404.65 TOTAL nfs_commit_queue_time
12787.93 +9.2% 13960.35 TOTAL nfs_commit_rtt_time
13519.94 +6.4% 14387.44 TOTAL nfs_commit_execute_time
3.1.0-rc9-ioless-full-nfs-ino-flushing-next-20111014+ 3.1.0-rc9-ioless-full-nfs-wakeup-wait-next-20111014+
------------------------ ------------------------
44.42 -0.8% 44.05 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
78.49 -8.0% 72.22 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
69.96 -2.4% 68.30 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
70.59 -3.8% 67.88 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
76.76 -8.7% 70.09 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
77.04 -6.9% 71.70 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
417.26 -5.5% 394.24 TOTAL write_bw
3.1.0-rc9-ioless-full-nfs-ino-flushing-next-20111014+ 3.1.0-rc9-ioless-full-nfs-wakeup-wait-next-20111014+
------------------------ ------------------------
2683.00 -1.8% 2634.00 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
811.00 -43.8% 456.00 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
1049.00 -16.7% 874.00 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
474.00 -8.4% 434.00 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
56.00 -23.2% 43.00 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
106.00 -17.0% 88.00 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
5179.00 -12.6% 4529.00 TOTAL nfs_nr_commits
3.1.0-rc9-ioless-full-nfs-ino-flushing-next-20111014+ 3.1.0-rc9-ioless-full-nfs-wakeup-wait-next-20111014+
------------------------ ------------------------
17641.00 -2.2% 17257.00 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
177296.00 -54.1% 81335.00 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
26346.00 +41.6% 37309.00 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
22279.00 +12.7% 25107.00 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
67612.00 -59.7% 27271.00 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
29292.00 -12.4% 25660.00 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
340466.00 -37.2% 213939.00 TOTAL nfs_nr_writes
3.1.0-rc9-ioless-full-nfs-ino-flushing-next-20111014+ 3.1.0-rc9-ioless-full-nfs-wakeup-wait-next-20111014+
------------------------ ------------------------
4.97 +1.1% 5.03 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
29.00 +63.5% 47.43 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
19.99 +17.0% 23.40 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
44.66 +5.3% 47.03 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
408.94 +18.6% 485.03 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
214.96 +12.3% 241.50 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
722.54 +17.6% 849.42 TOTAL nfs_commit_size
3.1.0-rc9-ioless-full-nfs-ino-flushing-next-20111014+ 3.1.0-rc9-ioless-full-nfs-wakeup-wait-next-20111014+
------------------------ ------------------------
0.76 +1.5% 0.77 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
0.13 +100.4% 0.27 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
0.80 -31.1% 0.55 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
0.95 -14.4% 0.81 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
0.34 +125.8% 0.76 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
0.78 +6.5% 0.83 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
3.75 +6.3% 3.99 TOTAL nfs_write_size
3.1.0-rc9-ioless-full-nfs-ino-flushing-next-20111014+ 3.1.0-rc9-ioless-full-nfs-wakeup-wait-next-20111014+
------------------------ ------------------------
460.14 -29.9% 322.63 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
718.69 +68.2% 1208.67 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
5203.60 -26.1% 3843.39 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
427.40 +93.4% 826.64 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
7369.68 -18.0% 6041.98 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
1297.87 -23.6% 992.02 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
15477.38 -14.5% 13235.34 TOTAL nfs_write_queue_time
3.1.0-rc9-ioless-full-nfs-ino-flushing-next-20111014+ 3.1.0-rc9-ioless-full-nfs-wakeup-wait-next-20111014+
------------------------ ------------------------
134.58 -8.9% 122.60 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
33.24 +48.8% 49.46 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
89.69 -19.4% 72.31 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
129.86 -35.3% 84.03 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
48.33 +239.4% 164.05 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
81.84 +13.1% 92.55 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
517.54 +13.0% 585.00 TOTAL nfs_write_rtt_time
3.1.0-rc9-ioless-full-nfs-ino-flushing-next-20111014+ 3.1.0-rc9-ioless-full-nfs-wakeup-wait-next-20111014+
------------------------ ------------------------
595.23 -25.1% 445.75 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
753.68 +67.4% 1261.46 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
5294.55 -26.0% 3917.11 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
559.44 +63.1% 912.46 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
7424.41 -16.2% 6221.64 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
1383.78 -21.3% 1089.67 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
16011.09 -13.5% 13848.09 TOTAL nfs_write_execute_time
3.1.0-rc9-ioless-full-nfs-ino-flushing-next-20111014+ 3.1.0-rc9-ioless-full-nfs-wakeup-wait-next-20111014+
------------------------ ------------------------
2.34 +2.5% 2.40 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
1.59 +488.4% 9.37 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
2.63 +47.0% 3.86 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
68.22 -6.5% 63.78 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
618.34 -52.9% 291.44 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
21.54 +56.9% 33.80 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
714.65 -43.4% 404.65 TOTAL nfs_commit_queue_time
3.1.0-rc9-ioless-full-nfs-ino-flushing-next-20111014+ 3.1.0-rc9-ioless-full-nfs-wakeup-wait-next-20111014+
------------------------ ------------------------
766.76 +1.7% 779.63 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
344.34 +49.3% 514.10 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
431.90 +15.0% 496.81 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
3743.78 +5.6% 3954.60 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
3699.59 +14.0% 4216.05 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
3801.55 +5.2% 3999.17 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
12787.93 +9.2% 13960.35 TOTAL nfs_commit_rtt_time
3.1.0-rc9-ioless-full-nfs-ino-flushing-next-20111014+ 3.1.0-rc9-ioless-full-nfs-wakeup-wait-next-20111014+
------------------------ ------------------------
769.38 +1.7% 782.27 NFS-thresh=100M/nfs-10dd-4k-32p-32768M-100M:10-X
346.06 +51.3% 523.52 NFS-thresh=100M/nfs-1dd-4k-32p-32768M-100M:10-X
434.96 +15.2% 501.07 NFS-thresh=100M/nfs-2dd-4k-32p-32768M-100M:10-X
3813.17 +5.4% 4020.45 NFS-thresh=1G/nfs-10dd-4k-32p-32768M-1024M:10-X
4318.75 +4.4% 4507.51 NFS-thresh=1G/nfs-1dd-4k-32p-32768M-1024M:10-X
3837.62 +5.6% 4052.61 NFS-thresh=1G/nfs-2dd-4k-32p-32768M-1024M:10-X
13519.94 +6.4% 14387.44 TOTAL nfs_commit_execute_time
Thanks,
Fengguang
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2011-10-20 3:59 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <20111003134228.090592370@intel.com>
[not found] ` <1318248846.14400.21.camel@laptop>
2011-10-10 13:07 ` [PATCH 00/11] IO-less dirty throttling v12 Wu Fengguang
2011-10-10 13:10 ` [RFC][PATCH 1/2] nfs: writeback pages wait queue Wu Fengguang
2011-10-10 13:11 ` [RFC][PATCH 2/2] nfs: scale writeback threshold proportional to dirty threshold Wu Fengguang
2011-10-18 8:53 ` Wu Fengguang
2011-10-18 8:59 ` Wu Fengguang
2011-10-20 2:49 ` Wu Fengguang
2011-10-18 8:51 ` [RFC][PATCH 1/2] nfs: writeback pages wait queue Wu Fengguang
2011-10-20 3:59 ` Wu Fengguang
2011-10-10 14:28 ` [PATCH 00/11] IO-less dirty throttling v12 Wu Fengguang
2011-10-17 3:03 ` Wu Fengguang
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).