From: Wu Fengguang <fengguang.wu@intel.com>
To: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Andrew Morton <akpm@linux-foundation.org>,
Jan Kara <jack@suse.cz>, "Li, Shaohua" <shaohua.li@intel.com>,
Christoph Hellwig <hch@lst.de>,
Dave Chinner <david@fromorbit.com>,
"Theodore Ts'o" <tytso@mit.edu>,
Chris Mason <chris.mason@oracle.com>, Mel Gorman <mel@csn.ul.ie>,
Rik van Riel <riel@redhat.com>,
KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>,
linux-mm <linux-mm@kvack.org>,
"linux-fsdevel@vger.kernel.org" <linux-fsdevel@vger.kernel.org>,
LKML <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH 06/13] writeback: bdi write bandwidth estimation
Date: Wed, 24 Nov 2010 22:06:10 +0800 [thread overview]
Message-ID: <20101124140610.GB8333@localhost> (raw)
In-Reply-To: <1290596302.2072.445.camel@laptop>
On Wed, Nov 24, 2010 at 06:58:22PM +0800, Peter Zijlstra wrote:
> On Wed, 2010-11-17 at 12:27 +0800, Wu Fengguang wrote:
>
> > @@ -555,8 +592,10 @@ static void balance_dirty_pages(struct a
> > pause = clamp_val(pause, 1, HZ/10);
> >
> > pause:
> > + bdi_update_write_bandwidth(bdi, &bw_time, &bw_written);
> > __set_current_state(TASK_INTERRUPTIBLE);
> > io_schedule_timeout(pause);
> > + bdi_update_write_bandwidth(bdi, &bw_time, &bw_written);
> >
> > /*
> > * The bdi thresh is somehow "soft" limit derived from the
>
> So its really a two part bandwidth calculation, the first call is:
>
> bdi_get_bandwidth()
>
> and the second call is:
>
> bdi_update_bandwidth()
>
> Would it make sense to actually implement it with two functions instead
> of overloading the functionality of the one function?
Thanks, it's good suggestion indeed. However after looking around, I
find it hard to split it up cleanly.. To make it clear, how about this
comment update?
Thanks,
Fengguang
---
--- linux-next.orig/mm/page-writeback.c 2010-11-24 19:05:01.000000000 +0800
+++ linux-next/mm/page-writeback.c 2010-11-24 22:01:43.000000000 +0800
@@ -554,6 +554,14 @@ out:
return a;
}
+/*
+ * This can be repeatedly called inside a long run loop, eg. by wb_writeback().
+ *
+ * On first invocation it will find *bw_written=0 and take the initial snapshot.
+ * On follow up calls it will update the bandwidth if
+ * - at least 10ms data have been collected
+ * - the bandwidth for the time range has not been updated in parallel by others
+ */
void bdi_update_write_bandwidth(struct backing_dev_info *bdi,
unsigned long *bw_time,
s64 *bw_written)
@@ -575,9 +583,12 @@ void bdi_update_write_bandwidth(struct b
* When there lots of tasks throttled in balance_dirty_pages(), they
* will each try to update the bandwidth for the same period, making
* the bandwidth drift much faster than the desired rate (as in the
- * single dirtier case). So do some rate limiting.
+ * single dirtier case).
+ *
+ * If someone changed bdi->write_bandwidth_update_time, he has done
+ * overlapped estimation with us. So start the next round of estimation.
*/
- if (jiffies - bdi->write_bandwidth_update_time < elapsed)
+ if (jiffies - bdi->write_bandwidth_update_time != elapsed)
goto snapshot;
written = percpu_counter_read(&bdi->bdi_stat[BDI_WRITTEN]) - *bw_written;
WARNING: multiple messages have this Message-ID (diff)
From: Wu Fengguang <fengguang.wu@intel.com>
To: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Andrew Morton <akpm@linux-foundation.org>,
Jan Kara <jack@suse.cz>, "Li, Shaohua" <shaohua.li@intel.com>,
Christoph Hellwig <hch@lst.de>,
Dave Chinner <david@fromorbit.com>, Theodore Ts'o <tytso@mit.edu>,
Chris Mason <chris.mason@oracle.com>, Mel Gorman <mel@csn.ul.ie>,
Rik van Riel <riel@redhat.com>,
KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>,
linux-mm <linux-mm@kvack.org>,
"linux-fsdevel@vger.kernel.org" <linux-fsdevel@vger.kernel.org>,
LKML <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH 06/13] writeback: bdi write bandwidth estimation
Date: Wed, 24 Nov 2010 22:06:10 +0800 [thread overview]
Message-ID: <20101124140610.GB8333@localhost> (raw)
In-Reply-To: <1290596302.2072.445.camel@laptop>
On Wed, Nov 24, 2010 at 06:58:22PM +0800, Peter Zijlstra wrote:
> On Wed, 2010-11-17 at 12:27 +0800, Wu Fengguang wrote:
>
> > @@ -555,8 +592,10 @@ static void balance_dirty_pages(struct a
> > pause = clamp_val(pause, 1, HZ/10);
> >
> > pause:
> > + bdi_update_write_bandwidth(bdi, &bw_time, &bw_written);
> > __set_current_state(TASK_INTERRUPTIBLE);
> > io_schedule_timeout(pause);
> > + bdi_update_write_bandwidth(bdi, &bw_time, &bw_written);
> >
> > /*
> > * The bdi thresh is somehow "soft" limit derived from the
>
> So its really a two part bandwidth calculation, the first call is:
>
> bdi_get_bandwidth()
>
> and the second call is:
>
> bdi_update_bandwidth()
>
> Would it make sense to actually implement it with two functions instead
> of overloading the functionality of the one function?
Thanks, it's good suggestion indeed. However after looking around, I
find it hard to split it up cleanly.. To make it clear, how about this
comment update?
Thanks,
Fengguang
---
--- linux-next.orig/mm/page-writeback.c 2010-11-24 19:05:01.000000000 +0800
+++ linux-next/mm/page-writeback.c 2010-11-24 22:01:43.000000000 +0800
@@ -554,6 +554,14 @@ out:
return a;
}
+/*
+ * This can be repeatedly called inside a long run loop, eg. by wb_writeback().
+ *
+ * On first invocation it will find *bw_written=0 and take the initial snapshot.
+ * On follow up calls it will update the bandwidth if
+ * - at least 10ms data have been collected
+ * - the bandwidth for the time range has not been updated in parallel by others
+ */
void bdi_update_write_bandwidth(struct backing_dev_info *bdi,
unsigned long *bw_time,
s64 *bw_written)
@@ -575,9 +583,12 @@ void bdi_update_write_bandwidth(struct b
* When there lots of tasks throttled in balance_dirty_pages(), they
* will each try to update the bandwidth for the same period, making
* the bandwidth drift much faster than the desired rate (as in the
- * single dirtier case). So do some rate limiting.
+ * single dirtier case).
+ *
+ * If someone changed bdi->write_bandwidth_update_time, he has done
+ * overlapped estimation with us. So start the next round of estimation.
*/
- if (jiffies - bdi->write_bandwidth_update_time < elapsed)
+ if (jiffies - bdi->write_bandwidth_update_time != elapsed)
goto snapshot;
written = percpu_counter_read(&bdi->bdi_stat[BDI_WRITTEN]) - *bw_written;
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom policy in Canada: sign http://dissolvethecrtc.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2010-11-24 14:06 UTC|newest]
Thread overview: 162+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-11-17 4:27 [PATCH 00/13] IO-less dirty throttling v2 Wu Fengguang
2010-11-17 4:27 ` Wu Fengguang
2010-11-17 4:27 ` Wu Fengguang
2010-11-17 4:27 ` [PATCH 01/13] writeback: IO-less balance_dirty_pages() Wu Fengguang
2010-11-17 4:27 ` Wu Fengguang
2010-11-17 4:27 ` Wu Fengguang
2010-11-17 10:34 ` Minchan Kim
2010-11-17 10:34 ` Minchan Kim
2010-11-22 2:01 ` Wu Fengguang
2010-11-22 2:01 ` Wu Fengguang
2010-11-22 2:01 ` Wu Fengguang
2010-11-17 23:08 ` Andrew Morton
2010-11-17 23:08 ` Andrew Morton
2010-11-17 23:08 ` Andrew Morton
2010-11-18 13:04 ` Peter Zijlstra
2010-11-18 13:04 ` Peter Zijlstra
2010-11-18 13:26 ` Wu Fengguang
2010-11-18 13:26 ` Wu Fengguang
2010-11-18 13:40 ` Peter Zijlstra
2010-11-18 13:40 ` Peter Zijlstra
2010-11-18 14:02 ` Wu Fengguang
2010-11-18 14:02 ` Wu Fengguang
[not found] ` <20101129151719.GA30590@localhost>
[not found] ` <1291064013.32004.393.camel@laptop>
[not found] ` <20101130043735.GA22947@localhost>
[not found] ` <1291156522.32004.1359.camel@laptop>
[not found] ` <1291156765.32004.1365.camel@laptop>
[not found] ` <20101201133818.GA13377@localhost>
2010-12-01 23:03 ` Andrew Morton
2010-12-01 23:03 ` Andrew Morton
2010-12-02 1:56 ` Wu Fengguang
2010-12-02 1:56 ` Wu Fengguang
2010-12-05 16:14 ` Wu Fengguang
2010-12-06 2:42 ` Ted Ts'o
2010-12-06 2:42 ` Ted Ts'o
2010-12-06 9:52 ` Dmitry
2010-12-06 9:52 ` Dmitry
2010-12-06 9:52 ` Dmitry
2010-12-06 12:34 ` Ted Ts'o
2010-12-06 12:34 ` Ted Ts'o
2010-11-17 4:27 ` [PATCH 02/13] writeback: consolidate variable names in balance_dirty_pages() Wu Fengguang
2010-11-17 4:27 ` Wu Fengguang
2010-11-17 4:27 ` [PATCH 03/13] writeback: per-task rate limit on balance_dirty_pages() Wu Fengguang
2010-11-17 4:27 ` Wu Fengguang
2010-11-17 4:27 ` Wu Fengguang
2010-11-17 14:39 ` Wu Fengguang
2010-11-17 14:39 ` Wu Fengguang
2010-11-24 10:23 ` Peter Zijlstra
2010-11-24 10:23 ` Peter Zijlstra
2010-11-24 10:43 ` Wu Fengguang
2010-11-24 10:43 ` Wu Fengguang
2010-11-24 10:49 ` Peter Zijlstra
2010-11-24 10:49 ` Peter Zijlstra
2010-11-17 4:27 ` [PATCH 04/13] writeback: prevent duplicate balance_dirty_pages_ratelimited() calls Wu Fengguang
2010-11-17 4:27 ` Wu Fengguang
2010-11-17 4:27 ` Wu Fengguang
2010-11-17 4:27 ` [PATCH 05/13] writeback: account per-bdi accumulated written pages Wu Fengguang
2010-11-17 4:27 ` Wu Fengguang
2010-11-17 4:27 ` Wu Fengguang
2010-11-24 10:26 ` Peter Zijlstra
2010-11-24 10:26 ` Peter Zijlstra
2010-11-24 10:44 ` Wu Fengguang
2010-11-24 10:44 ` Wu Fengguang
2010-11-17 4:27 ` [PATCH 06/13] writeback: bdi write bandwidth estimation Wu Fengguang
2010-11-17 4:27 ` Wu Fengguang
2010-11-17 4:27 ` Wu Fengguang
2010-11-17 23:08 ` Andrew Morton
2010-11-17 23:08 ` Andrew Morton
2010-11-17 23:08 ` Andrew Morton
2010-11-17 23:24 ` Peter Zijlstra
2010-11-17 23:24 ` Peter Zijlstra
2010-11-17 23:38 ` Andrew Morton
2010-11-17 23:38 ` Andrew Morton
2010-11-17 23:43 ` Peter Zijlstra
2010-11-17 23:43 ` Peter Zijlstra
2010-11-18 6:51 ` Wu Fengguang
2010-11-18 6:51 ` Wu Fengguang
2010-11-24 10:58 ` Peter Zijlstra
2010-11-24 10:58 ` Peter Zijlstra
2010-11-24 14:06 ` Wu Fengguang [this message]
2010-11-24 14:06 ` Wu Fengguang
2010-11-24 11:05 ` Peter Zijlstra
2010-11-24 11:05 ` Peter Zijlstra
2010-11-24 12:10 ` Wu Fengguang
2010-11-24 12:10 ` Wu Fengguang
2010-11-24 12:50 ` Peter Zijlstra
2010-11-24 12:50 ` Peter Zijlstra
2010-11-24 13:14 ` Wu Fengguang
2010-11-24 13:14 ` Wu Fengguang
2010-11-24 13:20 ` Wu Fengguang
2010-11-24 13:20 ` Wu Fengguang
2010-11-24 13:42 ` Peter Zijlstra
2010-11-24 13:42 ` Peter Zijlstra
2010-11-24 13:46 ` Wu Fengguang
2010-11-24 13:46 ` Wu Fengguang
2010-11-24 14:12 ` Peter Zijlstra
2010-11-24 14:12 ` Peter Zijlstra
2010-11-24 14:21 ` Wu Fengguang
2010-11-24 14:21 ` Wu Fengguang
2010-11-24 14:31 ` Peter Zijlstra
2010-11-24 14:31 ` Peter Zijlstra
2010-11-24 14:38 ` Wu Fengguang
2010-11-24 14:38 ` Wu Fengguang
2010-11-24 14:34 ` Wu Fengguang
2010-11-24 14:34 ` Wu Fengguang
2010-11-17 4:27 ` [PATCH 07/13] writeback: show bdi write bandwidth in debugfs Wu Fengguang
2010-11-17 4:27 ` Wu Fengguang
2010-11-17 4:27 ` Wu Fengguang
2010-11-17 4:27 ` [PATCH 08/13] writeback: quit throttling when bdi dirty pages dropped low Wu Fengguang
2010-11-17 4:27 ` Wu Fengguang
2010-11-17 4:27 ` Wu Fengguang
2010-11-24 11:13 ` Peter Zijlstra
2010-11-24 11:13 ` Peter Zijlstra
2010-11-24 12:30 ` Wu Fengguang
2010-11-24 12:30 ` Wu Fengguang
2010-11-24 12:46 ` Peter Zijlstra
2010-11-24 12:46 ` Peter Zijlstra
2010-11-24 12:59 ` Wu Fengguang
2010-11-24 12:59 ` Wu Fengguang
2010-11-17 4:27 ` [PATCH 09/13] writeback: reduce per-bdi dirty threshold ramp up time Wu Fengguang
2010-11-17 4:27 ` Wu Fengguang
2010-11-17 4:27 ` Wu Fengguang
2010-11-24 11:15 ` Peter Zijlstra
2010-11-24 11:15 ` Peter Zijlstra
2010-11-24 12:39 ` Wu Fengguang
2010-11-24 12:39 ` Wu Fengguang
2010-11-24 12:56 ` Peter Zijlstra
2010-11-24 12:56 ` Peter Zijlstra
2010-11-17 4:27 ` [PATCH 10/13] writeback: make reasonable gap between the dirty/background thresholds Wu Fengguang
2010-11-17 4:27 ` Wu Fengguang
2010-11-17 4:27 ` Wu Fengguang
2010-11-24 11:18 ` Peter Zijlstra
2010-11-24 11:18 ` Peter Zijlstra
2010-11-24 12:48 ` Wu Fengguang
2010-11-24 12:48 ` Wu Fengguang
2010-11-17 4:27 ` [PATCH 11/13] writeback: scale down max throttle bandwidth on concurrent dirtiers Wu Fengguang
2010-11-17 4:27 ` Wu Fengguang
2010-11-17 4:27 ` Wu Fengguang
2010-11-17 4:27 ` [PATCH 12/13] writeback: add trace event for balance_dirty_pages() Wu Fengguang
2010-11-17 4:27 ` Wu Fengguang
2010-11-17 4:27 ` Wu Fengguang
2010-11-17 4:41 ` Wu Fengguang
2010-11-17 4:41 ` Wu Fengguang
2010-11-17 4:27 ` [PATCH 13/13] writeback: make nr_to_write a per-file limit Wu Fengguang
2010-11-17 4:27 ` Wu Fengguang
2010-11-17 4:27 ` Wu Fengguang
2010-11-17 23:03 ` [PATCH 00/13] IO-less dirty throttling v2 Andrew Morton
2010-11-17 23:03 ` Andrew Morton
2010-11-17 23:03 ` Andrew Morton
2010-11-18 2:06 ` Dave Chinner
2010-11-18 2:06 ` Dave Chinner
2010-11-18 2:09 ` Andrew Morton
2010-11-18 2:09 ` Andrew Morton
2010-11-18 3:21 ` Dave Chinner
2010-11-18 3:21 ` Dave Chinner
2010-11-18 3:34 ` Andrew Morton
2010-11-18 3:34 ` Andrew Morton
2010-11-18 7:27 ` Dave Chinner
2010-11-18 7:27 ` Dave Chinner
2010-11-18 7:33 ` Andrew Morton
2010-11-18 7:33 ` Andrew Morton
2010-11-19 3:11 ` Dave Chinner
2010-11-19 3:11 ` Dave Chinner
2010-11-24 11:12 ` Avi Kivity
2010-11-24 11:12 ` Avi Kivity
-- strict thread matches above, loose matches on Subject: below --
2010-11-17 3:58 [PATCH 06/13] writeback: bdi write bandwidth estimation Wu Fengguang
2010-11-17 3:58 ` Wu Fengguang
2010-11-17 3:58 ` Wu Fengguang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20101124140610.GB8333@localhost \
--to=fengguang.wu@intel.com \
--cc=a.p.zijlstra@chello.nl \
--cc=akpm@linux-foundation.org \
--cc=chris.mason@oracle.com \
--cc=david@fromorbit.com \
--cc=hch@lst.de \
--cc=jack@suse.cz \
--cc=kosaki.motohiro@jp.fujitsu.com \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mel@csn.ul.ie \
--cc=riel@redhat.com \
--cc=shaohua.li@intel.com \
--cc=tytso@mit.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.