* [PATCH 08/13] writeback: quit throttling when bdi dirty pages dropped low
@ 2010-11-17 3:58 Wu Fengguang
0 siblings, 0 replies; 6+ messages in thread
From: Wu Fengguang @ 2010-11-17 3:58 UTC (permalink / raw)
To: Andrew Morton
Cc: Theodore Ts'o, Wu Fengguang, Dave Chinner, Jan Kara,
Peter Zijlstra, Mel Gorman, Rik van Riel, KOSAKI Motohiro,
Chris Mason, Christoph Hellwig, linux-mm, linux-fsdevel, LKML
Andrew,
References: <20101117035821.000579293@intel.com>
Content-Disposition: inline; filename=writeback-bdi-throttle-break.patch
Tests show that bdi_thresh may take minutes to ramp up on a typical
desktop. The time should be improvable but cannot be eliminated totally.
So when (background_thresh + dirty_thresh)/2 is reached and
balance_dirty_pages() starts to throttle the task, it will suddenly find
the (still low and ramping up) bdi_thresh is exceeded _excessively_. Here
we definitely don't want to stall the task for one minute (when it's
writing to USB stick). So introduce an alternative way to break out of
the loop when the bdi dirty/write pages has dropped by a reasonable
amount.
When dirty_background_ratio is set close to dirty_ratio, bdi_thresh may
also be constantly exceeded due to the task_dirty_limit() gap. This is
addressed by another patch to lower the background threshold when
necessary.
It will take at least 100ms before trying to break out.
Note that this opens the chance that during normal operation, a huge
number of slow dirtiers writing to a really slow device might manage to
outrun bdi_thresh. But the risk is pretty low. It takes at least one
100ms sleep loop to break out, and the global limit is still enforced.
Signed-off-by: Wu Fengguang <fengguang.wu@intel.com>
---
mm/page-writeback.c | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
--- linux-next.orig/mm/page-writeback.c 2010-11-15 12:52:34.000000000 +0800
+++ linux-next/mm/page-writeback.c 2010-11-15 13:08:16.000000000 +0800
@@ -526,6 +526,7 @@ static void balance_dirty_pages(struct a
{
long nr_reclaimable;
long nr_dirty, bdi_dirty; /* = file_dirty + writeback + unstable_nfs */
+ long bdi_prev_dirty = 0;
unsigned long background_thresh;
unsigned long dirty_thresh;
unsigned long bdi_thresh;
@@ -578,6 +579,25 @@ static void balance_dirty_pages(struct a
bdi_stat(bdi, BDI_WRITEBACK);
}
+ /*
+ * bdi_thresh takes time to ramp up from the initial 0,
+ * especially for slow devices.
+ *
+ * It's possible that at the moment dirty throttling starts,
+ * bdi_dirty = nr_dirty
+ * = (background_thresh + dirty_thresh) / 2
+ * >> bdi_thresh
+ * Then the task could be blocked for a dozen second to flush
+ * all the exceeded (bdi_dirty - bdi_thresh) pages. So offer a
+ * complementary way to break out of the loop when 250ms worth
+ * of dirty pages have been cleaned during our pause time.
+ */
+ if (nr_dirty < dirty_thresh &&
+ bdi_prev_dirty - bdi_dirty >
+ bdi->write_bandwidth >> (PAGE_CACHE_SHIFT + 2))
+ break;
+ bdi_prev_dirty = bdi_dirty;
+
if (bdi_dirty >= bdi_thresh) {
pause = HZ/10;
goto pause;
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom policy in Canada: sign http://dissolvethecrtc.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH 08/13] writeback: quit throttling when bdi dirty pages dropped low
2010-11-17 4:27 [PATCH 00/13] IO-less dirty throttling v2 Wu Fengguang
@ 2010-11-17 4:27 ` Wu Fengguang
2010-11-24 11:13 ` Peter Zijlstra
0 siblings, 1 reply; 6+ messages in thread
From: Wu Fengguang @ 2010-11-17 4:27 UTC (permalink / raw)
To: Andrew Morton
Cc: Jan Kara, Wu Fengguang, Christoph Hellwig, Dave Chinner,
Theodore Ts'o, Chris Mason, Peter Zijlstra, Mel Gorman,
Rik van Riel, KOSAKI Motohiro, linux-mm, linux-fsdevel, LKML
[-- Attachment #1: writeback-bdi-throttle-break.patch --]
[-- Type: text/plain, Size: 2878 bytes --]
Tests show that bdi_thresh may take minutes to ramp up on a typical
desktop. The time should be improvable but cannot be eliminated totally.
So when (background_thresh + dirty_thresh)/2 is reached and
balance_dirty_pages() starts to throttle the task, it will suddenly find
the (still low and ramping up) bdi_thresh is exceeded _excessively_. Here
we definitely don't want to stall the task for one minute (when it's
writing to USB stick). So introduce an alternative way to break out of
the loop when the bdi dirty/write pages has dropped by a reasonable
amount.
When dirty_background_ratio is set close to dirty_ratio, bdi_thresh may
also be constantly exceeded due to the task_dirty_limit() gap. This is
addressed by another patch to lower the background threshold when
necessary.
It will take at least 100ms before trying to break out.
Note that this opens the chance that during normal operation, a huge
number of slow dirtiers writing to a really slow device might manage to
outrun bdi_thresh. But the risk is pretty low. It takes at least one
100ms sleep loop to break out, and the global limit is still enforced.
Signed-off-by: Wu Fengguang <fengguang.wu@intel.com>
---
mm/page-writeback.c | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
--- linux-next.orig/mm/page-writeback.c 2010-11-15 12:52:34.000000000 +0800
+++ linux-next/mm/page-writeback.c 2010-11-15 13:08:16.000000000 +0800
@@ -526,6 +526,7 @@ static void balance_dirty_pages(struct a
{
long nr_reclaimable;
long nr_dirty, bdi_dirty; /* = file_dirty + writeback + unstable_nfs */
+ long bdi_prev_dirty = 0;
unsigned long background_thresh;
unsigned long dirty_thresh;
unsigned long bdi_thresh;
@@ -578,6 +579,25 @@ static void balance_dirty_pages(struct a
bdi_stat(bdi, BDI_WRITEBACK);
}
+ /*
+ * bdi_thresh takes time to ramp up from the initial 0,
+ * especially for slow devices.
+ *
+ * It's possible that at the moment dirty throttling starts,
+ * bdi_dirty = nr_dirty
+ * = (background_thresh + dirty_thresh) / 2
+ * >> bdi_thresh
+ * Then the task could be blocked for a dozen second to flush
+ * all the exceeded (bdi_dirty - bdi_thresh) pages. So offer a
+ * complementary way to break out of the loop when 250ms worth
+ * of dirty pages have been cleaned during our pause time.
+ */
+ if (nr_dirty < dirty_thresh &&
+ bdi_prev_dirty - bdi_dirty >
+ bdi->write_bandwidth >> (PAGE_CACHE_SHIFT + 2))
+ break;
+ bdi_prev_dirty = bdi_dirty;
+
if (bdi_dirty >= bdi_thresh) {
pause = HZ/10;
goto pause;
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom policy in Canada: sign http://dissolvethecrtc.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 08/13] writeback: quit throttling when bdi dirty pages dropped low
2010-11-17 4:27 ` [PATCH 08/13] writeback: quit throttling when bdi dirty pages dropped low Wu Fengguang
@ 2010-11-24 11:13 ` Peter Zijlstra
2010-11-24 12:30 ` Wu Fengguang
0 siblings, 1 reply; 6+ messages in thread
From: Peter Zijlstra @ 2010-11-24 11:13 UTC (permalink / raw)
To: Wu Fengguang
Cc: Andrew Morton, Jan Kara, Christoph Hellwig, Dave Chinner,
Theodore Ts'o, Chris Mason, Mel Gorman, Rik van Riel,
KOSAKI Motohiro, linux-mm, linux-fsdevel, LKML
On Wed, 2010-11-17 at 12:27 +0800, Wu Fengguang wrote:
> @@ -578,6 +579,25 @@ static void balance_dirty_pages(struct a
> bdi_stat(bdi, BDI_WRITEBACK);
> }
>
> + /*
> + * bdi_thresh takes time to ramp up from the initial 0,
> + * especially for slow devices.
> + *
> + * It's possible that at the moment dirty throttling starts,
> + * bdi_dirty = nr_dirty
> + * = (background_thresh + dirty_thresh) / 2
> + * >> bdi_thresh
> + * Then the task could be blocked for a dozen second to flush
> + * all the exceeded (bdi_dirty - bdi_thresh) pages. So offer a
> + * complementary way to break out of the loop when 250ms worth
> + * of dirty pages have been cleaned during our pause time.
> + */
> + if (nr_dirty < dirty_thresh &&
> + bdi_prev_dirty - bdi_dirty >
> + bdi->write_bandwidth >> (PAGE_CACHE_SHIFT + 2))
> + break;
> + bdi_prev_dirty = bdi_dirty;
> +
> if (bdi_dirty >= bdi_thresh) {
> pause = HZ/10;
> goto pause;
So we're testing to see if during our pause time (<=100ms) we've written
out 250ms worth of pages (given our current bandwidth estimation),
right?
(1/4th of bandwidth in bytes/s is bytes per 0.25s)
(and in your recent patches you've changed the bw to pages/s so I take
it the PAGE_CACHE_SIZE will be gone from all these sites).
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom policy in Canada: sign http://dissolvethecrtc.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 08/13] writeback: quit throttling when bdi dirty pages dropped low
2010-11-24 11:13 ` Peter Zijlstra
@ 2010-11-24 12:30 ` Wu Fengguang
2010-11-24 12:46 ` Peter Zijlstra
0 siblings, 1 reply; 6+ messages in thread
From: Wu Fengguang @ 2010-11-24 12:30 UTC (permalink / raw)
To: Peter Zijlstra
Cc: Andrew Morton, Jan Kara, Christoph Hellwig, Dave Chinner,
Theodore Ts'o, Chris Mason, Mel Gorman, Rik van Riel,
KOSAKI Motohiro, linux-mm, linux-fsdevel@vger.kernel.org, LKML
On Wed, Nov 24, 2010 at 07:13:53PM +0800, Peter Zijlstra wrote:
> On Wed, 2010-11-17 at 12:27 +0800, Wu Fengguang wrote:
>
> > @@ -578,6 +579,25 @@ static void balance_dirty_pages(struct a
> > bdi_stat(bdi, BDI_WRITEBACK);
> > }
> >
> > + /*
> > + * bdi_thresh takes time to ramp up from the initial 0,
> > + * especially for slow devices.
> > + *
> > + * It's possible that at the moment dirty throttling starts,
> > + * bdi_dirty = nr_dirty
> > + * = (background_thresh + dirty_thresh) / 2
> > + * >> bdi_thresh
> > + * Then the task could be blocked for a dozen second to flush
> > + * all the exceeded (bdi_dirty - bdi_thresh) pages. So offer a
> > + * complementary way to break out of the loop when 250ms worth
> > + * of dirty pages have been cleaned during our pause time.
> > + */
> > + if (nr_dirty < dirty_thresh &&
> > + bdi_prev_dirty - bdi_dirty >
> > + bdi->write_bandwidth >> (PAGE_CACHE_SHIFT + 2))
> > + break;
> > + bdi_prev_dirty = bdi_dirty;
> > +
> > if (bdi_dirty >= bdi_thresh) {
> > pause = HZ/10;
> > goto pause;
>
>
> So we're testing to see if during our pause time (<=100ms) we've written
> out 250ms worth of pages (given our current bandwidth estimation),
> right?
>
> (1/4th of bandwidth in bytes/s is bytes per 0.25s)
Right.
> (and in your recent patches you've changed the bw to pages/s so I take
> it the PAGE_CACHE_SIZE will be gone from all these sites).
Yeah. Actually I did one more fix after that. The break is designed
mainly to help single task case. It helps less for concurrent dirtier
cases, however for long run servers I guess they don't really care
some boot time lags.
For the 1-dd case, it looks better to lower the break threshold to
125ms. After all, it's not easy for the dirty pages to drop by 250ms
worth of data when you only slept 200ms (note: the max pause time has
been doubled mainly for servers).
- if (nr_dirty < dirty_thresh &&
- bdi_prev_dirty - bdi_dirty > (long)bdi->write_bandwidth / 4)
+ if (nr_dirty <= dirty_thresh &&
+ bdi_prev_dirty - bdi_dirty > (long)bdi->write_bandwidth / 8)
break;
Thanks,
Fengguang
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom policy in Canada: sign http://dissolvethecrtc.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 08/13] writeback: quit throttling when bdi dirty pages dropped low
2010-11-24 12:30 ` Wu Fengguang
@ 2010-11-24 12:46 ` Peter Zijlstra
2010-11-24 12:59 ` Wu Fengguang
0 siblings, 1 reply; 6+ messages in thread
From: Peter Zijlstra @ 2010-11-24 12:46 UTC (permalink / raw)
To: Wu Fengguang
Cc: Andrew Morton, Jan Kara, Christoph Hellwig, Dave Chinner,
Theodore Ts'o, Chris Mason, Mel Gorman, Rik van Riel,
KOSAKI Motohiro, linux-mm, linux-fsdevel@vger.kernel.org, LKML
On Wed, 2010-11-24 at 20:30 +0800, Wu Fengguang wrote:
>
> For the 1-dd case, it looks better to lower the break threshold to
> 125ms. After all, it's not easy for the dirty pages to drop by 250ms
> worth of data when you only slept 200ms (note: the max pause time has
> been doubled mainly for servers).
>
> - if (nr_dirty < dirty_thresh &&
> - bdi_prev_dirty - bdi_dirty > (long)bdi->write_bandwidth / 4)
> + if (nr_dirty <= dirty_thresh &&
> + bdi_prev_dirty - bdi_dirty > (long)bdi->write_bandwidth / 8)
> break;
Hrm, but 125ms worth in 200ms is rather easy, you'd want to keep that
limit above what the pause should give you, right?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom policy in Canada: sign http://dissolvethecrtc.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH 08/13] writeback: quit throttling when bdi dirty pages dropped low
2010-11-24 12:46 ` Peter Zijlstra
@ 2010-11-24 12:59 ` Wu Fengguang
0 siblings, 0 replies; 6+ messages in thread
From: Wu Fengguang @ 2010-11-24 12:59 UTC (permalink / raw)
To: Peter Zijlstra
Cc: Andrew Morton, Jan Kara, Christoph Hellwig, Dave Chinner,
Theodore Ts'o, Chris Mason, Mel Gorman, Rik van Riel,
KOSAKI Motohiro, linux-mm, linux-fsdevel@vger.kernel.org, LKML
On Wed, Nov 24, 2010 at 08:46:51PM +0800, Peter Zijlstra wrote:
> On Wed, 2010-11-24 at 20:30 +0800, Wu Fengguang wrote:
> >
> > For the 1-dd case, it looks better to lower the break threshold to
> > 125ms. After all, it's not easy for the dirty pages to drop by 250ms
> > worth of data when you only slept 200ms (note: the max pause time has
> > been doubled mainly for servers).
> >
> > - if (nr_dirty < dirty_thresh &&
> > - bdi_prev_dirty - bdi_dirty > (long)bdi->write_bandwidth / 4)
> > + if (nr_dirty <= dirty_thresh &&
> > + bdi_prev_dirty - bdi_dirty > (long)bdi->write_bandwidth / 8)
> > break;
>
> Hrm, but 125ms worth in 200ms is rather easy, you'd want to keep that
> limit above what the pause should give you, right?
Yeah, I exactly mean to quit the loop after sleeping 200ms.
200ms pause already seem large..
I'll leave the safeguard to the "nr_dirty <= dirty_thresh" test :)
This trace shows the problem it tries to solve. Here on fresh boot,
bdi_dirty=106600 which is much larger than bdi_limit=13525.
The same situation may happen if some task quickly eats lots of
anonymous memory, then the global/bdi limits will be knock down
suddenly and it takes time to bring down bdi_dirty.
# TASK-PID CPU# TIMESTAMP FUNCTION
# | | | | |
<...>-2722 [005] 39.623216: balance_dirty_pages: bdi=8:0 bdi_dirty=106600 bdi_limit=13525 task_limit=13423 task_weight=12% task_gap=-11022% bdi_gap=-5504% dirtied=268 bw=0 ratelimit=0 period=0 think=0 pause=200
<...>-2720 [004] 39.623222: balance_dirty_pages: bdi=8:0 bdi_dirty=106600 bdi_limit=13513 task_limit=13441 task_weight=9% task_gap=-11029% bdi_gap=-5510% dirtied=267 bw=0 ratelimit=0 period=0 think=0 pause=200
<...>-2717 [003] 39.623666: balance_dirty_pages: bdi=8:0 bdi_dirty=106560 bdi_limit=13742 task_limit=13659 task_weight=10% task_gap=-10815% bdi_gap=-5402% dirtied=268 bw=0 ratelimit=0 period=0 think=0 pause=200
<...>-2718 [004] 39.623990: balance_dirty_pages: bdi=8:0 bdi_dirty=106560 bdi_limit=13979 task_limit=13913 task_weight=8% task_gap=-10603% bdi_gap=-5297% dirtied=272 bw=0 ratelimit=0 period=0 think=0 pause=200
<...>-2715 [007] 39.626472: balance_dirty_pages: bdi=8:0 bdi_dirty=106360 bdi_limit=15312 task_limit=15211 task_weight=11% task_gap=-9523% bdi_gap=-4756% dirtied=268 bw=0 ratelimit=0 period=0 think=0 pause=200
<...>-2723 [001] 39.628951: balance_dirty_pages: bdi=8:0 bdi_dirty=106560 bdi_limit=15749 task_limit=15640 task_weight=11% task_gap=-9236% bdi_gap=-4612% dirtied=269 bw=0 ratelimit=0 period=0 think=0 pause=200
Thanks,
Fengguang
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom policy in Canada: sign http://dissolvethecrtc.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2010-11-24 12:59 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2010-11-17 3:58 [PATCH 08/13] writeback: quit throttling when bdi dirty pages dropped low Wu Fengguang
-- strict thread matches above, loose matches on Subject: below --
2010-11-17 4:27 [PATCH 00/13] IO-less dirty throttling v2 Wu Fengguang
2010-11-17 4:27 ` [PATCH 08/13] writeback: quit throttling when bdi dirty pages dropped low Wu Fengguang
2010-11-24 11:13 ` Peter Zijlstra
2010-11-24 12:30 ` Wu Fengguang
2010-11-24 12:46 ` Peter Zijlstra
2010-11-24 12:59 ` Wu Fengguang
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).