From: Nick Piggin <npiggin@suse.de>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: Dave Chinner <david@fromorbit.com>,
linux-kernel@vger.kernel.org, xfs@oss.sgi.com,
linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org,
tytso@mit.edu, jens.axboe@oracle.com
Subject: Re: [PATCH 6/6] writeback: limit write_cache_pages integrity scanning to current EOF
Date: Fri, 28 May 2010 15:06:55 +1000 [thread overview]
Message-ID: <20100528050655.GY22536@laptop> (raw)
In-Reply-To: <20100527143341.d4258798.akpm@linux-foundation.org>
On Thu, May 27, 2010 at 02:33:41PM -0700, Andrew Morton wrote:
> On Tue, 25 May 2010 20:54:12 +1000
> Dave Chinner <david@fromorbit.com> wrote:
>
> > From: Dave Chinner <dchinner@redhat.com>
> >
> > sync can currently take a really long time if a concurrent writer is
> > extending a file. The problem is that the dirty pages on the address
> > space grow in the same direction as write_cache_pages scans, so if
> > the writer keeps ahead of writeback, the writeback will not
> > terminate until the writer stops adding dirty pages.
...
> That being said, I think the patch is insufficient. If I create an
> enormous (possibly sparse) file with a 16TB hole (or a run of clean
> pages) in the middle and then start busily writing into that hole (run
> of clean pages), the problem will still occur.
Yep.
> One obvious fix for that (a) would be to add another radix-tree tag and
> do two passes across the radix-tree.
Yes this is the method I tried. Jan has taken it further and should
have the latest patches around. A good test case for the starvation
would be helpful.
> Another fix (b) would be to track the number of dirty pages per
> adddress_space, and only write that number of pages.
>
> Another fix would be to work out how the code handled this situation
> before we broke it, and restore that in some fashion. I guess fix (b)
> above kinda does that.
I took that out (and offered fix a in replacement but it was turned
down at the time). Because b stands for broken.
IIRC we were writing out no more than 2x the dirty pages of the file
during sync. The problem with that is more pages can be dirtied after
we calculate the number, and then we might write out those newly dirty
pages and miss old dirty pages.
next prev parent reply other threads:[~2010-05-28 5:06 UTC|newest]
Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-05-25 10:54 [PATCH 0/6] writeback: tracing and fixes Dave Chinner
2010-05-25 10:54 ` [PATCH 1/6] writeback: initial tracing support Dave Chinner
2010-05-25 11:13 ` Christoph Hellwig
2010-05-27 21:32 ` Andrew Morton
2010-05-28 0:44 ` Dave Chinner
2010-05-28 1:20 ` Steven Rostedt
2010-05-28 1:18 ` Steven Rostedt
2010-05-28 7:45 ` Christoph Hellwig
2010-05-25 10:54 ` [PATCH 2/6] writeback: Add tracing to balance_dirty_pages Dave Chinner
2010-05-25 11:13 ` Christoph Hellwig
2010-05-25 10:54 ` [PATCH 3/6] ext4: Use our own write_cache_pages() Dave Chinner
2010-05-25 13:06 ` tytso
2010-05-25 22:42 ` Dave Chinner
2010-05-25 10:54 ` [PATCH 4/6] writeback: pay attention to wbc->nr_to_write in write_cache_pages Dave Chinner
2010-05-25 11:11 ` Christoph Hellwig
2010-05-27 21:32 ` Andrew Morton
2010-05-28 0:56 ` Dave Chinner
2010-05-25 10:54 ` [PATCH 5/6] xfs: remove nr_to_write writeback windup Dave Chinner
2010-05-25 11:14 ` Christoph Hellwig
2010-05-25 10:54 ` [PATCH 6/6] writeback: limit write_cache_pages integrity scanning to current EOF Dave Chinner
2010-05-27 21:33 ` Andrew Morton
2010-05-28 1:23 ` Dave Chinner
2010-05-28 5:06 ` Nick Piggin [this message]
2010-06-01 15:54 ` Jan Kara
-- strict thread matches above, loose matches on Subject: below --
2010-06-03 23:55 [PATCH 0/6] writeback: tracing and fixes V3 Dave Chinner
2010-06-03 23:55 ` [PATCH 6/6] writeback: limit write_cache_pages integrity scanning to current EOF Dave Chinner
2010-06-04 7:52 ` Christoph Hellwig
2010-06-04 7:56 ` Nick Piggin
2010-06-08 0:38 [PATCH 0/6] writeback: tracing and fixes V4 Dave Chinner
2010-06-08 0:38 ` [PATCH 6/6] writeback: limit write_cache_pages integrity scanning to current EOF Dave Chinner
2010-06-08 5:38 ` Nick Piggin
2010-06-08 6:59 ` Dave Chinner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20100528050655.GY22536@laptop \
--to=npiggin@suse.de \
--cc=akpm@linux-foundation.org \
--cc=david@fromorbit.com \
--cc=jens.axboe@oracle.com \
--cc=linux-ext4@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=tytso@mit.edu \
--cc=xfs@oss.sgi.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).