Re: [PATCH 01/17] writeback: introduce .tagged_sync for the WB_SYNC_NONE sync stage

[Wu Fengguang <fengguang.wu@intel.com>]

On Mon, May 16, 2011 at 07:43:06AM +0800, Dave Chinner wrote:
> On Fri, May 13, 2011 at 10:56:08AM +0800, Wu Fengguang wrote:
> > On Fri, May 13, 2011 at 06:40:13AM +0800, Dave Chinner wrote:
> > > On Thu, May 12, 2011 at 09:57:07PM +0800, Wu Fengguang wrote:
> > > > sync(2) is performed in two stages: the WB_SYNC_NONE sync and the
> > > > WB_SYNC_ALL sync. Tag the first stage with wbc.tagged_sync and do
> > > > livelock prevention for it, too.
> > > > 
> > > > Note that writeback_inodes_sb() is called by not only sync(), they are
> > > > treated the same because the other callers also need livelock prevention.
> > > > 
> > > > Impact:  It changes the order in which pages/inodes are synced to disk.
> > > > Now in the WB_SYNC_NONE stage, it won't proceed to write the next inode
> > > > until finished with the current inode.
> > > 
> > > What about all the filesystems that implement their own
> > > .writepages()/write_cache_pages() functions or have
> > > have special code that checks WB_SYNC_ALL in .writepages (e.g. gfs2,
> > > ext4, btrfs and perhaps others). Don't they all need to be aware of
> > > this tagged_sync field?
> > 
> > Right, good point. Currently only ext4 is updated. The other
> > filesystems --- afs, btrfs, cifs, gfs2 --- do not even use
> > PAGECACHE_TAG_TOWRITE for livelock prevention. My plan was to add
> > PAGECACHE_TAG_TOWRITE and tagged_sync code to them as the next step,
> > when tagged_sync is accepted and proved to work fine.
> 
> Where "proved to work fine" can mean "caused regressions for certain
> filesystems"? I mean, for btrfs it means that the bio is submitted
> with WRITE rather than WRITE_SYNC, which causes subtle changes of
> behaviour in the elevator. that could cause strange regressions that
> are very hard to isolate.

Hmm, where is the relevant btrfs code? It seems that you assumed
WB_SYNC_ALL semantics in .tagged_sync, however the latter merely means
"tag all dirty pages with PAGECACHE_TAG_TOWRITE and write them out".

> Hence regardless of whether filesystems use PAGECACHE_TAG_TOWRITE
> or not, filesystems are checking for synchronous writeback for
> a reason. If we now have two different ways of signalling sync
> writeback they need to know about them.

See above, shall we rename .tagged_sync to .tagged_write?

> Which just raised the question in my mind - why did you add a new
> field rather than a new sync_mode definition? After all, this is a
> new sync control, and it seems clumsy to me to have two separate
> control fields for defining sync behaviour...

Yeah I considered that too. The main problem is, it somehow overloads
the sync mode enum and some filesystems already assumed two modes only
by using (sync_mode == WB_SYNC_ALL) and (sync_mode != WB_SYNC_NONE)
interchangeably.

For example, if adding another mode

        WB_SYNC_NONE            // WRITE, don't wait
+       WB_SYNC_NONE_TAGGED     // WRITE, don't wait, use PAGECACHE_TAG_TOWRITE
        WB_SYNC_ALL             // WRITE_SYNC,  wait, use PAGECACHE_TAG_TOWRITE

The btrfs code will unnecessarily wait on WB_SYNC_NONE_TAGGED:

                        if (wbc->sync_mode != WB_SYNC_NONE) {
                                if (PageWriteback(page))
                                        flush_fn(data);
                                wait_on_page_writeback(page);
                        }

We can fix btrfs trivially, however if there are out of tree
filesystems, they'll break silently..

Thanks,
Fengguang