Re: [PATCH] ext4: Flush any pending end_io requests before O_direct read on dioread_nolock

Ted Ts'o
2011-08-19T20:57:26Z

On Thu, Aug 18, 2011 at 06:28:45PM -0700, Jiaying Zhang wrote:
> @@ -800,12 +800,17 @@ ssize_t ext4_ind_direct_IO(int rw, struct kiocb *iocb,
> }
>
> retry:
> - if (rw == READ && ext4_should_dioread_nolock(inode))
> + if (rw == READ && ext4_should_dioread_nolock(inode)) {
> + if (unlikely(!list_empty(&ei->i_completed_io_list))) {
> + mutex_lock(&inode->i_mutex);
> + ext4_flush_completed_IO(inode);
> + mutex_unlock(&inode->i_mutex);
> + }

Doesn't this largely invalidate the reasons for using dioread_nolock
in the first place, which was to avoid taking the i_mutex for
performance reasons? If we are willing to solve the problem this way,
I wonder if we be better off just simply telling users to disable
dioread_nolock if they care about cache consistency between DIO reads
and buffered writes?

(Yes, I do understand that in the hopefully common case where a user
is not trying to do buffered writes and DIO reads at the same time,
we'll just take and release the mutex very quickly, but still, it's
got to have a performance impact.)

I seem to recall a conversation I had with Stephen Tweedie over a
decade ago, where he noted that many other Unix systems made
absolutely no cache consistency guarantees with respect to DIO and the
page cache, but he wanted to set a higher standard for Linux. Which
is fair enough, but I wonder if for the case of dioread_nolock, since
its raison d'etre is to avoid the i_mutex lock, to simply just say
that one of the side effects of dioread_nolock is that (for now) the
cache consistency guarantees are repealed if this mount option is
chosen.

What do folks think?

- Ted