Re: [PATCH] xfs: properly serialise fallocate against AIO+DIO

[Dave Chinner <david@fromorbit.com>]

On Tue, Oct 29, 2019 at 02:48:50PM +1100, Dave Chinner wrote:
> From: Dave Chinner <dchinner@redhat.com>
> 
> AIO+DIO can extend the file size on IO completion, and it holds
> no inode locks while the IO is in flight. Therefore, a race
> condition exists in file size updates if we do something like this:
> 
> aio-thread			fallocate-thread
> 
> lock inode
> submit IO beyond inode->i_size
> unlock inode
> .....
> 				lock inode
> 				break layouts
> 				if (off + len > inode->i_size)
> 					new_size = off + len
> 				.....
> 				inode_dio_wait()
> 				<blocks>
> .....
> completes
> inode->i_size updated
> inode_dio_done()
> ....
> 				<wakes>
> 				<does stuff no long beyond EOF>
> 				if (new_size)
> 					xfs_vn_setattr(inode, new_size)
> 
> 
> Yup, that attempt to extend the file size in the fallocate code
> turns into a truncate - it removes the whatever the aio write
> allocated and put to disk, and reduced the inode size back down to
> where the fallocate operation ends.
> 
> Fundamentally, xfs_file_fallocate()  not compatible with racing
> AIO+DIO completions, so we need to move the inode_dio_wait() call
> up to where the lock the inode and break the layouts.
> 
> Secondly, storing the inode size and then using it unchecked without
> holding the ILOCK is not safe; we can only do such a thing if we've
> locked out and drained all IO and other modification operations,
> which we don't do initially in xfs_file_fallocate.
> 
> It should be noted that some of the fallocate operations are
> compound operations - they are made up of multiple manipulations
> that may zero data, and so we may need to flush and invalidate the
> file multiple times during an operation. However, we only need to
> lock out IO and other space manipulation operations once, as that
> lockout is maintained until the entire fallocate operation has been
> completed.
> 
> Signed-off-by: Dave Chinner <dchinner@redhat.com>

Just a note for anyone considering backporting this patch - I
intentionally did not mark it for stable kernel backports because it
is not just a simple backport.

i.e. while it may apply cleanly to older kernels, this is based on
the current xfs for-next tree and so is based on Christoph's
xfs_ioc_space() redirection to fallocate() patch set. That means the
changes to xfs_ioc_space() are tiny and trivial. Backporting this to
kernels that don't have Christoph's patch set will require adding
all the flush/invalidation calls that I added to
xfs_file_fallocate().

i.e. all the older XFS_IOC_UNRESVSP, XFS_IOC_ZERO_RANGE, etc
interfaces look to have the same serialisation problem against
AIO-DIO writes, and so older kernels will need them fixed as well.
That code is not in this patch.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com