Re: splice vs truncate lockdep splat

[Dave Chinner <david@fromorbit.com>]

On Sun, Mar 20, 2016 at 09:43:01AM -0400, Sage Weil wrote:
> Hi all,
> 
> For the last few kernel releases we've been seeing this pretty regularly:
> 

[snip]

Yup, splice still has locking problems. Always has, and the
write_iter rework a few releases ago just moved the warnings
elsewhere.

>  #012-> #2 (&pipe->mutex/1){+.+.+.}:

xfs_file_splice_read
  xfs_iolock(XFS_IOLOCK_SHARED)
    ....
    pipe_lock(pipe)

XFS takes the XFS_IOLOCK_SHARED here to serialise splice read
against truncate.

>  #012-> #1 (&(&ip->i_iolock)->mr_lock){++++++}:

mutex_lock(inode->i_mutex)
....
  xfs_vn_setattr
    xfs_iolock(XFS_IOLOCK_EXCL)

Truncate lock order for IO serialisation (no pipe lock).

>  #012-> #0 (&sb->s_type->i_mutex_key#19){+.+.+.}:

iter_file_splice_write
  pipe_lock(pipe)
    xfs_file_buffered_aio_write
      mutex_lock(inode->i_mutex)
        xfs_iolock(XFS_IOLOCK_EXCL)

XFS takes i_mutex (required for buffered IO path) and
XFS_IOLOCK_EXCL (for atomic write semantics and truncate exclusion),
but the splice write path has put this under the pipe lock....

i.e. because the read/write path take the IOLOCK different ways,
lockdep seems to think it's OK. Hence IO testing doesn't report
issues, but the moment you throw in another operation that takes
both the i_mutex and XFS_IOLOCK_EXCL, lockdep will through a lock
order violation report.

> We're not actually doing racing truncate and splice (or any racing 
> operations for that matter) to the same files, so we don't actually hit 
> this, but the lockdep warning is enough to make our test runs fail, and 
> we'd rather not whitelist this if it's something that should get fixed 
> anyway.

We can't fix it in XFS - the high level splice infrastructure which
that read IO serialisation is done entirely based on page cache page
locking and so can hold the pipe lock across ->splice_read callouts.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs