[GIT PULL] Ceph updates for 3.1-rc1

[GIT PULL] Ceph updates for 3.1-rc1

[Sage Weil]

Hi Linus,

Please pull the following Ceph updates from

  git://git.kernel.org/pub/scm/linux/kernel/git/sage/ceph-client.git for-linus

Lots of different things here.  There is some code cleanup with the file 
flags, a few snapshot metadata writeback fixes, a fix for unnecessary 
timeouts on heavily loaded clusters, and a fix for dentry leases.  There 
are many fixes for bugs Al turned up: a fix and cleanup in the open intent 
code, and several fixes for d_parent use without the appropriate locks (a 
full code audit caught a few more).  RBD devices now clean up on the 
server when they are unmapped, rbd request sizes are now larger, and the 
ceph readahead window is set up properly.

Thanks!
sage


Greg Farnum (1):
      ceph: report f_bfree based on kb_avail rather than diffing.

Josh Durgin (1):
      rbd: set blk_queue request sizes to object size

Sage Weil (19):
      ceph: add flags field to file_info
      ceph: add F_SYNC file flag to force sync (non-O_DIRECT) io
      ceph: use flag bit for at_end readdir flag
      ceph: fix snap writeback when racing with writes
      ceph: only queue capsnap if caps are dirty
      libceph: don't time out osd requests that haven't been received
      ceph: avoid carrying Fw cap during write into page cache
      ceph: fix bad parent_inode calc in ceph_lookup_open
      ceph: only link open operations to directory unsafe list if O_CREAT|O_TRUNC
      ceph: fix ceph_lookup_open intent usage
      ceph: ignore lease mask
      ceph: set dir complete frag after adding capability
      ceph: handle racing calls to ceph_init_dentry
      ceph: protect access to d_parent
      ceph: protect d_parent access in ceph_d_revalidate
      ceph: avoid d_parent in ceph_dentry_hash; fix ceph_encode_fh() hashing bug
      ceph: document locking for ceph_set_dentry_offset
      ceph: explicitly reference rename old_dentry parent dir in request
      ceph: document unlocked d_parent accesses

Yehuda Sadeh (2):
      rbd: cancel watch request when releasing the device
      ceph: set up readahead size when rsize is not passed

 drivers/block/rbd.c            |   46 +++++++++++++++-
 fs/ceph/debugfs.c              |    2 +-
 fs/ceph/dir.c                  |  116 +++++++++++++++++++++++++---------------
 fs/ceph/export.c               |   24 +++++---
 fs/ceph/file.c                 |   61 +++++++++++++++------
 fs/ceph/inode.c                |   48 +++++++++-------
 fs/ceph/ioctl.c                |   15 +++++-
 fs/ceph/ioctl.h                |    1 +
 fs/ceph/mds_client.c           |   56 +++++++++++---------
 fs/ceph/mds_client.h           |    3 +-
 fs/ceph/snap.c                 |   25 +++++++--
 fs/ceph/super.c                |    7 ++-
 fs/ceph/super.h                |   20 +++----
 fs/ceph/xattr.c                |    8 ++-
 include/linux/ceph/messenger.h |    1 +
 net/ceph/messenger.c           |   12 ++---
 net/ceph/osd_client.c          |    6 ++
 17 files changed, 306 insertions(+), 145 deletions(-)

Re: [GIT PULL] Ceph updates for 3.1-rc1

[Christoph Hellwig]

On Tue, Jul 26, 2011 at 01:36:37PM -0700, Sage Weil wrote:
> Sage Weil (19):
>       ceph: add flags field to file_info
>       ceph: add F_SYNC file flag to force sync (non-O_DIRECT) io

Can you document what the exact semantics of this flag are, and how
they different from using O_SYNC/O_DSYNC.

Re: [GIT PULL] Ceph updates for 3.1-rc1

[Sage Weil]

On Sun, 31 Jul 2011, Christoph Hellwig wrote:
> On Tue, Jul 26, 2011 at 01:36:37PM -0700, Sage Weil wrote:
> > Sage Weil (19):
> >       ceph: add flags field to file_info
> >       ceph: add F_SYNC file flag to force sync (non-O_DIRECT) io
> 
> Can you document what the exact semantics of this flag are, and how
> they different from using O_SYNC/O_DSYNC.

I should document this... in the fs/ceph/ioctl.h?  Not sure where this 
normally goes.

The CEPH_F_SYNC flag forces synchronous reads/writes that bypass the page 
cache.  It's similar to O_DIRECT, but makes a copy of the data before 
doing the IO and does not have the alignment restrictions.

The real value is that it forces us to take a code path that's hard to 
trigger otherwise.  When a single client writes to a file, we can do 
buffered IO.  Even with O_SYNC/D_SYNC this just causes a flush after every 
write.  When multiple clients open a file for read/write, though, ceph 
bypasses the page cache and does synchronous IO to the server.  It's 
similar to the O_DIRECT path, but not quite, and doesn't get good test 
coverage without an easy way to use it.  

A ceph ioctl sets the CEPH_F_SYNC flag in the file private struct.  It's 
similar to the CEPH_F_LAZYIO ioctl, but with opposite results; that one 
avoids sync io despite multi-client write sharing (for applications who 
know what they're doing).

sage