[PATCH v3 00/21] NFS/RDMA client patches for 3.17

[Chuck Lever <chuck.lever@oracle.com>]

The main purpose of this series is to address connection drop
recovery issues by fixing FRMR re-use to make it less likely the
client will deadlock due to a memory management operation error.

Some clean-ups and other fixes are present as well.

See topic branch nfs-rdma-for-3.17 in

git://git.linux-nfs.org/projects/cel/cel-2.6.git

I tested with NFSv3 and NFSv4 on all three supported memory
registration modes. Used cthon04, iozone, and dbench with both
Solaris and Linux NFS/RDMA servers. Used xfstests with Linux.

v3:
Only two substantive changes:

- Patch 08/21 now uses generic IB helpers for managing FRMR
  rkeys

- Add Tested-by: from Steve Wise


v2:
Many patches from v1 have been written or replaced.

The MW ref counting approach in v1 is abandoned. Instead, I've
eliminated signaling FAST_REG_MR and LOCAL_INV, and added
appropriate recovery mechanisms after a transport reconnect that
should prevent rkey dis-synchrony entirely.

A couple of optimizations have been added, including:

- Allocating each MW separately rather than carving each out of a
  large piece of contiguous memory

- Now that the receive CQ upcall handler dequeues a bundle of CQEs
  at once, fire off the reply handler tasklet just once per upcall
  to reduce context switches and how often hard IRQs are disabled

Jury is still out on the latter.

--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com