[PATCH 0/38] pnfsd: Use layout state to track layouts and misc.

[PATCH 0/38] pnfsd: Use layout state to track layouts and misc.

[Benny Halevy]

The following patchset includes:
- API usage changes that originated in v3.8 around the use of nfsd_net
  to search clientid and stateid.
  - temporary removal of notify_deviceid and get_state pnfs callback API
    until we implement a mechanism to look up global stateids and clientids
    provided by the file system.
- rewriting of the layout tracking mechanism, using
  the layout_state structure to keep a list of all derived layout
  and enlisting the layout_state on its respective file and client rather
  than the layout structure themselves.
- various bug fixes

API related reverts:
[PATCH 01/38] Revert "SQUASHME: pnfsd: Fix compiler warning when
[PATCH 02/38] Revert "pnfsd: Clean up all stateids on DS on
[PATCH 03/38] Revert "pnfsd: update ds stateid from mds"
[PATCH 04/38] pnfsd-lexp: remove get_state
[PATCH 05/38] Revert "FIXME pnfsd: gfs2 temp turn off DS stateid
[PATCH 06/38] Revert "pnfsd: nfs4_preprocess_pnfs_ds_stateid"
[PATCH 07/38] Revert "pnfsd: Add way of determining if a client has
[PATCH 08/38] Revert "pnfsd: Notify device ID changes"
[PATCH 09/38] Revert "pnfsd: filelayout: file layout specific
[PATCH 10/38] Revert "pnfsd: Use ds_state_lock to protect retrieving
[PATCH 11/38] Revert "pnfsd: ds state"
[PATCH 12/38] Revert "pnfsd: decode getdeviceinfo notify types."
[PATCH 13/38] Revert "pnfsd: set_device_notify"
[PATCH 14/38] FIXME: temporarily disable create_layout_recall_list

API related changes:
[PATCH 15/38] SQUASHME: pnfsd: use nfsd_net for layoutget starting
[PATCH 16/38] SQUASHME: pnfsd: use nfsd4_init_callback starting v3.8
[PATCH 17/38] SQUASHME: use nfsd_net for layoutreturn starting v3.8

bug fixes:
[PATCH 18/38] SQUASHME: pnfsd: fix destroy_layout_state
[PATCH 19/38] SQUASHME: pnfsd: get rid of depdency on nfsd_serv
[PATCH 20/38] SQUASHME: pnfsd: merge_layout needs to acquire the
[PATCH 21/38] SQUASHME: pnfsd: return bool from merge_layout and fix
[PATCH 22/38] SQUASHME: pnfsd: hanging nfs4_layoutrecall on the
[PATCH 23/38] DEBUG: pnfsd: ASSERT_LAYOUT_LOCKED
[PATCH 24/38] pnfsd: cause oops when stop the nfs server

layout tracking:
[PATCH 25/38] pnfsd: layout state: rename fi_layout_states member
[PATCH 26/38] pnfsd: layout state: move initialization of ls_roc
[PATCH 27/38] pnfsd: layout state: introduce ls_file backpointer
[PATCH 28/38] pnfsd: layout state: introduce ls_client backpointer
[PATCH 29/38] pnfsd: layout state: hang layouts on layout state
[PATCH 30/38] pnfsd-lexp: fixup recall_layout
[PATCH 31/38] pnfsd: use the layout stateid for layout recalls
[PATCH 32/38] pnfsd: cosmetic fixes
[PATCH 33/38] pnfsd: pass layout stateid down to fs on layoutget
[PATCH 34/38] pnfsd: pass layout stateid down to fs on layoutreturn
[PATCH 35/38] pnfsd: use layout stateid for is_layout_recalled
[PATCH 36/38] pnfsd: deprecate cbl_seg.clientid
[PATCH 37/38] pnfsd: don't use clientid in merge_layout
[PATCH 38/38] pnfsd: remove clientid member from struct

Re: [PATCH 0/38] pnfsd: Use layout state to track layouts and misc.

[J. Bruce Fields]

On Thu, May 09, 2013 at 11:51:03AM +0300, Benny Halevy wrote:
> The following patchset includes:
> - API usage changes that originated in v3.8 around the use of nfsd_net
>   to search clientid and stateid.
>   - temporary removal of notify_deviceid and get_state pnfs callback API
>     until we implement a mechanism to look up global stateids and clientids
>     provided by the file system.
> - rewriting of the layout tracking mechanism, using
>   the layout_state structure to keep a list of all derived layout
>   and enlisting the layout_state on its respective file and client rather
>   than the layout structure themselves.
> - various bug fixes

Were there supposed to be patches following this?

Also, is there some minimal subset of the pnfsd patches that it would
make sense to start merging some day?

--b.

> 
> API related reverts:
> [PATCH 01/38] Revert "SQUASHME: pnfsd: Fix compiler warning when
> [PATCH 02/38] Revert "pnfsd: Clean up all stateids on DS on
> [PATCH 03/38] Revert "pnfsd: update ds stateid from mds"
> [PATCH 04/38] pnfsd-lexp: remove get_state
> [PATCH 05/38] Revert "FIXME pnfsd: gfs2 temp turn off DS stateid
> [PATCH 06/38] Revert "pnfsd: nfs4_preprocess_pnfs_ds_stateid"
> [PATCH 07/38] Revert "pnfsd: Add way of determining if a client has
> [PATCH 08/38] Revert "pnfsd: Notify device ID changes"
> [PATCH 09/38] Revert "pnfsd: filelayout: file layout specific
> [PATCH 10/38] Revert "pnfsd: Use ds_state_lock to protect retrieving
> [PATCH 11/38] Revert "pnfsd: ds state"
> [PATCH 12/38] Revert "pnfsd: decode getdeviceinfo notify types."
> [PATCH 13/38] Revert "pnfsd: set_device_notify"
> [PATCH 14/38] FIXME: temporarily disable create_layout_recall_list
> 
> API related changes:
> [PATCH 15/38] SQUASHME: pnfsd: use nfsd_net for layoutget starting
> [PATCH 16/38] SQUASHME: pnfsd: use nfsd4_init_callback starting v3.8
> [PATCH 17/38] SQUASHME: use nfsd_net for layoutreturn starting v3.8
> 
> bug fixes:
> [PATCH 18/38] SQUASHME: pnfsd: fix destroy_layout_state
> [PATCH 19/38] SQUASHME: pnfsd: get rid of depdency on nfsd_serv
> [PATCH 20/38] SQUASHME: pnfsd: merge_layout needs to acquire the
> [PATCH 21/38] SQUASHME: pnfsd: return bool from merge_layout and fix
> [PATCH 22/38] SQUASHME: pnfsd: hanging nfs4_layoutrecall on the
> [PATCH 23/38] DEBUG: pnfsd: ASSERT_LAYOUT_LOCKED
> [PATCH 24/38] pnfsd: cause oops when stop the nfs server
> 
> layout tracking:
> [PATCH 25/38] pnfsd: layout state: rename fi_layout_states member
> [PATCH 26/38] pnfsd: layout state: move initialization of ls_roc
> [PATCH 27/38] pnfsd: layout state: introduce ls_file backpointer
> [PATCH 28/38] pnfsd: layout state: introduce ls_client backpointer
> [PATCH 29/38] pnfsd: layout state: hang layouts on layout state
> [PATCH 30/38] pnfsd-lexp: fixup recall_layout
> [PATCH 31/38] pnfsd: use the layout stateid for layout recalls
> [PATCH 32/38] pnfsd: cosmetic fixes
> [PATCH 33/38] pnfsd: pass layout stateid down to fs on layoutget
> [PATCH 34/38] pnfsd: pass layout stateid down to fs on layoutreturn
> [PATCH 35/38] pnfsd: use layout stateid for is_layout_recalled
> [PATCH 36/38] pnfsd: deprecate cbl_seg.clientid
> [PATCH 37/38] pnfsd: don't use clientid in merge_layout
> [PATCH 38/38] pnfsd: remove clientid member from struct
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 0/38] pnfsd: Use layout state to track layouts and misc.

[Benny Halevy]

On 2013-05-10 20:29, J. Bruce Fields wrote:
> On Thu, May 09, 2013 at 11:51:03AM +0300, Benny Halevy wrote:
>> The following patchset includes:
>> - API usage changes that originated in v3.8 around the use of nfsd_net
>>   to search clientid and stateid.
>>   - temporary removal of notify_deviceid and get_state pnfs callback API
>>     until we implement a mechanism to look up global stateids and clientids
>>     provided by the file system.
>> - rewriting of the layout tracking mechanism, using
>>   the layout_state structure to keep a list of all derived layout
>>   and enlisting the layout_state on its respective file and client rather
>>   than the layout structure themselves.
>> - various bug fixes
> 
> Were there supposed to be patches following this?

Duh. I had a typo in the To address. Resent.

> 
> Also, is there some minimal subset of the pnfsd patches that it would
> make sense to start merging some day?

Absolutely.  Let me squash this patchset in and we can look at then
cleaned up pnfsd patchset.  The minimum use case would be either exofs or
the dlm use case.  We can technically separate out the layout recall path
but it won't be real without it.

Benny

> 
> --b.
> 
>>
>> API related reverts:
>> [PATCH 01/38] Revert "SQUASHME: pnfsd: Fix compiler warning when
>> [PATCH 02/38] Revert "pnfsd: Clean up all stateids on DS on
>> [PATCH 03/38] Revert "pnfsd: update ds stateid from mds"
>> [PATCH 04/38] pnfsd-lexp: remove get_state
>> [PATCH 05/38] Revert "FIXME pnfsd: gfs2 temp turn off DS stateid
>> [PATCH 06/38] Revert "pnfsd: nfs4_preprocess_pnfs_ds_stateid"
>> [PATCH 07/38] Revert "pnfsd: Add way of determining if a client has
>> [PATCH 08/38] Revert "pnfsd: Notify device ID changes"
>> [PATCH 09/38] Revert "pnfsd: filelayout: file layout specific
>> [PATCH 10/38] Revert "pnfsd: Use ds_state_lock to protect retrieving
>> [PATCH 11/38] Revert "pnfsd: ds state"
>> [PATCH 12/38] Revert "pnfsd: decode getdeviceinfo notify types."
>> [PATCH 13/38] Revert "pnfsd: set_device_notify"
>> [PATCH 14/38] FIXME: temporarily disable create_layout_recall_list
>>
>> API related changes:
>> [PATCH 15/38] SQUASHME: pnfsd: use nfsd_net for layoutget starting
>> [PATCH 16/38] SQUASHME: pnfsd: use nfsd4_init_callback starting v3.8
>> [PATCH 17/38] SQUASHME: use nfsd_net for layoutreturn starting v3.8
>>
>> bug fixes:
>> [PATCH 18/38] SQUASHME: pnfsd: fix destroy_layout_state
>> [PATCH 19/38] SQUASHME: pnfsd: get rid of depdency on nfsd_serv
>> [PATCH 20/38] SQUASHME: pnfsd: merge_layout needs to acquire the
>> [PATCH 21/38] SQUASHME: pnfsd: return bool from merge_layout and fix
>> [PATCH 22/38] SQUASHME: pnfsd: hanging nfs4_layoutrecall on the
>> [PATCH 23/38] DEBUG: pnfsd: ASSERT_LAYOUT_LOCKED
>> [PATCH 24/38] pnfsd: cause oops when stop the nfs server
>>
>> layout tracking:
>> [PATCH 25/38] pnfsd: layout state: rename fi_layout_states member
>> [PATCH 26/38] pnfsd: layout state: move initialization of ls_roc
>> [PATCH 27/38] pnfsd: layout state: introduce ls_file backpointer
>> [PATCH 28/38] pnfsd: layout state: introduce ls_client backpointer
>> [PATCH 29/38] pnfsd: layout state: hang layouts on layout state
>> [PATCH 30/38] pnfsd-lexp: fixup recall_layout
>> [PATCH 31/38] pnfsd: use the layout stateid for layout recalls
>> [PATCH 32/38] pnfsd: cosmetic fixes
>> [PATCH 33/38] pnfsd: pass layout stateid down to fs on layoutget
>> [PATCH 34/38] pnfsd: pass layout stateid down to fs on layoutreturn
>> [PATCH 35/38] pnfsd: use layout stateid for is_layout_recalled
>> [PATCH 36/38] pnfsd: deprecate cbl_seg.clientid
>> [PATCH 37/38] pnfsd: don't use clientid in merge_layout
>> [PATCH 38/38] pnfsd: remove clientid member from struct
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

[PATCH 01/38] Revert "SQUASHME: pnfsd: Fix compiler warning when CONFIG_SUNRPC_DEBUG is not set"

[Benny Halevy]

This reverts commit 93275cbb789cd18eda5f9fa91469fb9c28ed8896.
---
 fs/nfsd/nfs4pnfsds.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/fs/nfsd/nfs4pnfsds.c b/fs/nfsd/nfs4pnfsds.c
index ab67523..2732f20 100644
--- a/fs/nfsd/nfs4pnfsds.c
+++ b/fs/nfsd/nfs4pnfsds.c
@@ -170,9 +170,10 @@ static int stateid_hashval(stateid_t *stidp)
 	hashval = stateid_hashval(stidp);
 	list_for_each_entry(local, &ds_stid_hashtbl[hashval], ds_hash)
 		if (!memcmp(&local->ds_stid.si_opaque, &stidp->si_opaque, sizeof(stidp->si_opaque))) {
+			stateid_t *sid = &local->ds_stid;
 			dprintk("NFSD: %s <-- %p ds_flags %lx " STATEID_FMT "\n",
 				__func__, local, local->ds_flags,
-				STATEID_VAL(&local->ds_stid));
+				STATEID_VAL(sid));
 			return local;
 		}
 	return NULL;
-- 
1.7.11.7

[PATCH 02/38] Revert "pnfsd: Clean up all stateids on DS on shutdown"

[Benny Halevy]

This reverts commit 82d356f0294487cfc6b98d16ddea74d67eabddb4.

Conflicts:
	fs/nfsd/state.h
---
 fs/nfsd/nfs4pnfsds.c | 19 -------------------
 fs/nfsd/nfs4state.c  |  1 -
 fs/nfsd/state.h      |  2 --
 3 files changed, 22 deletions(-)

diff --git a/fs/nfsd/nfs4pnfsds.c b/fs/nfsd/nfs4pnfsds.c
index 2732f20..13fe811 100644
--- a/fs/nfsd/nfs4pnfsds.c
+++ b/fs/nfsd/nfs4pnfsds.c
@@ -274,25 +274,6 @@ static int stateid_hashval(stateid_t *stidp)
 	kref_get(&dsp->ds_ref);
 }
 
-void
-nfs4_pnfs_state_shutdown(void)
-{
-	struct pnfs_ds_stateid *dsp;
-	int i;
-
-	dprintk("pNFSD %s: -->\n", __func__);
-
-	ds_lock_state();
-	for (i = 0; i < STATEID_HASH_SIZE; i++) {
-		while (!list_empty(&ds_stid_hashtbl[i])) {
-			dsp = list_entry(ds_stid_hashtbl[i].next,
-					 struct pnfs_ds_stateid, ds_hash);
-			put_ds_stateid(dsp);
-		}
-	}
-	ds_unlock_state();
-}
-
 static struct pnfs_mds_id *
 alloc_init_mds_id(struct pnfs_get_state *gsp)
 {
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 3c25405..d2256d8 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -5090,7 +5090,6 @@ static int nfs4_state_create_net(struct net *net)
 
 	nfsd4_client_tracking_exit(net);
 	nfs4_state_destroy_net(net);
-	nfs4_pnfs_state_shutdown();
 }
 
 void
diff --git a/fs/nfsd/state.h b/fs/nfsd/state.h
index 9935edc..31d5d0f 100644
--- a/fs/nfsd/state.h
+++ b/fs/nfsd/state.h
@@ -531,7 +531,6 @@ extern struct nfs4_client_reclaim *nfs4_client_to_reclaim(const char *name,
 extern void pnfs_expire_client(struct nfs4_client *);
 extern void release_pnfs_ds_dev_list(struct nfs4_ol_stateid *);
 extern void nfs4_pnfs_state_init(void);
-extern void nfs4_pnfs_state_shutdown(void);
 extern void nfs4_ds_get_verifier(stateid_t *, struct super_block *, u32 *);
 extern int nfs4_preprocess_pnfs_ds_stateid(struct svc_fh *, stateid_t *);
 extern void pnfsd_roc(struct nfs4_client *clp, struct nfs4_file *fp);
@@ -540,7 +539,6 @@ static inline void nfsd4_free_pnfs_slabs(void) {}
 static inline int nfsd4_init_pnfs_slabs(void) { return 0; }
 static inline void pnfs_expire_client(struct nfs4_client *clp) {}
 static inline void release_pnfs_ds_dev_list(struct nfs4_ol_stateid *stp) {}
-static inline void nfs4_pnfs_state_shutdown(void) {}
 static inline void pnfsd_roc(struct nfs4_client *clp, struct nfs4_file *fp) {}
 #endif /* CONFIG_PNFSD */
 
-- 
1.7.11.7

[PATCH 03/38] Revert "pnfsd: update ds stateid from mds"

[Benny Halevy]

This reverts commit 9bf768bb5e8ea1d6c258fc59c779a26806fdbde9.
---
 fs/nfsd/nfs4pnfsds.c | 595 ---------------------------------------------------
 1 file changed, 595 deletions(-)
 delete mode 100644 fs/nfsd/nfs4pnfsds.c

diff --git a/fs/nfsd/nfs4pnfsds.c b/fs/nfsd/nfs4pnfsds.c
deleted file mode 100644
index 13fe811..0000000
--- a/fs/nfsd/nfs4pnfsds.c
+++ /dev/null
@@ -1,595 +0,0 @@
-/*
-*  linux/fs/nfsd/nfs4pnfsds.c
-*
-*  Copyright (c) 2005 The Regents of the University of Michigan.
-*  All rights reserved.
-*
-*  Andy Adamson <andros@umich.edu>
-*
-*  Redistribution and use in source and binary forms, with or without
-*  modification, are permitted provided that the following conditions
-*  are met:
-*
-*  1. Redistributions of source code must retain the above copyright
-*     notice, this list of conditions and the following disclaimer.
-*  2. Redistributions in binary form must reproduce the above copyright
-*     notice, this list of conditions and the following disclaimer in the
-*     documentation and/or other materials provided with the distribution.
-*  3. Neither the name of the University nor the names of its
-*     contributors may be used to endorse or promote products derived
-*     from this software without specific prior written permission.
-*
-*  THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED
-*  WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
-*  MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
-*  DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
-*  FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
-*  CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
-*  SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
-*  BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
-*  LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
-*  NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
-*  SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-*
-*/
-#if defined(CONFIG_PNFSD)
-
-#define NFSDDBG_FACILITY NFSDDBG_PNFS
-
-#include <linux/param.h>
-#include <linux/sunrpc/svc.h>
-#include <linux/sunrpc/debug.h>
-#include <linux/nfs4.h>
-#include <linux/exportfs.h>
-#include <linux/sched.h>
-
-#include "nfsd.h"
-#include "pnfsd.h"
-#include "state.h"
-
-/*
- * Hash tables for pNFS Data Server state
- *
- * mds_nodeid:	list of struct pnfs_mds_id one per Metadata server (MDS) using
- *		this data server (DS).
- *
- * mds_clid_hashtbl[]: uses clientid_hashval(), hash of all clientids obtained
- *			from any MDS.
- *
- * ds_stid_hashtbl[]: uses stateid_hashval(), hash of all stateids obtained
- *			from any MDS.
- *
- */
-/* Hash tables for clientid state */
-#define CLIENT_HASH_BITS                 4
-#define CLIENT_HASH_SIZE                (1 << CLIENT_HASH_BITS)
-#define CLIENT_HASH_MASK                (CLIENT_HASH_SIZE - 1)
-
-#define clientid_hashval(id) \
-	((id) & CLIENT_HASH_MASK)
-
-/* hash table for pnfs_ds_stateid */
-#define STATEID_HASH_BITS	10
-#define STATEID_HASH_SIZE	(1 << STATEID_HASH_BITS)
-#define STATEID_HASH_MASK	(STATEID_HASH_SIZE - 1)
-
-static int stateid_hashval(stateid_t *stidp)
-{
-	unsigned v = stidp->si_opaque.so_clid.cl_boot ^
-		     stidp->si_opaque.so_clid.cl_id ^
-		     stidp->si_opaque.so_id;
-	return v & STATEID_HASH_MASK;
-}
-
-static struct list_head mds_id_tbl;
-static struct list_head mds_clid_hashtbl[CLIENT_HASH_SIZE];
-static struct list_head ds_stid_hashtbl[STATEID_HASH_SIZE];
-
-static void put_ds_clientid(struct pnfs_ds_clientid *dcp);
-static void put_ds_mdsid(struct pnfs_mds_id *mdp);
-
-/* Mutex for data server state.  Needs to be separate from
- * mds state mutex since a node can be both mds and ds */
-static DEFINE_MUTEX(ds_mutex);
-static struct thread_info *ds_mutex_owner;
-
-static void
-ds_lock_state(void)
-{
-	mutex_lock(&ds_mutex);
-	ds_mutex_owner = current_thread_info();
-}
-
-static void
-ds_unlock_state(void)
-{
-	BUG_ON(ds_mutex_owner != current_thread_info());
-	ds_mutex_owner = NULL;
-	mutex_unlock(&ds_mutex);
-}
-
-static int
-cmp_clid(const clientid_t *cl1, const clientid_t *cl2)
-{
-	return (cl1->cl_boot == cl2->cl_boot) &&
-	       (cl1->cl_id == cl2->cl_id);
-}
-
-void
-nfs4_pnfs_state_init(void)
-{
-	int i;
-
-	for (i = 0; i < CLIENT_HASH_SIZE; i++)
-		INIT_LIST_HEAD(&mds_clid_hashtbl[i]);
-
-	for (i = 0; i < STATEID_HASH_SIZE; i++)
-		INIT_LIST_HEAD(&ds_stid_hashtbl[i]);
-
-	INIT_LIST_HEAD(&mds_id_tbl);
-}
-
-static struct pnfs_mds_id *
-find_pnfs_mds_id(u32 mdsid)
-{
-	struct pnfs_mds_id *local = NULL;
-
-	dprintk("pNFSD: %s\n", __func__);
-	list_for_each_entry(local, &mds_id_tbl, di_hash) {
-		if (local->di_mdsid == mdsid)
-			return local;
-	}
-	return NULL;
-}
-
-static struct pnfs_ds_clientid *
-find_pnfs_ds_clientid(const clientid_t *clid)
-{
-	struct pnfs_ds_clientid *local = NULL;
-	unsigned int hashval;
-
-	dprintk("pNFSD: %s\n", __func__);
-
-	hashval = clientid_hashval(clid->cl_id);
-	list_for_each_entry(local, &mds_clid_hashtbl[hashval], dc_hash) {
-		if (cmp_clid(&local->dc_mdsclid, clid))
-			return local;
-	}
-	return NULL;
-}
-
-/* FIXME: Can we use the server generic idr based stateid bookeeping introduced in 3.2? */
-static struct pnfs_ds_stateid *
-find_pnfs_ds_stateid(stateid_t *stidp)
-{
-	struct pnfs_ds_stateid *local = NULL;
-	unsigned int hashval;
-
-	dprintk("pNFSD: %s\n", __func__);
-
-	hashval = stateid_hashval(stidp);
-	list_for_each_entry(local, &ds_stid_hashtbl[hashval], ds_hash)
-		if (!memcmp(&local->ds_stid.si_opaque, &stidp->si_opaque, sizeof(stidp->si_opaque))) {
-			stateid_t *sid = &local->ds_stid;
-			dprintk("NFSD: %s <-- %p ds_flags %lx " STATEID_FMT "\n",
-				__func__, local, local->ds_flags,
-				STATEID_VAL(sid));
-			return local;
-		}
-	return NULL;
-}
-
-static void
-release_ds_mdsid(struct kref *kref)
-{
-	struct pnfs_mds_id *mdp =
-		container_of(kref, struct pnfs_mds_id, di_ref);
-	dprintk("pNFSD: %s\n", __func__);
-
-	list_del(&mdp->di_hash);
-	list_del(&mdp->di_mdsclid);
-	kfree(mdp);
-}
-
-static void
-release_ds_clientid(struct kref *kref)
-{
-	struct pnfs_ds_clientid *dcp =
-		container_of(kref, struct pnfs_ds_clientid, dc_ref);
-	struct pnfs_mds_id *mdp;
-	dprintk("pNFSD: %s\n", __func__);
-
-	mdp = find_pnfs_mds_id(dcp->dc_mdsid);
-	if (mdp)
-		put_ds_mdsid(mdp);
-
-	list_del(&dcp->dc_hash);
-	list_del(&dcp->dc_stateid);
-	list_del(&dcp->dc_permdsid);
-	kfree(dcp);
-}
-
-static void
-release_ds_stateid(struct kref *kref)
-{
-	struct pnfs_ds_stateid *dsp =
-		container_of(kref, struct pnfs_ds_stateid, ds_ref);
-	struct pnfs_ds_clientid *dcp;
-	dprintk("pNFS %s: dsp %p\n", __func__, dsp);
-
-	dcp = find_pnfs_ds_clientid(&dsp->ds_mdsclid);
-	if (dcp)
-		put_ds_clientid(dcp);
-
-	list_del(&dsp->ds_hash);
-	list_del(&dsp->ds_perclid);
-	kfree(dsp);
-}
-
-static void
-put_ds_clientid(struct pnfs_ds_clientid *dcp)
-{
-	dprintk("pNFS %s: dcp %p ref %d\n", __func__, dcp,
-		atomic_read(&dcp->dc_ref.refcount));
-	kref_put(&dcp->dc_ref, release_ds_clientid);
-}
-
-static void
-get_ds_clientid(struct pnfs_ds_clientid *dcp)
-{
-	dprintk("pNFS %s: dcp %p ref %d\n", __func__, dcp,
-		atomic_read(&dcp->dc_ref.refcount));
-	kref_get(&dcp->dc_ref);
-}
-
-static void
-put_ds_mdsid(struct pnfs_mds_id *mdp)
-{
-	dprintk("pNFS %s: mdp %p ref %d\n", __func__, mdp,
-		atomic_read(&mdp->di_ref.refcount));
-	kref_put(&mdp->di_ref, release_ds_mdsid);
-}
-
-static void
-get_ds_mdsid(struct pnfs_mds_id *mdp)
-{
-	dprintk("pNFS %s: mdp %p ref %d\n", __func__, mdp,
-		atomic_read(&mdp->di_ref.refcount));
-	kref_get(&mdp->di_ref);
-}
-
-static void
-put_ds_stateid(struct pnfs_ds_stateid *dsp)
-{
-	dprintk("pNFS %s: dsp %p ref %d\n", __func__, dsp,
-		atomic_read(&dsp->ds_ref.refcount));
-	kref_put(&dsp->ds_ref, release_ds_stateid);
-}
-
-static void
-get_ds_stateid(struct pnfs_ds_stateid *dsp)
-{
-	dprintk("pNFS %s: dsp %p ref %d\n", __func__, dsp,
-		atomic_read(&dsp->ds_ref.refcount));
-	kref_get(&dsp->ds_ref);
-}
-
-static struct pnfs_mds_id *
-alloc_init_mds_id(struct pnfs_get_state *gsp)
-{
-	struct pnfs_mds_id *mdp;
-
-	dprintk("pNFSD: %s\n", __func__);
-
-	mdp = kmalloc(sizeof(*mdp), GFP_KERNEL);
-	if (!mdp)
-		return NULL;
-	INIT_LIST_HEAD(&mdp->di_hash);
-	INIT_LIST_HEAD(&mdp->di_mdsclid);
-	list_add(&mdp->di_hash, &mds_id_tbl);
-	mdp->di_mdsid = gsp->dsid;
-	mdp->di_mdsboot = 0;
-	kref_init(&mdp->di_ref);
-	return mdp;
-}
-
-static struct pnfs_ds_clientid *
-alloc_init_ds_clientid(struct pnfs_get_state *gsp)
-{
-	struct pnfs_mds_id *mdp;
-	struct pnfs_ds_clientid *dcp;
-	clientid_t *clid = (clientid_t *)&gsp->clid;
-	unsigned int hashval = clientid_hashval(clid->cl_id);
-
-	dprintk("pNFSD: %s\n", __func__);
-
-	mdp = find_pnfs_mds_id(gsp->dsid);
-	if (!mdp) {
-		mdp = alloc_init_mds_id(gsp);
-		if (!mdp)
-			return NULL;
-	} else {
-		get_ds_mdsid(mdp);
-	}
-
-	dcp = kmalloc(sizeof(*dcp), GFP_KERNEL);
-	if (!dcp)
-		return NULL;
-
-	INIT_LIST_HEAD(&dcp->dc_hash);
-	INIT_LIST_HEAD(&dcp->dc_stateid);
-	INIT_LIST_HEAD(&dcp->dc_permdsid);
-	list_add(&dcp->dc_hash, &mds_clid_hashtbl[hashval]);
-	list_add(&dcp->dc_permdsid, &mdp->di_mdsclid);
-	dcp->dc_mdsclid = *clid;
-	kref_init(&dcp->dc_ref);
-	dcp->dc_mdsid = gsp->dsid;
-	return dcp;
-}
-
-static struct pnfs_ds_stateid *
-alloc_init_ds_stateid(struct svc_fh *cfh, stateid_t *stidp)
-{
-	struct pnfs_ds_stateid *dsp;
-
-	dprintk("pNFSD: %s\n", __func__);
-
-	dsp = kmalloc(sizeof(*dsp), GFP_KERNEL);
-	if (!dsp)
-		return dsp;
-
-	INIT_LIST_HEAD(&dsp->ds_hash);
-	INIT_LIST_HEAD(&dsp->ds_perclid);
-	memcpy(&dsp->ds_stid, stidp, sizeof(dsp->ds_stid));
-	fh_copy_shallow(&dsp->ds_fh, &cfh->fh_handle);
-	dsp->ds_access = 0;
-	dsp->ds_status = 0;
-	dsp->ds_flags = 0L;
-	kref_init(&dsp->ds_ref);
-	set_bit(DS_STATEID_NEW, &dsp->ds_flags);
-	clear_bit(DS_STATEID_VALID, &dsp->ds_flags);
-	clear_bit(DS_STATEID_ERROR, &dsp->ds_flags);
-	init_waitqueue_head(&dsp->ds_waitq);
-
-	list_add(&dsp->ds_hash, &ds_stid_hashtbl[stateid_hashval(stidp)]);
-	dprintk("pNFSD: %s <-- dsp %p\n", __func__, dsp);
-	return dsp;
-}
-
-static int
-update_ds_stateid(struct pnfs_ds_stateid *dsp, struct svc_fh *cfh,
-		  struct pnfs_get_state *gsp)
-{
-	struct pnfs_ds_clientid *dcp;
-	int new = 0;
-
-	dprintk("pNFSD: %s dsp %p\n", __func__, dsp);
-
-	dcp = find_pnfs_ds_clientid((clientid_t *)&gsp->clid);
-	if (!dcp) {
-		dcp = alloc_init_ds_clientid(gsp);
-		if (!dcp)
-			return 1;
-		new = 1;
-	}
-	if (test_bit(DS_STATEID_NEW, &dsp->ds_flags)) {
-		list_add(&dsp->ds_perclid, &dcp->dc_stateid);
-		if (!new)
-			get_ds_clientid(dcp);
-	}
-
-	memcpy(&dsp->ds_stid, &gsp->stid, sizeof(stateid_t));
-	dsp->ds_access = gsp->access;
-	dsp->ds_status = 0;
-	dsp->ds_verifier[0] = gsp->verifier[0];
-	dsp->ds_verifier[1] = gsp->verifier[1];
-	memcpy(&dsp->ds_mdsclid, &gsp->clid, sizeof(clientid_t));
-	set_bit(DS_STATEID_VALID, &dsp->ds_flags);
-	clear_bit(DS_STATEID_ERROR, &dsp->ds_flags);
-	clear_bit(DS_STATEID_NEW, &dsp->ds_flags);
-	return 0;
-}
-
-int
-nfs4_pnfs_cb_change_state(struct pnfs_get_state *gs)
-{
-	stateid_t *stid = (stateid_t *)&gs->stid;
-	struct pnfs_ds_stateid *dsp;
-
-	dprintk("pNFSD: %s stateid=" STATEID_FMT "\n", __func__,
-		STATEID_VAL(stid));
-
-	ds_lock_state();
-	dsp = find_pnfs_ds_stateid(stid);
-	if (dsp)
-		put_ds_stateid(dsp);
-	ds_unlock_state();
-
-	dprintk("pNFSD: %s dsp %p\n", __func__, dsp);
-
-	if (dsp)
-		return 0;
-	return -ENOENT;
-}
-
-/* Retrieves and validates stateid.
- * If stateid exists and its fields match, return it.
- * If stateid exists but either the generation or
- * ownerids don't match, check with mds to see if it is valid.
- * If the stateid doesn't exist, the first thread creates a
- * invalid *marker* stateid, then checks to see if the
- * stateid exists on the mds.  If so, it validates the *marker*
- * stateid and updates its fields.  Subsequent threads that
- * find the *marker* stateid wait until it is valid or an error
- * occurs.
- * Called with ds_state_lock.
- */
-static struct pnfs_ds_stateid *
-nfsv4_ds_get_state(struct svc_fh *cfh, stateid_t *stidp)
-{
-	struct inode *ino = cfh->fh_dentry->d_inode;
-	struct super_block *sb;
-	struct pnfs_ds_stateid *dsp = NULL;
-	struct pnfs_get_state gs = {
-		.access = 0,
-	};
-	int status = 0, waiter = 0;
-
-	dprintk("pNFSD: %s -->\n", __func__);
-
-	dsp = find_pnfs_ds_stateid(stidp);
-	/* Note: we assume same endianess on MDS and DS */
-	if (dsp && test_bit(DS_STATEID_VALID, &dsp->ds_flags) &&
-	    (stidp->si_generation == dsp->ds_stid.si_generation))
-		goto out_noput;
-
-	sb = ino->i_sb;
-	if (!sb || !sb->s_pnfs_op->get_state)
-		goto out_noput;
-
-	/* Uninitialize current state if it exists yet it doesn't match.
-	 * If it is already invalid, another thread is checking state */
-	if (dsp) {
-		if (!test_and_clear_bit(DS_STATEID_VALID, &dsp->ds_flags))
-			waiter = 1;
-	} else {
-		dsp = alloc_init_ds_stateid(cfh, stidp);
-		if (!dsp)
-			goto out_noput;
-	}
-
-	dprintk("pNFSD: %s Starting loop\n", __func__);
-	get_ds_stateid(dsp);
-	while (!test_bit(DS_STATEID_VALID, &dsp->ds_flags)) {
-		ds_unlock_state();
-
-		/* Another thread is checking the state */
-		if (waiter) {
-			dprintk("pNFSD: %s waiting\n", __func__);
-			wait_event_interruptible_timeout(dsp->ds_waitq,
-				(test_bit(DS_STATEID_VALID, &dsp->ds_flags) ||
-				 test_bit(DS_STATEID_ERROR, &dsp->ds_flags)),
-				 msecs_to_jiffies(1024));
-			dprintk("pNFSD: %s awake\n", __func__);
-			ds_lock_state();
-			if (test_bit(DS_STATEID_ERROR, &dsp->ds_flags))
-				goto out;
-
-			continue;
-		}
-
-		/* Validate stateid on mds */
-		dprintk("pNFSD: %s Checking state on MDS\n", __func__);
-		memcpy(&gs.stid, stidp, sizeof(gs.stid));
-		status = sb->s_pnfs_op->get_state(ino, &cfh->fh_handle, &gs);
-		dprintk("pNFSD: %s from MDS status %d\n", __func__, status);
-		ds_lock_state();
-		/* if !status and stateid is valid, update id and mark valid */
-		if (status || update_ds_stateid(dsp, cfh, &gs)) {
-			set_bit(DS_STATEID_ERROR, &dsp->ds_flags);
-			/* remove invalid stateid from list */
-			put_ds_stateid(dsp);
-			wake_up(&dsp->ds_waitq);
-			goto out;
-		}
-
-		wake_up(&dsp->ds_waitq);
-	}
-out:
-	if (dsp)
-		put_ds_stateid(dsp);
-out_noput:
-	if (dsp)
-		dprintk("pNFSD: %s <-- dsp %p ds_flags %lx " STATEID_FMT "\n",
-			__func__, dsp, dsp->ds_flags, STATEID_VAL(&dsp->ds_stid));
-	/* If error, return null */
-	if (dsp && test_bit(DS_STATEID_ERROR, &dsp->ds_flags))
-		dsp = NULL;
-	dprintk("pNFSD: %s <-- dsp %p\n", __func__, dsp);
-	return dsp;
-}
-
-int
-nfs4_preprocess_pnfs_ds_stateid(struct svc_fh *cfh, stateid_t *stateid)
-{
-	struct pnfs_ds_stateid *dsp;
-	int status = 0;
-
-	dprintk("pNFSD: %s --> " STATEID_FMT "\n", __func__,
-		STATEID_VAL(stateid));
-
-	/* Must release state lock while verifying stateid on mds */
-	nfs4_unlock_state();
-	ds_lock_state();
-	dsp = nfsv4_ds_get_state(cfh, stateid);
-	if (dsp) {
-		get_ds_stateid(dsp);
-		dprintk("pNFSD: %s Found " STATEID_FMT "\n", __func__,
-			STATEID_VAL(&dsp->ds_stid));
-
-		dprintk("NFSD: %s: dsp %p fh_size %u:%u "
-			"fh [%08x:%08x:%08x:%08x]:[%08x:%08x:%08x:%08x] "
-			"gen %x:%x\n",
-			__func__, dsp,
-			cfh->fh_handle.fh_size, dsp->ds_fh.fh_size,
-			((unsigned *)&cfh->fh_handle.fh_base)[0],
-			((unsigned *)&cfh->fh_handle.fh_base)[1],
-			((unsigned *)&cfh->fh_handle.fh_base)[2],
-			((unsigned *)&cfh->fh_handle.fh_base)[3],
-			((unsigned *)&dsp->ds_fh.fh_base)[0],
-			((unsigned *)&dsp->ds_fh.fh_base)[1],
-			((unsigned *)&dsp->ds_fh.fh_base)[2],
-			((unsigned *)&dsp->ds_fh.fh_base)[3],
-			stateid->si_generation, dsp->ds_stid.si_generation);
-	}
-
-	if (!dsp ||
-	    (cfh->fh_handle.fh_size != dsp->ds_fh.fh_size) ||
-	    (memcmp(&cfh->fh_handle.fh_base, &dsp->ds_fh.fh_base,
-		    dsp->ds_fh.fh_size) != 0) ||
-	    (stateid->si_generation > dsp->ds_stid.si_generation))
-		status = nfserr_bad_stateid;
-	else if (stateid->si_generation < dsp->ds_stid.si_generation)
-		status = nfserr_old_stateid;
-
-	if (dsp)
-		put_ds_stateid(dsp);
-	ds_unlock_state();
-	nfs4_lock_state();
-	dprintk("pNFSD: %s <-- status %d\n", __func__, be32_to_cpu(status));
-	return status;
-}
-
-void
-nfs4_ds_get_verifier(stateid_t *stateid, struct super_block *sb, u32 *p)
-{
-	struct pnfs_ds_stateid *dsp = NULL;
-
-	dprintk("pNFSD: %s --> stid %p\n", __func__, stateid);
-
-	ds_lock_state();
-	if (stateid != NULL) {
-		dsp = find_pnfs_ds_stateid(stateid);
-		if (dsp)
-			get_ds_stateid(dsp);
-	}
-
-	/* XXX: Should we fetch the stateid or wait if some other
-	 * thread is currently retrieving the stateid ? */
-	if (dsp && test_bit(DS_STATEID_VALID, &dsp->ds_flags)) {
-		*p++ = dsp->ds_verifier[0];
-		*p++ = dsp->ds_verifier[1];
-		put_ds_stateid(dsp);
-	} else {
-		/* must be on MDS */
-		ds_unlock_state();
-		sb->s_pnfs_op->get_verifier(sb, p);
-		ds_lock_state();
-		p += 2;
-	}
-	ds_unlock_state();
-	dprintk("pNFSD: %s <-- dsp %p\n", __func__, dsp);
-	return;
-}
-
-#endif /* CONFIG_PNFSD */
-- 
1.7.11.7

[PATCH 04/38] pnfsd-lexp: remove get_state

[Benny Halevy]

Signed-off-by: Benny Halevy <bhalevy@tonian.com>
---
 fs/nfsd/pnfsd_lexp.c | 7 -------
 1 file changed, 7 deletions(-)

diff --git a/fs/nfsd/pnfsd_lexp.c b/fs/nfsd/pnfsd_lexp.c
index bcdb80f..5d448c2 100644
--- a/fs/nfsd/pnfsd_lexp.c
+++ b/fs/nfsd/pnfsd_lexp.c
@@ -223,12 +223,6 @@ static int get_stripe_unit(int blocksize)
 	return 0;
 }
 
-static int pnfsd_lexp_get_state(struct inode *inode, struct knfsd_fh *fh,
-				struct pnfs_get_state *p)
-{
-	return 0;	/* just use the current stateid */
-}
-
 static struct pnfs_export_operations pnfsd_lexp_ops = {
 	.layout_type = pnfsd_lexp_layout_type,
 	.get_device_info = pnfsd_lexp_get_device_info,
@@ -236,7 +230,6 @@ static int pnfsd_lexp_get_state(struct inode *inode, struct knfsd_fh *fh,
 	.layout_get = pnfsd_lexp_layout_get,
 	.layout_commit = pnfsd_lexp_layout_commit,
 	.layout_return = pnfsd_lexp_layout_return,
-	.get_state = pnfsd_lexp_get_state,
 };
 
 void
-- 
1.7.11.7

[PATCH 05/38] Revert "FIXME pnfsd: gfs2 temp turn off DS stateid processing."

[Benny Halevy]

This reverts commit 9f47c0b97353c433f6c83a1bc665ea76eb12e25f.
---
 fs/nfsd/nfs4state.c | 7 -------
 1 file changed, 7 deletions(-)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index d2256d8..84fa1af 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -3556,15 +3556,8 @@ __be32 nfsd4_lookup_stateid(stateid_t *stateid, unsigned char typemask,
 		if (ZERO_STATEID(stateid) || ONE_STATEID(stateid))
 			status = nfserr_bad_stateid;
 		else
-#ifdef CONFIG_GFS2_FS_LOCKING_DLM
-		{
-			dprintk("%s Don't check DS stateid\n", __func__);
-			return 0;
-		}
-#else /* CONFIG_GFS2_FS_LOCKING_DLM */
 			status = nfs4_preprocess_pnfs_ds_stateid(current_fh,
 								 stateid);
-#endif /* CONFIG_GFS2_FS_LOCKING_DLM */
 		goto out;
 	}
 #endif /* CONFIG_PNFSD */
-- 
1.7.11.7

[PATCH 06/38] Revert "pnfsd: nfs4_preprocess_pnfs_ds_stateid"

[Benny Halevy]

This reverts commit 83d662282b879bc018b8a48229d67218d45b16fa.

Conflicts:
	fs/nfsd/state.h
---
 fs/nfsd/nfs4state.c | 12 ++----------
 fs/nfsd/state.h     |  2 --
 2 files changed, 2 insertions(+), 12 deletions(-)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 84fa1af..b404d33 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -3551,16 +3551,8 @@ __be32 nfsd4_lookup_stateid(stateid_t *stateid, unsigned char typemask,
 	if (grace_disallows_io(net, ino))
 		return nfserr_grace;
 
-#if defined(CONFIG_PNFSD)
-	if (pnfs_fh_is_ds(&current_fh->fh_handle)) {
-		if (ZERO_STATEID(stateid) || ONE_STATEID(stateid))
-			status = nfserr_bad_stateid;
-		else
-			status = nfs4_preprocess_pnfs_ds_stateid(current_fh,
-								 stateid);
-		goto out;
-	}
-#endif /* CONFIG_PNFSD */
+	if (pnfs_fh_is_ds(&current_fh->fh_handle))
+		return 0;
 
 	if (ZERO_STATEID(stateid) || ONE_STATEID(stateid))
 		return check_special_stateids(net, current_fh, stateid, flags);
diff --git a/fs/nfsd/state.h b/fs/nfsd/state.h
index 31d5d0f..788e5e8 100644
--- a/fs/nfsd/state.h
+++ b/fs/nfsd/state.h
@@ -531,8 +531,6 @@ extern struct nfs4_client_reclaim *nfs4_client_to_reclaim(const char *name,
 extern void pnfs_expire_client(struct nfs4_client *);
 extern void release_pnfs_ds_dev_list(struct nfs4_ol_stateid *);
 extern void nfs4_pnfs_state_init(void);
-extern void nfs4_ds_get_verifier(stateid_t *, struct super_block *, u32 *);
-extern int nfs4_preprocess_pnfs_ds_stateid(struct svc_fh *, stateid_t *);
 extern void pnfsd_roc(struct nfs4_client *clp, struct nfs4_file *fp);
 #else /* CONFIG_PNFSD */
 static inline void nfsd4_free_pnfs_slabs(void) {}
-- 
1.7.11.7

[PATCH 07/38] Revert "pnfsd: Add way of determining if a client has a device."

[Benny Halevy]

This reverts commit 3573c750c335ddff65c063f12bda1b54b288a674.
---
 fs/nfsd/nfs4pnfsd.c | 35 -----------------------------------
 fs/nfsd/nfs4proc.c  |  5 -----
 fs/nfsd/nfs4state.c |  1 -
 fs/nfsd/pnfsd.h     |  2 --
 fs/nfsd/state.h     |  1 -
 fs/nfsd/xdr4.h      |  1 -
 6 files changed, 45 deletions(-)

diff --git a/fs/nfsd/nfs4pnfsd.c b/fs/nfsd/nfs4pnfsd.c
index 35f43cf..8653719 100644
--- a/fs/nfsd/nfs4pnfsd.c
+++ b/fs/nfsd/nfs4pnfsd.c
@@ -114,35 +114,6 @@ struct sbid_tracker {
 	return 0;
 }
 
-/* XXX: Need to implement the notify types and track which
- * clients have which devices. */
-void pnfs_set_device_notify(clientid_t *clid, unsigned int types)
-{
-	struct nfs4_client *clp;
-	dprintk("%s: -->\n", __func__);
-
-	nfs4_lock_state();
-	/* Indicate that client has a device so we can only notify
-	 * the correct clients */
-	clp = find_confirmed_client(clid, true);
-	if (clp) {
-		atomic_inc(&clp->cl_deviceref);
-		dprintk("%s: Incr device count (clnt %p) to %d\n",
-			__func__, clp, atomic_read(&clp->cl_deviceref));
-	}
-	nfs4_unlock_state();
-}
-
-/* Clear notifications for this client
- * XXX: Do we need to loop through a clean up all
- *      krefs when nfsd cleans up the client? */
-void pnfs_clear_device_notify(struct nfs4_client *clp)
-{
-	atomic_dec(&clp->cl_deviceref);
-	dprintk("%s: Decr device count (clnt %p) to %d\n",
-		__func__, clp, atomic_read(&clp->cl_deviceref));
-}
-
 /*
  * Note: must be called under the state lock
  */
@@ -1593,9 +1564,6 @@ struct create_device_notify_list_arg {
 	struct nfs4_notify_device *cbnd;
 	struct create_device_notify_list_arg *arg = p;
 
-	if (atomic_read(&clp->cl_deviceref) <= 0)
-		return 0;
-
 	cbnd = kzalloc(sizeof(*cbnd), GFP_KERNEL);
 	if (!cbnd)
 		return -ENOMEM;
@@ -1633,7 +1601,6 @@ int nfsd_device_notify_cb(struct super_block *sb,
 			  struct nfsd4_pnfs_cb_dev_list *ndl)
 {
 	struct nfs4_notify_device *cbnd;
-	struct nfs4_client *nd_client;
 	unsigned int notify_num = 0;
 	int status = 0;
 	struct list_head todolist;
@@ -1653,9 +1620,7 @@ int nfsd_device_notify_cb(struct super_block *sb,
 		cbnd = list_entry(todolist.next, struct nfs4_notify_device,
 				  nd_perclnt);
 		list_del_init(&cbnd->nd_perclnt);
-		nd_client = cbnd->nd_client;
 		nfsd4_cb_notify_device(cbnd);
-		pnfs_clear_device_notify(nd_client);
 		notify_num++;
 	}
 
diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index 4a79281..2f1786a 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -1143,7 +1143,6 @@ static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write)
 {
 	struct super_block *sb;
 	int status;
-	clientid_t clid;
 
 	dprintk("%s: layout_type %u dev_id %llx:%llx maxcnt %u\n",
 	       __func__, gdp->gd_layout_type, gdp->gd_devid.sbid,
@@ -1166,10 +1165,6 @@ static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write)
 
 	/* Set up arguments so device can be retrieved at encode time */
 	gdp->gd_sb = sb;
-
-	/* Update notifications */
-	copy_clientid(&clid, cstate->session);
-	pnfs_set_device_notify(&clid, gdp->gd_notify_types);
 out:
 	return status;
 }
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index b404d33..664c7cb 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -1324,7 +1324,6 @@ static struct nfs4_client *create_client(struct xdr_netobj name,
 #if defined(CONFIG_PNFSD)
 	INIT_LIST_HEAD(&clp->cl_layouts);
 	INIT_LIST_HEAD(&clp->cl_layoutrecalls);
-	atomic_set(&clp->cl_deviceref, 0);
 #endif /* CONFIG_PNFSD */
 	INIT_LIST_HEAD(&clp->cl_lru);
 	INIT_LIST_HEAD(&clp->cl_callbacks);
diff --git a/fs/nfsd/pnfsd.h b/fs/nfsd/pnfsd.h
index 35859ff..5b3657c 100644
--- a/fs/nfsd/pnfsd.h
+++ b/fs/nfsd/pnfsd.h
@@ -131,8 +131,6 @@ int nfsd_layout_recall_cb(struct super_block *, struct inode *,
 int nfsd_device_notify_cb(struct super_block *,
 			  struct nfsd4_pnfs_cb_dev_list *);
 void nfsd4_cb_notify_device(struct nfs4_notify_device *);
-void pnfs_set_device_notify(clientid_t *, unsigned int types);
-void pnfs_clear_device_notify(struct nfs4_client *);
 
 static inline struct nfs4_layout_state *layoutstateid(struct nfs4_stid *s)
 {
diff --git a/fs/nfsd/state.h b/fs/nfsd/state.h
index 788e5e8..774927c 100644
--- a/fs/nfsd/state.h
+++ b/fs/nfsd/state.h
@@ -285,7 +285,6 @@ struct nfs4_client {
 	struct list_head	cl_layouts;	/* outstanding layouts */
 	struct list_head	cl_layoutrecalls; /* outstanding layoutrecall
 						     callbacks */
-	atomic_t		cl_deviceref;	/* Num outstanding devs */
 #endif /* CONFIG_PNFSD */
 };
 
diff --git a/fs/nfsd/xdr4.h b/fs/nfsd/xdr4.h
index 617e195..459d8ba 100644
--- a/fs/nfsd/xdr4.h
+++ b/fs/nfsd/xdr4.h
@@ -432,7 +432,6 @@ struct nfsd4_pnfs_getdevinfo {
 	struct nfsd4_pnfs_deviceid gd_devid;	/* request */
 	u32			gd_layout_type;	/* request */
 	u32			gd_maxcount;	/* request */
-	u32			gd_notify_types;/* request */
 	struct super_block	*gd_sb;
 };
 
-- 
1.7.11.7

[PATCH 08/38] Revert "pnfsd: Notify device ID changes"

[Benny Halevy]

This reverts commit 1a4f72df8cd69c8175677feaf7dd498edb6ff675.

Conflicts:
	fs/nfsd/nfs4callback.c
	fs/nfsd/nfs4pnfsd.c
	include/linux/nfsd/nfsd4_pnfs.h
---
 fs/nfsd/export.c                |   1 -
 fs/nfsd/nfs4callback.c          | 172 ----------------------------------------
 fs/nfsd/nfs4pnfsd.c             |  76 ------------------
 fs/nfsd/pnfsd.h                 |  13 ---
 include/linux/nfsd/nfsd4_pnfs.h |  14 ----
 5 files changed, 276 deletions(-)

diff --git a/fs/nfsd/export.c b/fs/nfsd/export.c
index e54dccf..698ca5c 100644
--- a/fs/nfsd/export.c
+++ b/fs/nfsd/export.c
@@ -346,7 +346,6 @@ static int svc_export_upcall(struct cache_detail *cd, struct cache_head *h)
 #if defined(CONFIG_PNFSD)
 static struct pnfsd_cb_operations pnfsd_cb_op = {
 	.cb_layout_recall = nfsd_layout_recall_cb,
-	.cb_device_notify = nfsd_device_notify_cb,
 
 	.cb_get_state = nfs4_pnfs_cb_get_state,
 	.cb_change_state = nfs4_pnfs_cb_change_state,
diff --git a/fs/nfsd/nfs4callback.c b/fs/nfsd/nfs4callback.c
index 89a671f..3ab44f7 100644
--- a/fs/nfsd/nfs4callback.c
+++ b/fs/nfsd/nfs4callback.c
@@ -52,7 +52,6 @@ enum {
 	NFSPROC4_CLNT_CB_RECALL,
 	NFSPROC4_CLNT_CB_SEQUENCE,
 	NFSPROC4_CLNT_CB_LAYOUT,
-	NFSPROC4_CLNT_CB_DEVICE,
 };
 
 #define NFS4_MAXTAGLEN		20
@@ -85,12 +84,6 @@ enum {
 #define NFS4_dec_cb_layout_sz		(cb_compound_dec_hdr_sz  +      \
 					cb_sequence_dec_sz +            \
 					op_dec_sz)
-#define NFS4_enc_cb_device_sz		(cb_compound_enc_hdr_sz +       \
-					cb_sequence_enc_sz +            \
-					1 + 6)
-#define NFS4_dec_cb_device_sz		(cb_compound_dec_hdr_sz  +      \
-					cb_sequence_dec_sz +            \
-					op_dec_sz)
 
 struct nfs4_cb_compound_hdr {
 	/* args */
@@ -460,67 +453,6 @@ static void encode_cb_layout4args(struct xdr_stream *xdr,
 	}
 	hdr->nops++;
 }
-
-/*
- * CB_NOTIFY_DEVICEID4args
- *
- *     typedef opaque notifylist4<>;
- *
- *     struct notify4 {
- *             bitmap4         notify_mask;
- *             notifylist4     notify_vals;
- *     };
- *
- *     struct CB_NOTIFY_DEVICEID4args {
- *             notify4 cnda_changes<>;
- *     };
- */
-static void encode_cb_device4args(struct xdr_stream *xdr,
-				  const struct nfs4_notify_device *nd,
-				  struct nfs4_cb_compound_hdr *hdr)
-{
-	u32 *p;
-	int i;
-	int len					= nd->nd_list->cbd_len;
-	struct nfsd4_pnfs_cb_dev_item *cbd	= nd->nd_list->cbd_list;
-
-	dprintk("NFSD %s: --> num %d\n", __func__, len);
-
-	BUG_ON(hdr->minorversion == 0);
-
-	p = xdr_reserve_space(xdr, 2 * 4);
-	*p++ = cpu_to_be32(OP_CB_NOTIFY_DEVICEID);
-	/* notify4 cnda_changes<>; */
-	*p = cpu_to_be32(len);
-	for (i = 0; i < len; i++) {
-		dprintk("%s: nt %d lt %d devid x%llx-x%llx im %d i %d\n",
-			__func__, cbd[i].cbd_notify_type,
-			cbd[i].cbd_layout_type,
-			cbd[i].cbd_devid.sbid,
-			cbd[i].cbd_devid.devid,
-			cbd[i].cbd_immediate, i);
-
-		BUG_ON(cbd[i].cbd_notify_type != NOTIFY_DEVICEID4_CHANGE &&
-		       cbd[i].cbd_notify_type != NOTIFY_DEVICEID4_DELETE);
-		p = xdr_reserve_space(xdr, 4 * 4 + 2 * 8);
-		/* bitmap4         notify_mask; */
-		*p++ = cpu_to_be32(1);
-		*p++ = cpu_to_be32(cbd[i].cbd_notify_type);
-		/* opaque     notify_vals<>; */
-		if (cbd[i].cbd_notify_type == NOTIFY_DEVICEID4_CHANGE)
-			*p++ = cpu_to_be32(24);
-		else
-			*p++ = cpu_to_be32(20);
-		*p++ = cpu_to_be32(cbd[i].cbd_layout_type);
-		p = nfsd4_encode_deviceid(p, &cbd[i].cbd_devid);
-
-		if (cbd[i].cbd_notify_type == NOTIFY_DEVICEID4_CHANGE) {
-			p = xdr_reserve_space(xdr, 4);
-			*p = cpu_to_be32(cbd[i].cbd_immediate);
-		}
-	}
-	hdr->nops++;
-}
 #endif /* CONFIG_PNFSD */
 
 /*
@@ -703,22 +635,6 @@ static void nfs4_xdr_enc_cb_layout(struct rpc_rqst *req,
 	encode_cb_layout4args(xdr, args, &hdr);
 	encode_cb_nops(&hdr);
 }
-
-static void nfs4_xdr_enc_cb_device(struct rpc_rqst *req,
-				   struct xdr_stream *xdr,
-				   const struct nfsd4_callback *cb)
-{
-	struct nfs4_notify_device *args =  cb->cb_op;
-	struct nfs4_cb_compound_hdr hdr = {
-		.ident = 0,
-		.minorversion = cb->cb_minorversion,
-	};
-
-	encode_cb_compound4args(xdr, &hdr);
-	encode_cb_sequence4args(xdr, cb, &hdr);
-	encode_cb_device4args(xdr, args, &hdr);
-	encode_cb_nops(&hdr);
-}
 #endif /* CONFIG_PNFSD */
 
 /*
@@ -791,31 +707,6 @@ static int nfs4_xdr_dec_cb_layout(struct rpc_rqst *rqstp,
 out:
 	return status;
 }
-
-static int nfs4_xdr_dec_cb_device(struct rpc_rqst *rqstp,
-				  struct xdr_stream *xdr,
-				  struct nfsd4_callback *cb)
-{
-	struct nfs4_cb_compound_hdr hdr;
-	enum nfsstat4 nfserr;
-	int status;
-
-	status = decode_cb_compound4res(xdr, &hdr);
-	if (unlikely(status))
-		goto out;
-	if (cb) {
-		status = decode_cb_sequence4res(xdr, cb);
-		if (unlikely(status))
-			goto out;
-	}
-	status = decode_cb_op_status(xdr, OP_CB_NOTIFY_DEVICEID, &nfserr);
-	if (unlikely(status))
-		goto out;
-	if (unlikely(nfserr != NFS4_OK))
-		status = nfs_cb_stat_to_errno(nfserr);
-out:
-	return status;
-}
 #endif /* CONFIG_PNFSD */
 
 /*
@@ -837,7 +728,6 @@ static int nfs4_xdr_dec_cb_device(struct rpc_rqst *rqstp,
 	PROC(CB_RECALL,	COMPOUND,	cb_recall,	cb_recall),
 #if defined(CONFIG_PNFSD)
 	PROC(CB_LAYOUT,	COMPOUND,	cb_layout,	cb_layout),
-	PROC(CB_DEVICE,	COMPOUND,	cb_device,	cb_device),
 #endif
 };
 
@@ -1431,66 +1321,4 @@ static void nfsd4_cb_layout_release(void *calldata)
 	cb->cb_ops = &nfsd4_cb_layout_ops;
 	run_nfsd4_cb(cb);
 }
-
-static void nfsd4_cb_device_prepare(struct rpc_task *task, void *calldata)
-{
-	struct nfsd4_callback *cb = calldata;
-	struct nfs4_notify_device *cbnd = container_of(cb, struct nfs4_notify_device, nd_recall);
-
-	nfsd4_cb_prepare_sequence(task, cb, cbnd->nd_client);
-}
-
-static void nfsd4_cb_device_done(struct rpc_task *task, void *calldata)
-{
-	struct nfsd4_callback *cb = calldata;
-	struct nfs4_notify_device *cbnd = container_of(cb, struct nfs4_notify_device, nd_recall);
-	struct nfs4_client *clp = cbnd->nd_client;
-
-	nfsd4_cb_done_sequence(task, clp);
-
-	dprintk("%s: clp %p cb_client %p: status %d\n",
-	       __func__,
-	       clp,
-	       clp->cl_cb_client,
-	       task->tk_status);
-
-	if (task->tk_status == -EIO) {
-		/* Network partition? */
-		nfsd4_mark_cb_down(clp, task->tk_status);
-	}
-	cb->cb_done = true;
-}
-
-static void nfsd4_cb_device_release(void *calldata)
-{
-	struct nfsd4_callback *cb = calldata;
-	struct nfs4_notify_device *cbnd = container_of(cb, struct nfs4_notify_device, nd_recall);
-
-	kfree(cbnd);
-}
-
-static const struct rpc_call_ops nfsd4_cb_device_ops = {
-	.rpc_call_prepare = nfsd4_cb_device_prepare,
-	.rpc_call_done = nfsd4_cb_device_done,
-	.rpc_release = nfsd4_cb_device_release,
-};
-
-/*
- * Called with state lock.
- */
-void
-nfsd4_cb_notify_device(struct nfs4_notify_device *cbnd)
-{
-	struct nfsd4_callback *cb = &cbnd->nd_recall;
-
-	cb->cb_op = cbnd;
-	cb->cb_clp = cbnd->nd_client;
-	cb->cb_msg.rpc_proc = &nfs4_cb_procedures[NFSPROC4_CLNT_CB_DEVICE];
-	cb->cb_msg.rpc_argp = cb;
-	cb->cb_msg.rpc_resp = cb;
-	cb->cb_msg.rpc_cred = callback_cred;
-
-	cb->cb_ops = &nfsd4_cb_device_ops;
-	run_nfsd4_cb(cb);
-}
 #endif /* CONFIG_PNFSD */
diff --git a/fs/nfsd/nfs4pnfsd.c b/fs/nfsd/nfs4pnfsd.c
index 8653719..85d0733 100644
--- a/fs/nfsd/nfs4pnfsd.c
+++ b/fs/nfsd/nfs4pnfsd.c
@@ -1552,79 +1552,3 @@ struct create_recall_list_arg {
 {
 	return _nfsd_layout_recall_cb(sb, inode, cbl, false);
 }
-
-struct create_device_notify_list_arg {
-	struct list_head *todolist;
-	struct nfsd4_pnfs_cb_dev_list *ndl;
-};
-
-static int
-create_device_notify_per_cl(struct nfs4_client *clp, void *p)
-{
-	struct nfs4_notify_device *cbnd;
-	struct create_device_notify_list_arg *arg = p;
-
-	cbnd = kzalloc(sizeof(*cbnd), GFP_KERNEL);
-	if (!cbnd)
-		return -ENOMEM;
-
-	cbnd->nd_list = arg->ndl;
-	cbnd->nd_client = clp;
-	INIT_WORK(&cbnd->nd_recall.cb_work, nfsd4_do_callback_rpc);
-	list_add(&cbnd->nd_perclnt, arg->todolist);
-	return 0;
-}
-
-/* Create a list of clients to send device notifications. */
-int
-create_device_notify_list(struct list_head *todolist,
-			  struct nfsd4_pnfs_cb_dev_list *ndl)
-{
-	int status;
-	struct create_device_notify_list_arg arg = {
-		.todolist = todolist,
-		.ndl = ndl,
-	};
-
-	nfs4_lock_state();
-	status = filter_confirmed_clients(create_device_notify_per_cl, &arg);
-	nfs4_unlock_state();
-
-	return status;
-}
-
-/*
- * For each client that a device, send a device notification.
- * XXX: Need to track which clients have which devices.
- */
-int nfsd_device_notify_cb(struct super_block *sb,
-			  struct nfsd4_pnfs_cb_dev_list *ndl)
-{
-	struct nfs4_notify_device *cbnd;
-	unsigned int notify_num = 0;
-	int status = 0;
-	struct list_head todolist;
-
-	BUG_ON(!ndl || ndl->cbd_len == 0 || !ndl->cbd_list);
-
-	dprintk("NFSD %s: cbl %p len %u\n", __func__, ndl, ndl->cbd_len);
-
-	if (nfsd_serv == NULL)
-		return -ENOENT;
-
-	INIT_LIST_HEAD(&todolist);
-
-	status = create_device_notify_list(&todolist, ndl);
-
-	while (!list_empty(&todolist)) {
-		cbnd = list_entry(todolist.next, struct nfs4_notify_device,
-				  nd_perclnt);
-		list_del_init(&cbnd->nd_perclnt);
-		nfsd4_cb_notify_device(cbnd);
-		notify_num++;
-	}
-
-	dprintk("NFSD %s: status %d clients %u\n",
-		__func__, status, notify_num);
-	return status;
-}
diff --git a/fs/nfsd/pnfsd.h b/fs/nfsd/pnfsd.h
index 5b3657c..458e095 100644
--- a/fs/nfsd/pnfsd.h
+++ b/fs/nfsd/pnfsd.h
@@ -101,16 +101,6 @@ struct pnfs_mds_id {
 	struct kref		di_ref;
 };
 
-/* notify device request (from exported filesystem) */
-struct nfs4_notify_device {
-	struct nfsd4_pnfs_cb_dev_list  *nd_list;
-	struct nfs4_client	       *nd_client;
-	struct list_head	        nd_perclnt;
-
-	/* nfsd internal */
-	struct nfsd4_callback		nd_recall;
-};
-
 u64 find_create_sbid(struct super_block *);
 struct super_block *find_sbid_id(u64);
 __be32 nfs4_pnfs_get_layout(struct nfsd4_pnfs_layoutget *, struct exp_xdr_stream *);
@@ -128,9 +118,6 @@ int _nfsd_layout_recall_cb(struct super_block *, struct inode *,
 			  bool with_nfs4_state_lock);
 int nfsd_layout_recall_cb(struct super_block *, struct inode *,
 			  struct nfsd4_pnfs_cb_layout *);
-int nfsd_device_notify_cb(struct super_block *,
-			  struct nfsd4_pnfs_cb_dev_list *);
-void nfsd4_cb_notify_device(struct nfs4_notify_device *);
 
 static inline struct nfs4_layout_state *layoutstateid(struct nfs4_stid *s)
 {
diff --git a/include/linux/nfsd/nfsd4_pnfs.h b/include/linux/nfsd/nfsd4_pnfs.h
index ecb412e..95b8b6e 100644
--- a/include/linux/nfsd/nfsd4_pnfs.h
+++ b/include/linux/nfsd/nfsd4_pnfs.h
@@ -253,18 +253,6 @@ struct nfs4_layoutrecall {
 	struct nfsd4_callback		clr_recall;
 };
 
-struct nfsd4_pnfs_cb_dev_item {
-	u32			cbd_notify_type;	/* request */
-	u32			cbd_layout_type;	/* request */
-	struct nfsd4_pnfs_deviceid cbd_devid;		/* request */
-	u32			cbd_immediate;		/* request */
-};
-
-struct nfsd4_pnfs_cb_dev_list {
-	u32				cbd_len;  /* request */
-	struct nfsd4_pnfs_cb_dev_item  *cbd_list; /* request */
-};
-
 /*
  * callbacks provided by the nfsd
  */
@@ -272,8 +260,6 @@ struct pnfsd_cb_operations {
 	/* Generic callbacks */
 	int (*cb_layout_recall) (struct super_block *, struct inode *,
 				 struct nfsd4_pnfs_cb_layout *);
-	int (*cb_device_notify) (struct super_block *,
-				 struct nfsd4_pnfs_cb_dev_list *);
 
 	/* pNFS Files layout specific callbacks */
 
-- 
1.7.11.7

[PATCH 09/38] Revert "pnfsd: filelayout: file layout specific callbacks"

[Benny Halevy]

This reverts commit c89fe09b70d3e5391db3b645f91ee10d75a31350.
---
 include/linux/nfsd/nfsd4_pnfs.h | 7 -------
 1 file changed, 7 deletions(-)

diff --git a/include/linux/nfsd/nfsd4_pnfs.h b/include/linux/nfsd/nfsd4_pnfs.h
index 95b8b6e..1e6ef66 100644
--- a/include/linux/nfsd/nfsd4_pnfs.h
+++ b/include/linux/nfsd/nfsd4_pnfs.h
@@ -260,13 +260,6 @@ struct pnfsd_cb_operations {
 	/* Generic callbacks */
 	int (*cb_layout_recall) (struct super_block *, struct inode *,
 				 struct nfsd4_pnfs_cb_layout *);
-
-	/* pNFS Files layout specific callbacks */
-
-	/* Callback from fs on MDS only */
-	int (*cb_get_state) (struct super_block *, struct pnfs_get_state *);
-	/* Callback from fs on DS only */
-	int (*cb_change_state) (struct pnfs_get_state *);
 };
 
 #endif /* _LINUX_NFSD_NFSD4_PNFS_H */
-- 
1.7.11.7

[PATCH 10/38] Revert "pnfsd: Use ds_state_lock to protect retrieving verifier."

[Benny Halevy]

This reverts commit 8ee9cf66f92878c5add818cdeb6b8a9937056f85.

Conflicts:
	fs/nfsd/pnfsd.h
---
 fs/nfsd/nfs4proc.c              | 16 +++-------------
 fs/nfsd/nfs4xdr.c               |  6 ------
 fs/nfsd/pnfsd.h                 |  5 ++---
 include/linux/nfsd/nfsd4_pnfs.h |  2 --
 4 files changed, 5 insertions(+), 24 deletions(-)

diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index 2f1786a..c9befff 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -499,19 +499,11 @@ static __be32 nfsd_check_obj_isreg(struct svc_fh *fh)
 			   &access->ac_supported);
 }
 
-static void nfsd4_get_verifier(struct super_block *sb, nfs4_verifier *verifier, struct net *nn)
+static void gen_boot_verifier(nfs4_verifier *verifier, struct net *net)
 {
 	__be32 verf[2];
 	struct nfsd_net *nn = net_generic(net, nfsd_net_id);
 
-#if defined(CONFIG_PNFSD)
-	if (sb->s_pnfs_op && sb->s_pnfs_op->get_verifier) {
-		nfs4_ds_get_verifier(NULL, sb,
-				     (u32 *)verifier->data);
-		return;
-	}
-#endif /* CONFIG_PNFSD */
-
 	verf[0] = (__be32)nn->nfssvc_boot.tv_sec;
 	verf[1] = (__be32)nn->nfssvc_boot.tv_usec;
 	memcpy(verifier->data, verf, sizeof(verifier->data));
@@ -521,8 +513,7 @@ static void nfsd4_get_verifier(struct super_block *sb, nfs4_verifier *verifier,
 nfsd4_commit(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	     struct nfsd4_commit *commit)
 {
-	nfsd4_get_verifier(cstate->current_fh.fh_dentry->d_inode->i_sb,
-			   &commit->co_verf, SVC_NET(rqstp));
+	gen_boot_verifier(&commit->co_verf, SVC_NET(rqstp));
 	return nfsd_commit(rqstp, &cstate->current_fh, commit->co_offset,
 			     commit->co_count);
 }
@@ -953,8 +944,7 @@ static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write)
 
 	cnt = write->wr_buflen;
 	write->wr_how_written = write->wr_stable_how;
-	nfsd4_get_verifier(cstate->current_fh.fh_dentry->d_inode->i_sb,
-			   &write->wr_verifier, SVC_NET(rqstp));
+	gen_boot_verifier(&write->wr_verifier, SVC_NET(rqstp));
 
 	nvecs = fill_in_write_vector(rqstp->rq_vec, write);
 	WARN_ON_ONCE(nvecs > ARRAY_SIZE(rqstp->rq_vec));
diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index 33f4c3e..a552202 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -2843,9 +2843,6 @@ static __be32 nfsd4_encode_bind_conn_to_session(struct nfsd4_compoundres *resp,
 	if (!nfserr) {
 		RESERVE_SPACE(NFS4_VERIFIER_SIZE);
 		WRITEMEM(commit->co_verf.data, NFS4_VERIFIER_SIZE);
-		dprintk("NFSD: nfsd4_encode_commit: verifier %x:%x\n",
-			((u32 *)(&commit->co_verf.data))[0],
-			((u32 *)(&commit->co_verf.data))[1]);
 		ADJUST_ARGS();
 	}
 	return nfserr;
@@ -3439,9 +3436,6 @@ static __be32 nfsd4_encode_bind_conn_to_session(struct nfsd4_compoundres *resp,
 		WRITE32(write->wr_bytes_written);
 		WRITE32(write->wr_how_written);
 		WRITEMEM(write->wr_verifier.data, NFS4_VERIFIER_SIZE);
-		dprintk("NFSD: nfsd4_encode_write: verifier %x:%x\n",
-			((u32 *)(&write->wr_verifier.data))[0],
-			((u32 *)(&write->wr_verifier.data))[1]);
 		ADJUST_ARGS();
 	}
 	return nfserr;
diff --git a/fs/nfsd/pnfsd.h b/fs/nfsd/pnfsd.h
index 458e095..6178c17 100644
--- a/fs/nfsd/pnfsd.h
+++ b/fs/nfsd/pnfsd.h
@@ -101,14 +101,13 @@ struct pnfs_mds_id {
 	struct kref		di_ref;
 };
 
+int nfs4_pnfs_cb_get_state(struct super_block *, struct pnfs_get_state *);
+int nfs4_pnfs_cb_change_state(struct pnfs_get_state *);
 u64 find_create_sbid(struct super_block *);
 struct super_block *find_sbid_id(u64);
 __be32 nfs4_pnfs_get_layout(struct nfsd4_pnfs_layoutget *, struct exp_xdr_stream *);
 int nfs4_pnfs_return_layout(struct super_block *, struct svc_fh *,
 					struct nfsd4_pnfs_layoutreturn *);
-int nfs4_pnfs_cb_get_state(struct super_block *, struct pnfs_get_state *);
-int nfs4_pnfs_cb_change_state(struct pnfs_get_state *);
-void nfs4_ds_get_verifier(stateid_t *, struct super_block *, u32 *);
 int put_layoutrecall(struct nfs4_layoutrecall *);
 void nomatching_layout(struct nfs4_layoutrecall *);
 void *layoutrecall_done(struct nfs4_layoutrecall *);
diff --git a/include/linux/nfsd/nfsd4_pnfs.h b/include/linux/nfsd/nfsd4_pnfs.h
index 1e6ef66..0aab899 100644
--- a/include/linux/nfsd/nfsd4_pnfs.h
+++ b/include/linux/nfsd/nfsd4_pnfs.h
@@ -222,8 +222,6 @@ struct pnfs_export_operations {
 
 	/* pNFS Files layout specific operations */
 
-	/* Get the write verifier for DS (called on MDS only) */
-	void (*get_verifier) (struct super_block *, u32 *p);
 	/* Call fs on DS only */
 	int (*get_state) (struct inode *, struct knfsd_fh *,
 			  struct pnfs_get_state *);
-- 
1.7.11.7

[PATCH 11/38] Revert "pnfsd: ds state"

[Benny Halevy]

This reverts commit 5aba2b7eeb8d96f2647ed54a98bf5ac2e71e916f.

Conflicts:
	fs/nfsd/Makefile
	fs/nfsd/nfs4pnfsd.c
	fs/nfsd/pnfsd.h
	fs/nfsd/state.h
	include/linux/nfsd/nfsd4_pnfs.h
---
 fs/nfsd/Makefile                |  2 +-
 fs/nfsd/export.c                |  3 --
 fs/nfsd/nfs4pnfsd.c             | 69 -----------------------------------------
 fs/nfsd/nfs4state.c             | 10 ------
 fs/nfsd/pnfsd.h                 | 45 ---------------------------
 fs/nfsd/state.h                 | 15 ---------
 include/linux/nfs4.h            |  1 -
 include/linux/nfsd/nfsd4_pnfs.h | 17 ----------
 include/uapi/linux/nfs4.h       |  1 -
 9 files changed, 1 insertion(+), 162 deletions(-)

diff --git a/fs/nfsd/Makefile b/fs/nfsd/Makefile
index 06f8732..5488e7e 100644
--- a/fs/nfsd/Makefile
+++ b/fs/nfsd/Makefile
@@ -12,5 +12,5 @@ nfsd-$(CONFIG_NFSD_V3)	+= nfs3proc.o nfs3xdr.o
 nfsd-$(CONFIG_NFSD_V3_ACL) += nfs3acl.o
 nfsd-$(CONFIG_NFSD_V4)	+= nfs4proc.o nfs4xdr.o nfs4state.o nfs4idmap.o \
 			   nfs4acl.o nfs4callback.o nfs4recover.o
-nfsd-$(CONFIG_PNFSD)	+= nfs4pnfsd.o nfs4pnfsdlm.o nfs4pnfsds.o
+nfsd-$(CONFIG_PNFSD)	+= nfs4pnfsd.o nfs4pnfsdlm.o
 nfsd-$(CONFIG_PNFSD_LOCAL_EXPORT) += pnfsd_lexp.o
diff --git a/fs/nfsd/export.c b/fs/nfsd/export.c
index 698ca5c..5abe7d3 100644
--- a/fs/nfsd/export.c
+++ b/fs/nfsd/export.c
@@ -346,9 +346,6 @@ static int svc_export_upcall(struct cache_detail *cd, struct cache_head *h)
 #if defined(CONFIG_PNFSD)
 static struct pnfsd_cb_operations pnfsd_cb_op = {
 	.cb_layout_recall = nfsd_layout_recall_cb,
-
-	.cb_get_state = nfs4_pnfs_cb_get_state,
-	.cb_change_state = nfs4_pnfs_cb_change_state,
 };
 #endif /* CONFIG_PNFSD */
 
diff --git a/fs/nfsd/nfs4pnfsd.c b/fs/nfsd/nfs4pnfsd.c
index 85d0733..2be124a 100644
--- a/fs/nfsd/nfs4pnfsd.c
+++ b/fs/nfsd/nfs4pnfsd.c
@@ -505,37 +505,6 @@ struct super_block *
 }
 
 /*
- * get_state() and cb_get_state() are
- */
-void
-release_pnfs_ds_dev_list(struct nfs4_ol_stateid *stp)
-{
-	struct pnfs_ds_dev_entry *ddp;
-
-	while (!list_empty(&stp->st_pnfs_ds_id)) {
-		ddp = list_entry(stp->st_pnfs_ds_id.next,
-				 struct pnfs_ds_dev_entry, dd_dev_entry);
-		list_del(&ddp->dd_dev_entry);
-		kfree(ddp);
-	}
-}
-
-static int
-nfs4_add_pnfs_ds_dev(struct nfs4_ol_stateid *stp, u32 dsid)
-{
-	struct pnfs_ds_dev_entry *ddp;
-
-	ddp = kmalloc(sizeof(*ddp), GFP_KERNEL);
-	if (!ddp)
-		return -ENOMEM;
-
-	INIT_LIST_HEAD(&ddp->dd_dev_entry);
-	list_add(&ddp->dd_dev_entry, &stp->st_pnfs_ds_id);
-	ddp->dd_dsid = dsid;
-	return 0;
-}
-
-/*
  * are two octet ranges overlapping?
  * start1            last1
  *   |-----------------|
@@ -1124,44 +1093,6 @@ int nfs4_pnfs_return_layout(struct super_block *sb, struct svc_fh *current_fh,
 	return status;
 }
 
-/*
- * PNFS Metadata server export operations callback for get_state
- *
- * called by the cluster fs when it receives a get_state() from a data
- * server.
- * returns status, or pnfs_get_state* with pnfs_get_state->status set.
- *
- */
-int
-nfs4_pnfs_cb_get_state(struct super_block *sb, struct pnfs_get_state *arg)
-{
-	struct nfs4_stid *stid;
-	struct nfs4_ol_stateid *stp;
-	int status = -EINVAL;
-	stateid_t *stateid = (stateid_t *)&arg->stid;
-
-	dprintk("NFSD: %s sid=" STATEID_FMT " ino %llu\n", __func__,
-		STATEID_VAL(stateid), arg->ino);
-
-	nfs4_lock_state();
-	status = nfsd4_lookup_stateid(stateid, NFS4_OPEN_STID|NFS4_LOCK_STID,
-	                              &stid, true);
-	if (!status) {
-		stp = container_of(stid, struct nfs4_ol_stateid, st_stid);
-		/* arg->dsid is the Data server id, set by the cluster fs */
-		status = nfs4_add_pnfs_ds_dev(stp, arg->dsid);
-		if (status)
-			goto out;
-
-		arg->access = stp->st_access_bmap;
-		*(clientid_t *)&arg->clid =
-			stp->st_stateowner->so_client->cl_clientid;
-	}
-out:
-	nfs4_unlock_state();
-	return status;
-}
-
 static bool
 cl_has_file_layout(struct nfs4_client *clp, struct nfs4_file *fp,
 		   stateid_t *lsid, struct nfsd4_pnfs_cb_layout *cbl)
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 664c7cb..0871146 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -531,7 +531,6 @@ static void unhash_generic_stateid(struct nfs4_ol_stateid *stp)
 {
 	list_del(&stp->st_perfile);
 	list_del(&stp->st_perstateowner);
-	release_pnfs_ds_dev_list(stp);
 }
 
 static void close_generic_stateid(struct nfs4_ol_stateid *stp)
@@ -2519,9 +2518,6 @@ static void init_open_stateid(struct nfs4_ol_stateid *stp, struct nfs4_file *fp,
 
 	nfsd4_init_stid(&stp->st_stid, clp, NFS4_OPEN_STID);
 	INIT_LIST_HEAD(&stp->st_lockowners);
-#if defined(CONFIG_PNFSD)
-	INIT_LIST_HEAD(&stp->st_pnfs_ds_id);
-#endif /* CONFIG_PNFSD */
 	list_add(&stp->st_perstateowner, &oo->oo_owner.so_stateids);
 	list_add(&stp->st_perfile, &fp->fi_stateids);
 	stp->st_stateowner = &oo->oo_owner;
@@ -4108,9 +4104,6 @@ static void hash_lockowner(struct nfs4_lockowner *lo, unsigned int strhashval, s
 	if (stp == NULL)
 		return NULL;
 	nfsd4_init_stid(&stp->st_stid, clp, NFS4_LOCK_STID);
-#if defined(CONFIG_PNFSD)
-	INIT_LIST_HEAD(&stp->st_pnfs_ds_id);
-#endif /* CONFIG_PNFSD */
 	list_add(&stp->st_perfile, &fp->fi_stateids);
 	list_add(&stp->st_perstateowner, &lo->lo_owner.so_stateids);
 	stp->st_stateowner = &lo->lo_owner;
@@ -4872,9 +4865,6 @@ struct nfs4_client *nfsd_find_client(struct sockaddr_storage *addr, size_t addr_
 		INIT_LIST_HEAD(&file_hashtbl[i]);
 	}
 	INIT_LIST_HEAD(&del_recall_lru);
-#if defined(CONFIG_PNFSD)
-	nfs4_pnfs_state_init();
-#endif /* CONFIG_PNFSD */
 }
 
 /*
diff --git a/fs/nfsd/pnfsd.h b/fs/nfsd/pnfsd.h
index 6178c17..2405d47 100644
--- a/fs/nfsd/pnfsd.h
+++ b/fs/nfsd/pnfsd.h
@@ -58,51 +58,6 @@ struct nfs4_layout {
 	struct nfsd4_layout_seg		lo_seg;
 };
 
-struct pnfs_inval_state {
-	struct knfsd_fh		mdsfh; /* needed only by invalidate all */
-	stateid_t		stid;
-	clientid_t		clid;
-	u32			status;
-};
-
-/* pNFS Data Server state */
-#define DS_STATEID_VALID   0
-#define DS_STATEID_ERROR   1
-#define DS_STATEID_NEW     2
-
-struct pnfs_ds_stateid {
-	struct list_head	ds_hash;        /* ds_stateid hash entry */
-	struct list_head	ds_perclid;     /* per client hash entry */
-	stateid_t		ds_stid;
-	struct knfsd_fh		ds_fh;
-	unsigned long		ds_access;
-	u32			ds_status;      /* from MDS */
-	u32			ds_verifier[2]; /* from MDS */
-	wait_queue_head_t	ds_waitq;
-	unsigned long		ds_flags;
-	struct kref		ds_ref;
-	clientid_t		ds_mdsclid;
-};
-
-struct pnfs_ds_clientid {
-	struct list_head	dc_hash;        /* mds_clid_hashtbl entry */
-	struct list_head	dc_stateid;     /* ds_stateid head */
-	struct list_head	dc_permdsid;    /* per mdsid hash entry */
-	clientid_t		dc_mdsclid;
-	struct kref		dc_ref;
-	uint32_t		dc_mdsid;
-};
-
-struct pnfs_mds_id {
-	struct list_head	di_hash;        /* mds_nodeid list entry */
-	struct list_head	di_mdsclid;     /* mds_clientid head */
-	uint32_t		di_mdsid;
-	time_t			di_mdsboot;	/* mds boot time */
-	struct kref		di_ref;
-};
-
-int nfs4_pnfs_cb_get_state(struct super_block *, struct pnfs_get_state *);
-int nfs4_pnfs_cb_change_state(struct pnfs_get_state *);
 u64 find_create_sbid(struct super_block *);
 struct super_block *find_sbid_id(u64);
 __be32 nfs4_pnfs_get_layout(struct nfsd4_pnfs_layoutget *, struct exp_xdr_stream *);
diff --git a/fs/nfsd/state.h b/fs/nfsd/state.h
index 774927c..fac8550 100644
--- a/fs/nfsd/state.h
+++ b/fs/nfsd/state.h
@@ -446,24 +446,12 @@ static inline struct file *find_any_file(struct nfs4_file *f)
 		return f->fi_fds[O_RDONLY];
 }
 
-#if defined(CONFIG_PNFSD)
-/* pNFS Metadata server state */
-
-struct pnfs_ds_dev_entry {
-	struct list_head	dd_dev_entry; /* st_pnfs_ds_id entry */
-	u32			dd_dsid;
-};
-#endif /* CONFIG_PNFSD */
-
 /* "ol" stands for "Open or Lock".  Better suggestions welcome. */
 struct nfs4_ol_stateid {
 	struct nfs4_stid    st_stid; /* must be first field */
 	struct list_head              st_perfile;
 	struct list_head              st_perstateowner;
 	struct list_head              st_lockowners;
-#if defined(CONFIG_PNFSD)
-	struct list_head              st_pnfs_ds_id;
-#endif /* CONFIG_PNFSD */
 	struct nfs4_stateowner      * st_stateowner;
 	struct nfs4_file            * st_file;
 	unsigned long                 st_access_bmap;
@@ -528,14 +516,11 @@ extern struct nfs4_client_reclaim *nfs4_client_to_reclaim(const char *name,
 extern int nfsd4_init_pnfs_slabs(void);
 extern void nfsd4_free_pnfs_slabs(void);
 extern void pnfs_expire_client(struct nfs4_client *);
-extern void release_pnfs_ds_dev_list(struct nfs4_ol_stateid *);
-extern void nfs4_pnfs_state_init(void);
 extern void pnfsd_roc(struct nfs4_client *clp, struct nfs4_file *fp);
 #else /* CONFIG_PNFSD */
 static inline void nfsd4_free_pnfs_slabs(void) {}
 static inline int nfsd4_init_pnfs_slabs(void) { return 0; }
 static inline void pnfs_expire_client(struct nfs4_client *clp) {}
-static inline void release_pnfs_ds_dev_list(struct nfs4_ol_stateid *stp) {}
 static inline void pnfsd_roc(struct nfs4_client *clp, struct nfs4_file *fp) {}
 #endif /* CONFIG_PNFSD */
 
diff --git a/include/linux/nfs4.h b/include/linux/nfs4.h
index 59cc833..f6315ce 100644
--- a/include/linux/nfs4.h
+++ b/include/linux/nfs4.h
@@ -34,7 +34,6 @@ struct nfs4_fsid {
 };
 
 typedef struct { char data[NFS4_VERIFIER_SIZE]; } nfs4_verifier;
-typedef struct { char data[NFS4_CLIENTID_SIZE]; } nfs4_clientid;
 
 struct nfs_stateid4 {
 	__be32 seqid;
diff --git a/include/linux/nfsd/nfsd4_pnfs.h b/include/linux/nfsd/nfsd4_pnfs.h
index 0aab899..d86e1b7 100644
--- a/include/linux/nfsd/nfsd4_pnfs.h
+++ b/include/linux/nfsd/nfsd4_pnfs.h
@@ -127,17 +127,6 @@ struct nfsd4_pnfs_layoutreturn_arg {
 	enum layoutreturn_flags	lr_flags;	/* internal flags */
 };
 
-/* pNFS Metadata to Data server state communication */
-struct pnfs_get_state {
-	u32			dsid;		/* request */
-	u64			ino;		/* request */
-	nfs4_stateid		stid;		/* request;response */
-	nfs4_clientid		clid;		/* response */
-	u32			access;		/* response */
-	u32			stid_gen;	/* response */
-	u32			verifier[2];	/* response */
-};
-
 /*
  * pNFS export operations vector.
  *
@@ -219,12 +208,6 @@ struct pnfs_export_operations {
 
 	/* Can layout segments be merged for this layout type? */
 	int (*can_merge_layouts) (u32 layout_type);
-
-	/* pNFS Files layout specific operations */
-
-	/* Call fs on DS only */
-	int (*get_state) (struct inode *, struct knfsd_fh *,
-			  struct pnfs_get_state *);
 };
 
 struct nfsd4_pnfs_cb_layout {
diff --git a/include/uapi/linux/nfs4.h b/include/uapi/linux/nfs4.h
index d81912a..028f5fc 100644
--- a/include/uapi/linux/nfs4.h
+++ b/include/uapi/linux/nfs4.h
@@ -17,7 +17,6 @@
 
 #define NFS4_BITMAP_SIZE	2
 #define NFS4_VERIFIER_SIZE	8
-#define NFS4_CLIENTID_SIZE	8
 #define NFS4_STATEID_SEQID_SIZE 4
 #define NFS4_STATEID_OTHER_SIZE 12
 #define NFS4_STATEID_SIZE	(NFS4_STATEID_SEQID_SIZE + NFS4_STATEID_OTHER_SIZE)
-- 
1.7.11.7

[PATCH 12/38] Revert "pnfsd: decode getdeviceinfo notify types."

[Benny Halevy]

This reverts commit 367bfe8e26e0753cad8eb6855f4254f803720c9c.
---
 fs/nfsd/nfs4xdr.c | 8 ++------
 1 file changed, 2 insertions(+), 6 deletions(-)

diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index a552202..596a2a5 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -1474,12 +1474,8 @@ static __be32 nfsd4_decode_reclaim_complete(struct nfsd4_compoundargs *argp, str
 	READ32(gdev->gd_layout_type);
 	READ32(gdev->gd_maxcount);
 	READ32(num);
-	if (num) {
-		READ_BUF(4);
-		READ32(gdev->gd_notify_types);
-	} else {
-		gdev->gd_notify_types = 0;
-	}
+	if (num)
+		READ_BUF(4); /* TODO: for now, just skip notify_types */
 
 	DECODE_TAIL;
 }
-- 
1.7.11.7

[PATCH 13/38] Revert "pnfsd: set_device_notify"

[Benny Halevy]

This reverts commit e31921a7fa0a13119e359b8ec200fb4047c0527f.
---
 fs/nfsd/nfs4xdr.c               | 37 +++++++++++++------------------------
 include/linux/nfsd/nfsd4_pnfs.h | 10 ----------
 2 files changed, 13 insertions(+), 34 deletions(-)

diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index 596a2a5..b157eae 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -3766,8 +3766,15 @@ static __be32 nfsd4_encode_bind_conn_to_session(struct nfsd4_compoundres *resp,
 
 	nfserr = sb->s_pnfs_op->get_device_info(sb, &xdr, gdev->gd_layout_type,
 						&gdev->gd_devid);
-	if (nfserr)
-		goto err;
+	if (nfserr) {
+		/* Rewind to the beginning */
+		p = p_in;
+		ADJUST_ARGS();
+		if (nfserr == -ETOOSMALL)
+			goto toosmall;
+		printk(KERN_ERR "%s: export ERROR %d\n", __func__, nfserr);
+		goto out;
+	}
 
 	/* The file system should never write 0 bytes without
 	 * returning an error
@@ -3782,21 +3789,11 @@ static __be32 nfsd4_encode_bind_conn_to_session(struct nfsd4_compoundres *resp,
 	ADJUST_ARGS();
 
 handle_notifications:
-	/* Encode supported device notifications */
+	/* Encode supported device notifications.
+	 * Note: Currently none are supported.
+	 */
 	RESERVE_SPACE(4);
-	if (sb->s_pnfs_op->set_device_notify) {
-		struct pnfs_devnotify_arg dn_args;
-
-		dn_args.dn_layout_type = gdev->gd_layout_type;
-		dn_args.dn_devid = gdev->gd_devid;
-		dn_args.dn_notify_types = gdev->gd_notify_types;
-		nfserr = sb->s_pnfs_op->set_device_notify(sb, &dn_args);
-		if (nfserr)
-			goto err;
-		WRITE32(dn_args.dn_notify_types);
-	} else {
-		WRITE32(0);
-	}
+	WRITE32(0);
 	ADJUST_ARGS();
 
 out:
@@ -3807,14 +3804,6 @@ static __be32 nfsd4_encode_bind_conn_to_session(struct nfsd4_compoundres *resp,
 	WRITE32((p_save ? (xdr.p - p_save) * 4 : 0) + type_notify_len);
 	ADJUST_ARGS();
 	goto out;
-err:
-	/* Rewind to the beginning */
-	p = p_in;
-	ADJUST_ARGS();
-	if (nfserr == -ETOOSMALL)
-		goto toosmall;
-	printk(KERN_ERR "%s: export ERROR %d\n", __func__, nfserr);
-	goto out;
 }
 
 static __be32
diff --git a/include/linux/nfsd/nfsd4_pnfs.h b/include/linux/nfsd/nfsd4_pnfs.h
index d86e1b7..a49d5f6 100644
--- a/include/linux/nfsd/nfsd4_pnfs.h
+++ b/include/linux/nfsd/nfsd4_pnfs.h
@@ -58,13 +58,6 @@ struct nfsd4_pnfs_dev_iter_res {
 	u32		gd_eof;		/* response */
 };
 
-/* Arguments for set_device_notify */
-struct pnfs_devnotify_arg {
-	struct nfsd4_pnfs_deviceid dn_devid;	/* request */
-	u32 dn_layout_type;			/* request */
-	u32 dn_notify_types;			/* request/response */
-};
-
 struct nfsd4_layout_seg {
 	u64	clientid;
 	u32	layout_type;
@@ -160,9 +153,6 @@ struct pnfs_export_operations {
 				u32 layout_type,
 				struct nfsd4_pnfs_dev_iter_res *);
 
-	int (*set_device_notify) (struct super_block *,
-				  struct pnfs_devnotify_arg *);
-
 	/* Retrieve and encode a layout for inode onto the xdr stream.
 	 * arg->minlength is the minimum number of accessible bytes required
 	 *   by the client.
-- 
1.7.11.7

[PATCH 14/38] FIXME: temporarily disable create_layout_recall_list in 3.8

[Benny Halevy]

Need struct net * for calling filter_confirmed_clients.
Will be replaced by new layout recall API based on the layout stateid
rather than clientid for identifying a specific client
to recall from.

Signed-off-by: Benny Halevy <bhalevy@tonian.com>
---
 fs/nfsd/nfs4pnfsd.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/fs/nfsd/nfs4pnfsd.c b/fs/nfsd/nfs4pnfsd.c
index 2be124a..0fa0e8b 100644
--- a/fs/nfsd/nfs4pnfsd.c
+++ b/fs/nfsd/nfs4pnfsd.c
@@ -1310,6 +1310,7 @@ struct create_recall_list_arg {
 	};
 	int status = 0;
 
+#if 0
 	dprintk("%s: -->\n", __func__);
 
 	/* If client given by fs, just do single client */
@@ -1331,6 +1332,7 @@ struct create_recall_list_arg {
 out:
 	*todo_len = arg.todo_count;
 	dprintk("%s: <-- list len %u status %d\n", __func__, *todo_len, status);
+#endif
 	return status;
 }
 
-- 
1.7.11.7

[PATCH 15/38] SQUASHME: pnfsd: use nfsd_net for layoutget starting v3.8

[Benny Halevy]

---
 fs/nfsd/nfs4pnfsd.c | 10 +++++++---
 fs/nfsd/nfs4xdr.c   |  2 +-
 fs/nfsd/pnfsd.h     |  2 +-
 fs/nfsd/state.h     |  4 ++--
 4 files changed, 11 insertions(+), 7 deletions(-)

diff --git a/fs/nfsd/nfs4pnfsd.c b/fs/nfsd/nfs4pnfsd.c
index 0fa0e8b..a15100b 100644
--- a/fs/nfsd/nfs4pnfsd.c
+++ b/fs/nfsd/nfs4pnfsd.c
@@ -22,6 +22,7 @@
  *****************************************************************************/
 
 #include "pnfsd.h"
+#include "netns.h"
 
 #define NFSDDBG_FACILITY                NFSDDBG_PNFS
 
@@ -195,7 +196,8 @@ struct sbid_tracker {
 	status = nfsd4_lookup_stateid(stateid,
 				      (NFS4_OPEN_STID | NFS4_LOCK_STID |
 				       NFS4_DELEG_STID | NFS4_LAYOUT_STID),
-				      &stid, true);
+				      &stid, true,
+				      net_generic(clp->net, nfsd_net_id));
 	if (status)
 		goto out;
 
@@ -633,7 +635,8 @@ struct super_block *
 }
 
 __be32
-nfs4_pnfs_get_layout(struct nfsd4_pnfs_layoutget *lgp,
+nfs4_pnfs_get_layout(struct svc_rqst *rqstp,
+		     struct nfsd4_pnfs_layoutget *lgp,
 		     struct exp_xdr_stream *xdr)
 {
 	u32 status;
@@ -685,7 +688,8 @@ struct super_block *
 
 	nfs4_lock_state();
 	fp = find_alloc_file(ino, lgp->lg_fhp);
-	clp = find_confirmed_client((clientid_t *)&lgp->lg_seg.clientid, true);
+	clp = find_confirmed_client((clientid_t *)&lgp->lg_seg.clientid, true,
+				    net_generic(SVC_NET(rqstp), nfsd_net_id));
 	dprintk("pNFS %s: fp %p clp %p\n", __func__, fp, clp);
 	if (!fp || !clp) {
 		nfserr = nfserr_inval;
diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index b157eae..992e425 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -3848,7 +3848,7 @@ static __be32 nfsd4_encode_bind_conn_to_session(struct nfsd4_compoundres *resp,
 		xdr.end = xdr.p + exp_xdr_qwords(maxcount & ~3);
 
 	/* Retrieve, encode, and merge layout; process stateid */
-	nfserr = nfs4_pnfs_get_layout(lgp, &xdr);
+	nfserr = nfs4_pnfs_get_layout(resp->rqstp, lgp, &xdr);
 	if (nfserr)
 		goto err;
 
diff --git a/fs/nfsd/pnfsd.h b/fs/nfsd/pnfsd.h
index 2405d47..d40694b 100644
--- a/fs/nfsd/pnfsd.h
+++ b/fs/nfsd/pnfsd.h
@@ -60,7 +60,7 @@ struct nfs4_layout {
 
 u64 find_create_sbid(struct super_block *);
 struct super_block *find_sbid_id(u64);
-__be32 nfs4_pnfs_get_layout(struct nfsd4_pnfs_layoutget *, struct exp_xdr_stream *);
+__be32 nfs4_pnfs_get_layout(struct svc_rqst *, struct nfsd4_pnfs_layoutget *, struct exp_xdr_stream *);
 int nfs4_pnfs_return_layout(struct super_block *, struct svc_fh *,
 					struct nfsd4_pnfs_layoutreturn *);
 int put_layoutrecall(struct nfs4_layoutrecall *);
diff --git a/fs/nfsd/state.h b/fs/nfsd/state.h
index fac8550..2e5b600 100644
--- a/fs/nfsd/state.h
+++ b/fs/nfsd/state.h
@@ -504,12 +504,12 @@ extern struct nfs4_client_reclaim *nfs4_client_to_reclaim(const char *name,
 extern struct nfs4_file *find_alloc_file(struct inode *, struct svc_fh *);
 extern void put_nfs4_file(struct nfs4_file *);
 extern void get_nfs4_file(struct nfs4_file *);
-extern struct nfs4_client *find_confirmed_client(clientid_t *, bool sessions);
+extern struct nfs4_client *find_confirmed_client(clientid_t *, bool sessions, struct nfsd_net *);
 extern struct nfs4_stid *nfs4_alloc_stid(struct nfs4_client *cl, struct kmem_cache *slab);
 extern void nfsd4_init_stid(struct nfs4_stid *, struct nfs4_client *, unsigned char type);
 extern void nfsd4_unhash_stid(struct nfs4_stid *);
 extern struct nfs4_stid *find_stateid(struct nfs4_client *, stateid_t *);
-extern __be32 nfsd4_lookup_stateid(stateid_t *, unsigned char typemask, struct nfs4_stid **, bool sessions);
+extern __be32 nfsd4_lookup_stateid(stateid_t *, unsigned char typemask, struct nfs4_stid **, bool sessions, struct nfsd_net *);
 extern int filter_confirmed_clients(int (* func)(struct nfs4_client *, void *), void *);
 
 #if defined(CONFIG_PNFSD)
-- 
1.7.11.7

[PATCH 16/38] SQUASHME: pnfsd: use nfsd4_init_callback starting v3.8

[Benny Halevy]

Signed-off-by: Benny Halevy <bhalevy@tonian.com>
---
 fs/nfsd/nfs4pnfsd.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/fs/nfsd/nfs4pnfsd.c b/fs/nfsd/nfs4pnfsd.c
index a15100b..e516eba 100644
--- a/fs/nfsd/nfs4pnfsd.c
+++ b/fs/nfsd/nfs4pnfsd.c
@@ -454,7 +454,7 @@ struct super_block *
 
 	kref_init(&clr->clr_ref);
 	INIT_LIST_HEAD(&clr->clr_perclnt);
-	INIT_WORK(&clr->clr_recall.cb_work, nfsd4_do_callback_rpc);
+	nfsd4_init_callback(&clr->clr_recall);
 
 	dprintk("NFSD %s return %p\n", __func__, clr);
 	return clr;
-- 
1.7.11.7

[PATCH 17/38] SQUASHME: use nfsd_net for layoutreturn starting v3.8

[Benny Halevy]

---
 fs/nfsd/nfs4pnfsd.c |  7 +++++--
 fs/nfsd/nfs4proc.c  |  2 +-
 fs/nfsd/nfs4state.c | 18 ------------------
 fs/nfsd/pnfsd.h     |  3 +--
 fs/nfsd/state.h     |  1 -
 5 files changed, 7 insertions(+), 24 deletions(-)

diff --git a/fs/nfsd/nfs4pnfsd.c b/fs/nfsd/nfs4pnfsd.c
index e516eba..96206d9 100644
--- a/fs/nfsd/nfs4pnfsd.c
+++ b/fs/nfsd/nfs4pnfsd.c
@@ -1002,7 +1002,9 @@ struct super_block *
 	       lo_seg_overlapping(&clr->cb.cbl_seg, &lrp->args.lr_seg);
 }
 
-int nfs4_pnfs_return_layout(struct super_block *sb, struct svc_fh *current_fh,
+int nfs4_pnfs_return_layout(struct svc_rqst *rqstp,
+			    struct super_block *sb,
+			    struct svc_fh *current_fh,
 			    struct nfsd4_pnfs_layoutreturn *lrp)
 {
 	int status = 0;
@@ -1019,7 +1021,8 @@ int nfs4_pnfs_return_layout(struct super_block *sb, struct svc_fh *current_fh,
 	dprintk("NFSD: %s\n", __func__);
 
 	nfs4_lock_state();
-	clp = find_confirmed_client((clientid_t *)&lrp->args.lr_seg.clientid, true);
+	clp = find_confirmed_client((clientid_t *)&lrp->args.lr_seg.clientid,
+				    true, net_generic(SVC_NET(rqstp), nfsd_net_id));
 	if (!clp)
 		goto out;
 
diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index c9befff..4fd5549 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -1327,7 +1327,7 @@ static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write)
 	/* Set clientid from sessionid */
 	copy_clientid((clientid_t *)&lrp->args.lr_seg.clientid, cstate->session);
 	lrp->lrs_present = 0;
-	status = nfs4_pnfs_return_layout(sb, current_fh, lrp);
+	status = nfs4_pnfs_return_layout(rqstp, sb, current_fh, lrp);
 out:
 	dprintk("pNFS %s: status %d return_type 0x%x lrs_present %d\n",
 		__func__, status, lrp->args.lr_return_type, lrp->lrs_present);
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 0871146..1170f8a 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -1456,24 +1456,6 @@ static bool clp_used_exchangeid(struct nfs4_client *clp)
 	return find_clp_in_name_tree(name, &nn->unconf_name_tree);
 }
 
-int
-filter_confirmed_clients(int (* func)(struct nfs4_client *, void *),
-			 void *arg)
-{
-	struct nfs4_client *clp, *next;
-	int i, status = 0;
-
-	for (i = 0; i < CLIENT_HASH_SIZE; i++)
-		list_for_each_entry_safe (clp, next, &conf_str_hashtbl[i],
-					  cl_strhash) {
-			status = func(clp, arg);
-			if (status)
-				break;
-		}
-
-	return status;
-}
-
 static void
 gen_callback(struct nfs4_client *clp, struct nfsd4_setclientid *se, struct svc_rqst *rqstp)
 {
diff --git a/fs/nfsd/pnfsd.h b/fs/nfsd/pnfsd.h
index d40694b..2507d31 100644
--- a/fs/nfsd/pnfsd.h
+++ b/fs/nfsd/pnfsd.h
@@ -61,8 +61,7 @@ struct nfs4_layout {
 u64 find_create_sbid(struct super_block *);
 struct super_block *find_sbid_id(u64);
 __be32 nfs4_pnfs_get_layout(struct svc_rqst *, struct nfsd4_pnfs_layoutget *, struct exp_xdr_stream *);
-int nfs4_pnfs_return_layout(struct super_block *, struct svc_fh *,
-					struct nfsd4_pnfs_layoutreturn *);
+int nfs4_pnfs_return_layout(struct svc_rqst *, struct super_block *, struct svc_fh *, struct nfsd4_pnfs_layoutreturn *);
 int put_layoutrecall(struct nfs4_layoutrecall *);
 void nomatching_layout(struct nfs4_layoutrecall *);
 void *layoutrecall_done(struct nfs4_layoutrecall *);
diff --git a/fs/nfsd/state.h b/fs/nfsd/state.h
index 2e5b600..5c613ba 100644
--- a/fs/nfsd/state.h
+++ b/fs/nfsd/state.h
@@ -510,7 +510,6 @@ extern struct nfs4_client_reclaim *nfs4_client_to_reclaim(const char *name,
 extern void nfsd4_unhash_stid(struct nfs4_stid *);
 extern struct nfs4_stid *find_stateid(struct nfs4_client *, stateid_t *);
 extern __be32 nfsd4_lookup_stateid(stateid_t *, unsigned char typemask, struct nfs4_stid **, bool sessions, struct nfsd_net *);
-extern int filter_confirmed_clients(int (* func)(struct nfs4_client *, void *), void *);
 
 #if defined(CONFIG_PNFSD)
 extern int nfsd4_init_pnfs_slabs(void);
-- 
1.7.11.7

[PATCH 18/38] SQUASHME: pnfsd: fix destroy_layout_state

[Benny Halevy]

squash into "pnfsd: layout state allocation"

Signed-off-by: Benny Halevy <bhalevy@tonian.com>
---
 fs/nfsd/nfs4pnfsd.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/fs/nfsd/nfs4pnfsd.c b/fs/nfsd/nfs4pnfsd.c
index 96206d9..24c19b3 100644
--- a/fs/nfsd/nfs4pnfsd.c
+++ b/fs/nfsd/nfs4pnfsd.c
@@ -155,7 +155,7 @@ struct sbid_tracker {
 		list_del(&ls->ls_perfile);
 		spin_unlock(&layout_lock);
 	}
-	kfree(ls);
+	kmem_cache_free(layout_state_slab, ls);
 }
 
 /*
-- 
1.7.11.7

[PATCH 19/38] SQUASHME: pnfsd: get rid of depdency on nfsd_serv starting 3.8

[Benny Halevy]

Signed-off-by: Benny Halevy <bhalevy@tonian.com>
---
 fs/nfsd/nfs4pnfsd.c | 5 -----
 1 file changed, 5 deletions(-)

diff --git a/fs/nfsd/nfs4pnfsd.c b/fs/nfsd/nfs4pnfsd.c
index 24c19b3..48104fb 100644
--- a/fs/nfsd/nfs4pnfsd.c
+++ b/fs/nfsd/nfs4pnfsd.c
@@ -1446,11 +1446,6 @@ struct create_recall_list_arg {
 	       cbl->cbl_seg.iomode != IOMODE_RW &&
 	       cbl->cbl_seg.iomode != IOMODE_ANY);
 
-	if (nfsd_serv == NULL) {
-		dprintk("NFSD nfsd_layout_recall_cb: nfsd_serv == NULL\n");
-		return -ENOENT;
-	}
-
 	if (!with_nfs4_state_lock)
 		nfs4_lock_state();
 	status = -ENOENT;
-- 
1.7.11.7

[PATCH 20/38] SQUASHME: pnfsd: merge_layout needs to acquire the layout_lock for traversing fi_layouts

[Benny Halevy]

Signed-off-by: Benny Halevy <bhalevy@tonian.com>
---
 fs/nfsd/nfs4pnfsd.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/fs/nfsd/nfs4pnfsd.c b/fs/nfsd/nfs4pnfsd.c
index 48104fb..0bde7b7 100644
--- a/fs/nfsd/nfs4pnfsd.c
+++ b/fs/nfsd/nfs4pnfsd.c
@@ -622,6 +622,7 @@ struct super_block *
 {
 	struct nfs4_layout *lp = NULL;
 
+	spin_lock(&layout_lock);
 	list_for_each_entry (lp, &fp->fi_layouts, lo_perfile)
 		if (lp->lo_seg.layout_type == seg->layout_type &&
 		    lp->lo_seg.clientid == seg->clientid &&
@@ -630,6 +631,7 @@ struct super_block *
 			extend_layout(&lp->lo_seg, seg);
 			break;
 		}
+	spin_unlock(&layout_lock);
 
 	return lp;
 }
-- 
1.7.11.7

[PATCH 21/38] SQUASHME: pnfsd: return bool from merge_layout and fix not found path

[Benny Halevy]

This also fixes the false return when no layout is found
(previously a non NULL pointer was returned)

Signed-off-by: Benny Halevy <bhalevy@tonian.com>
---
 fs/nfsd/nfs4pnfsd.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/fs/nfsd/nfs4pnfsd.c b/fs/nfsd/nfs4pnfsd.c
index 0bde7b7..ff688cb 100644
--- a/fs/nfsd/nfs4pnfsd.c
+++ b/fs/nfsd/nfs4pnfsd.c
@@ -578,7 +578,7 @@ struct super_block *
 /*
  * are two octet ranges overlapping or adjacent?
  */
-static int
+static bool
 lo_seg_mergeable(struct nfsd4_layout_seg *l1, struct nfsd4_layout_seg *l2)
 {
 	u64 start1 = l1->offset;
@@ -615,25 +615,26 @@ struct super_block *
 		      lo_end : lo_end - lo_start;
 }
 
-static struct nfs4_layout *
+static bool
 merge_layout(struct nfs4_file *fp,
 	     struct nfs4_client *clp,
 	     struct nfsd4_layout_seg *seg)
 {
-	struct nfs4_layout *lp = NULL;
+	bool ret = false;
+	struct nfs4_layout *lp;
 
 	spin_lock(&layout_lock);
 	list_for_each_entry (lp, &fp->fi_layouts, lo_perfile)
 		if (lp->lo_seg.layout_type == seg->layout_type &&
 		    lp->lo_seg.clientid == seg->clientid &&
 		    lp->lo_seg.iomode == seg->iomode &&
-		    lo_seg_mergeable(&lp->lo_seg, seg)) {
+		    (ret = lo_seg_mergeable(&lp->lo_seg, seg))) {
 			extend_layout(&lp->lo_seg, seg);
 			break;
 		}
 	spin_unlock(&layout_lock);
 
-	return lp;
+	return ret;
 }
 
 __be32
-- 
1.7.11.7

[PATCH 22/38] SQUASHME: pnfsd: hanging nfs4_layoutrecall on the client's list must be under the layout_lock

[Benny Halevy]

Signed-off-by: Benny Halevy <bhalevy@tonian.com>
---
 fs/nfsd/nfs4pnfsd.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/fs/nfsd/nfs4pnfsd.c b/fs/nfsd/nfs4pnfsd.c
index ff688cb..7c454d2 100644
--- a/fs/nfsd/nfs4pnfsd.c
+++ b/fs/nfsd/nfs4pnfsd.c
@@ -1416,8 +1416,10 @@ struct create_recall_list_arg {
 		pending->parent = parent;
 		get_layoutrecall(pending);
 		/* Add to list so corresponding layoutreturn can find req */
+		spin_lock(&layout_lock);
 		list_add(&pending->clr_perclnt,
 			 &pending->clr_client->cl_layoutrecalls);
+		spin_unlock(&layout_lock);
 
 		nfsd4_cb_layout(pending);
 		--todo_len;
-- 
1.7.11.7

[PATCH 23/38] DEBUG: pnfsd: ASSERT_LAYOUT_LOCKED

[Benny Halevy]

Signed-off-by: Benny Halevy <bhalevy@tonian.com>
---
 fs/nfsd/nfs4pnfsd.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/fs/nfsd/nfs4pnfsd.c b/fs/nfsd/nfs4pnfsd.c
index 7c454d2..d9bf099 100644
--- a/fs/nfsd/nfs4pnfsd.c
+++ b/fs/nfsd/nfs4pnfsd.c
@@ -28,6 +28,8 @@
 
 static DEFINE_SPINLOCK(layout_lock);
 
+#define ASSERT_LAYOUT_LOCKED()	assert_spin_locked(&layout_lock);
+
 /*
  * Layout state - NFSv4.1 pNFS
  */
-- 
1.7.11.7

[PATCH 24/38] pnfsd: cause oops when stop the nfs server

[Benny Halevy]

From: chenditang <chendt.fnst@cn.fujitsu.com>

mount nfs dir in the client, and then restart the NFS service in MDS,
that will cause oops for client_mutex_owner is NULL in the
destroy_client() function.

kernel BUG at fs/nfsd/nfs4state.c:1130!
invalid opcode: 0000 [#1] SMP
Modules linked in: nfsd(OF) lockd exportfs nfs_acl auth_rpcgss autofs4
dlm sctp libcrc32c configfs sunrpc be2iscsi iscsi_boot_sysfs bnx2i cnic
uio cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio ib_iser rdma_cm ib_cm iw_cm
ib_sa ib_mad ib_core ib_addr ipv6 iscsi_tcp libiscsi_tcp libiscsi
scsi_transport_iscsi dm_mirror dm_region_hash dm_log dm_mod ppdev
parport_pc parport microcode pcspkr i2c_piix4 i2c_core e1000 sg ext4(F)
mbcache(F) jbd2(F) sr_mod(F) cdrom(F) sd_mod(F) crc_t10dif(F)
pata_acpi(F) ata_generic(F) ata_piix(F) ahci(F) libahci(F) [last
unloaded: speedstep_lib]
CPU 0
Pid: 2893, comm: nfsd Tainted: GF O 3.8.0-rc4_fl+ #2 innotek GmbH
VirtualBox/VirtualBox
RIP: 0010:[<ffffffffa045a4cf>] [<ffffffffa045a4cf>]
destroy_client+0x2ff/0x330 [nfsd]
RSP: 0018:ffff8800256b9d38 EFLAGS: 00010203
RAX: 0000000000000010 RBX: ffff880025630000 RCX: 0000000000001440
RDX: 00000000000025fc RSI: 0000000000000082 RDI: 0000000000000246
RBP: ffff8800256b9d88 R08: ffffffff81cdfba0 R09: 0000000000007768
R10: 00000000000001b1 R11: 00000000000001b1 R12: ffff880037988400
R13: ffff88003d58ea80 R14: ffffffff81ab2080 R15: 0000000000000000
FS: 0000000000000000(0000) GS:ffff88003fc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00000000006d39d8 CR3: 00000000255be000 CR4: 00000000000006f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process nfsd (pid: 2893, threadinfo ffff8800256b8000, task ffff88003d58ea80)
Stack:
ffff8800256b9d78 ffff88003d51bdc0 ffff8800256b9d58 ffff880037988400
ffffffff81ab2080 ffff880037988400 0000000000000020 ffffffffa04709b0
ffffffff81ab2080 0000000000000000 ffff8800256b9dc8 ffffffffa045a555
Call Trace:
[<ffffffffa045a555>] nfs4_state_destroy_net+0x55/0x120 [nfsd]
[<ffffffffa045a718>] nfs4_state_shutdown_net+0xf8/0x140 [nfsd]
[<ffffffffa04349b0>] ? nfsd_pool_stats_release+0x50/0x50 [nfsd]
[<ffffffffa0434835>] nfsd_shutdown_net+0x35/0x60 [nfsd]
[<ffffffffa04348ad>] nfsd_last_thread+0x4d/0x80 [nfsd]
[<ffffffffa037f3e5>] svc_shutdown_net+0x35/0x40 [sunrpc]
[<ffffffffa0434935>] nfsd_destroy+0x55/0x80 [nfsd]
[<ffffffffa0434ab4>] nfsd+0x104/0x130 [nfsd]
[<ffffffffa04349b0>] ? nfsd_pool_stats_release+0x50/0x50 [nfsd]
[<ffffffff8107951e>] kthread+0xce/0xe0
[<ffffffff81079450>] ? kthread_freezable_should_stop+0x70/0x70
[<ffffffff8155c56c>] ret_from_fork+0x7c/0xb0
[<ffffffff81079450>] ? kthread_freezable_should_stop+0x70/0x70
Code: 38 d1 e0 48 89 df e8 81 38 d1 e0 e9 2f ff ff ff 0f 1f 40 00 48 c7
c7 72 8a 46 a0 31 c0 e8 86 6b 0f e1 e9 70 fd ff ff 0f 0b eb fe <0f> 0b
eb fe 0f 0b 66 66 2e 0f 1f 84 00 00 00 00 00 eb f3 0f 0b
RIP [<ffffffffa045a4cf>] destroy_client+0x2ff/0x330 [nfsd]
RSP <ffff8800256b9d38>
---[ end trace c2d9f251eabc7c2d ]---

Signed-off-by: chendt.fnst <chendt.fnst@cn.fujitsu.com>
Reviewed-by: fanchaoting <fanchaoting@cn.fujitsu.com>
Signed-off-by: Benny Halevy <bhalevy@tonian.com>
---
 fs/nfsd/nfs4state.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 1170f8a..282be50 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -4937,6 +4937,8 @@ static int nfs4_state_create_net(struct net *net)
 	struct nfsd_net *nn = net_generic(net, nfsd_net_id);
 	struct rb_node *node, *tmp;
 
+	nfs4_lock_state();
+
 	for (i = 0; i < CLIENT_HASH_SIZE; i++) {
 		while (!list_empty(&nn->conf_id_hashtbl[i])) {
 			clp = list_entry(nn->conf_id_hashtbl[i].next, struct nfs4_client, cl_idhash);
@@ -4953,6 +4955,7 @@ static int nfs4_state_create_net(struct net *net)
 		destroy_client(clp);
 	}
 
+	nfs4_unlock_state();
 	kfree(nn->sessionid_hashtbl);
 	kfree(nn->lockowner_ino_hashtbl);
 	kfree(nn->ownerstr_hashtbl);
-- 
1.7.11.7

[PATCH 25/38] pnfsd: layout state: rename fi_layout_states member

[Benny Halevy]

Signed-off-by: Benny Halevy <bhalevy@tonian.com>
---
 fs/nfsd/nfs4pnfsd.c | 2 +-
 fs/nfsd/nfs4state.c | 2 +-
 fs/nfsd/state.h     | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/fs/nfsd/nfs4pnfsd.c b/fs/nfsd/nfs4pnfsd.c
index d9bf099..1b31432 100644
--- a/fs/nfsd/nfs4pnfsd.c
+++ b/fs/nfsd/nfs4pnfsd.c
@@ -133,7 +133,7 @@ struct sbid_tracker {
 	nfsd4_init_stid(&new->ls_stid, clp, NFS4_LAYOUT_STID);
 	INIT_LIST_HEAD(&new->ls_perfile);
 	spin_lock(&layout_lock);
-	list_add(&new->ls_perfile, &fp->fi_layout_states);
+	list_add(&new->ls_perfile, &fp->fi_lo_states);
 	spin_unlock(&layout_lock);
 	new->ls_roc = false;
 	return new;
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 282be50..12eed1d 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -2364,7 +2364,7 @@ static void nfsd4_init_file(struct nfs4_file *fp, struct inode *ino,
 	memset(fp->fi_access, 0, sizeof(fp->fi_access));
 #if defined(CONFIG_PNFSD)
 	INIT_LIST_HEAD(&fp->fi_layouts);
-	INIT_LIST_HEAD(&fp->fi_layout_states);
+	INIT_LIST_HEAD(&fp->fi_lo_states);
 	fp->fi_fsid.major = current_fh->fh_export->ex_fsid;
 	fp->fi_fsid.minor = 0;
 	fp->fi_fhlen = current_fh->fh_handle.fh_size;
diff --git a/fs/nfsd/state.h b/fs/nfsd/state.h
index 5c613ba..c185025 100644
--- a/fs/nfsd/state.h
+++ b/fs/nfsd/state.h
@@ -412,7 +412,7 @@ struct nfs4_file {
 	bool			fi_had_conflict;
 #if defined(CONFIG_PNFSD)
 	struct list_head	fi_layouts;
-	struct list_head	fi_layout_states;
+	struct list_head	fi_lo_states;
 	/* used by layoutget / layoutrecall */
 	struct nfs4_fsid	fi_fsid;
 	u32			fi_fhlen;
-- 
1.7.11.7

[PATCH 26/38] pnfsd: layout state: move initialization of ls_roc

[Benny Halevy]

initialize prior to enlisting on fi_lo_states

Signed-off-by: Benny Halevy <bhalevy@tonian.com>
---
 fs/nfsd/nfs4pnfsd.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/fs/nfsd/nfs4pnfsd.c b/fs/nfsd/nfs4pnfsd.c
index 1b31432..6e0e9d1 100644
--- a/fs/nfsd/nfs4pnfsd.c
+++ b/fs/nfsd/nfs4pnfsd.c
@@ -132,10 +132,10 @@ struct sbid_tracker {
 	kref_init(&new->ls_ref);
 	nfsd4_init_stid(&new->ls_stid, clp, NFS4_LAYOUT_STID);
 	INIT_LIST_HEAD(&new->ls_perfile);
+	new->ls_roc = false;
 	spin_lock(&layout_lock);
 	list_add(&new->ls_perfile, &fp->fi_lo_states);
 	spin_unlock(&layout_lock);
-	new->ls_roc = false;
 	return new;
 }
 
-- 
1.7.11.7

[PATCH 27/38] pnfsd: layout state: introduce ls_file backpointer

[Benny Halevy]

Signed-off-by: Benny Halevy <bhalevy@tonian.com>
---
 fs/nfsd/nfs4pnfsd.c | 3 +++
 fs/nfsd/pnfsd.h     | 1 +
 2 files changed, 4 insertions(+)

diff --git a/fs/nfsd/nfs4pnfsd.c b/fs/nfsd/nfs4pnfsd.c
index 6e0e9d1..b147427 100644
--- a/fs/nfsd/nfs4pnfsd.c
+++ b/fs/nfsd/nfs4pnfsd.c
@@ -132,6 +132,8 @@ struct sbid_tracker {
 	kref_init(&new->ls_ref);
 	nfsd4_init_stid(&new->ls_stid, clp, NFS4_LAYOUT_STID);
 	INIT_LIST_HEAD(&new->ls_perfile);
+	get_nfs4_file(fp);	/* released on destroy_layout_state */
+	new->ls_file = fp;
 	new->ls_roc = false;
 	spin_lock(&layout_lock);
 	list_add(&new->ls_perfile, &fp->fi_lo_states);
@@ -157,6 +159,7 @@ struct sbid_tracker {
 		list_del(&ls->ls_perfile);
 		spin_unlock(&layout_lock);
 	}
+	put_nfs4_file(ls->ls_file);
 	kmem_cache_free(layout_state_slab, ls);
 }
 
diff --git a/fs/nfsd/pnfsd.h b/fs/nfsd/pnfsd.h
index 2507d31..6301c51 100644
--- a/fs/nfsd/pnfsd.h
+++ b/fs/nfsd/pnfsd.h
@@ -45,6 +45,7 @@ struct nfs4_layout_state {
 	struct nfs4_stid	ls_stid; /* must be first field */
 	struct kref		ls_ref;
 	struct list_head	ls_perfile;
+	struct nfs4_file	*ls_file;
 	bool			ls_roc;
 };
 
-- 
1.7.11.7

[PATCH 28/38] pnfsd: layout state: introduce ls_client backpointer

[Benny Halevy]

And enlist the layout state on the client cl_lo_states list.

Signed-off-by: Benny Halevy <bhalevy@tonian.com>
---
 fs/nfsd/nfs4pnfsd.c | 16 +++++++++++++++-
 fs/nfsd/nfs4state.c |  1 +
 fs/nfsd/pnfsd.h     |  2 ++
 fs/nfsd/state.h     |  1 +
 4 files changed, 19 insertions(+), 1 deletion(-)

diff --git a/fs/nfsd/nfs4pnfsd.c b/fs/nfsd/nfs4pnfsd.c
index b147427..50b5208 100644
--- a/fs/nfsd/nfs4pnfsd.c
+++ b/fs/nfsd/nfs4pnfsd.c
@@ -131,11 +131,14 @@ struct sbid_tracker {
 		return new;
 	kref_init(&new->ls_ref);
 	nfsd4_init_stid(&new->ls_stid, clp, NFS4_LAYOUT_STID);
+	INIT_LIST_HEAD(&new->ls_perclnt);
 	INIT_LIST_HEAD(&new->ls_perfile);
+	new->ls_client = clp;
 	get_nfs4_file(fp);	/* released on destroy_layout_state */
 	new->ls_file = fp;
 	new->ls_roc = false;
 	spin_lock(&layout_lock);
+	list_add(&new->ls_perclnt, &clp->cl_lo_states);
 	list_add(&new->ls_perfile, &fp->fi_lo_states);
 	spin_unlock(&layout_lock);
 	return new;
@@ -147,6 +150,17 @@ struct sbid_tracker {
 	kref_get(&ls->ls_ref);
 }
 
+/*
+ * Note: always called under the layout_lock
+ */
+static void
+unhash_layout_state(struct nfs4_layout_state *ls)
+{
+	ASSERT_LAYOUT_LOCKED();
+	list_del_init(&ls->ls_perclnt);
+	list_del_init(&ls->ls_perfile);
+}
+
 static void
 destroy_layout_state(struct kref *kref)
 {
@@ -156,7 +170,7 @@ struct sbid_tracker {
 	nfsd4_unhash_stid(&ls->ls_stid);
 	if (!list_empty(&ls->ls_perfile)) {
 		spin_lock(&layout_lock);
-		list_del(&ls->ls_perfile);
+		unhash_layout_state(ls);
 		spin_unlock(&layout_lock);
 	}
 	put_nfs4_file(ls->ls_file);
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 12eed1d..4f40327 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -1322,6 +1322,7 @@ static struct nfs4_client *create_client(struct xdr_netobj name,
 	INIT_LIST_HEAD(&clp->cl_delegations);
 #if defined(CONFIG_PNFSD)
 	INIT_LIST_HEAD(&clp->cl_layouts);
+	INIT_LIST_HEAD(&clp->cl_lo_states);
 	INIT_LIST_HEAD(&clp->cl_layoutrecalls);
 #endif /* CONFIG_PNFSD */
 	INIT_LIST_HEAD(&clp->cl_lru);
diff --git a/fs/nfsd/pnfsd.h b/fs/nfsd/pnfsd.h
index 6301c51..454ef23 100644
--- a/fs/nfsd/pnfsd.h
+++ b/fs/nfsd/pnfsd.h
@@ -44,6 +44,8 @@
 struct nfs4_layout_state {
 	struct nfs4_stid	ls_stid; /* must be first field */
 	struct kref		ls_ref;
+	struct list_head	ls_perclnt;
+	struct nfs4_client	*ls_client;
 	struct list_head	ls_perfile;
 	struct nfs4_file	*ls_file;
 	bool			ls_roc;
diff --git a/fs/nfsd/state.h b/fs/nfsd/state.h
index c185025..e026e0d 100644
--- a/fs/nfsd/state.h
+++ b/fs/nfsd/state.h
@@ -283,6 +283,7 @@ struct nfs4_client {
 	struct net		*net;
 #if defined(CONFIG_PNFSD)
 	struct list_head	cl_layouts;	/* outstanding layouts */
+	struct list_head	cl_lo_states;	/* outstanding layout states */
 	struct list_head	cl_layoutrecalls; /* outstanding layoutrecall
 						     callbacks */
 #endif /* CONFIG_PNFSD */
-- 
1.7.11.7

[PATCH 29/38] pnfsd: layout state: hang layouts on layout state

[Benny Halevy]

Rather than list the layout directly on the corresponding file and client
list the layout state on the file and client and list all layout segments
associated with the state on the respective layout state.

Use the layout state list (lo_layouts) for looking up the layout.

Signed-off-by: Benny Halevy <bhalevy@tonian.com>
---
 fs/nfsd/nfs4pnfsd.c | 157 ++++++++++++++++++++++++++--------------------------
 fs/nfsd/nfs4state.c |   2 -
 fs/nfsd/pnfsd.h     |   6 +-
 fs/nfsd/state.h     |   2 -
 4 files changed, 79 insertions(+), 88 deletions(-)

diff --git a/fs/nfsd/nfs4pnfsd.c b/fs/nfsd/nfs4pnfsd.c
index 50b5208..d3a0bde 100644
--- a/fs/nfsd/nfs4pnfsd.c
+++ b/fs/nfsd/nfs4pnfsd.c
@@ -133,6 +133,7 @@ struct sbid_tracker {
 	nfsd4_init_stid(&new->ls_stid, clp, NFS4_LAYOUT_STID);
 	INIT_LIST_HEAD(&new->ls_perclnt);
 	INIT_LIST_HEAD(&new->ls_perfile);
+	INIT_LIST_HEAD(&new->ls_layouts);
 	new->ls_client = clp;
 	get_nfs4_file(fp);	/* released on destroy_layout_state */
 	new->ls_file = fp;
@@ -276,13 +277,6 @@ static void update_layout_stateid_locked(struct nfs4_layout_state *ls, stateid_t
 		__func__, sid->si_generation, ls);
 }
 
-static void update_layout_stateid(struct nfs4_layout_state *ls, stateid_t *sid)
-{
-	spin_lock(&layout_lock);
-	update_layout_stateid_locked(ls, sid);
-	spin_unlock(&layout_lock);
-}
-
 static void update_layout_roc(struct nfs4_layout_state *ls, bool roc)
 {
 	if (roc) {
@@ -295,35 +289,28 @@ static void update_layout_roc(struct nfs4_layout_state *ls, bool roc)
 static void
 init_layout(struct nfs4_layout *lp,
 	    struct nfs4_layout_state *ls,
-	    struct nfs4_file *fp,
-	    struct nfs4_client *clp,
 	    struct svc_fh *current_fh,
 	    struct nfsd4_layout_seg *seg,
 	    stateid_t *stateid)
 {
-	dprintk("pNFS %s: lp %p ls %p clp %p fp %p ino %p\n", __func__,
-		lp, ls, clp, fp, fp->fi_inode);
+	dprintk("pNFS %s: lp %p ls %p ino %lu\n", __func__,
+		lp, ls, ls->ls_file->fi_inode->i_ino);
 
-	get_nfs4_file(fp);
-	lp->lo_client = clp;
-	lp->lo_file = fp;
 	memcpy(&lp->lo_seg, seg, sizeof(lp->lo_seg));
 	get_layout_state(ls);		/* put on destroy_layout */
 	lp->lo_state = ls;
-	update_layout_stateid(ls, stateid);
-	list_add_tail(&lp->lo_perclnt, &clp->cl_layouts);
-	list_add_tail(&lp->lo_perfile, &fp->fi_layouts);
+	spin_lock(&layout_lock);
+	update_layout_stateid_locked(ls, stateid);
+	list_add_tail(&lp->lo_perstate, &ls->ls_layouts);
+	spin_unlock(&layout_lock);
 	dprintk("pNFS %s end\n", __func__);
 }
 
-/*
- * Note: always called under the layout_lock
- */
 static void
 dequeue_layout(struct nfs4_layout *lp)
 {
-	list_del(&lp->lo_perclnt);
-	list_del(&lp->lo_perfile);
+	ASSERT_LAYOUT_LOCKED();
+	list_del_init(&lp->lo_perstate);
 }
 
 /*
@@ -332,20 +319,15 @@ static void update_layout_roc(struct nfs4_layout_state *ls, bool roc)
 static void
 destroy_layout(struct nfs4_layout *lp)
 {
-	struct nfs4_client *clp;
-	struct nfs4_file *fp;
 	struct nfs4_layout_state *ls;
 
-	clp = lp->lo_client;
-	fp = lp->lo_file;
 	ls = lp->lo_state;
-	dprintk("pNFS %s: lp %p clp %p fp %p ino %p\n",
-		__func__, lp, clp, fp, fp->fi_inode);
+	dprintk("pNFS %s: lp %p ls %p ino %lu\n",
+		__func__, lp, ls, ls->ls_file->fi_inode->i_ino);
 
 	kmem_cache_free(pnfs_layout_slab, lp);
 	/* release references taken by init_layout */
 	put_layout_state(ls);
-	put_nfs4_file(fp);
 }
 
 static void fs_layout_return(struct inode *ino,
@@ -635,15 +617,13 @@ struct super_block *
 }
 
 static bool
-merge_layout(struct nfs4_file *fp,
-	     struct nfs4_client *clp,
-	     struct nfsd4_layout_seg *seg)
+merge_layout(struct nfs4_layout_state *ls, struct nfsd4_layout_seg *seg)
 {
 	bool ret = false;
 	struct nfs4_layout *lp;
 
 	spin_lock(&layout_lock);
-	list_for_each_entry (lp, &fp->fi_layouts, lo_perfile)
+	list_for_each_entry (lp, &ls->ls_layouts, lo_perstate)
 		if (lp->lo_seg.layout_type == seg->layout_type &&
 		    lp->lo_seg.clientid == seg->clientid &&
 		    lp->lo_seg.iomode == seg->iomode &&
@@ -795,11 +775,11 @@ struct super_block *
 	 * Can the new layout be merged into an existing one?
 	 * If so, free unused layout struct
 	 */
-	if (can_merge && merge_layout(fp, clp, &res.lg_seg))
+	if (can_merge && merge_layout(ls, &res.lg_seg))
 		goto out_freelayout;
 
 	/* Can't merge, so let's initialize this new layout */
-	init_layout(lp, ls, fp, clp, lgp->lg_fhp, &res.lg_seg, &lgp->lg_sid);
+	init_layout(lp, ls, lgp->lg_fhp, &res.lg_seg, &lgp->lg_sid);
 out_unlock:
 	if (ls)
 		put_layout_state(ls);
@@ -883,8 +863,8 @@ struct super_block *
 		return;
 	}
 
-	list_for_each_entry_safe(lo, nextlp, lo_destroy_list, lo_perfile) {
-		struct inode *inode = lo->lo_file->fi_inode;
+	list_for_each_entry_safe(lo, nextlp, lo_destroy_list, lo_perstate) {
+		struct inode *inode = lo->lo_state->ls_file->fi_inode;
 		struct nfsd4_pnfs_layoutreturn lr;
 		bool empty;
 		int lr_flags = flags;
@@ -894,39 +874,39 @@ struct super_block *
 		lr.args.lr_seg = lo->lo_seg;
 
 		spin_lock(&layout_lock);
-		if (list_empty(&lo->lo_file->fi_layouts))
+		if (list_empty(&lo->lo_state->ls_file->fi_lo_states))
 			lr_flags |= LR_FLAG_EMPTY;
 		spin_unlock(&layout_lock);
 
-		list_del(&lo->lo_perfile);
+		list_del(&lo->lo_perstate);
 		empty = list_empty(lo_destroy_list);
 
 		fs_layout_return(inode, &lr, lr_flags, empty ? cb_cookie : NULL);
 
-		destroy_layout(lo); /* this will put the lo_file */
+		destroy_layout(lo);
 	}
 }
 
+/*
+ * Return layouts for RETURN_FILE
+ */
 static int
-pnfs_return_file_layouts(struct nfs4_client *clp, struct nfs4_file *fp,
-			 struct nfsd4_pnfs_layoutreturn *lrp,
+pnfs_return_file_layouts(struct nfsd4_pnfs_layoutreturn *lrp,
 			 struct nfs4_layout_state *ls,
 			 struct list_head *lo_destroy_list)
 {
 	int layouts_found = 0;
 	struct nfs4_layout *lp, *nextlp;
 
-	dprintk("%s: clp %p fp %p\n", __func__, clp, fp);
+	dprintk("%s: ls %p\n", __func__, ls);
 	lrp->lrs_present = 0;
 	spin_lock(&layout_lock);
-	list_for_each_entry_safe (lp, nextlp, &fp->fi_layouts, lo_perfile) {
-		dprintk("%s: lp %p client %p,%p lo_type %x,%x iomode %d,%d\n",
-			__func__, lp,
-			lp->lo_client, clp,
+	list_for_each_entry_safe (lp, nextlp, &ls->ls_layouts, lo_perstate) {
+		dprintk("%s: lp %p ls %p inode %lu lo_type %x,%x iomode %d,%d\n",
+			__func__, lp, lp->lo_state,
+			lp->lo_state->ls_file->fi_inode->i_ino,
 			lp->lo_seg.layout_type, lrp->args.lr_seg.layout_type,
 			lp->lo_seg.iomode, lrp->args.lr_seg.iomode);
-		if (lp->lo_client != clp)
-			continue;
 		if (lp->lo_seg.layout_type != lrp->args.lr_seg.layout_type ||
 		    (lp->lo_seg.iomode != lrp->args.lr_seg.iomode &&
 		     lrp->args.lr_seg.iomode != IOMODE_ANY) ||
@@ -938,7 +918,7 @@ struct super_block *
 		trim_layout(&lp->lo_seg, &lrp->args.lr_seg);
 		if (!lp->lo_seg.length) {
 			dequeue_layout(lp);
-			list_add_tail(&lp->lo_perfile, lo_destroy_list);
+			list_add_tail(&lp->lo_perstate, lo_destroy_list);
 		} else
 			lrp->lrs_present = 1;
 	}
@@ -949,28 +929,36 @@ struct super_block *
 	return layouts_found;
 }
 
+/*
+ * Return layouts for RETURN_FSID or RETURN_ALL
+ */
 static int
 pnfs_return_client_layouts(struct nfs4_client *clp,
 			   struct nfsd4_pnfs_layoutreturn *lrp, u64 ex_fsid,
 			   struct list_head *lo_destroy_list)
 {
 	int layouts_found = 0;
+	struct nfs4_layout_state *ls, *nextls;
 	struct nfs4_layout *lp, *nextlp;
 
 	spin_lock(&layout_lock);
-	list_for_each_entry_safe (lp, nextlp, &clp->cl_layouts, lo_perclnt) {
-		if (lrp->args.lr_seg.layout_type != lp->lo_seg.layout_type ||
-		   (lrp->args.lr_seg.iomode != lp->lo_seg.iomode &&
-		    lrp->args.lr_seg.iomode != IOMODE_ANY))
-			continue;
-
+	list_for_each_entry_safe (ls, nextls, &clp->cl_lo_states, ls_perclnt) {
 		if (lrp->args.lr_return_type == RETURN_FSID &&
-		    !same_fsid_major(&lp->lo_file->fi_fsid, ex_fsid))
+		    !same_fsid_major(&ls->ls_file->fi_fsid, ex_fsid))
 			continue;
 
-		layouts_found++;
-		dequeue_layout(lp);
-		list_add_tail(&lp->lo_perfile, lo_destroy_list);
+		list_for_each_entry_safe (lp, nextlp, &ls->ls_layouts, lo_perstate) {
+			if (lrp->args.lr_seg.layout_type != lp->lo_seg.layout_type)
+				break;
+
+			if (lrp->args.lr_seg.iomode != lp->lo_seg.iomode &&
+			    lrp->args.lr_seg.iomode != IOMODE_ANY)
+				continue;
+
+			layouts_found++;
+			dequeue_layout(lp);
+			list_add_tail(&lp->lo_perstate, lo_destroy_list);
+		}
 	}
 	spin_unlock(&layout_lock);
 
@@ -1071,8 +1059,7 @@ int nfs4_pnfs_return_layout(struct svc_rqst *rqstp,
 			goto out_put_file;
 
 		/* update layouts */
-		layouts_found = pnfs_return_file_layouts(clp, fp, lrp, ls,
-							 &lo_destroy_list);
+		layouts_found = pnfs_return_file_layouts(lrp, ls, &lo_destroy_list);
 		put_layout_state(ls);
 		if (!lrp->lrs_present)
 			lr_flags |= LR_FLAG_CL_EMPTY;
@@ -1202,8 +1189,13 @@ int nfs4_pnfs_return_layout(struct svc_rqst *rqstp,
 		clr->clr_client, clr->clr_file);
 
 	if (clr->cb.cbl_recall_type == RETURN_FILE) {
-		pnfs_return_file_layouts(clr->clr_client, clr->clr_file, &lr,
-					 NULL, &lo_destroy_list);
+		struct nfs4_layout_state *ls;
+
+		list_for_each_entry (ls, &clr->clr_client->cl_lo_states, ls_perclnt)
+			if (ls->ls_file == clr->clr_file) {
+				pnfs_return_file_layouts(&lr, ls, &lo_destroy_list);
+				break;
+			}
 		if (!lr.lrs_present)
 			lr_flags |= LR_FLAG_CL_EMPTY;
 	} else {
@@ -1222,6 +1214,15 @@ int nfs4_pnfs_return_layout(struct svc_rqst *rqstp,
 			     recall_cookie);
 }
 
+void strip_layout_state(struct nfs4_layout_state *ls, struct list_head *lo_destroy_list)
+{
+	unhash_layout_state(ls);
+	if (!list_empty(&ls->ls_layouts)) {
+		list_splice(&ls->ls_layouts, lo_destroy_list);
+		INIT_LIST_HEAD(&ls->ls_layouts);
+	}
+}
+
 /* Return On Close:
  *   Look for all layouts of @fp that belong to @clp, if ROC is set, remove
  *   the layout and simulate a layout_return. Surly the client has forgotten
@@ -1229,30 +1230,27 @@ int nfs4_pnfs_return_layout(struct svc_rqst *rqstp,
  */
 void pnfsd_roc(struct nfs4_client *clp, struct nfs4_file *fp)
 {
-	struct nfs4_layout *lo, *nextlp;
+	struct nfs4_layout_state *ls, *nextls;
 	LIST_HEAD(lo_destroy_list);
 
 	/* TODO: We need to also free layout recalls like pnfs_expire_client */
 	dprintk("%s: fp=%p clp=%p", __func__, fp, clp);
 	spin_lock(&layout_lock);
-	list_for_each_entry_safe (lo, nextlp, &fp->fi_layouts, lo_perfile) {
-		/* Check for a match */
-		if (lo->lo_client != clp)
+	list_for_each_entry_safe (ls, nextls, &fp->fi_lo_states, ls_perfile) {
+		if (ls->ls_client != clp)
 			continue;
 
-		/* Return the layout */
-		list_del_init(&lo->lo_state->ls_perfile);	/* just to be on the safe side */
-		dequeue_layout(lo);
-		list_add_tail(&lo->lo_perfile, &lo_destroy_list);
+		strip_layout_state(ls, &lo_destroy_list);
+		break;
 	}
 	spin_unlock(&layout_lock);
 
-	pnfsd_return_lo_list(&lo_destroy_list, NULL, NULL, LR_FLAG_INTERN, NULL);
+	pnfsd_return_lo_list(&lo_destroy_list, fp->fi_inode, NULL, LR_FLAG_INTERN, NULL);
 }
 
 void pnfs_expire_client(struct nfs4_client *clp)
 {
-	struct nfs4_layout *lo, *nextlo;
+	struct nfs4_layout_state *ls, *nextls;
 	LIST_HEAD(lo_destroy_list);
 
 	for (;;) {
@@ -1275,12 +1273,11 @@ void pnfs_expire_client(struct nfs4_client *clp)
 	}
 
 	spin_lock(&layout_lock);
-	list_for_each_entry_safe (lo, nextlo, &clp->cl_layouts, lo_perclnt) {
-		BUG_ON(lo->lo_client != clp);
-		dequeue_layout(lo);
-		list_add_tail(&lo->lo_perfile, &lo_destroy_list);
-		dprintk("%s: inode %lu lp %p clp %p\n", __func__,
-			lo->lo_file->fi_inode->i_ino, lo, clp);
+	list_for_each_entry_safe (ls, nextls, &clp->cl_lo_states, ls_perclnt) {
+		BUG_ON(ls->ls_client != clp);
+		strip_layout_state(ls, &lo_destroy_list);
+		dprintk("%s: inode %lu ls %p clp %p\n", __func__,
+			ls->ls_file->fi_inode->i_ino, ls, clp);
 	}
 	spin_unlock(&layout_lock);
 
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 4f40327..faef919 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -1321,7 +1321,6 @@ static struct nfs4_client *create_client(struct xdr_netobj name,
 	INIT_LIST_HEAD(&clp->cl_openowners);
 	INIT_LIST_HEAD(&clp->cl_delegations);
 #if defined(CONFIG_PNFSD)
-	INIT_LIST_HEAD(&clp->cl_layouts);
 	INIT_LIST_HEAD(&clp->cl_lo_states);
 	INIT_LIST_HEAD(&clp->cl_layoutrecalls);
 #endif /* CONFIG_PNFSD */
@@ -2364,7 +2363,6 @@ static void nfsd4_init_file(struct nfs4_file *fp, struct inode *ino,
 	memset(fp->fi_fds, 0, sizeof(fp->fi_fds));
 	memset(fp->fi_access, 0, sizeof(fp->fi_access));
 #if defined(CONFIG_PNFSD)
-	INIT_LIST_HEAD(&fp->fi_layouts);
 	INIT_LIST_HEAD(&fp->fi_lo_states);
 	fp->fi_fsid.major = current_fh->fh_export->ex_fsid;
 	fp->fi_fsid.minor = 0;
diff --git a/fs/nfsd/pnfsd.h b/fs/nfsd/pnfsd.h
index 454ef23..453f951 100644
--- a/fs/nfsd/pnfsd.h
+++ b/fs/nfsd/pnfsd.h
@@ -48,15 +48,13 @@ struct nfs4_layout_state {
 	struct nfs4_client	*ls_client;
 	struct list_head	ls_perfile;
 	struct nfs4_file	*ls_file;
+	struct list_head	ls_layouts;
 	bool			ls_roc;
 };
 
 /* outstanding layout */
 struct nfs4_layout {
-	struct list_head		lo_perfile;	/* hash by f_id */
-	struct list_head		lo_perclnt;	/* hash by clientid */
-	struct nfs4_file		*lo_file;	/* backpointer */
-	struct nfs4_client		*lo_client;
+	struct list_head		lo_perstate;
 	struct nfs4_layout_state	*lo_state;
 	struct nfsd4_layout_seg		lo_seg;
 };
diff --git a/fs/nfsd/state.h b/fs/nfsd/state.h
index e026e0d..ab4a136 100644
--- a/fs/nfsd/state.h
+++ b/fs/nfsd/state.h
@@ -282,7 +282,6 @@ struct nfs4_client {
 						/* wait here for slots */
 	struct net		*net;
 #if defined(CONFIG_PNFSD)
-	struct list_head	cl_layouts;	/* outstanding layouts */
 	struct list_head	cl_lo_states;	/* outstanding layout states */
 	struct list_head	cl_layoutrecalls; /* outstanding layoutrecall
 						     callbacks */
@@ -412,7 +411,6 @@ struct nfs4_file {
 	struct inode		*fi_inode;
 	bool			fi_had_conflict;
 #if defined(CONFIG_PNFSD)
-	struct list_head	fi_layouts;
 	struct list_head	fi_lo_states;
 	/* used by layoutget / layoutrecall */
 	struct nfs4_fsid	fi_fsid;
-- 
1.7.11.7

[PATCH 30/38] pnfsd-lexp: fixup recall_layout

[Benny Halevy]

Signed-off-by: Benny Halevy <bhalevy@tonian.com>
---
 fs/nfsd/pnfsd_lexp.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/fs/nfsd/pnfsd_lexp.c b/fs/nfsd/pnfsd_lexp.c
index 5d448c2..79a4bc1 100644
--- a/fs/nfsd/pnfsd_lexp.c
+++ b/fs/nfsd/pnfsd_lexp.c
@@ -253,7 +253,7 @@ static int get_stripe_unit(int blocksize)
 static bool
 has_layout(struct nfs4_file *fp)
 {
-	return !list_empty(&fp->fi_layouts);
+	return !list_empty(&fp->fi_lo_states);
 }
 
 /*
-- 
1.7.11.7

[PATCH 31/38] pnfsd: use the layout stateid for layout recalls

[Benny Halevy]

Implement for RETURN_FILE only at this stage.
A non-null stateid indicates a particular client to recall the layout from.
A null stateid indicates a recall from all clients holding a layout for the
specified file.

Signed-off-by: Benny Halevy <bhalevy@tonian.com>
---
 fs/nfsd/nfs4pnfsd.c             | 169 ++++++++++++++--------------------------
 fs/nfsd/pnfsd.h                 |   1 +
 fs/nfsd/state.h                 |  13 ++++
 include/linux/nfsd/nfsd4_pnfs.h |  21 +++++
 4 files changed, 95 insertions(+), 109 deletions(-)

diff --git a/fs/nfsd/nfs4pnfsd.c b/fs/nfsd/nfs4pnfsd.c
index d3a0bde..27f717f 100644
--- a/fs/nfsd/nfs4pnfsd.c
+++ b/fs/nfsd/nfs4pnfsd.c
@@ -434,7 +434,6 @@ struct super_block *
  */
 static struct nfs4_layoutrecall *
 alloc_init_layoutrecall(struct nfsd4_pnfs_cb_layout *cbl,
-			struct nfs4_client *clp,
 			struct nfs4_file *lrfile)
 {
 	struct nfs4_layoutrecall *clr;
@@ -449,7 +448,6 @@ struct super_block *
 	memset(clr, 0, sizeof(*clr));
 	if (lrfile)
 		get_nfs4_file(lrfile);
-	clr->clr_client = clp;
 	clr->clr_file = lrfile;
 	clr->cb = *cbl;
 
@@ -490,6 +488,9 @@ struct super_block *
 	return kref_put(&clr->clr_ref, destroy_layoutrecall);
 }
 
+/*
+ * Note: must be called under the layout lock
+ */
 void *
 layoutrecall_done(struct nfs4_layoutrecall *clr)
 {
@@ -1109,67 +1110,6 @@ int nfs4_pnfs_return_layout(struct svc_rqst *rqstp,
 	return status;
 }
 
-static bool
-cl_has_file_layout(struct nfs4_client *clp, struct nfs4_file *fp,
-		   stateid_t *lsid, struct nfsd4_pnfs_cb_layout *cbl)
-{
-	struct nfs4_layout *lo;
-	bool ret = false;
-
-	spin_lock(&layout_lock);
-	list_for_each_entry(lo, &fp->fi_layouts, lo_perfile) {
-		if (same_clid(&lo->lo_client->cl_clientid, &clp->cl_clientid) &&
-		    lo_seg_overlapping(&cbl->cbl_seg, &lo->lo_seg) &&
-		    (cbl->cbl_seg.iomode & lo->lo_seg.iomode))
-			goto found;
-	}
-	goto unlock;
-found:
-	/* Im going to send a recall on this latout update state */
-	update_layout_stateid_locked(lo->lo_state, lsid);
-	ret = true;
-unlock:
-	spin_unlock(&layout_lock);
-	return ret;
-}
-
-static int
-cl_has_fsid_layout(struct nfs4_client *clp, struct nfs4_fsid *fsid)
-{
-	int found = 0;
-	struct nfs4_layout *lp;
-
-	/* note: minor version unused */
-	spin_lock(&layout_lock);
-	list_for_each_entry(lp, &clp->cl_layouts, lo_perclnt)
-		if (lp->lo_file->fi_fsid.major == fsid->major) {
-			found = 1;
-			break;
-		}
-	spin_unlock(&layout_lock);
-	return found;
-}
-
-static int
-cl_has_any_layout(struct nfs4_client *clp)
-{
-	return !list_empty(&clp->cl_layouts);
-}
-
-static int
-cl_has_layout(struct nfs4_client *clp, struct nfsd4_pnfs_cb_layout *cbl,
-	      struct nfs4_file *lrfile, stateid_t *lsid)
-{
-	switch (cbl->cbl_recall_type) {
-	case RETURN_FILE:
-		return cl_has_file_layout(clp, lrfile, lsid, cbl);
-	case RETURN_FSID:
-		return cl_has_fsid_layout(clp, &cbl->cbl_fsid);
-	default:
-		return cl_has_any_layout(clp);
-	}
-}
-
 /*
  * Called without the layout_lock.
  */
@@ -1292,33 +1232,40 @@ struct create_recall_list_arg {
 };
 
 /*
- * look for matching layout for the given client
- * and add a pending layout recall to the todo list
- * if found any.
- * returns:
- *   0 if layouts found or negative error.
+ * Note: must be called under the layout lock
  */
-static int
-lo_recall_per_client(struct nfs4_client *clp, void *p)
+static struct nfs4_layout_state *
+should_recall_file_layout(struct nfsd4_pnfs_cb_layout *cbl,
+			  struct nfs4_file *fp)
 {
-	stateid_t lsid;
-	struct nfs4_layoutrecall *pending;
-	struct create_recall_list_arg *arg = p;
+	struct nfs4_layout_state *ls, *ret = NULL;
+	stateid_t *stid = (stateid_t *)&cbl->cbl_sid;
+	struct nfs4_layout *lo;
 
-	memset(&lsid, 0, sizeof(lsid));
-	if (!cl_has_layout(clp, arg->cbl, arg->lrfile, &lsid))
-		return 0;
+	dprintk("%s: ino=%lu clientid=%llux iomode=%u", __func__,
+		fp->fi_inode->i_ino, cbl->cbl_seg.clientid,
+		cbl->cbl_seg.iomode);
 
-	/* Matching put done by layoutreturn */
-	pending = alloc_init_layoutrecall(arg->cbl, clp, arg->lrfile);
-	/* out of memory, drain todo queue */
-	if (!pending)
-		return -ENOMEM;
+	list_for_each_entry (ls, &fp->fi_lo_states, ls_perfile) {
+		if (!is_null_stid(stid) &&
+		    !same_stid(stid, &ls->ls_stid.sc_stateid))
+			continue;
 
-	*(stateid_t *)&pending->cb.cbl_sid = lsid;
-	list_add(&pending->clr_perclnt, arg->todolist);
-	arg->todo_count++;
-	return 0;
+		if (cbl->cbl_seg.clientid &&
+		    !same_clid(&ls->ls_client->cl_clientid,
+			       (clientid_t *)&cbl->cbl_seg.clientid))
+			continue;
+
+		list_for_each_entry (lo, &ls->ls_layouts, lo_perstate)
+			if (cbl->cbl_seg.layout_type == lo->lo_seg.layout_type &&
+			    lo_seg_overlapping(&cbl->cbl_seg, &lo->lo_seg) &&
+			    (cbl->cbl_seg.iomode & lo->lo_seg.iomode)) {
+				ret = ls;
+				break;
+			}
+	}
+
+	return ret;
 }
 
 /* Create a layoutrecall structure for each client based on the
@@ -1328,37 +1275,41 @@ struct create_recall_list_arg {
 			  struct nfsd4_pnfs_cb_layout *cbl,
 			  struct nfs4_file *lrfile)
 {
-	struct nfs4_client *clp;
-	struct create_recall_list_arg arg = {
-		.cbl = cbl,
-		.lrfile = lrfile,
-		.todolist = todolist,
-	};
+	struct nfs4_layout_state *ls;
+	struct nfs4_layoutrecall *pending;
 	int status = 0;
 
-#if 0
 	dprintk("%s: -->\n", __func__);
 
-	/* If client given by fs, just do single client */
-	if (cbl->cbl_seg.clientid) {
-		clp = find_confirmed_client((clientid_t *)&cbl->cbl_seg.clientid, true);
-		if (!clp) {
-			status = -ENOENT;
-			dprintk("%s: clientid %llx not found\n", __func__,
-				(unsigned long long)cbl->cbl_seg.clientid);
-			goto out;
-		}
+	/* We do not support wildcard recalls yet */
+	if (cbl->cbl_recall_type != RETURN_FILE)
+		return -EOPNOTSUPP;
 
-		status = lo_recall_per_client(clp, &arg);
-	} else {
-		/* Check all clients for layout matches */
-		status = filter_confirmed_clients(lo_recall_per_client, &arg);
+	/* Matching put done by layoutreturn */
+	pending = alloc_init_layoutrecall(cbl, lrfile);
+	if (!pending)
+		return -ENOMEM;
+
+	switch (cbl->cbl_recall_type) {
+	case RETURN_FILE:
+		spin_lock(&layout_lock);
+		ls = should_recall_file_layout(cbl, lrfile);
+		if (ls) {
+			update_layout_stateid_locked(ls,
+					(stateid_t *)&pending->cb.cbl_sid);
+			pending->clr_client = ls->ls_client;
+			list_add(&pending->clr_perclnt, todolist);
+			(*todo_len)++;
+		}
+		spin_unlock(&layout_lock);
+		break;
+	case RETURN_FSID:
+	default:
+		WARN_ON(1);
+		return -EINVAL;	/* not supported yet */
 	}
 
-out:
-	*todo_len = arg.todo_count;
 	dprintk("%s: <-- list len %u status %d\n", __func__, *todo_len, status);
-#endif
 	return status;
 }
 
@@ -1380,7 +1331,7 @@ struct create_recall_list_arg {
 		pending = list_entry(todolist->next, struct nfs4_layoutrecall,
 				     clr_perclnt);
 
-		parent = alloc_init_layoutrecall(&pending->cb, NULL,
+		parent = alloc_init_layoutrecall(&pending->cb,
 						 pending->clr_file);
 		if (unlikely(!parent)) {
 			/* We want forward progress. If parent cannot be
diff --git a/fs/nfsd/pnfsd.h b/fs/nfsd/pnfsd.h
index 453f951..159fe94 100644
--- a/fs/nfsd/pnfsd.h
+++ b/fs/nfsd/pnfsd.h
@@ -49,6 +49,7 @@ struct nfs4_layout_state {
 	struct list_head	ls_perfile;
 	struct nfs4_file	*ls_file;
 	struct list_head	ls_layouts;
+	struct list_head	ls_lo_recalls;
 	bool			ls_roc;
 };
 
diff --git a/fs/nfsd/state.h b/fs/nfsd/state.h
index ab4a136..9057c86 100644
--- a/fs/nfsd/state.h
+++ b/fs/nfsd/state.h
@@ -62,6 +62,19 @@
 	stateid_opaque_t        si_opaque;
 } stateid_t;
 
+static inline bool
+is_null_stid(stateid_t *stid)
+{
+	return stid->si_opaque.so_id == 0;
+}
+
+static inline int
+same_stid(stateid_t *stid1, stateid_t *stid2)
+{
+	return !memcmp(&stid1->si_opaque, &stid2->si_opaque,
+		       NFS4_STATEID_OTHER_SIZE);
+}
+
 #define STATEID_FMT	"(%08x/%08x/%08x/%08x)"
 #define STATEID_VAL(s) \
 	(s)->si_opaque.so_clid.cl_boot, \
diff --git a/include/linux/nfsd/nfsd4_pnfs.h b/include/linux/nfsd/nfsd4_pnfs.h
index a49d5f6..a10bcf2 100644
--- a/include/linux/nfsd/nfsd4_pnfs.h
+++ b/include/linux/nfsd/nfsd4_pnfs.h
@@ -200,6 +200,27 @@ struct pnfs_export_operations {
 	int (*can_merge_layouts) (u32 layout_type);
 };
 
+/*
+ * @cbl_recall_type	Indicates RETURN_FILE, RETURN_FSID, or RETURN_ALL
+ *
+ * @cbl_seg		Indicates a the layout_type and iomode to recall,
+ * 			IOMODE_READ, IOMODE_RW, or IOMODE_ANY.
+ * 			For RETURN_FILE, offset and length can be given to recall
+ * 			a particular range.  To recall the layout for the whole
+ * 			file, offset is set to 0 and length to NFS4_MAX_UINT64.
+ *
+ * cbl_layoutchanged	Set to true to provide a hint to the client not to
+ * 			attempt flushing dirty data to the data servers
+ * 			using the recalled layout.
+ *
+ * cbl_sid		For RETURN_FILE, non-zero stateid indicates a particular
+ * 			stateid (file/clientid tuple) to recall.
+ *
+ * cbl_fsid		For RETURN_FSID, indicated the fsid to recall.
+ *
+ * cbl_cookie		A private file system value to be given on the final
+ *			layoutreturn completing the layout recall.
+ */
 struct nfsd4_pnfs_cb_layout {
 	u32			cbl_recall_type;	/* request */
 	struct nfsd4_layout_seg cbl_seg;		/* request */
-- 
1.7.11.7

[PATCH 32/38] pnfsd: cosmetic fixes

[Benny Halevy]

---
 fs/nfsd/nfs4pnfsd.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/fs/nfsd/nfs4pnfsd.c b/fs/nfsd/nfs4pnfsd.c
index 27f717f..7a5ba09 100644
--- a/fs/nfsd/nfs4pnfsd.c
+++ b/fs/nfsd/nfs4pnfsd.c
@@ -1268,8 +1268,10 @@ struct create_recall_list_arg {
 	return ret;
 }
 
-/* Create a layoutrecall structure for each client based on the
- * original structure. */
+/*
+ * Create a layoutrecall structure for each client based on the
+ * original structure.
+ */
 int
 create_layout_recall_list(struct list_head *todolist, unsigned *todo_len,
 			  struct nfsd4_pnfs_cb_layout *cbl,
@@ -1348,7 +1350,7 @@ struct create_recall_list_arg {
 				put_layoutrecall(pending);
 			}
 			--todo_len;
-				status = -ENOMEM;
+			status = -ENOMEM;
 		}
 	}
 
-- 
1.7.11.7

[PATCH 33/38] pnfsd: pass layout stateid down to fs on layoutget

[Benny Halevy]

rather than the clientid which is deprecated from the fs pnfs api

Signed-off-by: Benny Halevy <bhalevy@tonian.com>
---
 fs/nfsd/nfs4pnfsd.c             | 7 +++++--
 fs/nfsd/nfs4proc.c              | 3 +--
 fs/nfsd/xdr4.h                  | 1 +
 include/linux/nfsd/nfsd4_pnfs.h | 1 +
 4 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/fs/nfsd/nfs4pnfsd.c b/fs/nfsd/nfs4pnfsd.c
index 7a5ba09..1a65f85 100644
--- a/fs/nfsd/nfs4pnfsd.c
+++ b/fs/nfsd/nfs4pnfsd.c
@@ -691,7 +691,7 @@ struct super_block *
 
 	nfs4_lock_state();
 	fp = find_alloc_file(ino, lgp->lg_fhp);
-	clp = find_confirmed_client((clientid_t *)&lgp->lg_seg.clientid, true,
+	clp = find_confirmed_client(&lgp->lg_clientid, true,
 				    net_generic(SVC_NET(rqstp), nfsd_net_id));
 	dprintk("pNFS %s: fp %p clp %p\n", __func__, fp, clp);
 	if (!fp || !clp) {
@@ -718,10 +718,13 @@ struct super_block *
 		goto out_unlock;
 	}
 
+	memcpy(&args.lg_sid, &lgp->lg_sid, sizeof(args.lg_sid));
+
 	dprintk("pNFS %s: pre-export type 0x%x maxcount %Zd "
-		"iomode %u offset %llu length %llu\n",
+		"ls %p stateid " STATEID_FMT " iomode %u offset %llu length %llu\n",
 		__func__, lgp->lg_seg.layout_type,
 		exp_xdr_qbytes(xdr->end - xdr->p),
+		ls, STATEID_VAL(&lgp->lg_sid),
 		lgp->lg_seg.iomode, lgp->lg_seg.offset, lgp->lg_seg.length);
 
 	/* FIXME: need to eliminate the use of the state lock */
diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index 4fd5549..e962dc6 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -1193,9 +1193,8 @@ static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write)
 		goto out;
 	}
 
-	/* Set up arguments so layout can be retrieved at encode time */
 	lgp->lg_fhp = current_fh;
-	copy_clientid((clientid_t *)&lgp->lg_seg.clientid, cstate->session);
+	copy_clientid(&lgp->lg_clientid, cstate->session);
 	status = nfs_ok;
 out:
 	return status;
diff --git a/fs/nfsd/xdr4.h b/fs/nfsd/xdr4.h
index 459d8ba..1862b16 100644
--- a/fs/nfsd/xdr4.h
+++ b/fs/nfsd/xdr4.h
@@ -452,6 +452,7 @@ struct nfsd4_pnfs_layoutget {
 	stateid_t		lg_sid;		/* request/response */
 	struct nfsd4_layout_seg	lg_seg;		/* request/response */
 	u32			lg_roc;		/* response */
+	clientid_t		lg_clientid;
 };
 
 struct nfsd4_pnfs_layoutcommit {
diff --git a/include/linux/nfsd/nfsd4_pnfs.h b/include/linux/nfsd/nfsd4_pnfs.h
index a10bcf2..2f311ac 100644
--- a/include/linux/nfsd/nfsd4_pnfs.h
+++ b/include/linux/nfsd/nfsd4_pnfs.h
@@ -81,6 +81,7 @@ struct nfsd4_pnfs_layoutget_arg {
 	u64			lg_minlength;
 	u64			lg_sbid;
 	const struct knfsd_fh	*lg_fh;
+	nfs4_stateid		lg_sid;
 };
 
 struct nfsd4_pnfs_layoutget_res {
-- 
1.7.11.7

[PATCH 34/38] pnfsd: pass layout stateid down to fs on layoutreturn

[Benny Halevy]

rather than the clientid which is deprecated from the fs pnfs api

Signed-off-by: Benny Halevy <bhalevy@tonian.com>
---
 fs/nfsd/nfs4pnfsd.c             | 8 +++++---
 fs/nfsd/nfs4proc.c              | 2 +-
 fs/nfsd/nfs4xdr.c               | 4 ++--
 fs/nfsd/xdr4.h                  | 2 +-
 include/linux/nfsd/nfsd4_pnfs.h | 1 +
 5 files changed, 10 insertions(+), 7 deletions(-)

diff --git a/fs/nfsd/nfs4pnfsd.c b/fs/nfsd/nfs4pnfsd.c
index 1a65f85..0b256f5 100644
--- a/fs/nfsd/nfs4pnfsd.c
+++ b/fs/nfsd/nfs4pnfsd.c
@@ -927,7 +927,7 @@ struct super_block *
 			lrp->lrs_present = 1;
 	}
 	if (ls && layouts_found && lrp->lrs_present)
-		update_layout_stateid_locked(ls, &lrp->lr_sid);
+		update_layout_stateid_locked(ls, (stateid_t *)&lrp->args.lr_sid);
 	spin_unlock(&layout_lock);
 
 	return layouts_found;
@@ -1035,7 +1035,7 @@ int nfs4_pnfs_return_layout(struct svc_rqst *rqstp,
 	dprintk("NFSD: %s\n", __func__);
 
 	nfs4_lock_state();
-	clp = find_confirmed_client((clientid_t *)&lrp->args.lr_seg.clientid,
+	clp = find_confirmed_client(&lrp->lr_clientid,
 				    true, net_generic(SVC_NET(rqstp), nfsd_net_id));
 	if (!clp)
 		goto out;
@@ -1058,7 +1058,9 @@ int nfs4_pnfs_return_layout(struct svc_rqst *rqstp,
 
 		/* Check the stateid */
 		dprintk("%s PROCESS LO_STATEID inode %p\n", __func__, ino);
-		status = nfs4_process_layout_stateid(clp, fp, &lrp->lr_sid, &ls, false);
+		status = nfs4_process_layout_stateid(clp, fp,
+						     (stateid_t *)&lrp->args.lr_sid,
+						     &ls, false);
 		if (status)
 			goto out_put_file;
 
diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index e962dc6..cd52393 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -1324,7 +1324,7 @@ static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write)
 	}
 
 	/* Set clientid from sessionid */
-	copy_clientid((clientid_t *)&lrp->args.lr_seg.clientid, cstate->session);
+	copy_clientid(&lrp->lr_clientid, cstate->session);
 	lrp->lrs_present = 0;
 	status = nfs4_pnfs_return_layout(rqstp, sb, current_fh, lrp);
 out:
diff --git a/fs/nfsd/nfs4xdr.c b/fs/nfsd/nfs4xdr.c
index 992e425..070e25a 100644
--- a/fs/nfsd/nfs4xdr.c
+++ b/fs/nfsd/nfs4xdr.c
@@ -1560,7 +1560,7 @@ static __be32 nfsd4_decode_reclaim_complete(struct nfsd4_compoundargs *argp, str
 		READ_BUF(16);
 		READ64(lrp->args.lr_seg.offset);
 		READ64(lrp->args.lr_seg.length);
-		nfsd4_decode_stateid(argp, &lrp->lr_sid);
+		nfsd4_decode_stateid(argp, (stateid_t *)&lrp->args.lr_sid);
 		READ_BUF(4);
 		READ32(lrp->args.lrf_body_len);
 		if (lrp->args.lrf_body_len > 0) {
@@ -3925,7 +3925,7 @@ static __be32 nfsd4_encode_bind_conn_to_session(struct nfsd4_compoundres *resp,
 	WRITE32(lrp->lrs_present != 0);    /* got stateid? */
 	ADJUST_ARGS();
 	if (lrp->lrs_present)
-		nfsd4_encode_stateid(resp, &lrp->lr_sid);
+		nfsd4_encode_stateid(resp, (stateid_t *)&lrp->args.lr_sid);
 out:
 	return nfserr;
 }
diff --git a/fs/nfsd/xdr4.h b/fs/nfsd/xdr4.h
index 1862b16..802a151 100644
--- a/fs/nfsd/xdr4.h
+++ b/fs/nfsd/xdr4.h
@@ -463,7 +463,7 @@ struct nfsd4_pnfs_layoutcommit {
 
 struct nfsd4_pnfs_layoutreturn {
 	struct nfsd4_pnfs_layoutreturn_arg args;
-	stateid_t		lr_sid;		/* request/resopnse */
+	clientid_t		lr_clientid;	/* request */
 	u32			lrs_present;	/* response */
 };
 
diff --git a/include/linux/nfsd/nfsd4_pnfs.h b/include/linux/nfsd/nfsd4_pnfs.h
index 2f311ac..2e09065 100644
--- a/include/linux/nfsd/nfsd4_pnfs.h
+++ b/include/linux/nfsd/nfsd4_pnfs.h
@@ -117,6 +117,7 @@ struct nfsd4_pnfs_layoutreturn_arg {
 	u32			lr_reclaim;	/* request */
 	u32			lrf_body_len;	/* request */
 	void			*lrf_body;	/* request */
+	nfs4_stateid		lr_sid;		/* request/response */
 	void			*lr_cookie;	/* fs private */
 	enum layoutreturn_flags	lr_flags;	/* internal flags */
 };
-- 
1.7.11.7

[PATCH 35/38] pnfsd: use layout stateid for is_layout_recalled

[Benny Halevy]

rather than clientid

Signed-off-by: Benny Halevy <bhalevy@tonian.com>
---
 fs/nfsd/nfs4pnfsd.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/fs/nfsd/nfs4pnfsd.c b/fs/nfsd/nfs4pnfsd.c
index 0b256f5..0ef351d 100644
--- a/fs/nfsd/nfs4pnfsd.c
+++ b/fs/nfsd/nfs4pnfsd.c
@@ -548,10 +548,10 @@ struct super_block *
  */
 static int
 is_layout_recalled(struct nfs4_client *clp,
-		   struct svc_fh *current_fh,
-		   struct nfsd4_layout_seg *seg)
+		   struct nfsd4_pnfs_layoutget *lgp)
 {
 	struct nfs4_layoutrecall *clr;
+	struct nfsd4_layout_seg *seg = &lgp->lg_seg;
 
 	spin_lock(&layout_lock);
 	list_for_each_entry (clr, &clp->cl_layoutrecalls, clr_perclnt) {
@@ -560,13 +560,13 @@ struct super_block *
 		if (clr->cb.cbl_recall_type == RETURN_ALL)
 			goto found;
 		if (clr->cb.cbl_recall_type == RETURN_FSID) {
-			if (same_fsid(&clr->cb.cbl_fsid, current_fh))
+			if (same_fsid(&clr->cb.cbl_fsid, lgp->lg_fhp))
 				goto found;
 			else
 				continue;
 		}
 		BUG_ON(clr->cb.cbl_recall_type != RETURN_FILE);
-		if (clr->cb.cbl_seg.clientid == seg->clientid &&
+		if (same_stid((stateid_t *)&clr->cb.cbl_sid, &lgp->lg_sid) &&
 		    lo_seg_overlapping(&clr->cb.cbl_seg, seg))
 			goto found;
 	}
@@ -704,7 +704,7 @@ struct super_block *
 	if (nfserr)
 		goto out_unlock;
 
-	if (is_layout_recalled(clp, lgp->lg_fhp, &lgp->lg_seg)) {
+	if (is_layout_recalled(clp, lgp)) {
 		nfserr = nfserr_recallconflict;
 		goto out_unlock;
 	}
-- 
1.7.11.7

[PATCH 36/38] pnfsd: deprecate cbl_seg.clientid

[Benny Halevy]

Use only cbl_sid for identification

Signed-off-by: Benny Halevy <bhalevy@tonian.com>
---
 fs/nfsd/nfs4pnfsd.c | 9 ++-------
 1 file changed, 2 insertions(+), 7 deletions(-)

diff --git a/fs/nfsd/nfs4pnfsd.c b/fs/nfsd/nfs4pnfsd.c
index 0ef351d..45111fe 100644
--- a/fs/nfsd/nfs4pnfsd.c
+++ b/fs/nfsd/nfs4pnfsd.c
@@ -1247,8 +1247,8 @@ struct create_recall_list_arg {
 	stateid_t *stid = (stateid_t *)&cbl->cbl_sid;
 	struct nfs4_layout *lo;
 
-	dprintk("%s: ino=%lu clientid=%llux iomode=%u", __func__,
-		fp->fi_inode->i_ino, cbl->cbl_seg.clientid,
+	dprintk("%s: ino=%lu stateid=" STATEID_FMT " iomode=%u", __func__,
+		fp->fi_inode->i_ino, STATEID_VAL(stid),
 		cbl->cbl_seg.iomode);
 
 	list_for_each_entry (ls, &fp->fi_lo_states, ls_perfile) {
@@ -1256,11 +1256,6 @@ struct create_recall_list_arg {
 		    !same_stid(stid, &ls->ls_stid.sc_stateid))
 			continue;
 
-		if (cbl->cbl_seg.clientid &&
-		    !same_clid(&ls->ls_client->cl_clientid,
-			       (clientid_t *)&cbl->cbl_seg.clientid))
-			continue;
-
 		list_for_each_entry (lo, &ls->ls_layouts, lo_perstate)
 			if (cbl->cbl_seg.layout_type == lo->lo_seg.layout_type &&
 			    lo_seg_overlapping(&cbl->cbl_seg, &lo->lo_seg) &&
-- 
1.7.11.7

[PATCH 37/38] pnfsd: don't use clientid in merge_layout

[Benny Halevy]

The layout state is already associated with the clientid
so there is no need to recheck

Signed-off-by: Benny Halevy <bhalevy@tonian.com>
---
 fs/nfsd/nfs4pnfsd.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/fs/nfsd/nfs4pnfsd.c b/fs/nfsd/nfs4pnfsd.c
index 45111fe..0c05a22 100644
--- a/fs/nfsd/nfs4pnfsd.c
+++ b/fs/nfsd/nfs4pnfsd.c
@@ -626,7 +626,6 @@ struct super_block *
 	spin_lock(&layout_lock);
 	list_for_each_entry (lp, &ls->ls_layouts, lo_perstate)
 		if (lp->lo_seg.layout_type == seg->layout_type &&
-		    lp->lo_seg.clientid == seg->clientid &&
 		    lp->lo_seg.iomode == seg->iomode &&
 		    (ret = lo_seg_mergeable(&lp->lo_seg, seg))) {
 			extend_layout(&lp->lo_seg, seg);
-- 
1.7.11.7

[PATCH 38/38] pnfsd: remove clientid member from struct nfsd4_layout_seg

[Benny Halevy]

Use layout stateid for identification instead.

Signed-off-by: Benny Halevy <bhalevy@tonian.com>
---
 fs/nfsd/nfs4proc.c              | 2 --
 include/linux/nfsd/nfsd4_pnfs.h | 1 -
 2 files changed, 3 deletions(-)

diff --git a/fs/nfsd/nfs4proc.c b/fs/nfsd/nfs4proc.c
index cd52393..5bbd1a3 100644
--- a/fs/nfsd/nfs4proc.c
+++ b/fs/nfsd/nfs4proc.c
@@ -1243,8 +1243,6 @@ static int fill_in_write_vector(struct kvec *vec, struct nfsd4_write *write)
 		__func__, lcp->args.lc_newoffset, lcp->args.lc_last_wr + 1,
 		ino->i_size);
 
-	/* Set clientid from sessionid */
-	copy_clientid((clientid_t *)&lcp->args.lc_seg.clientid, cstate->session);
 	lcp->res.lc_size_chg = 0;
 	if (sb->s_pnfs_op->layout_commit) {
 		status = sb->s_pnfs_op->layout_commit(ino, &lcp->args, &lcp->res);
diff --git a/include/linux/nfsd/nfsd4_pnfs.h b/include/linux/nfsd/nfsd4_pnfs.h
index 2e09065..fd7bad9 100644
--- a/include/linux/nfsd/nfsd4_pnfs.h
+++ b/include/linux/nfsd/nfsd4_pnfs.h
@@ -59,7 +59,6 @@ struct nfsd4_pnfs_dev_iter_res {
 };
 
 struct nfsd4_layout_seg {
-	u64	clientid;
 	u32	layout_type;
 	u32	iomode;
 	u64	offset;
-- 
1.7.11.7