[PATCHSET 0/8] pnfsd-exofs: New set of exofs pnfs export patches

[PATCHSET 0/8] pnfsd-exofs: New set of exofs pnfs export patches

[Boaz Harrosh]

Benny hi

I'm submitting a new SQUASHED set of exofs patches. These should completely
replace the patches you have now in the pnfsd-exofs{-all} branch

If anyone is curios I'll send a SQUASHME patchset as reply to this mail.

I Think the pnfsd-exofs branch should advance down to before the pnfsd-exp-all.
It should be the first or second candidate for inclusion.

BTW: You forgot to branch the pnfs-blocks branch last time.

I think that now, since most of the client code is in-tree and all the common code
is in. It would be easier to work with a tree that has two separate branches for the
1.block-layout and 2.server branches. And the pnfs-all-latest which merges the two.

something like:


                |----pnfsd(including gfs2_---pnfsd-exofs---pnfsd-exp----pnfsd-spnfs----pnfsd-spnfs-blocks ---|
nfs-linux-next--|                                                                                            + --- pnfs-all-latest
                | --------pnfs-blocks------------------------------------------------------------------------|

* "+" means a merge

It will be much easier to develop this way

usual list of patches:

[PATCH 1/8] nfs_xdr: Move nfs4_string definition out of #ifdef CONFIG_NFS_V4
[PATCH 2/8] exofs: pnfs-tree: Remove pnfs-osd private definitions
[PATCH 3/8] pnfs-obj: pnfs_osd_xdr: Remove dead code and cleanup
 
    Intended for upstream

[PATCH 4/8] pnfsd-exofs: pnfs objects server XDR functions
[PATCH 5/8] pnfsd-exofs: Add pNFS export support
[PATCH 6/8] pnfsd-exofs: get_device_info implementation.
[PATCH 7/8] pnfsd-exofs: Recall layouts on truncate
[PATCH 8/8] pnfsd-exofs: layoutreturn pnfs-obj information decoding

   New exofs pnfs export patches

Thanks
Boaz

Re: [PATCHSET 0/8] pnfsd-exofs: New set of exofs pnfs export patches

[Benny Halevy]

On 2011-06-14 16:53, Boaz Harrosh wrote:
> 
> Benny hi
> 
> I'm submitting a new SQUASHED set of exofs patches. These should completely
> replace the patches you have now in the pnfsd-exofs{-all} branch
> 

Great, thanks!

> If anyone is curios I'll send a SQUASHME patchset as reply to this mail.
> 
> I Think the pnfsd-exofs branch should advance down to before the pnfsd-exp-all.
> It should be the first or second candidate for inclusion.

makes sense

> 
> BTW: You forgot to branch the pnfs-blocks branch last time.

Already fixed... (pnfs-block is there)

> 
> I think that now, since most of the client code is in-tree and all the common code
> is in. It would be easier to work with a tree that has two separate branches for the
> 1.block-layout and 2.server branches. And the pnfs-all-latest which merges the two.
> 
> something like:
> 
> 
>                 |----pnfsd(including gfs2_---pnfsd-exofs---pnfsd-exp----pnfsd-spnfs----pnfsd-spnfs-blocks ---|
> nfs-linux-next--|                                                                                            + --- pnfs-all-latest
>                 | --------pnfs-blocks------------------------------------------------------------------------|

Good point, I'll try it out.

> 
> * "+" means a merge
> 
> It will be much easier to develop this way
> 
> usual list of patches:
> 
> [PATCH 1/8] nfs_xdr: Move nfs4_string definition out of #ifdef CONFIG_NFS_V4
> [PATCH 2/8] exofs: pnfs-tree: Remove pnfs-osd private definitions
> [PATCH 3/8] pnfs-obj: pnfs_osd_xdr: Remove dead code and cleanup
>  
>     Intended for upstream
> 
> [PATCH 4/8] pnfsd-exofs: pnfs objects server XDR functions
> [PATCH 5/8] pnfsd-exofs: Add pNFS export support
> [PATCH 6/8] pnfsd-exofs: get_device_info implementation.
> [PATCH 7/8] pnfsd-exofs: Recall layouts on truncate
> [PATCH 8/8] pnfsd-exofs: layoutreturn pnfs-obj information decoding
> 
>    New exofs pnfs export patches

Where are the patches? :)

Benny

> 
> Thanks
> Boaz
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/8] nfs_xdr: Move nfs4_string definition out of #ifdef CONFIG_NFS_V4

[Boaz Harrosh]

exofs file system wants to use pnfs_osd_xdr.h file instead of
redefining pnfs-objects types in it's private "pnfs.h" headr.

Before we do the switch we must make sure pnfs_osd_xdr.h is
compilable also under NFS versions smaller than 4.1. Since now
it is needed regardless of version, by the exofs code.

nfs4_string is not the only nfs4 type out in the global scope.

Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
---
 include/linux/nfs_xdr.h |   10 +++++-----
 1 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h
index 0a21844..7372921 100644
--- a/include/linux/nfs_xdr.h
+++ b/include/linux/nfs_xdr.h
@@ -774,6 +774,11 @@ struct nfs3_getaclres {
 	struct posix_acl *	acl_default;
 };
 
+struct nfs4_string {
+	unsigned int len;
+	char *data;
+};
+
 #ifdef CONFIG_NFS_V4
 
 typedef u64 clientid4;
@@ -964,11 +969,6 @@ struct nfs4_server_caps_res {
 	struct nfs4_sequence_res	seq_res;
 };
 
-struct nfs4_string {
-	unsigned int len;
-	char *data;
-};
-
 #define NFS4_PATHNAME_MAXCOMPONENTS 512
 struct nfs4_pathname {
 	unsigned int ncomponents;
-- 
1.7.3.4

[PATCH 2/8] exofs: pnfs-tree: Remove pnfs-osd private definitions

[Boaz Harrosh]

Now that pnfs-osd has hit mainline we can remove exofs's
private header. (And the FIXME comment)

Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
---
 fs/exofs/exofs.h |    6 +-----
 fs/exofs/pnfs.h  |   45 ---------------------------------------------
 2 files changed, 1 insertions(+), 50 deletions(-)
 delete mode 100644 fs/exofs/pnfs.h

diff --git a/fs/exofs/exofs.h b/fs/exofs/exofs.h
index c965806..e103dbd 100644
--- a/fs/exofs/exofs.h
+++ b/fs/exofs/exofs.h
@@ -36,13 +36,9 @@
 #include <linux/fs.h>
 #include <linux/time.h>
 #include <linux/backing-dev.h>
+#include <linux/pnfs_osd_xdr.h>
 #include "common.h"
 
-/* FIXME: Remove once pnfs hits mainline
- * #include <linux/exportfs/pnfs_osd_xdr.h>
- */
-#include "pnfs.h"
-
 #define EXOFS_ERR(fmt, a...) printk(KERN_ERR "exofs: " fmt, ##a)
 
 #ifdef CONFIG_EXOFS_DEBUG
diff --git a/fs/exofs/pnfs.h b/fs/exofs/pnfs.h
deleted file mode 100644
index c52e988..0000000
--- a/fs/exofs/pnfs.h
+++ /dev/null
@@ -1,45 +0,0 @@
-/*
- * Copyright (C) 2008, 2009
- * Boaz Harrosh <bharrosh@panasas.com>
- *
- * This file is part of exofs.
- *
- * exofs is free software; you can redistribute it and/or modify it under the
- * terms of the GNU General Public License  version 2 as published by the Free
- * Software Foundation.
- *
- */
-
-/* FIXME: Remove this file once pnfs hits mainline */
-
-#ifndef __EXOFS_PNFS_H__
-#define __EXOFS_PNFS_H__
-
-#if ! defined(__PNFS_OSD_XDR_H__)
-
-enum pnfs_iomode {
-	IOMODE_READ = 1,
-	IOMODE_RW = 2,
-	IOMODE_ANY = 3,
-};
-
-/* Layout Structure */
-enum pnfs_osd_raid_algorithm4 {
-	PNFS_OSD_RAID_0		= 1,
-	PNFS_OSD_RAID_4		= 2,
-	PNFS_OSD_RAID_5		= 3,
-	PNFS_OSD_RAID_PQ	= 4     /* Reed-Solomon P+Q */
-};
-
-struct pnfs_osd_data_map {
-	u32	odm_num_comps;
-	u64	odm_stripe_unit;
-	u32	odm_group_width;
-	u32	odm_group_depth;
-	u32	odm_mirror_cnt;
-	u32	odm_raid_algorithm;
-};
-
-#endif /* ! defined(__PNFS_OSD_XDR_H__) */
-
-#endif /* __EXOFS_PNFS_H__ */
-- 
1.7.3.4

[PATCH 4/8] pnfsd-exofs: pnfs objects server XDR functions

[Boaz Harrosh]

* Server support for encoding a pnfs_osd_layout4 XDR stream.

* Implementation of pnfs_osd_xdr_encode_deviceaddr().
  Which takes a pnfs_osd_deviceaddr structure and encodes
  it onto an XDR stream.

* Implementation of pnfs_osd_xdr_decode_ioerr() used by
  servers in layout_return notifications.

* Implementation of pnfs_osd_xdr_decode_layoutupdate()
  used by servers in layout_commit notifications.

* Add pnfs_osd_xdr_srv.c to fs/exportfs/Makefile conditional
  on CONFIG_EXPORTFS_OSD_LAYOUT selected

* Add CONFIG_EXPORTFS_OSD_LAYOUT to fs/Kconfig. It is
  prompt-less needs to be selected by osd filesystems

Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>

TODO: We can now remove the olo_comps pointer from
      struct pnfs_osd_layout neither the server nor client use
      it any more. (rename it to pnfs_osd_layout_hdr)

Signed-off-by: Boaz Harrosh <Boaz Harrosh bharrosh@panasas.com>
---
 fs/Kconfig                            |    8 +
 fs/exportfs/Makefile                  |    1 +
 fs/exportfs/pnfs_osd_xdr_srv.c        |  287 +++++++++++++++++++++++++++++++++
 include/linux/nfsd/pnfs_osd_xdr_srv.h |   40 +++++
 4 files changed, 336 insertions(+), 0 deletions(-)
 create mode 100644 fs/exportfs/pnfs_osd_xdr_srv.c
 create mode 100644 include/linux/nfsd/pnfs_osd_xdr_srv.h

diff --git a/fs/Kconfig b/fs/Kconfig
index 320670e..4980006 100644
--- a/fs/Kconfig
+++ b/fs/Kconfig
@@ -56,6 +56,14 @@ config EXPORTFS_FILE_LAYOUT
 	  Exportfs support for the NFSv4.1 files layout type.
 	  Must be automatically selected by supporting filesystems.
 
+config EXPORTFS_OSD_LAYOUT
+	bool
+	depends on PNFSD && EXPORTFS
+	help
+	  Exportfs support for the NFSv4.1 objects layout type.
+	  Must be automatically selected by supporting osd
+	  filesystems.
+
 config FILE_LOCKING
 	bool "Enable POSIX file locking API" if EXPERT
 	default y
diff --git a/fs/exportfs/Makefile b/fs/exportfs/Makefile
index 658207d..f6ea13f 100644
--- a/fs/exportfs/Makefile
+++ b/fs/exportfs/Makefile
@@ -5,3 +5,4 @@ obj-$(CONFIG_EXPORTFS) += exportfs.o
 
 exportfs-y				:= expfs.o
 exportfs-$(CONFIG_EXPORTFS_FILE_LAYOUT)	+= nfs4filelayoutxdr.o
+exportfs-$(CONFIG_EXPORTFS_OSD_LAYOUT)	+= pnfs_osd_xdr_srv.o
diff --git a/fs/exportfs/pnfs_osd_xdr_srv.c b/fs/exportfs/pnfs_osd_xdr_srv.c
new file mode 100644
index 0000000..12a3bda
--- /dev/null
+++ b/fs/exportfs/pnfs_osd_xdr_srv.c
@@ -0,0 +1,287 @@
+/*
+ *  Object-Based pNFS Layout XDR layer for the Server side
+ *
+ *  Copyright (C) 2007 and on Panasas Inc.
+ *  All rights reserved.
+ *
+ *  Benny Halevy <bhalevy@panasas.com>
+ *  Boaz Harrosh <bharrosh@panasas.com>
+ *
+ *  This program is free software; you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License version 2
+ *  See the file COPYING included with this distribution for more details.
+ *
+ *  Redistribution and use in source and binary forms, with or without
+ *  modification, are permitted provided that the following conditions
+ *  are met:
+ *
+ *  1. Redistributions of source code must retain the above copyright
+ *     notice, this list of conditions and the following disclaimer.
+ *  2. Redistributions in binary form must reproduce the above copyright
+ *     notice, this list of conditions and the following disclaimer in the
+ *     documentation and/or other materials provided with the distribution.
+ *  3. Neither the name of the Panasas company nor the names of its
+ *     contributors may be used to endorse or promote products derived
+ *     from this software without specific prior written permission.
+ *
+ *  THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESS OR IMPLIED
+ *  WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+ *  MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ *  DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
+ *  FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ *  CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ *  SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
+ *  BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+ *  LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
+ *  NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+ *  SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <linux/nfsd/nfsd4_pnfs.h>
+
+#include "linux/nfsd/pnfs_osd_xdr_srv.h"
+
+/*
+ * struct pnfs_osd_data_map {
+ *	u32	odm_num_comps;
+ *	u64	odm_stripe_unit;
+ *	u32	odm_group_width;
+ *	u32	odm_group_depth;
+ *	u32	odm_mirror_cnt;
+ *	u32	odm_raid_algorithm;
+ * };
+ */
+static enum nfsstat4 pnfs_osd_xdr_encode_data_map(
+	struct exp_xdr_stream *xdr,
+	struct pnfs_osd_data_map *data_map)
+{
+	__be32 *p = exp_xdr_reserve_qwords(xdr, 1+2+1+1+1+1);
+
+	if (!p)
+		return NFS4ERR_TOOSMALL;
+
+	p = exp_xdr_encode_u32(p, data_map->odm_num_comps);
+	p = exp_xdr_encode_u64(p, data_map->odm_stripe_unit);
+	p = exp_xdr_encode_u32(p, data_map->odm_group_width);
+	p = exp_xdr_encode_u32(p, data_map->odm_group_depth);
+	p = exp_xdr_encode_u32(p, data_map->odm_mirror_cnt);
+	p = exp_xdr_encode_u32(p, data_map->odm_raid_algorithm);
+
+	return 0;
+}
+
+/*
+ * struct pnfs_osd_objid {
+ *	struct nfs4_deviceid	oid_device_id;
+ *	u64			oid_partition_id;
+ *	u64			oid_object_id;
+ * };
+ */
+static inline enum nfsstat4 pnfs_osd_xdr_encode_objid(
+	struct exp_xdr_stream *xdr,
+	struct pnfs_osd_objid *object_id)
+{
+	__be32 *p = exp_xdr_reserve_qwords(xdr, 2+2+2+2);
+	struct nfsd4_pnfs_deviceid *dev_id =
+		(struct nfsd4_pnfs_deviceid *)&object_id->oid_device_id;
+
+	if (!p)
+		return NFS4ERR_TOOSMALL;
+
+	p = exp_xdr_encode_u64(p, dev_id->sbid);
+	p = exp_xdr_encode_u64(p, dev_id->devid);
+	p = exp_xdr_encode_u64(p, object_id->oid_partition_id);
+	p = exp_xdr_encode_u64(p, object_id->oid_object_id);
+
+	return 0;
+}
+
+/*
+ * enum pnfs_osd_cap_key_sec4 {
+ *	PNFS_OSD_CAP_KEY_SEC_NONE = 0,
+ *	PNFS_OSD_CAP_KEY_SEC_SSV  = 1
+ * };
+ *
+ * struct pnfs_osd_object_cred {
+ *	struct pnfs_osd_objid		oc_object_id;
+ *	u32				oc_osd_version;
+ *	u32				oc_cap_key_sec;
+ *	struct pnfs_osd_opaque_cred	oc_cap_key
+ *	struct pnfs_osd_opaque_cred	oc_cap;
+ * };
+ */
+enum nfsstat4 pnfs_osd_xdr_encode_layout_cred(
+	struct exp_xdr_stream *xdr,
+	struct pnfs_osd_object_cred *olo_comp)
+{
+	__be32 *p;
+	enum nfsstat4 err;
+
+	err = pnfs_osd_xdr_encode_objid(xdr, &olo_comp->oc_object_id);
+	if (err)
+		return err;
+
+	p = exp_xdr_reserve_space(xdr, 3*4 + 4+olo_comp->oc_cap.cred_len);
+	if (!p)
+		return NFS4ERR_TOOSMALL;
+
+	p = exp_xdr_encode_u32(p, olo_comp->oc_osd_version);
+
+	/* No sec for now */
+	p = exp_xdr_encode_u32(p, PNFS_OSD_CAP_KEY_SEC_NONE);
+	p = exp_xdr_encode_u32(p, 0); /* opaque oc_capability_key<> */
+
+	exp_xdr_encode_opaque(p, olo_comp->oc_cap.cred,
+			      olo_comp->oc_cap.cred_len);
+
+	return 0;
+}
+EXPORT_SYMBOL(pnfs_osd_xdr_encode_layout_cred);
+
+/*
+ * struct pnfs_osd_layout {
+ *	struct pnfs_osd_data_map	olo_map;
+ *	u32				olo_comps_index;
+ *	u32				olo_num_comps;
+ *	struct pnfs_osd_object_cred	*olo_comps;
+ * };
+ */
+enum nfsstat4 pnfs_osd_xdr_encode_layout_hdr(
+	struct exp_xdr_stream *xdr,
+	struct pnfs_osd_layout *pol)
+{
+	__be32 *p;
+	enum nfsstat4 err;
+
+	err = pnfs_osd_xdr_encode_data_map(xdr, &pol->olo_map);
+	if (err)
+		return err;
+
+	p = exp_xdr_reserve_qwords(xdr, 2);
+	if (!p)
+		return NFS4ERR_TOOSMALL;
+
+	p = exp_xdr_encode_u32(p, pol->olo_comps_index);
+	p = exp_xdr_encode_u32(p, pol->olo_num_comps);
+
+	return 0;
+}
+EXPORT_SYMBOL(pnfs_osd_xdr_encode_layout_hdr);
+
+static enum nfsstat4 _encode_string(struct exp_xdr_stream *xdr,
+			  const struct nfs4_string *str)
+{
+	__be32 *p = exp_xdr_reserve_space(xdr, 4 + str->len);
+
+	if (!p)
+		return NFS4ERR_TOOSMALL;
+	exp_xdr_encode_opaque(p, str->data, str->len);
+	return 0;
+}
+
+/* struct pnfs_osd_deviceaddr {
+ *	struct pnfs_osd_targetid	oda_targetid;
+ *	struct pnfs_osd_targetaddr	oda_targetaddr;
+ *	u8				oda_lun[8];
+ *	struct nfs4_string		oda_systemid;
+ *	struct pnfs_osd_object_cred	oda_root_obj_cred;
+ *	struct nfs4_string		oda_osdname;
+ * };
+ */
+enum nfsstat4 pnfs_osd_xdr_encode_deviceaddr(
+	struct exp_xdr_stream *xdr, struct pnfs_osd_deviceaddr *devaddr)
+{
+	__be32 *p;
+	enum nfsstat4 err;
+
+	p = exp_xdr_reserve_space(xdr, 4 + 4 + sizeof(devaddr->oda_lun));
+	if (!p)
+		return NFS4ERR_TOOSMALL;
+
+	/* Empty oda_targetid */
+	p = exp_xdr_encode_u32(p, OBJ_TARGET_ANON);
+
+	/* Empty oda_targetaddr for now */
+	p = exp_xdr_encode_u32(p, 0);
+
+	/* oda_lun */
+	exp_xdr_encode_bytes(p, devaddr->oda_lun, sizeof(devaddr->oda_lun));
+
+	err = _encode_string(xdr, &devaddr->oda_systemid);
+	if (err)
+		return err;
+
+	err = pnfs_osd_xdr_encode_layout_cred(xdr,
+					      &devaddr->oda_root_obj_cred);
+	if (err)
+		return err;
+
+	err = _encode_string(xdr, &devaddr->oda_osdname);
+	if (err)
+		return err;
+
+	return 0;
+}
+EXPORT_SYMBOL(pnfs_osd_xdr_encode_deviceaddr);
+
+/*
+ * struct pnfs_osd_layoutupdate {
+ *	u32	dsu_valid;
+ *	s64	dsu_delta;
+ *	u32	olu_ioerr_flag;
+ * };
+ */
+__be32 *
+pnfs_osd_xdr_decode_layoutupdate(struct pnfs_osd_layoutupdate *lou, __be32 *p)
+{
+	lou->dsu_valid = be32_to_cpu(*p++);
+	if (lou->dsu_valid)
+		p = xdr_decode_hyper(p, &lou->dsu_delta);
+	lou->olu_ioerr_flag = be32_to_cpu(*p++);
+	return p;
+}
+EXPORT_SYMBOL(pnfs_osd_xdr_decode_layoutupdate);
+
+/*
+ * struct pnfs_osd_objid {
+ *	struct nfs4_deviceid	oid_device_id;
+ *	u64			oid_partition_id;
+ *	u64			oid_object_id;
+ * }; xdr size 32
+ */
+static inline __be32 *
+pnfs_osd_xdr_decode_objid(__be32 *p, struct pnfs_osd_objid *objid)
+{
+	/* FIXME: p = xdr_decode_fixed(...) */
+	memcpy(objid->oid_device_id.data, p, sizeof(objid->oid_device_id.data));
+	p += XDR_QUADLEN(sizeof(objid->oid_device_id.data));
+
+	p = xdr_decode_hyper(p, &objid->oid_partition_id);
+	p = xdr_decode_hyper(p, &objid->oid_object_id);
+	return p;
+}
+
+/*
+ * struct pnfs_osd_ioerr {
+ *	struct pnfs_osd_objid	oer_component;
+ *	u64			oer_comp_offset;
+ *	u64			oer_comp_length;
+ *	u32			oer_iswrite;
+ *	u32			oer_errno;
+ * }; xdr size 32 + 24
+ */
+bool pnfs_osd_xdr_decode_ioerr(struct pnfs_osd_ioerr *ioerr,
+			       struct exp_xdr_stream *xdr)
+{
+	__be32 *p = exp_xdr_reserve_space(xdr, 32 + 24);
+	if (!p)
+		return false;
+
+	p = pnfs_osd_xdr_decode_objid(p, &ioerr->oer_component);
+	p = xdr_decode_hyper(p, &ioerr->oer_comp_offset);
+	p = xdr_decode_hyper(p, &ioerr->oer_comp_length);
+	ioerr->oer_iswrite = be32_to_cpu(*p++);
+	ioerr->oer_errno = be32_to_cpu(*p);
+	return true;
+}
+EXPORT_SYMBOL(pnfs_osd_xdr_decode_ioerr);
diff --git a/include/linux/nfsd/pnfs_osd_xdr_srv.h b/include/linux/nfsd/pnfs_osd_xdr_srv.h
new file mode 100644
index 0000000..fab244e
--- /dev/null
+++ b/include/linux/nfsd/pnfs_osd_xdr_srv.h
@@ -0,0 +1,40 @@
+/*
+ * pnfs-objects Server XDR definitions and API
+ *
+ * Copyright (C) from 2011 Panasas Inc.  All rights reserved.
+ *
+ * Authors:
+ *   Boaz Harrosh <bharrosh@panasas.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2
+ *
+ */
+#ifndef __PNFS_OSD_XDR_SRV_H__
+#define __PNFS_OSD_XDR_SRV_H__
+
+#include <linux/pnfs_osd_xdr.h>
+#include <linux/exp_xdr.h>
+
+/* Layout encoding */
+enum nfsstat4 pnfs_osd_xdr_encode_layout_hdr(
+	struct exp_xdr_stream *xdr,
+	struct pnfs_osd_layout *layout);
+
+enum nfsstat4 pnfs_osd_xdr_encode_layout_cred(
+	struct exp_xdr_stream *xdr,
+	struct pnfs_osd_object_cred *cred);
+
+/* deviceaddr encoding */
+enum nfsstat4 pnfs_osd_xdr_encode_deviceaddr(
+	struct exp_xdr_stream *xdr, struct pnfs_osd_deviceaddr *devaddr);
+
+/* layout_commit decoding */
+__be32 *pnfs_osd_xdr_decode_layoutupdate(
+	struct pnfs_osd_layoutupdate *lou, __be32 *p);
+
+/* layout_return decoding */
+bool pnfs_osd_xdr_decode_ioerr(
+	struct pnfs_osd_ioerr *ioerr, struct exp_xdr_stream *xdr);
+
+#endif /* __PNFS_OSD_XDR_SRV_H__ */
-- 
1.7.3.4

[PATCH 5/8] pnfsd-exofs: Add pNFS export support

[Boaz Harrosh]

Add a new file export.c that implements the pnfs_export_operations.
And set a pointer to it in the super_block.

Implemented: LAYOUTGET, simple LAYOUTCOMMIT, simple LAYOUTRETURN

missing: GETDEVICEINFO, Recalls. These will come in next patches

export.c inclusion is dependent on CONFIG_PNFSD defined. If not
then the few used functions from export.c are defined empty
in exofs.h.

In that case of PNFSD EXPORTFS_OSD_LAYOUT must be selected
in fs/exofs/Kconfig

Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
---
 fs/exofs/Kbuild   |    1 +
 fs/exofs/Kconfig  |    1 +
 fs/exofs/exofs.h  |    7 ++
 fs/exofs/export.c |  198 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 fs/exofs/super.c  |    1 +
 5 files changed, 208 insertions(+), 0 deletions(-)
 create mode 100644 fs/exofs/export.c

diff --git a/fs/exofs/Kbuild b/fs/exofs/Kbuild
index 2d0f757..5458546 100644
--- a/fs/exofs/Kbuild
+++ b/fs/exofs/Kbuild
@@ -13,4 +13,5 @@
 #
 
 exofs-y := ios.o inode.o file.o symlink.o namei.o dir.o super.o
+exofs-$(CONFIG_PNFSD) +=  export.o
 obj-$(CONFIG_EXOFS_FS) += exofs.o
diff --git a/fs/exofs/Kconfig b/fs/exofs/Kconfig
index 86194b2..77c677f 100644
--- a/fs/exofs/Kconfig
+++ b/fs/exofs/Kconfig
@@ -1,6 +1,7 @@
 config EXOFS_FS
 	tristate "exofs: OSD based file system support"
 	depends on SCSI_OSD_ULD
+	select EXPORTFS_OSD_LAYOUT if PNFSD
 	help
 	  EXOFS is a file system that uses an OSD storage device,
 	  as its backing storage.
diff --git a/fs/exofs/exofs.h b/fs/exofs/exofs.h
index e103dbd..3469d77 100644
--- a/fs/exofs/exofs.h
+++ b/fs/exofs/exofs.h
@@ -301,4 +301,11 @@ extern const struct inode_operations exofs_special_inode_operations;
 extern const struct inode_operations exofs_symlink_inode_operations;
 extern const struct inode_operations exofs_fast_symlink_inode_operations;
 
+/* export.c */
+#ifdef CONFIG_PNFSD
+void exofs_init_export(struct super_block *sb);
+#else
+static inline void exofs_init_export(struct super_block *sb) {}
+#endif
+
 #endif
diff --git a/fs/exofs/export.c b/fs/exofs/export.c
new file mode 100644
index 0000000..0a068b7
--- /dev/null
+++ b/fs/exofs/export.c
@@ -0,0 +1,198 @@
+/*
+ * export.c - Implementation of the pnfs_export_operations
+ *
+ * Copyright (C) 2009 Panasas Inc.
+ * All rights reserved.
+ *
+ * Boaz Harrosh <bharrosh@panasas.com>
+ *
+ * This file is part of exofs.
+ *
+ * exofs is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation.  Since it is based on ext2, and the only
+ * valid version of GPL for the Linux kernel is version 2, the only valid
+ * version of GPL for exofs is version 2.
+ *
+ * exofs is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with exofs; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+ */
+
+#include <linux/nfsd/nfsd4_pnfs.h>
+#include "exofs.h"
+
+#include "linux/nfsd/pnfs_osd_xdr_srv.h"
+
+static int exofs_layout_type(struct super_block *sb)
+{
+	return LAYOUT_OSD2_OBJECTS;
+}
+
+static void set_dev_id(struct nfs4_deviceid *pnfs_devid, u64 sbid, u64 devid)
+{
+	struct nfsd4_pnfs_deviceid *dev_id =
+		(struct nfsd4_pnfs_deviceid *)pnfs_devid;
+
+	dev_id->sbid  = sbid;
+	dev_id->devid = devid;
+}
+
+static enum nfsstat4 exofs_layout_get(
+	struct inode *inode,
+	struct exp_xdr_stream *xdr,
+	const struct nfsd4_pnfs_layoutget_arg *args,
+	struct nfsd4_pnfs_layoutget_res *res)
+{
+	struct exofs_i_info *oi = exofs_i(inode);
+	struct exofs_sb_info *sbi = inode->i_sb->s_fs_info;
+	struct exofs_layout *el = &sbi->layout;
+	struct pnfs_osd_object_cred *creds = NULL;
+	struct pnfs_osd_layout layout;
+	__be32 *start;
+	unsigned i;
+	enum nfsstat4 nfserr;
+
+	res->lg_seg.offset = 0;
+	res->lg_seg.length = NFS4_MAX_UINT64;
+	res->lg_seg.iomode = IOMODE_RW;
+	res->lg_return_on_close = true; /* TODO: unused but will be soon */
+
+	/* skip opaque size, will be filled-in later */
+	start = exp_xdr_reserve_qwords(xdr, 1);
+	if (!start) {
+		nfserr = NFS4ERR_TOOSMALL;
+		goto out;
+	}
+
+	/* Fill in a pnfs_osd_layout struct */
+	layout.olo_map = sbi->data_map;
+	layout.olo_comps_index = 0;
+	layout.olo_num_comps = el->s_numdevs;
+	layout.olo_comps = creds;
+
+	nfserr = pnfs_osd_xdr_encode_layout_hdr(xdr, &layout);
+	if (unlikely(nfserr))
+		goto out;
+
+	/* Encode layout components */
+	for (i = 0; i < el->s_numdevs; i++) {
+		struct pnfs_osd_object_cred cred;
+		osd_id id = exofs_oi_objno(oi);
+		unsigned dev = exofs_layout_od_id(el, id, i);
+
+		set_dev_id(&cred.oc_object_id.oid_device_id, args->lg_sbid,
+			   dev);
+		cred.oc_object_id.oid_partition_id = el->s_pid;
+		cred.oc_object_id.oid_object_id = id;
+		cred.oc_osd_version = osd_dev_is_ver1(el->s_ods[dev]) ?
+						PNFS_OSD_VERSION_1 :
+						PNFS_OSD_VERSION_2;
+		cred.oc_cap_key_sec = PNFS_OSD_CAP_KEY_SEC_NONE;
+
+		cred.oc_cap_key.cred_len	= 0;
+		cred.oc_cap_key.cred		= NULL;
+
+		cred.oc_cap.cred_len	= OSD_CAP_LEN;
+		cred.oc_cap.cred	= oi->i_cred;
+		nfserr = pnfs_osd_xdr_encode_layout_cred(xdr, &cred);
+		if (unlikely(nfserr))
+			goto out;
+	}
+
+	exp_xdr_encode_opaque_len(start, xdr->p);
+	nfserr = NFS4_OK;
+	/* TODO: Takes the inode ref here, add to inode's layouts list */
+
+out:
+	kfree(creds);
+	EXOFS_DBGMSG("(0x%lx) nfserr=%u xdr_bytes=%zu\n",
+		     inode->i_ino, nfserr, exp_xdr_qbytes(xdr->p - start));
+	return nfserr;
+}
+
+/* NOTE: inode mutex must NOT be held */
+static int exofs_layout_commit(
+	struct inode *inode,
+	const struct nfsd4_pnfs_layoutcommit_arg *args,
+	struct nfsd4_pnfs_layoutcommit_res *res)
+{
+	struct timespec mtime;
+	loff_t i_size;
+
+	/* NOTE: I would love to call inode_setattr here
+	 *	 but i cannot since this will cause an eventual vmtruncate,
+	 *	 which will cause a layout_recall. So open code the i_size
+	 *	 and mtime/atime changes under i_mutex.
+	 */
+	mutex_lock_nested(&inode->i_mutex, I_MUTEX_NORMAL);
+
+	if (args->lc_mtime.seconds) {
+		mtime.tv_sec = args->lc_mtime.seconds;
+		mtime.tv_nsec = args->lc_mtime.nseconds;
+
+		/* layout commit may only make time bigger, since there might
+		 * be reordering of the notifications and it might arrive after
+		 * A local change.
+		 * TODO: if mtime > ctime then we know set_attr did an mtime
+		 * in the future. and we can let this update through
+		 */
+		if (0 <= timespec_compare(&mtime, &inode->i_mtime))
+			mtime = inode->i_mtime;
+	} else {
+		mtime = current_fs_time(inode->i_sb);
+	}
+
+	/* TODO: Will below work? since mark_inode_dirty has it's own
+	 *       Time handling
+	 */
+	inode->i_atime = inode->i_mtime = mtime;
+
+	i_size = i_size_read(inode);
+	if (args->lc_newoffset) {
+		loff_t new_size = args->lc_last_wr + 1;
+
+		if (i_size < new_size) {
+			i_size_write(inode, i_size = new_size);
+			res->lc_size_chg = 1;
+			res->lc_newsize = new_size;
+		}
+	}
+	/* TODO: else { i_size = osd_get_object_length() } */
+
+/* TODO: exofs does not currently use the osd_xdr part of the layout_commit */
+
+	mark_inode_dirty_sync(inode);
+
+	mutex_unlock(&inode->i_mutex);
+	EXOFS_DBGMSG("(0x%lx) i_size=0x%llx lcp->off=0x%llx\n",
+		     inode->i_ino, i_size, args->lc_last_wr);
+	return 0;
+}
+
+static int exofs_layout_return(
+	struct inode *inode,
+	const struct nfsd4_pnfs_layoutreturn_arg *args)
+{
+	/* TODO: Decode the pnfs_osd_ioerr if lrf_body_len > 0 */
+
+	/* TODO: When layout_get takes the inode ref put_ref here */
+	return 0;
+}
+
+struct pnfs_export_operations exofs_pnfs_ops = {
+	.layout_type	= exofs_layout_type,
+	.layout_get	= exofs_layout_get,
+	.layout_commit	= exofs_layout_commit,
+	.layout_return	= exofs_layout_return,
+};
+
+void exofs_init_export(struct super_block *sb)
+{
+	sb->s_pnfs_op = &exofs_pnfs_ops;
+}
diff --git a/fs/exofs/super.c b/fs/exofs/super.c
index 06065bd..96ecb15 100644
--- a/fs/exofs/super.c
+++ b/fs/exofs/super.c
@@ -770,6 +770,7 @@ static int exofs_fill_super(struct super_block *sb, void *data, int silent)
 	sb->s_fs_info = sbi;
 	sb->s_op = &exofs_sops;
 	sb->s_export_op = &exofs_export_ops;
+	exofs_init_export(sb);
 	root = exofs_iget(sb, EXOFS_ROOT_ID - EXOFS_OBJ_OFF);
 	if (IS_ERR(root)) {
 		EXOFS_ERR("ERROR: exofs_iget failed\n");
-- 
1.7.3.4

[PATCH 6/8] pnfsd-exofs: get_device_info implementation.

[Boaz Harrosh]

Implement a .get_device_info operation from the
pnfs_export_operations.

Retrieve to system_id and osdname for the requested
devices.

TODO: Return also UNC hints for the device login
information.

Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
---
 fs/exofs/export.c |   54 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 54 insertions(+), 0 deletions(-)

diff --git a/fs/exofs/export.c b/fs/exofs/export.c
index 0a068b7..23c1666 100644
--- a/fs/exofs/export.c
+++ b/fs/exofs/export.c
@@ -185,11 +185,65 @@ static int exofs_layout_return(
 	return 0;
 }
 
+int exofs_get_device_info(struct super_block *sb, struct exp_xdr_stream *xdr,
+			  u32 layout_type,
+			  const struct nfsd4_pnfs_deviceid *devid)
+{
+	struct exofs_sb_info *sbi = sb->s_fs_info;
+	struct pnfs_osd_deviceaddr devaddr;
+	const struct osd_dev_info *odi;
+	u64 devno = devid->devid;
+	__be32 *start;
+	int err;
+
+	memset(&devaddr, 0, sizeof(devaddr));
+
+	if (unlikely(devno >= sbi->layout.s_numdevs)) {
+		EXOFS_DBGMSG("Error: Device((%llx,%llx) does not exist\n",
+			     devid->sbid, devno);
+		return -ENODEV;
+	}
+
+	odi = osduld_device_info(sbi->layout.s_ods[devno]);
+
+	devaddr.oda_systemid.len = odi->systemid_len;
+	devaddr.oda_systemid.data = (void *)odi->systemid; /* !const cast */
+
+	devaddr.oda_osdname.len = odi->osdname_len ;
+	devaddr.oda_osdname.data = (void *)odi->osdname;/* !const cast */
+
+	/* skip opaque size, will be filled-in later */
+	start = exp_xdr_reserve_qwords(xdr, 1);
+	if (!start) {
+		err = -ETOOSMALL;
+		goto err;
+	}
+
+	err = pnfs_osd_xdr_encode_deviceaddr(xdr, &devaddr);
+	if (err) {
+		err = -ETOOSMALL;
+		goto err;
+	}
+
+	exp_xdr_encode_opaque_len(start, xdr->p);
+
+	EXOFS_DBGMSG("xdr_bytes=%Zu devid=(%llx,%llx) osdname-%s\n",
+		     exp_xdr_qbytes(xdr->p - start), devid->sbid, devno,
+		     odi->osdname);
+	return 0;
+
+err:
+	EXOFS_DBGMSG("Error: err=%d at_byte=%zu\n",
+		     err, exp_xdr_qbytes(xdr->p - start));
+	return err;
+}
+
 struct pnfs_export_operations exofs_pnfs_ops = {
 	.layout_type	= exofs_layout_type,
 	.layout_get	= exofs_layout_get,
 	.layout_commit	= exofs_layout_commit,
 	.layout_return	= exofs_layout_return,
+	.get_device_info = exofs_get_device_info,
 };
 
 void exofs_init_export(struct super_block *sb)
-- 
1.7.3.4

[PATCH 8/8] pnfsd-exofs: layoutreturn pnfs-obj information decoding

[Boaz Harrosh]

Use pnfs_osd_xdr_decode_ioerr() in exofs_layout_return,
and print the received errors.

TODO: Call exofs_file_recovery() on errors that need it.
      (When it's implemented)

Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
---
 fs/exofs/export.c |   22 +++++++++++++++++++++-
 1 files changed, 21 insertions(+), 1 deletions(-)

diff --git a/fs/exofs/export.c b/fs/exofs/export.c
index c0705df..8281c85 100644
--- a/fs/exofs/export.c
+++ b/fs/exofs/export.c
@@ -229,11 +229,31 @@ static int exofs_layout_commit(
 	return 0;
 }
 
+static void exofs_handle_error(struct pnfs_osd_ioerr *ioerr)
+{
+	EXOFS_ERR("exofs_handle_error: errno=%d is_write=%d obj=0x%llx "
+		  "offset=0x%llx length=0x%llx\n",
+		  ioerr->oer_errno, ioerr->oer_iswrite,
+		  _LLU(ioerr->oer_component.oid_object_id),
+		  _LLU(ioerr->oer_comp_offset),
+		  _LLU(ioerr->oer_comp_length));
+}
+
 static int exofs_layout_return(
 	struct inode *inode,
 	const struct nfsd4_pnfs_layoutreturn_arg *args)
 {
-	/* TODO: Decode the pnfs_osd_ioerr if lrf_body_len > 0 */
+	struct exp_xdr_stream xdr = {
+		.p = args->lrf_body,
+		.end = args->lrf_body + exp_xdr_qwords(args->lrf_body_len),
+	};
+	struct pnfs_osd_ioerr ioerr;
+
+	EXOFS_DBGMSG("(0x%lx) cookie %p body_len %d\n",
+		     inode->i_ino, args->lr_cookie, args->lrf_body_len);
+
+	while (pnfs_osd_xdr_decode_ioerr(&ioerr, &xdr))
+		exofs_handle_error(&ioerr);
 
 	if (args->lr_cookie) {
 		struct exofs_i_info *oi = exofs_i(inode);
-- 
1.7.3.4

[PATCH 3/8] pnfs-obj: pnfs_osd_xdr: Remove dead code and cleanup

[Boaz Harrosh]

From: Boaz Harrosh <Boaz Harrosh bharrosh@panasas.com>

* Some leftovers from ancient times.
* This file will only define common types and client API.
  Remove server from comments

Signed-off-by: Boaz Harrosh <Boaz Harrosh bharrosh@panasas.com>
---
 include/linux/pnfs_osd_xdr.h |   31 ++-----------------------------
 1 files changed, 2 insertions(+), 29 deletions(-)

diff --git a/include/linux/pnfs_osd_xdr.h b/include/linux/pnfs_osd_xdr.h
index 76efbdd..435dd5f 100644
--- a/include/linux/pnfs_osd_xdr.h
+++ b/include/linux/pnfs_osd_xdr.h
@@ -41,9 +41,6 @@
 
 #include <linux/nfs_fs.h>
 #include <linux/nfs_page.h>
-#include <scsi/osd_protocol.h>
-
-#define PNFS_OSD_OSDNAME_MAXSIZE 256
 
 /*
  * draft-ietf-nfsv4-minorversion-22
@@ -99,12 +96,6 @@ struct pnfs_osd_objid {
 #define _DEVID_HI(oid_device_id) \
 	(unsigned long long)be64_to_cpup(((__be64 *)(oid_device_id)->data) + 1)
 
-static inline int
-pnfs_osd_objid_xdr_sz(void)
-{
-	return (NFS4_DEVICEID4_SIZE / 4) + 2 + 2;
-}
-
 enum pnfs_osd_version {
 	PNFS_OSD_MISSING              = 0,
 	PNFS_OSD_VERSION_1            = 1,
@@ -189,8 +180,6 @@ struct pnfs_osd_targetid {
 	struct nfs4_string		oti_scsi_device_id;
 };
 
-enum { PNFS_OSD_TARGETID_MAX = 1 + PNFS_OSD_OSDNAME_MAXSIZE / 4 };
-
 /*   struct netaddr4 {
  *       // see struct rpcb in RFC1833
  *       string r_netid<>;    // network id
@@ -207,12 +196,6 @@ struct pnfs_osd_targetaddr {
 	struct pnfs_osd_net_addr	ota_netaddr;
 };
 
-enum {
-	NETWORK_ID_MAX = 16 / 4,
-	UNIVERSAL_ADDRESS_MAX = 64 / 4,
-	PNFS_OSD_TARGETADDR_MAX = 3 +  NETWORK_ID_MAX + UNIVERSAL_ADDRESS_MAX,
-};
-
 struct pnfs_osd_deviceaddr {
 	struct pnfs_osd_targetid	oda_targetid;
 	struct pnfs_osd_targetaddr	oda_targetaddr;
@@ -222,15 +205,6 @@ struct pnfs_osd_deviceaddr {
 	struct nfs4_string		oda_osdname;
 };
 
-enum {
-	ODA_OSDNAME_MAX = PNFS_OSD_OSDNAME_MAXSIZE / 4,
-	PNFS_OSD_DEVICEADDR_MAX =
-		PNFS_OSD_TARGETID_MAX + PNFS_OSD_TARGETADDR_MAX +
-		2 /*oda_lun*/ +
-		1 + OSD_SYSTEMID_LEN +
-		1 + ODA_OSDNAME_MAX,
-};
-
 /* LAYOUTCOMMIT: layoutupdate */
 
 /*   union pnfs_osd_deltaspaceused4 switch (bool dsu_valid) {
@@ -279,7 +253,7 @@ struct pnfs_osd_ioerr {
 	u32			oer_errno;
 };
 
-/* OSD XDR API */
+/* OSD XDR Client API */
 /* Layout helpers */
 /* Layout decoding is done in two parts:
  * 1. First Call pnfs_osd_xdr_decode_layout_map to read in only the header part
@@ -337,8 +311,7 @@ extern int
 pnfs_osd_xdr_encode_layoutupdate(struct xdr_stream *xdr,
 				 struct pnfs_osd_layoutupdate *lou);
 
-/* osd_ioerror encoding/decoding (layout_return) */
-/* Client */
+/* osd_ioerror encoding (layout_return) */
 extern __be32 *pnfs_osd_xdr_ioerr_reserve_space(struct xdr_stream *xdr);
 extern void pnfs_osd_xdr_encode_ioerr(__be32 *p, struct pnfs_osd_ioerr *ioerr);
 
-- 
1.7.3.4

[PATCH 7/8] pnfsd-exofs: Recall layouts on truncate

[Boaz Harrosh]

From: Boaz Harrosh <Boaz Harrosh bharrosh@panasas.com>

* Define a new exofs_inode_recall_layout() which receives a function
  pointer. The function pointer is called when all layouts have returned,
  and before any new will be given.

* At exofs_layout_return() when all layouts recalled have returned,
  above exofs_inode_recall_layout() is awaken.

* At exofs_truncate() recall all layouts on the file.

Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
[added u64 todo_data param starting 2.6.36]
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
[pnfsd-exofs: Fix exofs_inode_recall_layout() sending 2 recalls]
    A misunderstanding of the wait_event_interruptible() macro has caused
    exofs to always issue two recalls in succession.

    Using __wait_event_interruptible() would fix that. But I don't like
    how this code is structured so change it around completely to be more
    flat and clear.
Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
Signed-off-by: Benny Halevy <bhalevy@panasas.com>
---
 fs/exofs/exofs.h  |   14 ++++++
 fs/exofs/export.c |  131 +++++++++++++++++++++++++++++++++++++++++++++++++++-
 fs/exofs/inode.c  |    7 ++-
 3 files changed, 147 insertions(+), 5 deletions(-)

diff --git a/fs/exofs/exofs.h b/fs/exofs/exofs.h
index 3469d77..45c9fe5 100644
--- a/fs/exofs/exofs.h
+++ b/fs/exofs/exofs.h
@@ -99,6 +99,7 @@ struct exofs_sb_info {
 struct exofs_i_info {
 	struct inode   vfs_inode;          /* normal in-memory inode          */
 	wait_queue_head_t i_wq;            /* wait queue for inode            */
+	spinlock_t     i_layout_lock;      /* lock for layout/return/recall   */
 	unsigned long  i_flags;            /* various atomic flags            */
 	uint32_t       i_data[EXOFS_IDATA];/*short symlink names and device #s*/
 	uint32_t       i_dir_start_lookup; /* which page to start lookup      */
@@ -162,6 +163,9 @@ static inline unsigned exofs_io_state_size(unsigned numdevs)
  */
 #define OBJ_2BCREATED	0	/* object will be created soon*/
 #define OBJ_CREATED	1	/* object has been created on the osd*/
+/* Below are not used atomic but reuse the same i_flags */
+#define OBJ_LAYOUT_IS_GIVEN  2  /* inode has given layouts to clients*/
+#define OBJ_IN_LAYOUT_RECALL 3  /* inode is in the middle of a layout recall*/
 
 static inline int obj_2bcreated(struct exofs_i_info *oi)
 {
@@ -302,9 +306,19 @@ extern const struct inode_operations exofs_symlink_inode_operations;
 extern const struct inode_operations exofs_fast_symlink_inode_operations;
 
 /* export.c */
+typedef int (exofs_recall_fn)(struct inode *inode, u64 data);
 #ifdef CONFIG_PNFSD
+int exofs_inode_recall_layout(struct inode *inode, enum pnfs_iomode iomode,
+			      exofs_recall_fn todo, u64 todo_data);
 void exofs_init_export(struct super_block *sb);
 #else
+static inline int
+exofs_inode_recall_layout(struct inode *inode, enum pnfs_iomode iomode,
+exofs_recall_fn todo, u64 todo_data)
+{
+	return todo(inode, todo_data);
+}
+
 static inline void exofs_init_export(struct super_block *sb) {}
 #endif
 
diff --git a/fs/exofs/export.c b/fs/exofs/export.c
index 23c1666..c0705df 100644
--- a/fs/exofs/export.c
+++ b/fs/exofs/export.c
@@ -43,6 +43,36 @@ static void set_dev_id(struct nfs4_deviceid *pnfs_devid, u64 sbid, u64 devid)
 	dev_id->devid = devid;
 }
 
+static int cb_layout_recall(struct inode *inode, enum pnfs_iomode iomode,
+			    u64 offset, u64 length, void *cookie)
+{
+	struct nfsd4_pnfs_cb_layout cbl;
+	struct pnfsd_cb_ctl cb_ctl;
+	int status;
+
+	memset(&cb_ctl, 0, sizeof(cb_ctl));
+	status = pnfsd_get_cb_op(&cb_ctl);
+	if (unlikely(status)) {
+		EXOFS_ERR("%s: nfsd unloaded!! inode (0x%lx) status=%d\n",
+			  __func__, inode->i_ino, status);
+		goto err;
+	}
+
+	memset(&cbl, 0, sizeof(cbl));
+	cbl.cbl_recall_type = RETURN_FILE;
+	cbl.cbl_seg.layout_type = LAYOUT_OSD2_OBJECTS;
+	cbl.cbl_seg.iomode = iomode;
+	cbl.cbl_seg.offset = offset;
+	cbl.cbl_seg.length = length;
+	cbl.cbl_cookie = cookie;
+
+	status = cb_ctl.cb_op->cb_layout_recall(inode->i_sb, inode, &cbl);
+	pnfsd_put_cb_op(&cb_ctl);
+
+err:
+	return status;
+}
+
 static enum nfsstat4 exofs_layout_get(
 	struct inode *inode,
 	struct exp_xdr_stream *xdr,
@@ -56,6 +86,7 @@ static enum nfsstat4 exofs_layout_get(
 	struct pnfs_osd_layout layout;
 	__be32 *start;
 	unsigned i;
+	bool in_recall;
 	enum nfsstat4 nfserr;
 
 	res->lg_seg.offset = 0;
@@ -106,8 +137,16 @@ static enum nfsstat4 exofs_layout_get(
 	}
 
 	exp_xdr_encode_opaque_len(start, xdr->p);
-	nfserr = NFS4_OK;
-	/* TODO: Takes the inode ref here, add to inode's layouts list */
+
+	spin_lock(&oi->i_layout_lock);
+	in_recall = test_bit(OBJ_IN_LAYOUT_RECALL, &oi->i_flags);
+	if (!in_recall) {
+		__set_bit(OBJ_LAYOUT_IS_GIVEN, &oi->i_flags);
+		nfserr = NFS4_OK;
+	} else {
+		nfserr = NFS4ERR_RECALLCONFLICT;
+	}
+	spin_unlock(&oi->i_layout_lock);
 
 out:
 	kfree(creds);
@@ -122,8 +161,23 @@ static int exofs_layout_commit(
 	const struct nfsd4_pnfs_layoutcommit_arg *args,
 	struct nfsd4_pnfs_layoutcommit_res *res)
 {
+	struct exofs_i_info *oi = exofs_i(inode);
 	struct timespec mtime;
 	loff_t i_size;
+	int in_recall;
+
+	/* In case of a recall we ignore the new size and mtime since they
+	 * are going to be changed again by truncate, and since we cannot take
+	 * the inode lock in that case.
+	 */
+	spin_lock(&oi->i_layout_lock);
+	in_recall = test_bit(OBJ_IN_LAYOUT_RECALL, &oi->i_flags);
+	spin_unlock(&oi->i_layout_lock);
+	if (in_recall) {
+		EXOFS_DBGMSG("(0x%lx) commit was called during recall\n",
+			     inode->i_ino);
+		return 0;
+	}
 
 	/* NOTE: I would love to call inode_setattr here
 	 *	 but i cannot since this will cause an eventual vmtruncate,
@@ -181,7 +235,20 @@ static int exofs_layout_return(
 {
 	/* TODO: Decode the pnfs_osd_ioerr if lrf_body_len > 0 */
 
-	/* TODO: When layout_get takes the inode ref put_ref here */
+	if (args->lr_cookie) {
+		struct exofs_i_info *oi = exofs_i(inode);
+		bool in_recall;
+
+		spin_lock(&oi->i_layout_lock);
+		in_recall = test_bit(OBJ_IN_LAYOUT_RECALL, &oi->i_flags);
+		__clear_bit(OBJ_LAYOUT_IS_GIVEN, &oi->i_flags);
+		spin_unlock(&oi->i_layout_lock);
+
+		/* TODO: how to communicate cookie with the waiter */
+		if (in_recall)
+			wake_up(&oi->i_wq); /* wakeup any recalls */
+	}
+
 	return 0;
 }
 
@@ -246,6 +313,64 @@ struct pnfs_export_operations exofs_pnfs_ops = {
 	.get_device_info = exofs_get_device_info,
 };
 
+static bool is_layout_returned(struct exofs_i_info *oi)
+{
+	bool layout_given;
+
+	spin_lock(&oi->i_layout_lock);
+	layout_given = test_bit(OBJ_LAYOUT_IS_GIVEN, &oi->i_flags);
+	spin_unlock(&oi->i_layout_lock);
+
+	return !layout_given;
+}
+
+int exofs_inode_recall_layout(struct inode *inode, enum pnfs_iomode iomode,
+			      exofs_recall_fn todo, u64 todo_data)
+{
+	struct exofs_i_info *oi = exofs_i(inode);
+	int layout_given;
+	int error = 0;
+
+	spin_lock(&oi->i_layout_lock);
+	layout_given = test_bit(OBJ_LAYOUT_IS_GIVEN, &oi->i_flags);
+	__set_bit(OBJ_IN_LAYOUT_RECALL, &oi->i_flags);
+	spin_unlock(&oi->i_layout_lock);
+
+	if (!layout_given)
+		goto exec;
+
+	for (;;) {
+		EXOFS_DBGMSG("(0x%lx) has_layout issue a recall\n",
+			     inode->i_ino);
+		error = cb_layout_recall(inode, iomode, 0, NFS4_MAX_UINT64,
+					 &oi->i_wq);
+		switch (error) {
+		case 0:
+		case -EAGAIN:
+			break;
+		case -ENOENT:
+			goto exec;
+		default:
+			goto err;
+		}
+
+		error = wait_event_interruptible(oi->i_wq,
+						 is_layout_returned(oi));
+		if (error)
+			goto err;
+	}
+
+exec:
+	error = todo(inode, todo_data);
+
+err:
+	spin_lock(&oi->i_layout_lock);
+	__clear_bit(OBJ_IN_LAYOUT_RECALL, &oi->i_flags);
+	spin_unlock(&oi->i_layout_lock);
+	EXOFS_DBGMSG("(0x%lx) return=>%d\n", inode->i_ino, error);
+	return error;
+}
+
 void exofs_init_export(struct super_block *sb)
 {
 	sb->s_pnfs_op = &exofs_pnfs_ops;
diff --git a/fs/exofs/inode.c b/fs/exofs/inode.c
index 8472c09..1365e72 100644
--- a/fs/exofs/inode.c
+++ b/fs/exofs/inode.c
@@ -847,8 +847,9 @@ static inline int exofs_inode_is_fast_symlink(struct inode *inode)
 const struct osd_attr g_attr_logical_length = ATTR_DEF(
 	OSD_APAGE_OBJECT_INFORMATION, OSD_ATTR_OI_LOGICAL_LENGTH, 8);
 
-static int _do_truncate(struct inode *inode, loff_t newsize)
+static int _do_truncate(struct inode *inode, u64 data)
 {
+	loff_t newsize = data;
 	struct exofs_i_info *oi = exofs_i(inode);
 	int ret;
 
@@ -885,7 +886,8 @@ int exofs_setattr(struct dentry *dentry, struct iattr *iattr)
 
 	if ((iattr->ia_valid & ATTR_SIZE) &&
 	    iattr->ia_size != i_size_read(inode)) {
-		error = _do_truncate(inode, iattr->ia_size);
+		error = exofs_inode_recall_layout(inode, IOMODE_ANY,
+						  _do_truncate, iattr->ia_size);
 		if (unlikely(error))
 			return error;
 	}
@@ -998,6 +1000,7 @@ static void __oi_init(struct exofs_i_info *oi)
 {
 	init_waitqueue_head(&oi->i_wq);
 	oi->i_flags = 0;
+	spin_lock_init(&oi->i_layout_lock);
 }
 /*
  * Fill in an inode read from the OSD and set it up for use
-- 
1.7.3.4

[PATCHSET 0/10] pnfsd-exofs: SQUASHMES that went into last changes

[Boaz Harrosh]

here are the list of changes done to the old tree

[PATCH 01/10] SQUASHME: pnfsd-exofs xdr_srv: Wrong type in comments
[PATCH 02/10] SQUASHME: pnfsd-exofs: Move all Server xdr definisions to separate header
[PATCH 03/10] SQUASHME: pnfsd-exofs: Prettify pnfs_osd_xdr_srv.h
[PATCH 04/10] SQUASHME: pnfsd-exofs: Change layout encoding to two pass
[PATCH 05/10] SQUASHME: pnfsd-exofs: osd_xdr_srv: Convert XDR API to return enum nfsstat4
[PATCH 06/10] SQUASHME: pnfsd-exofs: Server xdr definisions were moved to a separate header
[PATCH 07/10] SQUASHME: pnfsd-exofs: osd_xdr_srv header moved
[PATCH 08/10] SQUASHME: pnfsd-exofs: layout encoding was Changed to two pass
[PATCH 09/10] SQUASHME: pnfsd-exofs: Convert XDR API to return enum nfsstat4
[PATCH 09/10] SQUASHME: pnfsd-exofs: checkpatch love and a dprint


Boaz

[PATCH 01/10] SQUASHME: pnfsd-exofs xdr_srv: Wrong type in comments

[Boaz Harrosh]

Boaz
---
 fs/exportfs/pnfs_osd_xdr_srv.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/fs/exportfs/pnfs_osd_xdr_srv.c b/fs/exportfs/pnfs_osd_xdr_srv.c
index fc7d805..444798c 100644
--- a/fs/exportfs/pnfs_osd_xdr_srv.c
+++ b/fs/exportfs/pnfs_osd_xdr_srv.c
@@ -71,7 +71,7 @@ static int pnfs_osd_xdr_encode_data_map(
 
 /*
  * struct pnfs_osd_objid {
- * 	struct pnfs_deviceid	oid_device_id;
+ * 	struct nfs4_deviceid	oid_device_id;
  * 	u64			oid_partition_id;
  * 	u64			oid_object_id;
  * };
@@ -249,7 +249,7 @@ EXPORT_SYMBOL(pnfs_osd_xdr_decode_layoutupdate);
 
 /*
  * struct pnfs_osd_objid {
- * 	struct pnfs_deviceid	oid_device_id;
+ * 	struct nfs4_deviceid	oid_device_id;
  * 	u64			oid_partition_id;
  * 	u64			oid_object_id;
  * }; xdr size 32
-- 
1.7.3.4

[PATCH 02/10] SQUASHME: pnfsd-exofs: Move all Server xdr definisions to separate header

[Boaz Harrosh]

Server XDR definitions are defined in there own header

Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
---
 fs/exportfs/pnfs_osd_xdr_srv.c        |    3 ++-
 include/linux/nfsd/pnfs_osd_xdr_srv.h |   17 +++++++++++++++++
 2 files changed, 19 insertions(+), 1 deletions(-)
 create mode 100644 include/linux/nfsd/pnfs_osd_xdr_srv.h

diff --git a/fs/exportfs/pnfs_osd_xdr_srv.c b/fs/exportfs/pnfs_osd_xdr_srv.c
index 444798c..67564f6 100644
--- a/fs/exportfs/pnfs_osd_xdr_srv.c
+++ b/fs/exportfs/pnfs_osd_xdr_srv.c
@@ -38,7 +38,8 @@
  */
 
 #include <linux/nfsd/nfsd4_pnfs.h>
-#include <linux/pnfs_osd_xdr.h>
+
+#include "linux/nfsd/pnfs_osd_xdr_srv.h"
 
 /*
  * struct pnfs_osd_data_map {
diff --git a/include/linux/nfsd/pnfs_osd_xdr_srv.h b/include/linux/nfsd/pnfs_osd_xdr_srv.h
new file mode 100644
index 0000000..65de588
--- /dev/null
+++ b/include/linux/nfsd/pnfs_osd_xdr_srv.h
@@ -0,0 +1,17 @@
+#include <linux/pnfs_osd_xdr.h>
+#include <linux/exp_xdr.h>
+
+/* Layout encoding */
+extern int pnfs_osd_xdr_encode_layout(
+	struct exp_xdr_stream *xdr,
+	struct pnfs_osd_layout *layout);
+
+/* deviceaddr for Servers */
+extern int pnfs_osd_xdr_encode_deviceaddr(
+	struct exp_xdr_stream *xdr, struct pnfs_osd_deviceaddr *devaddr);
+
+extern __be32 * pnfs_osd_xdr_decode_layoutupdate(
+	struct pnfs_osd_layoutupdate *lou, __be32 *p);
+
+extern bool pnfs_osd_xdr_decode_ioerr(
+	struct pnfs_osd_ioerr *ioerr, struct exp_xdr_stream *xdr);
-- 
1.7.3.4

[PATCH 03/10] SQUASHME: pnfsd-exofs: Prettify pnfs_osd_xdr_srv.h

[Boaz Harrosh]

Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
---
 include/linux/nfsd/pnfs_osd_xdr_srv.h |   29 ++++++++++++++++++++++++-----
 1 files changed, 24 insertions(+), 5 deletions(-)

diff --git a/include/linux/nfsd/pnfs_osd_xdr_srv.h b/include/linux/nfsd/pnfs_osd_xdr_srv.h
index 65de588..256a7e07 100644
--- a/include/linux/nfsd/pnfs_osd_xdr_srv.h
+++ b/include/linux/nfsd/pnfs_osd_xdr_srv.h
@@ -1,17 +1,36 @@
+/*
+ * pnfs-objects Server XDR definitions and API
+ *
+ * Copyright (C) from 2011 Panasas Inc.  All rights reserved.
+ *
+ * Authors:
+ *   Boaz Harrosh <bharrosh@panasas.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2
+ *
+ */
+#ifndef __PNFS_OSD_XDR_SRV_H__
+#define __PNFS_OSD_XDR_SRV_H__
+
 #include <linux/pnfs_osd_xdr.h>
 #include <linux/exp_xdr.h>
 
 /* Layout encoding */
-extern int pnfs_osd_xdr_encode_layout(
+int pnfs_osd_xdr_encode_layout(
 	struct exp_xdr_stream *xdr,
 	struct pnfs_osd_layout *layout);
 
-/* deviceaddr for Servers */
-extern int pnfs_osd_xdr_encode_deviceaddr(
+/* deviceaddr encoding */
+int pnfs_osd_xdr_encode_deviceaddr(
 	struct exp_xdr_stream *xdr, struct pnfs_osd_deviceaddr *devaddr);
 
-extern __be32 * pnfs_osd_xdr_decode_layoutupdate(
+/* layout_commit decoding */
+__be32 * pnfs_osd_xdr_decode_layoutupdate(
 	struct pnfs_osd_layoutupdate *lou, __be32 *p);
 
-extern bool pnfs_osd_xdr_decode_ioerr(
+/* layout_return decoding */
+bool pnfs_osd_xdr_decode_ioerr(
 	struct pnfs_osd_ioerr *ioerr, struct exp_xdr_stream *xdr);
+
+#endif /* __PNFS_OSD_XDR_SRV_H__ */
-- 
1.7.3.4

[PATCH 04/10] SQUASHME: pnfsd-exofs: Change layout encoding to two pass

[Boaz Harrosh]

This way there is No need for dynamic allocation of the creds
array.

TODO: We can now remove the olo_comps pointer from
      struct pnfs_osd_layout neither the server nor client use
      it any more. (rename it to pnfs_osd_layout_hdr)

Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
---
 fs/exportfs/pnfs_osd_xdr_srv.c        |   16 +++++-----------
 include/linux/nfsd/pnfs_osd_xdr_srv.h |    6 +++++-
 2 files changed, 10 insertions(+), 12 deletions(-)

diff --git a/fs/exportfs/pnfs_osd_xdr_srv.c b/fs/exportfs/pnfs_osd_xdr_srv.c
index 67564f6..ec814c1 100644
--- a/fs/exportfs/pnfs_osd_xdr_srv.c
+++ b/fs/exportfs/pnfs_osd_xdr_srv.c
@@ -110,7 +110,7 @@ static inline int pnfs_osd_xdr_encode_objid(
  * 	struct pnfs_osd_opaque_cred	oc_cap;
  * };
  */
-static int pnfs_osd_xdr_encode_object_cred(
+int pnfs_osd_xdr_encode_layout_cred(
 	struct exp_xdr_stream *xdr,
 	struct pnfs_osd_object_cred *olo_comp)
 {
@@ -136,6 +136,7 @@ static int pnfs_osd_xdr_encode_object_cred(
 
 	return 0;
 }
+EXPORT_SYMBOL(pnfs_osd_xdr_encode_layout_cred);
 
 /*
  * struct pnfs_osd_layout {
@@ -145,12 +146,11 @@ static int pnfs_osd_xdr_encode_object_cred(
  * 	struct pnfs_osd_object_cred	*olo_comps;
  * };
  */
-int pnfs_osd_xdr_encode_layout(
+int pnfs_osd_xdr_encode_layout_hdr(
 	struct exp_xdr_stream *xdr,
 	struct pnfs_osd_layout *pol)
 {
 	__be32 *p;
-	u32 i;
 	int err;
 
 	err = pnfs_osd_xdr_encode_data_map(xdr, &pol->olo_map);
@@ -164,15 +164,9 @@ int pnfs_osd_xdr_encode_layout(
 	p = exp_xdr_encode_u32(p, pol->olo_comps_index);
 	p = exp_xdr_encode_u32(p, pol->olo_num_comps);
 
-	for (i = 0; i < pol->olo_num_comps; i++) {
-		err = pnfs_osd_xdr_encode_object_cred(xdr, &pol->olo_comps[i]);
-		if (err)
-			return err;
-	}
-
 	return 0;
 }
-EXPORT_SYMBOL(pnfs_osd_xdr_encode_layout);
+EXPORT_SYMBOL(pnfs_osd_xdr_encode_layout_hdr);
 
 static int _encode_string(struct exp_xdr_stream *xdr,
 			  const struct nfs4_string *str)
@@ -217,7 +211,7 @@ int pnfs_osd_xdr_encode_deviceaddr(
 	if (err)
 		return err;
 
-	err = pnfs_osd_xdr_encode_object_cred(xdr,
+	err = pnfs_osd_xdr_encode_layout_cred(xdr,
 					      &devaddr->oda_root_obj_cred);
 	if (err)
 		return err;
diff --git a/include/linux/nfsd/pnfs_osd_xdr_srv.h b/include/linux/nfsd/pnfs_osd_xdr_srv.h
index 256a7e07..e9a2187 100644
--- a/include/linux/nfsd/pnfs_osd_xdr_srv.h
+++ b/include/linux/nfsd/pnfs_osd_xdr_srv.h
@@ -17,10 +17,14 @@
 #include <linux/exp_xdr.h>
 
 /* Layout encoding */
-int pnfs_osd_xdr_encode_layout(
+int pnfs_osd_xdr_encode_layout_hdr(
 	struct exp_xdr_stream *xdr,
 	struct pnfs_osd_layout *layout);
 
+int pnfs_osd_xdr_encode_layout_cred(
+	struct exp_xdr_stream *xdr,
+	struct pnfs_osd_object_cred *cred);
+
 /* deviceaddr encoding */
 int pnfs_osd_xdr_encode_deviceaddr(
 	struct exp_xdr_stream *xdr, struct pnfs_osd_deviceaddr *devaddr);
-- 
1.7.3.4

[PATCH 05/10] SQUASHME: pnfsd-exofs: osd_xdr_srv: Convert XDR API to return enum nfsstat4

[Boaz Harrosh]

* Change pnfs_osd_xdr_srv API to return nfs4 errors

Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
---
 fs/exportfs/pnfs_osd_xdr_srv.c        |   30 +++++++++++++++---------------
 include/linux/nfsd/pnfs_osd_xdr_srv.h |    6 +++---
 2 files changed, 18 insertions(+), 18 deletions(-)

diff --git a/fs/exportfs/pnfs_osd_xdr_srv.c b/fs/exportfs/pnfs_osd_xdr_srv.c
index ec814c1..4d92f1c 100644
--- a/fs/exportfs/pnfs_osd_xdr_srv.c
+++ b/fs/exportfs/pnfs_osd_xdr_srv.c
@@ -51,14 +51,14 @@
  * 	u32	odm_raid_algorithm;
  * };
  */
-static int pnfs_osd_xdr_encode_data_map(
+static enum nfsstat4 pnfs_osd_xdr_encode_data_map(
 	struct exp_xdr_stream *xdr,
 	struct pnfs_osd_data_map *data_map)
 {
 	__be32 *p = exp_xdr_reserve_qwords(xdr, 1+2+1+1+1+1);
 
 	if (!p)
-		return -E2BIG;
+		return NFS4ERR_TOOSMALL;
 
 	p = exp_xdr_encode_u32(p, data_map->odm_num_comps);
 	p = exp_xdr_encode_u64(p, data_map->odm_stripe_unit);
@@ -77,7 +77,7 @@ static int pnfs_osd_xdr_encode_data_map(
  * 	u64			oid_object_id;
  * };
  */
-static inline int pnfs_osd_xdr_encode_objid(
+static inline enum nfsstat4 pnfs_osd_xdr_encode_objid(
 	struct exp_xdr_stream *xdr,
 	struct pnfs_osd_objid *object_id)
 {
@@ -86,7 +86,7 @@ static inline int pnfs_osd_xdr_encode_objid(
 		(struct nfsd4_pnfs_deviceid *)&object_id->oid_device_id;
 
 	if (!p)
-		return -E2BIG;
+		return NFS4ERR_TOOSMALL;
 
 	p = exp_xdr_encode_u64(p, dev_id->fsid);
 	p = exp_xdr_encode_u64(p, dev_id->devid);
@@ -110,12 +110,12 @@ static inline int pnfs_osd_xdr_encode_objid(
  * 	struct pnfs_osd_opaque_cred	oc_cap;
  * };
  */
-int pnfs_osd_xdr_encode_layout_cred(
+enum nfsstat4 pnfs_osd_xdr_encode_layout_cred(
 	struct exp_xdr_stream *xdr,
 	struct pnfs_osd_object_cred *olo_comp)
 {
 	__be32 *p;
-	int err;
+	enum nfsstat4 err;
 
 	err = pnfs_osd_xdr_encode_objid(xdr, &olo_comp->oc_object_id);
 	if (err)
@@ -123,7 +123,7 @@ int pnfs_osd_xdr_encode_layout_cred(
 
 	p = exp_xdr_reserve_space(xdr, 3*4 + 4+olo_comp->oc_cap.cred_len);
 	if (!p)
-		return -E2BIG;
+		return NFS4ERR_TOOSMALL;
 
 	p = exp_xdr_encode_u32(p, olo_comp->oc_osd_version);
 
@@ -146,12 +146,12 @@ EXPORT_SYMBOL(pnfs_osd_xdr_encode_layout_cred);
  * 	struct pnfs_osd_object_cred	*olo_comps;
  * };
  */
-int pnfs_osd_xdr_encode_layout_hdr(
+enum nfsstat4 pnfs_osd_xdr_encode_layout_hdr(
 	struct exp_xdr_stream *xdr,
 	struct pnfs_osd_layout *pol)
 {
 	__be32 *p;
-	int err;
+	enum nfsstat4 err;
 
 	err = pnfs_osd_xdr_encode_data_map(xdr, &pol->olo_map);
 	if (err)
@@ -159,7 +159,7 @@ int pnfs_osd_xdr_encode_layout_hdr(
 
 	p = exp_xdr_reserve_qwords(xdr, 2);
 	if (!p)
-		return -E2BIG;
+		return NFS4ERR_TOOSMALL;
 
 	p = exp_xdr_encode_u32(p, pol->olo_comps_index);
 	p = exp_xdr_encode_u32(p, pol->olo_num_comps);
@@ -168,13 +168,13 @@ int pnfs_osd_xdr_encode_layout_hdr(
 }
 EXPORT_SYMBOL(pnfs_osd_xdr_encode_layout_hdr);
 
-static int _encode_string(struct exp_xdr_stream *xdr,
+static enum nfsstat4 _encode_string(struct exp_xdr_stream *xdr,
 			  const struct nfs4_string *str)
 {
 	__be32 *p = exp_xdr_reserve_space(xdr, 4 + str->len);
 
 	if (!p)
-		return -E2BIG;
+		return NFS4ERR_TOOSMALL;
 	exp_xdr_encode_opaque(p, str->data, str->len);
 	return 0;
 }
@@ -188,15 +188,15 @@ static int _encode_string(struct exp_xdr_stream *xdr,
  * 	struct nfs4_string		oda_osdname;
  * };
  */
-int pnfs_osd_xdr_encode_deviceaddr(
+enum nfsstat4 pnfs_osd_xdr_encode_deviceaddr(
 	struct exp_xdr_stream *xdr, struct pnfs_osd_deviceaddr *devaddr)
 {
 	__be32 *p;
-	int err;
+	enum nfsstat4 err;
 
 	p = exp_xdr_reserve_space(xdr, 4 + 4 + sizeof(devaddr->oda_lun));
 	if (!p)
-		return -E2BIG;
+		return NFS4ERR_TOOSMALL;
 
 	/* Empty oda_targetid */
 	p = exp_xdr_encode_u32(p, OBJ_TARGET_ANON);
diff --git a/include/linux/nfsd/pnfs_osd_xdr_srv.h b/include/linux/nfsd/pnfs_osd_xdr_srv.h
index e9a2187..2909fbe 100644
--- a/include/linux/nfsd/pnfs_osd_xdr_srv.h
+++ b/include/linux/nfsd/pnfs_osd_xdr_srv.h
@@ -17,16 +17,16 @@
 #include <linux/exp_xdr.h>
 
 /* Layout encoding */
-int pnfs_osd_xdr_encode_layout_hdr(
+enum nfsstat4 pnfs_osd_xdr_encode_layout_hdr(
 	struct exp_xdr_stream *xdr,
 	struct pnfs_osd_layout *layout);
 
-int pnfs_osd_xdr_encode_layout_cred(
+enum nfsstat4 pnfs_osd_xdr_encode_layout_cred(
 	struct exp_xdr_stream *xdr,
 	struct pnfs_osd_object_cred *cred);
 
 /* deviceaddr encoding */
-int pnfs_osd_xdr_encode_deviceaddr(
+enum nfsstat4 pnfs_osd_xdr_encode_deviceaddr(
 	struct exp_xdr_stream *xdr, struct pnfs_osd_deviceaddr *devaddr);
 
 /* layout_commit decoding */
-- 
1.7.3.4

[PATCH 06/10] SQUASHME: pnfsd-exofs: Server xdr definisions were moved to a separate header

[Boaz Harrosh]

Include from export.c the server XDR header

Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
---
 fs/exofs/export.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/fs/exofs/export.c b/fs/exofs/export.c
index c8a7d46..20969bf 100644
--- a/fs/exofs/export.c
+++ b/fs/exofs/export.c
@@ -27,6 +27,8 @@
 #include <linux/nfsd/nfsd4_pnfs.h>
 #include "exofs.h"
 
+#include "../exportfs/pnfs_osd_xdr_srv.h"
+
 static int exofs_layout_type(struct super_block *sb)
 {
 	return LAYOUT_OSD2_OBJECTS;
-- 
1.7.3.4

[PATCH 07/10] SQUASHME: pnfsd-exofs: osd_xdr_srv header moved

[Boaz Harrosh]

Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
---
 fs/exofs/export.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/fs/exofs/export.c b/fs/exofs/export.c
index 20969bf..616cf5d 100644
--- a/fs/exofs/export.c
+++ b/fs/exofs/export.c
@@ -27,7 +27,7 @@
 #include <linux/nfsd/nfsd4_pnfs.h>
 #include "exofs.h"
 
-#include "../exportfs/pnfs_osd_xdr_srv.h"
+#include "linux/nfsd/pnfs_osd_xdr_srv.h"
 
 static int exofs_layout_type(struct super_block *sb)
 {
-- 
1.7.3.4

[PATCH 08/10] SQUASHME: pnfsd-exofs: layout encoding was Changed to two pass

[Boaz Harrosh]

This way there is No need for dynamic allocation of the creds
array.

Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
---
 fs/exofs/export.c |   55 ++++++++++++++++++++++++++---------------------------
 1 files changed, 27 insertions(+), 28 deletions(-)

diff --git a/fs/exofs/export.c b/fs/exofs/export.c
index 616cf5d..14852d8 100644
--- a/fs/exofs/export.c
+++ b/fs/exofs/export.c
@@ -70,44 +70,43 @@ static enum nfsstat4 exofs_layout_get(
 		goto out;
 	}
 
-	creds = kcalloc(el->s_numdevs, sizeof(*creds), GFP_KERNEL);
-	if (!creds) {
-		nfserr = NFS4ERR_LAYOUTTRYLATER;
-		goto out;
-	}
-
 	/* Fill in a pnfs_osd_layout struct */
 	layout.olo_map = sbi->data_map;
+	layout.olo_comps_index = 0;
+	layout.olo_num_comps = el->s_numdevs;
+	layout.olo_comps = creds;
+
+	err = pnfs_osd_xdr_encode_layout_hdr(xdr, &layout);
+	if (err) {
+		nfserr = NFS4ERR_TOOSMALL;
+		goto out;
+	}
 
+	/* Encode layout components */
 	for (i = 0; i < el->s_numdevs; i++) {
-		struct pnfs_osd_object_cred *cred = &creds[i];
+		struct pnfs_osd_object_cred cred;
 		osd_id id = exofs_oi_objno(oi);
 		unsigned dev = exofs_layout_od_id(el, id, i);
 
-		set_dev_id(&cred->oc_object_id.oid_device_id, args->lg_sbid,
+		set_dev_id(&cred.oc_object_id.oid_device_id, args->lg_sbid,
 			   dev);
-		cred->oc_object_id.oid_partition_id = el->s_pid;
-		cred->oc_object_id.oid_object_id = id;
-		cred->oc_osd_version = osd_dev_is_ver1(el->s_ods[dev]) ?
+		cred.oc_object_id.oid_partition_id = el->s_pid;
+		cred.oc_object_id.oid_object_id = id;
+		cred.oc_osd_version = osd_dev_is_ver1(el->s_ods[dev]) ?
 						PNFS_OSD_VERSION_1 :
 						PNFS_OSD_VERSION_2;
-		cred->oc_cap_key_sec = PNFS_OSD_CAP_KEY_SEC_NONE;
-
-		cred->oc_cap_key.cred_len	= 0;
-		cred->oc_cap_key.cred		= NULL;
+		cred.oc_cap_key_sec = PNFS_OSD_CAP_KEY_SEC_NONE;
 
-		cred->oc_cap.cred_len	= OSD_CAP_LEN;
-		cred->oc_cap.cred	= oi->i_cred;
-	}
-
-	layout.olo_comps_index = 0;
-	layout.olo_num_comps = el->s_numdevs;
-	layout.olo_comps = creds;
+		cred.oc_cap_key.cred_len	= 0;
+		cred.oc_cap_key.cred		= NULL;
 
-	err = pnfs_osd_xdr_encode_layout(xdr, &layout);
-	if (err) {
-		nfserr = NFS4ERR_TOOSMALL; /* FIXME: Change osd_xdr error codes */
-		goto out;
+		cred.oc_cap.cred_len	= OSD_CAP_LEN;
+		cred.oc_cap.cred	= oi->i_cred;
+		err = pnfs_osd_xdr_encode_layout_cred(xdr, &cred);
+		if (err) {
+			nfserr = NFS4ERR_TOOSMALL;
+			goto out;
+		}
 	}
 
 	exp_xdr_encode_opaque_len(start, xdr->p);
-- 
1.7.3.4

[PATCH 09/10] SQUASHME: pnfsd-exofs: Convert XDR API to return enum nfsstat4

[Boaz Harrosh]

Remove error translation from export.c, now that XDR API returns
the proper constants

Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
---
 fs/exofs/export.c |   20 +++++++++-----------
 1 files changed, 9 insertions(+), 11 deletions(-)

diff --git a/fs/exofs/export.c b/fs/exofs/export.c
index 88be441..c153ad6 100644
--- a/fs/exofs/export.c
+++ b/fs/exofs/export.c
@@ -86,7 +86,7 @@ static enum nfsstat4 exofs_layout_get(
 	struct pnfs_osd_layout layout;
 	__be32 *start;
 	bool in_recall;
-	int i, err;
+	int i;
 	enum nfsstat4 nfserr;
 
 	res->lg_seg.offset = 0;
@@ -107,11 +107,9 @@ static enum nfsstat4 exofs_layout_get(
 	layout.olo_num_comps = el->s_numdevs;
 	layout.olo_comps = creds;
 
-	err = pnfs_osd_xdr_encode_layout_hdr(xdr, &layout);
-	if (err) {
-		nfserr = NFS4ERR_TOOSMALL;
+	nfserr = pnfs_osd_xdr_encode_layout_hdr(xdr, &layout);
+	if (unlikely(nfserr))
 		goto out;
-	}
 
 	/* Encode layout components */
 	for (i = 0; i < el->s_numdevs; i++) {
@@ -133,11 +131,9 @@ static enum nfsstat4 exofs_layout_get(
 
 		cred.oc_cap.cred_len	= OSD_CAP_LEN;
 		cred.oc_cap.cred	= oi->i_cred;
-		err = pnfs_osd_xdr_encode_layout_cred(xdr, &cred);
-		if (err) {
-			nfserr = NFS4ERR_TOOSMALL;
+		nfserr = pnfs_osd_xdr_encode_layout_cred(xdr, &cred);
+		if (unlikely(nfserr))
 			goto out;
-		}
 	}
 
 	exp_xdr_encode_opaque_len(start, xdr->p);
@@ -303,13 +299,15 @@ int exofs_get_device_info(struct super_block *sb, struct exp_xdr_stream *xdr,
 	/* skip opaque size, will be filled-in later */
 	start = exp_xdr_reserve_qwords(xdr, 1);
 	if (!start) {
-		err = -E2BIG;
+		err = -ETOOSMALL;
 		goto err;
 	}
 
 	err = pnfs_osd_xdr_encode_deviceaddr(xdr, &devaddr);
-	if (err)
+	if (err) {
+		err = -ETOOSMALL;
 		goto err;
+	}
 
 	exp_xdr_encode_opaque_len(start, xdr->p);
 
-- 
1.7.3.4

[PATCH 10/10] SQUASHME: pnfsd-exofs: checkpatch love and a dprint

[Boaz Harrosh]

Singed-off-by: Boaz Harrosh <bharrosh@panasas.com>

---
diff --git a/fs/exofs/export.c b/fs/exofs/export.c
index c153ad6..8281c85 100644
--- a/fs/exofs/export.c
+++ b/fs/exofs/export.c
@@ -85,8 +85,8 @@ static enum nfsstat4 exofs_layout_get(
 	struct pnfs_osd_object_cred *creds = NULL;
 	struct pnfs_osd_layout layout;
 	__be32 *start;
+	unsigned i;
 	bool in_recall;
-	int i;
 	enum nfsstat4 nfserr;
 
 	res->lg_seg.offset = 0;
@@ -285,8 +285,11 @@ int exofs_get_device_info(struct super_block *sb, struct exp_xdr_stream *xdr,
 
 	memset(&devaddr, 0, sizeof(devaddr));
 
-	if (unlikely(devno >= sbi->layout.s_numdevs))
+	if (unlikely(devno >= sbi->layout.s_numdevs)) {
+		EXOFS_DBGMSG("Error: Device((%llx,%llx) does not exist\n",
+			     devid->sbid, devno);
 		return -ENODEV;
+	}
 
 	odi = osduld_device_info(sbi->layout.s_ods[devno]);
 
diff --git a/fs/exportfs/pnfs_osd_xdr_srv.c b/fs/exportfs/pnfs_osd_xdr_srv.c
index 01b0e53..12a3bda 100644
--- a/fs/exportfs/pnfs_osd_xdr_srv.c
+++ b/fs/exportfs/pnfs_osd_xdr_srv.c
@@ -43,12 +43,12 @@
 
 /*
  * struct pnfs_osd_data_map {
- * 	u32	odm_num_comps;
- * 	u64	odm_stripe_unit;
- * 	u32	odm_group_width;
- * 	u32	odm_group_depth;
- * 	u32	odm_mirror_cnt;
- * 	u32	odm_raid_algorithm;
+ *	u32	odm_num_comps;
+ *	u64	odm_stripe_unit;
+ *	u32	odm_group_width;
+ *	u32	odm_group_depth;
+ *	u32	odm_mirror_cnt;
+ *	u32	odm_raid_algorithm;
  * };
  */
 static enum nfsstat4 pnfs_osd_xdr_encode_data_map(
@@ -72,9 +72,9 @@ static enum nfsstat4 pnfs_osd_xdr_encode_data_map(
 
 /*
  * struct pnfs_osd_objid {
- * 	struct nfs4_deviceid	oid_device_id;
- * 	u64			oid_partition_id;
- * 	u64			oid_object_id;
+ *	struct nfs4_deviceid	oid_device_id;
+ *	u64			oid_partition_id;
+ *	u64			oid_object_id;
  * };
  */
 static inline enum nfsstat4 pnfs_osd_xdr_encode_objid(
@@ -98,16 +98,16 @@ static inline enum nfsstat4 pnfs_osd_xdr_encode_objid(
 
 /*
  * enum pnfs_osd_cap_key_sec4 {
- * 	PNFS_OSD_CAP_KEY_SEC_NONE = 0,
- * 	PNFS_OSD_CAP_KEY_SEC_SSV  = 1
+ *	PNFS_OSD_CAP_KEY_SEC_NONE = 0,
+ *	PNFS_OSD_CAP_KEY_SEC_SSV  = 1
  * };
  *
  * struct pnfs_osd_object_cred {
- * 	struct pnfs_osd_objid		oc_object_id;
- * 	u32				oc_osd_version;
- * 	u32				oc_cap_key_sec;
- * 	struct pnfs_osd_opaque_cred	oc_cap_key
- * 	struct pnfs_osd_opaque_cred	oc_cap;
+ *	struct pnfs_osd_objid		oc_object_id;
+ *	u32				oc_osd_version;
+ *	u32				oc_cap_key_sec;
+ *	struct pnfs_osd_opaque_cred	oc_cap_key
+ *	struct pnfs_osd_opaque_cred	oc_cap;
  * };
  */
 enum nfsstat4 pnfs_osd_xdr_encode_layout_cred(
@@ -140,10 +140,10 @@ EXPORT_SYMBOL(pnfs_osd_xdr_encode_layout_cred);
 
 /*
  * struct pnfs_osd_layout {
- * 	struct pnfs_osd_data_map	olo_map;
- * 	u32				olo_comps_index;
- * 	u32				olo_num_comps;
- * 	struct pnfs_osd_object_cred	*olo_comps;
+ *	struct pnfs_osd_data_map	olo_map;
+ *	u32				olo_comps_index;
+ *	u32				olo_num_comps;
+ *	struct pnfs_osd_object_cred	*olo_comps;
  * };
  */
 enum nfsstat4 pnfs_osd_xdr_encode_layout_hdr(
@@ -180,12 +180,12 @@ static enum nfsstat4 _encode_string(struct exp_xdr_stream *xdr,
 }
 
 /* struct pnfs_osd_deviceaddr {
- * 	struct pnfs_osd_targetid	oda_targetid;
- * 	struct pnfs_osd_targetaddr	oda_targetaddr;
- * 	u8				oda_lun[8];
- * 	struct nfs4_string		oda_systemid;
- * 	struct pnfs_osd_object_cred	oda_root_obj_cred;
- * 	struct nfs4_string		oda_osdname;
+ *	struct pnfs_osd_targetid	oda_targetid;
+ *	struct pnfs_osd_targetaddr	oda_targetaddr;
+ *	u8				oda_lun[8];
+ *	struct nfs4_string		oda_systemid;
+ *	struct pnfs_osd_object_cred	oda_root_obj_cred;
+ *	struct nfs4_string		oda_osdname;
  * };
  */
 enum nfsstat4 pnfs_osd_xdr_encode_deviceaddr(
@@ -226,9 +226,9 @@ EXPORT_SYMBOL(pnfs_osd_xdr_encode_deviceaddr);
 
 /*
  * struct pnfs_osd_layoutupdate {
- * 	u32	dsu_valid;
- * 	s64	dsu_delta;
- * 	u32	olu_ioerr_flag;
+ *	u32	dsu_valid;
+ *	s64	dsu_delta;
+ *	u32	olu_ioerr_flag;
  * };
  */
 __be32 *
@@ -244,9 +244,9 @@ EXPORT_SYMBOL(pnfs_osd_xdr_decode_layoutupdate);
 
 /*
  * struct pnfs_osd_objid {
- * 	struct nfs4_deviceid	oid_device_id;
- * 	u64			oid_partition_id;
- * 	u64			oid_object_id;
+ *	struct nfs4_deviceid	oid_device_id;
+ *	u64			oid_partition_id;
+ *	u64			oid_object_id;
  * }; xdr size 32
  */
 static inline __be32 *
@@ -263,11 +263,11 @@ pnfs_osd_xdr_decode_objid(__be32 *p, struct pnfs_osd_objid *objid)
 
 /*
  * struct pnfs_osd_ioerr {
- * 	struct pnfs_osd_objid	oer_component;
- * 	u64			oer_comp_offset;
- * 	u64			oer_comp_length;
- * 	u32			oer_iswrite;
- * 	u32			oer_errno;
+ *	struct pnfs_osd_objid	oer_component;
+ *	u64			oer_comp_offset;
+ *	u64			oer_comp_length;
+ *	u32			oer_iswrite;
+ *	u32			oer_errno;
  * }; xdr size 32 + 24
  */
 bool pnfs_osd_xdr_decode_ioerr(struct pnfs_osd_ioerr *ioerr,
diff --git a/include/linux/nfsd/pnfs_osd_xdr_srv.h b/include/linux/nfsd/pnfs_osd_xdr_srv.h
index 2909fbe..fab244e 100644
--- a/include/linux/nfsd/pnfs_osd_xdr_srv.h
+++ b/include/linux/nfsd/pnfs_osd_xdr_srv.h
@@ -30,7 +30,7 @@ enum nfsstat4 pnfs_osd_xdr_encode_deviceaddr(
 	struct exp_xdr_stream *xdr, struct pnfs_osd_deviceaddr *devaddr);
 
 /* layout_commit decoding */
-__be32 * pnfs_osd_xdr_decode_layoutupdate(
+__be32 *pnfs_osd_xdr_decode_layoutupdate(
 	struct pnfs_osd_layoutupdate *lou, __be32 *p);
 
 /* layout_return decoding */

Re: [PATCHSET 0/8] pnfsd-exofs: New set of exofs pnfs export patches

[Benny Halevy]

OK, I took this in under pnfs-all-3.0-rc3-2011-06-14-2
The new tree structure is:

master (at nfs-2.6/nfs-for-bakeathon, based off of v3.0-rc3)
	nfs-upstream
		pnfs-block
			pnfs-obj
	nfsd-upstream
		nfsd41-all
			pnfsd-all
				pnfsd-exofs-all
					pnfsd-lexp-all
						spnfs-all
							spnfs-block-all
								pnfs-all-latest
								+pnfs-obj (that's a merge :)

Benny

On 2011-06-14 17:00, Benny Halevy wrote:
> On 2011-06-14 16:53, Boaz Harrosh wrote:
>>
>> Benny hi
>>
>> I'm submitting a new SQUASHED set of exofs patches. These should completely
>> replace the patches you have now in the pnfsd-exofs{-all} branch
>>
> 
> Great, thanks!
> 
>> If anyone is curios I'll send a SQUASHME patchset as reply to this mail.
>>
>> I Think the pnfsd-exofs branch should advance down to before the pnfsd-exp-all.
>> It should be the first or second candidate for inclusion.
> 
> makes sense
> 
>>
>> BTW: You forgot to branch the pnfs-blocks branch last time.
> 
> Already fixed... (pnfs-block is there)
> 
>>
>> I think that now, since most of the client code is in-tree and all the common code
>> is in. It would be easier to work with a tree that has two separate branches for the
>> 1.block-layout and 2.server branches. And the pnfs-all-latest which merges the two.
>>
>> something like:
>>
>>
>>                 |----pnfsd(including gfs2_---pnfsd-exofs---pnfsd-exp----pnfsd-spnfs----pnfsd-spnfs-blocks ---|
>> nfs-linux-next--|                                                                                            + --- pnfs-all-latest
>>                 | --------pnfs-blocks------------------------------------------------------------------------|
> 
> Good point, I'll try it out.
> 
>>
>> * "+" means a merge
>>
>> It will be much easier to develop this way
>>
>> usual list of patches:
>>
>> [PATCH 1/8] nfs_xdr: Move nfs4_string definition out of #ifdef CONFIG_NFS_V4
>> [PATCH 2/8] exofs: pnfs-tree: Remove pnfs-osd private definitions
>> [PATCH 3/8] pnfs-obj: pnfs_osd_xdr: Remove dead code and cleanup
>>  
>>     Intended for upstream
>>
>> [PATCH 4/8] pnfsd-exofs: pnfs objects server XDR functions
>> [PATCH 5/8] pnfsd-exofs: Add pNFS export support
>> [PATCH 6/8] pnfsd-exofs: get_device_info implementation.
>> [PATCH 7/8] pnfsd-exofs: Recall layouts on truncate
>> [PATCH 8/8] pnfsd-exofs: layoutreturn pnfs-obj information decoding
>>
>>    New exofs pnfs export patches
> 
> Where are the patches? :)
> 
> Benny
> 
>>
>> Thanks
>> Boaz
>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html