(unknown),

(unknown),

[J. Bruce Fields]

J. Bruce Fields <bfields@fieldses.org> wrote:
> OK, good.  I'll revise and post a new series.  (Do people prefer
> another mailbomb or a git url?)

OK, I went for the former; if you'd rather get this out of git, you can

	git clone http://www.linux-nfs.org/~bfields/linux.git
	git checkout server-cluster-lease-api

The changes from the last version seem pretty trivial, but I've
compile-tested this only for now.

I'm ignoring the problem of breaking leases on unlink and rename.  I
think we should go ahead and do this part now--it's adequate for the
current lease semantics, and even more so for our current application
(just turning off leases selectively on some filesystems)--but I'd
really like to solve that problem eventually.

That's probably not going to happen until we get a cluster filesystem
with real lease support into the kernel....

Changes:
	- do away with the break_lease method.
	- rename __setlease to setlease, setlease to vfs_setlease, and
	  make sure it's setlease (the one that doesn't call into the
	  filesystem) that's exported.
	- rename ->set_lease to ->setlease.  (I don't really care which
	  we go with, it just seemed confusing when everything else was
	  already named without the underscore.)
	- Add a trivial patch that disables leases on nfs (as suggested
	  by a patch elsewhere from Peter Staubach)

--b.

(unknown)

[George Cameron]

subscribe linux-nfs

(unknown),

[Sagar Borikar]

unsubscribe

(unknown)

[Linus Torvalds]

Trond?

See 'http://lkml.org/lkml/2008/7/17/154'? It's been 10+ days, it's 
apparently still there.

		Linus

(unknown)

[Eric Sesterhenn]

hi,

with todays -git i see the following lockdep warning, that wasnt there
yesterday. It occurs after booting the box.

[   83.741022] Installing knfsd (copyright (C) 1996 okir-pn4DOG8n3UYbFoVRYvo4fw@public.gmane.org).
[   84.115838] NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state
recovery directory
[   84.133473] NFSD: starting 90-second grace period
[  174.132715] 
[  174.132724] =============================================
[  174.133004] [ INFO: possible recursive locking detected ]
[  174.133133] 2.6.28 #85
[  174.133232] ---------------------------------------------
[  174.133428] nfsd4/3693 is trying to acquire lock:
[  174.133548]  (&type->i_mutex_dir_key#2){--..}, at: [<c01c4212>]
vfs_fsync+0x62/0xe0
[  174.133991] 
[  174.133994] but task is already holding lock:
[  174.134251]  (&type->i_mutex_dir_key#2){--..}, at: [<d1c224f7>]
nfsd4_sync_rec_dir+0x17/0x40 [nfsd]
[  174.134774] 
[  174.134777] other info that might help us debug this:
[  174.134983] 4 locks held by nfsd4/3693:
[  174.135151]  #0:  (nfsd4){--..}, at: [<c0139d4a>]
run_workqueue+0x7a/0x1e0
[  174.135539]  #1:  ((laundromat_work).work){--..}, at: [<c0139d4a>]
run_workqueue+0x7a/0x1e0
[  174.135968]  #2:  (client_mutex){--..}, at: [<d1c1e9f6>]
laundromat_main+0x26/0x2a0 [nfsd]
[  174.136021]  #3:  (&type->i_mutex_dir_key#2){--..}, at: [<d1c224f7>]
nfsd4_sync_rec_dir+0x17/0x40 [nfsd]
[  174.136021] 
[  174.136021] stack backtrace:
[  174.136021] Pid: 3693, comm: nfsd4 Not tainted 2.6.28 #85
[  174.136021] Call Trace:
[  174.136021]  [<c05abdb6>] ? printk+0x18/0x1a
[  174.136021]  [<c014f626>] __lock_acquire+0xe76/0x1110
[  174.136021]  [<c014f934>] lock_acquire+0x74/0xa0
[  174.136021]  [<c01c4212>] ? vfs_fsync+0x62/0xe0
[  174.136021]  [<c05acfac>] mutex_lock_nested+0x8c/0x2c0
[  174.136021]  [<c01c4212>] ? vfs_fsync+0x62/0xe0
[  174.136021]  [<c01c4212>] ? vfs_fsync+0x62/0xe0
[  174.136021]  [<c01c4212>] vfs_fsync+0x62/0xe0
[  174.136021]  [<d1c074be>] nfsd_sync_dir+0xe/0x10 [nfsd]
[  174.136021]  [<d1c22501>] nfsd4_sync_rec_dir+0x21/0x40 [nfsd]
[  174.136021]  [<d1c22595>] nfsd4_recdir_purge_old+0x75/0x80 [nfsd]
[  174.136021]  [<d1c1ea1e>] laundromat_main+0x4e/0x2a0 [nfsd]
[  174.136021]  [<c0139d4a>] ? run_workqueue+0x7a/0x1e0
[  174.136021]  [<c0139dac>] run_workqueue+0xdc/0x1e0
[  174.136021]  [<c0139d4a>] ? run_workqueue+0x7a/0x1e0
[  174.136021]  [<d1c1e9d0>] ? laundromat_main+0x0/0x2a0 [nfsd]
[  174.136021]  [<c013a1a7>] worker_thread+0x87/0xf0
[  174.136021]  [<c013dac0>] ? autoremove_wake_function+0x0/0x50
[  174.136021]  [<c013a120>] ? worker_thread+0x0/0xf0
[  174.136021]  [<c013d80a>] kthread+0x3a/0x70
[  174.136021]  [<c013d7d0>] ? kthread+0x0/0x70
[  174.136021]  [<c0103cf3>] kernel_thread_helper+0x7/0x14

Greetings, Eric

(unknown)

[vivianofferplc013-+MseY0pMZYIAvxtiuMwx3w]

This is a Financial Service Announcement, we offer loan to all in need,ranging from $5000 to $800,000.00 USD. Our interest rate is 3% and our service and terms are dependable. any interested person should apply via email:lapoloanlender@gmail.com

(unknown),

[andros]

All layout drivers returned -1 or 0 for both get_read_threshold and
get_write_threshold policy operations. Remove the threshold policy ops and
the related helper functions.

Note that the threshold

0001-SQUASHME-pnfs_submit-remove-filelayout-get-threshold.patch
0002-SQUASHME-pnfs_submit-don-t-use-pnfs_use_read.patch
0003-SQUASHME-pnfs_submit-don-t-use-pnfs_use_write.patch
0004-SQUASHME-pnfs_submit-don-t-use-pnfs_use_write-for-co.patch
0005-SQUASHME-pnfs_submit-don-t-use-pg_threshold.patch
0006-SQUASHME-pnfs_submit-remove-pg_threshold.patch
0007-SQUASHME-pnfs_submit-don-t-use-below_threshold-in-re.patch
0008-SQUASHME-pnfs_submit-don-t-use-below_threshold-in-di.patch
0009-SQUASHME-pnfs_submit-remove-unused-threshold-calcula.patch
0010-SQUASHME-pnfs_submit-cleanup-filelayout-pg_test.patch

Note: these will not be restored.

Testing:
--------
CONFIG_NFS4_V1 set:
v4, v4.1, pNFS mounts Connectathon passes

CONFIG_NFS4_V1 not set:
v4 mount Connectathon passes

-->Andy

(unknown)

[Maurice Volaski]

subscribe nfs

(unknown),

[andros]

This is against the pnfs-submit branch of the 2.6.34 tree. They will need to be
applied against the 2.6.35-rc1 tree which I can do after comments.

RFC: I would like comments, especially on
0006-SQUASHME-pnfs-submit-move-layoutcommit-to-nfs_write_.patch.

Remove unused layoutcommit layoutdriver_io_operations. Will be restored
in post-submit patches
0001-SQUASHME-pnfs-submit-remove-setup_layoutcommit.patch
0002-SQUASHNE-pnfs-submit-remove-cleanup_layoutcommit.patch
0003-SQUASHME-pnfs-submit-remove-encode_layoutcommit.patch

A cleanup, and call the async error handler.
0004-SQUASHME-pnfs-submit-cleanup-layoutcommit-call.patch
0005-SQUASHME-pnfs-submit-handle-async-layoutcommit-error.patch

This next  patch moves the pnfs_layoutcommit_inode call to nfs_write_inode,
and it is the only call other than in layoutreturn. (removed calls in
__nfs4_close, nfs_commit_inode, nfs_wb_sync).

This is fine for the file layout, and I think it's OK for the object and
block layouts as well.

I left the LAYOUTCOMMIT call in nfs_write_inode a synchronous call, because
nfs_commit_unstable_pages sets the FLUSH_SYNC flag. Should this
be an asyc LAYOUTCOMMIT call?

pnfs_layoutcommit_inode is called after nfs_commit_unstable_pages() so that
if LAYOUTCOMMIT fails, the unstable pages have been processed..

The error handlers (sync and async) call nfs4_map_errors, so unhandled
errors (such as NFS4ERR_BADLAYOUT) get returned to nfs_write_ioode as -EIO.

Examining the write_inode call paths, I could not see where the -EIO would
be passed back to the application.  Testing with pynfs which I
had return NFS4ERR_BADLAYOUT to the layout commit call, shows the -EIO return
not stopping the client nor is the error reported back to the application.

We will add code to the error handlers for errors such as NFS4ERR_BADLAYOUT 
that require us to stop using and free the layout, and redo the I/O through
the MDS.

Anyway, review is much appreciated.

0006-SQUASHME-pnfs-submit-move-layoutcommit-to-nfs_write_.patch

Testing:
With CONFIG_NFS_V4_1 set
NFSv4.1/pnfs passed Connectathon against write enabled GFS2/pNFS. Note: there
were exactly the same number of LAYOUTCOMMITS sent as were sent with
pnfs_layoutcommit_inode being called from __nfs4_close (never happened),
nfs_commit_inode and nfs_wb_sync.

Passed Connectathon general test against pynfs file layout server with
the NFS4ERR_BADLAYOUT being returned on every third LAYOUTCOMMIT.


-->Andy

[PATCH 1/6] SQUASHME pnfs-submit: remove setup_layoutcommit

[andros]

From: Andy Adamson <andros@netapp.com>

Not used by the file layout driver

Signed-off-by: Andy Adamson <andros@netapp.com>
---
 fs/nfs/pnfs.c             |    7 -------
 include/linux/nfs4_pnfs.h |    2 --
 2 files changed, 0 insertions(+), 9 deletions(-)

diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c
index 8cc4412..23787ff 100644
--- a/fs/nfs/pnfs.c
+++ b/fs/nfs/pnfs.c
@@ -1944,7 +1944,6 @@ pnfs_layoutcommit_setup(struct inode *inode,
 			struct pnfs_layoutcommit_data *data,
 			loff_t write_begin_pos, loff_t write_end_pos, int sync)
 {
-	struct nfs_inode *nfsi = NFS_I(inode);
 	struct nfs_server *nfss = NFS_SERVER(inode);
 	int result = 0;
 
@@ -1970,12 +1969,6 @@ pnfs_layoutcommit_setup(struct inode *inode,
 	data->args.bitmask = nfss->attr_bitmask;
 	data->res.server = nfss;
 
-	/* Call layout driver to set the arguments.
-	 */
-	if (nfss->pnfs_curr_ld->ld_io_ops->setup_layoutcommit)
-		result = nfss->pnfs_curr_ld->ld_io_ops->setup_layoutcommit(
-				&nfsi->layout, &data->args);
-
 	dprintk("%s End Status %d\n", __func__, result);
 	return result;
 }
diff --git a/include/linux/nfs4_pnfs.h b/include/linux/nfs4_pnfs.h
index 84d2e95..cab9137 100644
--- a/include/linux/nfs4_pnfs.h
+++ b/include/linux/nfs4_pnfs.h
@@ -146,8 +146,6 @@ struct layoutdriver_io_operations {
 	struct pnfs_layout_segment * (*alloc_lseg) (struct pnfs_layout_type *layoutid, struct nfs4_pnfs_layoutget_res *lgr);
 	void (*free_lseg) (struct pnfs_layout_segment *lseg);
 
-	int (*setup_layoutcommit) (struct pnfs_layout_type *layoutid,
-				    struct pnfs_layoutcommit_arg *args);
 	void (*encode_layoutcommit) (struct pnfs_layout_type *layoutid,
 				     struct xdr_stream *xdr,
 				     const struct pnfs_layoutcommit_arg *args);
-- 
1.6.6

[PATCH 2/6] SQUASHNE pnfs-submit: remove cleanup_layoutcommit

[andros]

From: Andy Adamson <andros@netapp.com>

Not used by the file layout driver

Signed-off-by: Andy Adamson <andros@netapp.com>
---
 fs/nfs/nfs4proc.c         |    4 ++--
 fs/nfs/pnfs.c             |   24 ------------------------
 fs/nfs/pnfs.h             |    1 -
 include/linux/nfs4_pnfs.h |    3 ---
 4 files changed, 2 insertions(+), 30 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 8b375a7..2826ada 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -5600,7 +5600,7 @@ static int pnfs4_sync_layoutcommit(struct pnfs_layoutcommit_data *data)
 	} while (exception.retry);
 
 	data->status = err;
-	pnfs_layoutcommit_done(data);
+	put_rpccred(data->cred);
 	pnfs_layoutcommit_free(data);
 	return err;
 }
@@ -5625,7 +5625,7 @@ pnfs_layoutcommit_rpc_done(struct rpc_task *task, void *calldata)
 	struct nfs_server *server = NFS_SERVER(data->args.inode);
 
 	data->status = task->tk_status;
-	pnfs_layoutcommit_done(data);
+	put_rpccred(data->cred);
 
 	nfs4_sequence_done(server, &data->res.seq_res, task->tk_status);
 }
diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c
index 23787ff..def27f8 100644
--- a/fs/nfs/pnfs.c
+++ b/fs/nfs/pnfs.c
@@ -1912,30 +1912,6 @@ out:
 	return trypnfs;
 }
 
-/* Called on completion of layoutcommit */
-void
-pnfs_layoutcommit_done(struct pnfs_layoutcommit_data *data)
-{
-	struct nfs_server *nfss = NFS_SERVER(data->args.inode);
-	struct nfs_inode *nfsi = NFS_I(data->args.inode);
-
-	dprintk("%s: (status %d)\n", __func__, data->status);
-
-	if (data->status < 0)
-		printk(KERN_ERR "%s, Layoutcommit Failed! = %d\n",
-		       __func__, data->status);
-
-	/* TODO: Maybe we should avoid this by allowing the layout driver
-	 * to directly xdr its layout on the wire.
-	 */
-	if (nfss->pnfs_curr_ld->ld_io_ops->cleanup_layoutcommit)
-		nfss->pnfs_curr_ld->ld_io_ops->cleanup_layoutcommit(
-							&nfsi->layout,
-							&data->args,
-							data->status);
-	put_rpccred(data->cred);
-}
-
 /*
  * Set up the argument/result storage required for the RPC call.
  */
diff --git a/fs/nfs/pnfs.h b/fs/nfs/pnfs.h
index c89be78..43e02f8 100644
--- a/fs/nfs/pnfs.h
+++ b/fs/nfs/pnfs.h
@@ -46,7 +46,6 @@ enum pnfs_try_status _pnfs_try_to_read_data(struct nfs_read_data *,
 					    const struct rpc_call_ops *);
 int pnfs_initialize(void);
 void pnfs_uninitialize(void);
-void pnfs_layoutcommit_done(struct pnfs_layoutcommit_data *data);
 void pnfs_layoutcommit_free(struct pnfs_layoutcommit_data *data);
 int pnfs_layoutcommit_inode(struct inode *inode, int sync);
 void pnfs_update_last_write(struct nfs_inode *nfsi, loff_t offset, size_t extent);
diff --git a/include/linux/nfs4_pnfs.h b/include/linux/nfs4_pnfs.h
index cab9137..1b017fe 100644
--- a/include/linux/nfs4_pnfs.h
+++ b/include/linux/nfs4_pnfs.h
@@ -149,9 +149,6 @@ struct layoutdriver_io_operations {
 	void (*encode_layoutcommit) (struct pnfs_layout_type *layoutid,
 				     struct xdr_stream *xdr,
 				     const struct pnfs_layoutcommit_arg *args);
-	void (*cleanup_layoutcommit) (struct pnfs_layout_type *layoutid,
-				      struct pnfs_layoutcommit_arg *args,
-				      int status);
 	void (*encode_layoutreturn) (struct pnfs_layout_type *layoutid,
 				struct xdr_stream *xdr,
 				const struct nfs4_pnfs_layoutreturn_arg *args);
-- 
1.6.6

[PATCH 3/6] SQUASHME pnfs-submit: remove encode_layoutcommit

[andros]

From: Andy Adamson <andros@netapp.com>

Not used by the file layout driver

Signed-off-by: Andy Adamson <andros@netapp.com>
---
 fs/nfs/nfs4xdr.c          |   11 ++---------
 include/linux/nfs4_pnfs.h |    3 ---
 2 files changed, 2 insertions(+), 12 deletions(-)

diff --git a/fs/nfs/nfs4xdr.c b/fs/nfs/nfs4xdr.c
index d7d41e9..72fd799 100644
--- a/fs/nfs/nfs4xdr.c
+++ b/fs/nfs/nfs4xdr.c
@@ -1852,8 +1852,6 @@ encode_layoutcommit(struct xdr_stream *xdr,
 		    const struct pnfs_layoutcommit_arg *args,
 		    struct compound_hdr *hdr)
 {
-	struct layoutdriver_io_operations *ld_io_ops =
-			NFS_SERVER(args->inode)->pnfs_curr_ld->ld_io_ops;
 	__be32 *p;
 
 	dprintk("%s: %llu@%llu lbw: %llu type: %d\n", __func__,
@@ -1879,13 +1877,8 @@ encode_layoutcommit(struct xdr_stream *xdr,
 	p = reserve_space(xdr, 4);
 	*p = cpu_to_be32(args->layout_type);
 
-	if (ld_io_ops->encode_layoutcommit)
-		ld_io_ops->encode_layoutcommit(
-			&NFS_I(args->inode)->layout, xdr, args);
-	else {
-		p = reserve_space(xdr, 4);
-		xdr_encode_opaque(p, NULL, 0);
-	}
+	p = reserve_space(xdr, 4);
+	xdr_encode_opaque(p, NULL, 0);
 
 	hdr->nops++;
 	hdr->replen += decode_layoutcommit_maxsz;
diff --git a/include/linux/nfs4_pnfs.h b/include/linux/nfs4_pnfs.h
index 1b017fe..3c22168 100644
--- a/include/linux/nfs4_pnfs.h
+++ b/include/linux/nfs4_pnfs.h
@@ -146,9 +146,6 @@ struct layoutdriver_io_operations {
 	struct pnfs_layout_segment * (*alloc_lseg) (struct pnfs_layout_type *layoutid, struct nfs4_pnfs_layoutget_res *lgr);
 	void (*free_lseg) (struct pnfs_layout_segment *lseg);
 
-	void (*encode_layoutcommit) (struct pnfs_layout_type *layoutid,
-				     struct xdr_stream *xdr,
-				     const struct pnfs_layoutcommit_arg *args);
 	void (*encode_layoutreturn) (struct pnfs_layout_type *layoutid,
 				struct xdr_stream *xdr,
 				const struct nfs4_pnfs_layoutreturn_arg *args);
-- 
1.6.6

[PATCH 4/6] SQUASHME pnfs-submit: cleanup layoutcommit call

[andros]

From: Andy Adamson <andros@netapp.com>

pnfs_layoutcommit_inode prints its status upon exit.

Signed-off-by: Andy Adamson <andros@netapp.com>
---
 fs/nfs/pnfs.c |   11 ++---------
 1 files changed, 2 insertions(+), 9 deletions(-)

diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c
index def27f8..91e944a 100644
--- a/fs/nfs/pnfs.c
+++ b/fs/nfs/pnfs.c
@@ -766,15 +766,8 @@ _pnfs_return_layout(struct inode *ino, struct nfs4_pnfs_layout_segment *range,
 				!pnfs_return_layout_barrier(nfsi, &arg));
 		}
 
-		if (layoutcommit_needed(nfsi)) {
-			status = pnfs_layoutcommit_inode(ino, wait);
-			if (status) {
-				dprintk("%s: layoutcommit failed, status=%d. "
-					"Returning layout anyway\n",
-					__func__, status);
-				status = 0;
-			}
-		}
+		if (layoutcommit_needed(nfsi))
+			pnfs_layoutcommit_inode(ino, wait);
 	}
 send_return:
 	status = return_layout(ino, &arg, stateid, type, lo, wait);
-- 
1.6.6

[PATCH 5/6] SQUASHME pnfs-submit: handle async layoutcommit errors

[andros]

From: Andy Adamson <andros@netapp.com>

nfs4_async_handle_error handles session level and grace,delay errors.

NOTE: Layout specific errors (NFS4ERR_BADIOMODE, NFS4ERR_BAD_LAYOUT,
NFS4ERR_UNKNOWN_LAYOUTTYPE) need to be handled.

Signed-off-by: Andy Adamson <andros@netapp.com>
---
 fs/nfs/nfs4proc.c |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
index 2826ada..514c5c6 100644
--- a/fs/nfs/nfs4proc.c
+++ b/fs/nfs/nfs4proc.c
@@ -5628,6 +5628,9 @@ pnfs_layoutcommit_rpc_done(struct rpc_task *task, void *calldata)
 	put_rpccred(data->cred);
 
 	nfs4_sequence_done(server, &data->res.seq_res, task->tk_status);
+
+	if (nfs4_async_handle_error(task, server, NULL) == -EAGAIN)
+		nfs_restart_rpc(task, server->nfs_client);
 }
 
 static void pnfs_layoutcommit_release(void *lcdata)
-- 
1.6.6

[PATCH 6/6] SQUASHME pnfs-submit: move layoutcommit to nfs_write_inode

[andros]

From: Andy Adamson <andros@netapp.com>

The LAYOUTCOMMIT call indicates an update to the file meta data is needed,
and should be called when clearing the I_DIRTY_SYNC state.

A call to LAYOUTCOMMIT in nfs_write_inode replaces the calls in nfs_wb_all,
nfs_commit_inode, and __nfs4_close.

Signed-off-by: Andy Adamson <andros@netapp.com>
---
 fs/nfs/nfs4state.c |    2 --
 fs/nfs/write.c     |   22 ++++++++++------------
 2 files changed, 10 insertions(+), 14 deletions(-)

diff --git a/fs/nfs/nfs4state.c b/fs/nfs/nfs4state.c
index d0dbdd4..e1207fa 100644
--- a/fs/nfs/nfs4state.c
+++ b/fs/nfs/nfs4state.c
@@ -589,8 +589,6 @@ static void __nfs4_close(struct path *path, struct nfs4_state *state, fmode_t fm
 #ifdef CONFIG_NFS_V4_1
 		struct nfs_inode *nfsi = NFS_I(state->inode);
 
-		if (layoutcommit_needed(nfsi))
-			pnfs_layoutcommit_inode(state->inode, wait);
 		if (has_layout(nfsi) && nfsi->layout.roc_iomode) {
 			struct nfs4_pnfs_layout_segment range;
 
diff --git a/fs/nfs/write.c b/fs/nfs/write.c
index 0fd33cb..875d07f 100644
--- a/fs/nfs/write.c
+++ b/fs/nfs/write.c
@@ -1489,13 +1489,6 @@ static int nfs_commit_inode(struct inode *inode, int how)
 			wait_on_bit(&NFS_I(inode)->flags, NFS_INO_COMMIT,
 					nfs_wait_bit_killable,
 					TASK_KILLABLE);
-#ifdef CONFIG_NFS_V4_1
-		if (may_wait && layoutcommit_needed(NFS_I(inode))) {
-			error = pnfs_layoutcommit_inode(inode, 1);
-			if (error < 0)
-				return error;
-		}
-#endif /* CONFIG_NFS_V4_1 */
 	} else
 		nfs_commit_clear_lock(NFS_I(inode));
 out:
@@ -1545,7 +1538,16 @@ static int nfs_commit_unstable_pages(struct inode *inode, struct writeback_contr
 
 int nfs_write_inode(struct inode *inode, struct writeback_control *wbc)
 {
-	return nfs_commit_unstable_pages(inode, wbc);
+	int ret;
+	ret = nfs_commit_unstable_pages(inode, wbc);
+#ifdef CONFIG_NFS_V4_1
+	if (ret >= 0 && layoutcommit_needed(NFS_I(inode))) {
+		int err = pnfs_layoutcommit_inode(inode, 1);
+		if (err < 0)
+			ret = err;
+	}
+#endif /* CONFIG_NFS_V4_1 */
+	return ret;
 }
 
 /*
@@ -1562,10 +1564,6 @@ int nfs_wb_all(struct inode *inode)
 	};
 
 	ret = sync_inode(inode, &wbc);
-#ifdef CONFIG_NFS_V4_1
-	if (!ret && layoutcommit_needed(NFS_I(inode)))
-		ret = pnfs_layoutcommit_inode(inode, 1);
-#endif
 	return ret;
 }
 
-- 
1.6.6

[PATCH 0/6] pnfs-submit cleanup layoutcommit for file layout

[Andy Adamson]

This time sent with the subject....

On Jun 2, 2010, at 11:54 AM, andros@netapp.com wrote:

>
> This is against the pnfs-submit branch of the 2.6.34 tree. They will  
> need to be
> applied against the 2.6.35-rc1 tree which I can do after comments.
>
> RFC: I would like comments, especially on
> 0006-SQUASHME-pnfs-submit-move-layoutcommit-to-nfs_write_.patch.
>
> Remove unused layoutcommit layoutdriver_io_operations. Will be  
> restored
> in post-submit patches
> 0001-SQUASHME-pnfs-submit-remove-setup_layoutcommit.patch
> 0002-SQUASHNE-pnfs-submit-remove-cleanup_layoutcommit.patch
> 0003-SQUASHME-pnfs-submit-remove-encode_layoutcommit.patch
>
> A cleanup, and call the async error handler.
> 0004-SQUASHME-pnfs-submit-cleanup-layoutcommit-call.patch
> 0005-SQUASHME-pnfs-submit-handle-async-layoutcommit-error.patch
>
> This next  patch moves the pnfs_layoutcommit_inode call to  
> nfs_write_inode,
> and it is the only call other than in layoutreturn. (removed calls in
> __nfs4_close, nfs_commit_inode, nfs_wb_sync).
>
> This is fine for the file layout, and I think it's OK for the object  
> and
> block layouts as well.
>
> I left the LAYOUTCOMMIT call in nfs_write_inode a synchronous call,  
> because
> nfs_commit_unstable_pages sets the FLUSH_SYNC flag. Should this
> be an asyc LAYOUTCOMMIT call?
>
> pnfs_layoutcommit_inode is called after nfs_commit_unstable_pages()  
> so that
> if LAYOUTCOMMIT fails, the unstable pages have been processed..
>
> The error handlers (sync and async) call nfs4_map_errors, so unhandled
> errors (such as NFS4ERR_BADLAYOUT) get returned to nfs_write_ioode  
> as -EIO.
>
> Examining the write_inode call paths, I could not see where the -EIO  
> would
> be passed back to the application.  Testing with pynfs which I
> had return NFS4ERR_BADLAYOUT to the layout commit call, shows the - 
> EIO return
> not stopping the client nor is the error reported back to the  
> application.
>
> We will add code to the error handlers for errors such as  
> NFS4ERR_BADLAYOUT
> that require us to stop using and free the layout, and redo the I/O  
> through
> the MDS.
>
> Anyway, review is much appreciated.
>
> 0006-SQUASHME-pnfs-submit-move-layoutcommit-to-nfs_write_.patch
>
> Testing:
> With CONFIG_NFS_V4_1 set
> NFSv4.1/pnfs passed Connectathon against write enabled GFS2/pNFS.  
> Note: there
> were exactly the same number of LAYOUTCOMMITS sent as were sent with
> pnfs_layoutcommit_inode being called from __nfs4_close (never  
> happened),
> nfs_commit_inode and nfs_wb_sync.
>
> Passed Connectathon general test against pynfs file layout server with
> the NFS4ERR_BADLAYOUT being returned on every third LAYOUTCOMMIT.
>
>
> -->Andy
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs"  
> in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 4/6] SQUASHME pnfs-submit: cleanup layoutcommit call

[Boaz Harrosh]

On 06/02/2010 06:54 PM, andros@netapp.com wrote:
> From: Andy Adamson <andros@netapp.com>
> 
> pnfs_layoutcommit_inode prints its status upon exit.
> 
> Signed-off-by: Andy Adamson <andros@netapp.com>
> ---
>  fs/nfs/pnfs.c |   11 ++---------
>  1 files changed, 2 insertions(+), 9 deletions(-)
> 
> diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c
> index def27f8..91e944a 100644
> --- a/fs/nfs/pnfs.c
> +++ b/fs/nfs/pnfs.c
> @@ -766,15 +766,8 @@ _pnfs_return_layout(struct inode *ino, struct nfs4_pnfs_layout_segment *range,
>  				!pnfs_return_layout_barrier(nfsi, &arg));
>  		}
>  
> -		if (layoutcommit_needed(nfsi)) {
> -			status = pnfs_layoutcommit_inode(ino, wait);
> -			if (status) {
> -				dprintk("%s: layoutcommit failed, status=%d. "
> -					"Returning layout anyway\n",
> -					__func__, status);
> -				status = 0;
> -			}
> -		}
> +		if (layoutcommit_needed(nfsi))
> +			pnfs_layoutcommit_inode(ino, wait);

The print was not for the fail but for the status ignored by
layout_return. But ye we can do with out.

But please leave a big comment why we ignore the status from
pnfs_layoutcommit_inode.

Boaz
>  	}
>  send_return:
>  	status = return_layout(ino, &arg, stateid, type, lo, wait);

Re: [PATCH 6/6] SQUASHME pnfs-submit: move layoutcommit to nfs_write_inode

[Boaz Harrosh]

On 06/02/2010 06:54 PM, andros@netapp.com wrote:
> From: Andy Adamson <andros@netapp.com>
> 
> The LAYOUTCOMMIT call indicates an update to the file meta data is needed,
> and should be called when clearing the I_DIRTY_SYNC state.
> 
> A call to LAYOUTCOMMIT in nfs_write_inode replaces the calls in nfs_wb_all,
> nfs_commit_inode, and __nfs4_close.
> 
> Signed-off-by: Andy Adamson <andros@netapp.com>
> ---
>  fs/nfs/nfs4state.c |    2 --
>  fs/nfs/write.c     |   22 ++++++++++------------
>  2 files changed, 10 insertions(+), 14 deletions(-)
> 
> diff --git a/fs/nfs/nfs4state.c b/fs/nfs/nfs4state.c
> index d0dbdd4..e1207fa 100644
> --- a/fs/nfs/nfs4state.c
> +++ b/fs/nfs/nfs4state.c
> @@ -589,8 +589,6 @@ static void __nfs4_close(struct path *path, struct nfs4_state *state, fmode_t fm
>  #ifdef CONFIG_NFS_V4_1
>  		struct nfs_inode *nfsi = NFS_I(state->inode);
>  
> -		if (layoutcommit_needed(nfsi))
> -			pnfs_layoutcommit_inode(state->inode, wait);
>  		if (has_layout(nfsi) && nfsi->layout.roc_iomode) {
>  			struct nfs4_pnfs_layout_segment range;
>  
> diff --git a/fs/nfs/write.c b/fs/nfs/write.c
> index 0fd33cb..875d07f 100644
> --- a/fs/nfs/write.c
> +++ b/fs/nfs/write.c
> @@ -1489,13 +1489,6 @@ static int nfs_commit_inode(struct inode *inode, int how)
>  			wait_on_bit(&NFS_I(inode)->flags, NFS_INO_COMMIT,
>  					nfs_wait_bit_killable,
>  					TASK_KILLABLE);
> -#ifdef CONFIG_NFS_V4_1
> -		if (may_wait && layoutcommit_needed(NFS_I(inode))) {
> -			error = pnfs_layoutcommit_inode(inode, 1);
> -			if (error < 0)
> -				return error;
> -		}
> -#endif /* CONFIG_NFS_V4_1 */
>  	} else
>  		nfs_commit_clear_lock(NFS_I(inode));
>  out:
> @@ -1545,7 +1538,16 @@ static int nfs_commit_unstable_pages(struct inode *inode, struct writeback_contr
>  
>  int nfs_write_inode(struct inode *inode, struct writeback_control *wbc)
>  {
> -	return nfs_commit_unstable_pages(inode, wbc);
> +	int ret;
> +	ret = nfs_commit_unstable_pages(inode, wbc);
> +#ifdef CONFIG_NFS_V4_1
> +	if (ret >= 0 && layoutcommit_needed(NFS_I(inode))) {
> +		int err = pnfs_layoutcommit_inode(inode, 1);
> +		if (err < 0)
> +			ret = err;
> +	}
> +#endif /* CONFIG_NFS_V4_1 */

Here you should do better. Now that it's so easy for sure.

define an inline called pnfs_layoutcommit_inode that does
the if() and calls a _pnfs_layoutcommit_inode(). The later
is today's original function.

In header define that inline for the pnfs case and define
an empty one returning 0 for the not CONFIG_NFS_V4_1 case.

I had a patch like that but after your patch it is much
easier and makes more sense.

I would fold it into this patch.

Boaz
> +	return ret;
>  }
>  
>  /*
> @@ -1562,10 +1564,6 @@ int nfs_wb_all(struct inode *inode)
>  	};
>  
>  	ret = sync_inode(inode, &wbc);
> -#ifdef CONFIG_NFS_V4_1
> -	if (!ret && layoutcommit_needed(NFS_I(inode)))
> -		ret = pnfs_layoutcommit_inode(inode, 1);
> -#endif
>  	return ret;
>  }
>  

Re: [PATCH 0/6] pnfs-submit cleanup layoutcommit for file layout

[Boaz Harrosh]

On 06/02/2010 06:54 PM, andros@netapp.com wrote:
> This is against the pnfs-submit branch of the 2.6.34 tree. They will need to be
> applied against the 2.6.35-rc1 tree which I can do after comments.
> 
> RFC: I would like comments, especially on
> 0006-SQUASHME-pnfs-submit-move-layoutcommit-to-nfs_write_.patch.
> 
> Remove unused layoutcommit layoutdriver_io_operations. Will be restored
> in post-submit patches
> 0001-SQUASHME-pnfs-submit-remove-setup_layoutcommit.patch
> 0002-SQUASHNE-pnfs-submit-remove-cleanup_layoutcommit.patch

These two should be combined. The cleanup_ is to clean after
what's done in setup_.

> 0003-SQUASHME-pnfs-submit-remove-encode_layoutcommit.patch
> 

For example objects can do with this one only

> A cleanup, and call the async error handler.
> 0004-SQUASHME-pnfs-submit-cleanup-layoutcommit-call.patch
> 0005-SQUASHME-pnfs-submit-handle-async-layoutcommit-error.patch
> 
> This next  patch moves the pnfs_layoutcommit_inode call to nfs_write_inode,
> and it is the only call other than in layoutreturn. (removed calls in
> __nfs4_close, nfs_commit_inode, nfs_wb_sync).
> 
> This is fine for the file layout, and I think it's OK for the object and
> block layouts as well.
> 

It sounds very nice. It might have problems though. On the NFS_STABLE path
again. Because of this stupid thing I found that when returning NFS_STABLE
from writes, and no commits are called, then the internal i_size does not
get updated until after the layout commit has returned and the client detects
a change_attr on server. (Even if it was this client that caused the update)

But this should be fixed regardless. And currently I'm running with
commits on in objlayout. (Which reminds me to send the patch to Benny)

So yes I like this change a lot. It makes tons of sense to me as well.

> I left the LAYOUTCOMMIT call in nfs_write_inode a synchronous call, because
> nfs_commit_unstable_pages sets the FLUSH_SYNC flag. Should this
> be an asyc LAYOUTCOMMIT call?
> 

look at the struct writeback_control *wbc received, it has a flag which states
if this is sync or async do according to that flag. (Tell me if you don't find it)

> pnfs_layoutcommit_inode is called after nfs_commit_unstable_pages() so that
> if LAYOUTCOMMIT fails, the unstable pages have been processed..
> 
> The error handlers (sync and async) call nfs4_map_errors, so unhandled
> errors (such as NFS4ERR_BADLAYOUT) get returned to nfs_write_ioode as -EIO.
> 
> Examining the write_inode call paths, I could not see where the -EIO would
> be passed back to the application.  Testing with pynfs which I
> had return NFS4ERR_BADLAYOUT to the layout commit call, shows the -EIO return
> not stopping the client nor is the error reported back to the application.
> 
> We will add code to the error handlers for errors such as NFS4ERR_BADLAYOUT 
> that require us to stop using and free the layout, and redo the I/O through
> the MDS.
> 
> Anyway, review is much appreciated.
> 
> 0006-SQUASHME-pnfs-submit-move-layoutcommit-to-nfs_write_.patch
> 
> Testing:
> With CONFIG_NFS_V4_1 set
> NFSv4.1/pnfs passed Connectathon against write enabled GFS2/pNFS. Note: there
> were exactly the same number of LAYOUTCOMMITS sent as were sent with
> pnfs_layoutcommit_inode being called from __nfs4_close (never happened),
> nfs_commit_inode and nfs_wb_sync.
> 
> Passed Connectathon general test against pynfs file layout server with
> the NFS4ERR_BADLAYOUT being returned on every third LAYOUTCOMMIT.
> 

Andy you got this patchset all backwards. And they are not a set.

4,5,6 are to go in first and are intended for the full tree
and the .34 and .33 backport tree's as well. If I want to test
with them I'll need them stand alone un-conflicting.

Then 1+2,3 are something else and should be done on top of these above.
If they are self sustained and could be re applied on the to of the tree
as patch -R, then grate. If not then a "bring them back patch" could be
nice. without them we can't test any of this.

> 
> -->Andy
> 

Boaz

Re: [PATCH 0/6] pnfs-submit cleanup layoutcommit for file layout

[William A. (Andy) Adamson]

On Thu, Jun 3, 2010 at 4:10 AM, Boaz Harrosh <bharrosh@panasas.com> wro=
te:
> On 06/02/2010 06:54 PM, andros@netapp.com wrote:
>> This is against the pnfs-submit branch of the 2.6.34 tree. They will=
 need to be
>> applied against the 2.6.35-rc1 tree which I can do after comments.
>>
>> RFC: I would like comments, especially on
>> 0006-SQUASHME-pnfs-submit-move-layoutcommit-to-nfs_write_.patch.
>>
>> Remove unused layoutcommit layoutdriver_io_operations. Will be resto=
red
>> in post-submit patches
>> 0001-SQUASHME-pnfs-submit-remove-setup_layoutcommit.patch
>> 0002-SQUASHNE-pnfs-submit-remove-cleanup_layoutcommit.patch
>
> These two should be combined. The cleanup_ is to clean after
> what's done in setup_.
>
>> 0003-SQUASHME-pnfs-submit-remove-encode_layoutcommit.patch
>>
>
> For example objects can do with this one only

OK - makes sense as all three get squashed into the same patch.

>
>> A cleanup, and call the async error handler.
>> 0004-SQUASHME-pnfs-submit-cleanup-layoutcommit-call.patch
>> 0005-SQUASHME-pnfs-submit-handle-async-layoutcommit-error.patch
>>
>> This next =A0patch moves the pnfs_layoutcommit_inode call to nfs_wri=
te_inode,
>> and it is the only call other than in layoutreturn. (removed calls i=
n
>> __nfs4_close, nfs_commit_inode, nfs_wb_sync).
>>
>> This is fine for the file layout, and I think it's OK for the object=
 and
>> block layouts as well.
>>
>
> It sounds very nice. It might have problems though. On the NFS_STABLE=
 path
> again. Because of this stupid thing I found that when returning NFS_S=
TABLE
> from writes, and no commits are called, then the internal i_size does=
 not
> get updated until after the layout commit has returned and the client=
 detects
> a change_attr on server. (Even if it was this client that caused the =
update)
>
> But this should be fixed regardless. And currently I'm running with
> commits on in objlayout. (Which reminds me to send the patch to Benny=
)
>
> So yes I like this change a lot. It makes tons of sense to me as well=
=2E

Good.

>
>> I left the LAYOUTCOMMIT call in nfs_write_inode a synchronous call, =
because
>> nfs_commit_unstable_pages sets the FLUSH_SYNC flag. Should this
>> be an asyc LAYOUTCOMMIT call?
>>
>
> look at the struct writeback_control *wbc received, it has a flag whi=
ch states
> if this is sync or async do according to that flag. (Tell me if you d=
on't find it)

OK, thanks.

>
>> pnfs_layoutcommit_inode is called after nfs_commit_unstable_pages() =
so that
>> if LAYOUTCOMMIT fails, the unstable pages have been processed..
>>
>> The error handlers (sync and async) call nfs4_map_errors, so unhandl=
ed
>> errors (such as NFS4ERR_BADLAYOUT) get returned to nfs_write_ioode a=
s -EIO.
>>
>> Examining the write_inode call paths, I could not see where the -EIO=
 would
>> be passed back to the application. =A0Testing with pynfs which I
>> had return NFS4ERR_BADLAYOUT to the layout commit call, shows the -E=
IO return
>> not stopping the client nor is the error reported back to the applic=
ation.
>>
>> We will add code to the error handlers for errors such as NFS4ERR_BA=
DLAYOUT
>> that require us to stop using and free the layout, and redo the I/O =
through
>> the MDS.
>>
>> Anyway, review is much appreciated.
>>
>> 0006-SQUASHME-pnfs-submit-move-layoutcommit-to-nfs_write_.patch
>>
>> Testing:
>> With CONFIG_NFS_V4_1 set
>> NFSv4.1/pnfs passed Connectathon against write enabled GFS2/pNFS. No=
te: there
>> were exactly the same number of LAYOUTCOMMITS sent as were sent with
>> pnfs_layoutcommit_inode being called from __nfs4_close (never happen=
ed),
>> nfs_commit_inode and nfs_wb_sync.
>>
>> Passed Connectathon general test against pynfs file layout server wi=
th
>> the NFS4ERR_BADLAYOUT being returned on every third LAYOUTCOMMIT.
>>
>
> Andy you got this patchset all backwards. And they are not a set.
>
> 4,5,6 are to go in first and are intended for the full tree
> and the .34 and .33 backport tree's as well. If I want to test
> with them I'll need them stand alone un-conflicting.

Sure.

>
> Then 1+2,3 are something else and should be done on top of these abov=
e.
> If they are self sustained and could be re applied on the to of the t=
ree
> as patch -R, then grate. If not then a "bring them back patch" could =
be
> nice. without them we can't test any of this

Thanks for the review. I'll resend as requested so that you can test..

-->Andy

>
>>
>> -->Andy
>>
>
> Boaz
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" =
in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at =A0http://vger.kernel.org/majordomo-info.html
>

(unknown),

[Grant Mayor]

FUNDING NOTIFICATION.............I am Mr Grant Mayor. i am a loan lender
as well as an angel investor, I offer funds based on demands and i use the
best possible interest rate so that it can make both paties smile. I allow
my clients to decide the loan repayment duration. So i have my door for
funding opened to all that is in need, If you have been turned down be
many financial institutions, this is a great time for you to make things
up. If you need a stress free funing, kindly get back to me with a mail of
application for funding and you will get funded within 3 working days.
Send all mails of application to: to.grantfinancing-Re5JQEeQqe9fmgfxC/sS/w@public.gmane.org

I await your response.

Grant Mayor.

(unknown)

[Fredrik Pettersson]

unsubscribe linux-nfs

(unknown), ,

[Ming-Yang Lee]

Do you need a loan to pay your bills or to start up a business or for Xmas?.
Kindly apply now for a low rate loan of 3%. for more information contact:
ming.yangfundsservice-HzS2SGvssRM@public.gmane.org
We Await Your Response.
Mr Ming-Yang Lee

----------------------------------------------------------------
This message was sent using IMP, the Internet Messaging Program.



-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

(unknown)

[Silver gears]

(unknown)

[Christoph Neuhaus]

unsubscribe linux-nfs

(unknown),

[Mrs Mellisa Lewis.]

Contact My Lawyer For More Details,!! Barr jay mchenry for  
$14,258,000.00 tell him that i have will this money to  
you.Ref:(JJ/MMS/953/5015/GwrI/316us/uk For charity organization in  
your country.Email:(bjmfirm-N/I+a4o3xEsAvxtiuMwx3w@public.gmane.org) Tel: +44703 183 9543,God Bless  
You Mrs Mellisa Lewis.

(unknown),

[Allen and Violet Large]

Dear Sir/Madam,

This is my fifth times of writting you this email since last year till
date but no response from you.Hope you get this one, as this is a personal
email directed to you. My wife and I won a Jackpot Lotteryof $11.3 million
in July and have voluntarily decided to donate the sum of $500,000.00 USD
to you as part of our own charity project to improve the lot of 10 lucky
individuals all over the world. If you have received this email then you
are one of the lucky recipients and all you have to do is get back with us
so that we can send your details to the payout bank.Please note that you
have to contact my private email for more
informations(allen-violetlarge11-W+jws2TpwOvR7s880joybQ@public.gmane.org)

You can verify this by visiting the web pages below.



http://www.dailymail.co.uk/news/article-1326473/Canadian-couple-Allen-Violet-Large-away-entire-11-2m-lottery-win.html


Goodluck,
Allen and Violet Large
allen-violetlarge11-W+jws2TpwOvR7s880joybQ@public.gmane.org

(unknown),

[Christian Organization]

Good day,

We are Christian organization, we give loan to those who are dedicated
Christians, contact us at mercantilefinanceloanservice@yahoo.com

Regard

Mercantile

(unknown)

[MRS GRACE MANDA]

[-- Attachment #1: Type: text/plain, Size: 71 bytes --]









This is Mrs Grace Manda (  Please I need your Help is Urgent). 

[-- Attachment #2: Mrs Grace Manda.rtf --]
[-- Type: application/rtf, Size: 35796 bytes --]

(unknown),

[pepa6.es-sh/6fXdz2Rs]

Proposal,

Respond to my personal email;  mrs.zhangxiao1962@outlook.
com 


Yours Sincerely.
Mrs. Zhang Xiao (Accounts book Keeper)
Angang 
Steel Company Limited
396 Nan Zhong Hua Lu, Tie Dong District Anshan, 
Liaoning 114021, China.