[patch] lockd control of grace period for HA NFS

[patch] lockd control of grace period for HA NFS

[Marc Eshel]

Hi Neil

Can you please consider adding this patch to lockd which is can be very
helpful for HA NFS.
HA NFS server have to block NLM lock requests on all machines in the
cluster while they recover or failover one of the machines in the cluster.
The following patch will allow NFS server machine on which there was no
failure to go into grace period just to block new lock request until the IP
of the failing node is transferred to another node in the cluster. Only the
clients of the failing node will have to do reclaims their locks.
'echo 1 > /porc/sys/fs/nfs/nlm_grace' will put lockd in grace period and
'echo 0 > /porc/sys/fs/nfs/nlm_grace' will take it out of grace period but
it is not necessary to 'echo 0' since it will just come out of grace period
when the period ends.

Thanks, Marc.



--- fs/lockd/svc.c.orig 2004-11-22 16:59:13.000000000 -0800
+++ fs/lockd/svc.c      2004-11-23 17:15:18.980522064 -0800
@@ -45,6 +45,7 @@ static unsigned int           nlmsvc_users;
 static pid_t                 nlmsvc_pid;
 int                          nlmsvc_grace_period;
 unsigned long                nlmsvc_timeout;
+unsigned long                grace_period_expire;

 static DECLARE_MUTEX_LOCKED(lockd_start);
 static DECLARE_WAIT_QUEUE_HEAD(lockd_exit);
@@ -53,6 +54,7 @@ static DECLARE_WAIT_QUEUE_HEAD(lockd_exi
  * These can be set at insmod time (useful for NFS as root filesystem),
  * and also changed through the sysctl interface.  -- Jamie Lokier, Aug
  2003
  */
+static int                   nlm_grace;
 static unsigned long         nlm_grace_period;
 static unsigned long         nlm_timeout = LOCKD_DFLT_TIMEO;
 static int             nlm_udpport, nlm_tcpport;
@@ -62,6 +64,8 @@ static int              nlm_udpport, nlm_tcpport;
  */
 static const unsigned long   nlm_grace_period_min = 0;
 static const unsigned long   nlm_grace_period_max = 240;
+static const int       nlm_grace_min = 0;
+static const int       nlm_grace_max = 1;
 static const unsigned long   nlm_timeout_min = 3;
 static const unsigned long   nlm_timeout_max = 20;
 static const int       nlm_port_min = 0, nlm_port_max = 65535;
@@ -95,7 +99,6 @@ lockd(struct svc_rqst *rqstp)
 {
  struct svc_serv *serv = rqstp->rq_server;
  int       err = 0;
-     unsigned long grace_period_expire;

  /* Lock module and set up kernel thread */
  /* lockd_up is waiting for us to startup, so will
@@ -349,6 +352,28 @@ nlmsvc_dispatch(struct svc_rqst *rqstp,
  return 1;
 }

+int lockd_toggle_grace(ctl_table *table, int write, struct file *filp,
+                  void __user *buffer, size_t *lenp, loff_t *ppos)
+{
+     int rc;
+
+     if (!write)
+           nlm_grace = nlmsvc_grace_period;
+
+     rc = proc_dointvec_minmax(table, write, filp, buffer, lenp, ppos);
+     if (rc)
+           return rc;
+
+     if (write) {
+           dprintk("lockd: grace period %s\n", nlm_grace ? "on" : "off");
+           if (nlm_grace)
+                 grace_period_expire = set_grace_period();
+           else
+                 clear_grace_period();
+     }
+     return 0;
+}
+
 /*
  * Sysctl parameters (same as module parameters, different interface).
  */
@@ -369,6 +394,16 @@ static ctl_table nlm_sysctls[] = {
  },
  {
   .ctl_name      = CTL_UNNUMBERED,
+           .procname   = "nlm_grace",
+           .data       = &nlm_grace,
+           .maxlen           = sizeof(int),
+           .mode       = 0644,
+           .proc_handler     = &lockd_toggle_grace,
+           .extra1           = (int *) &nlm_grace_min,
+           .extra2           = (int *) &nlm_grace_max,
+     },
+     {
+           .ctl_name   = CTL_UNNUMBERED,
   .procname      = "nlm_timeout",
   .data          = &nlm_timeout,
   .maxlen        = sizeof(int),



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now. 
http://productguide.itmanagersjournal.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [patch] lockd control of grace period for HA NFS

[Trond Myklebust]

ty den 23.11.2004 Klokka 19:38 (-0800) skreiv Marc Eshel:
> Can you please consider adding this patch to lockd which is can be very
> helpful for HA NFS.

Why are you apparently inventing one system for NFSv2/v3 and a different
one for NFSv4 here?

The right way to do this is to have nfsd initiate NFSv4 state recovery,
and have it call lockd back in order to notify it of the "reboot
situation".

Cheers,
  Trond
-- 
Trond Myklebust <trond.myklebust@fys.uio.no>



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now. 
http://productguide.itmanagersjournal.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [patch] lockd control of grace period for HA NFS

[Marc Eshel]

Trond Myklebust <trond.myklebust@fys.uio.no> wrote on 11/23/2004 08:28:15
PM:

> ty den 23.11.2004 Klokka 19:38 (-0800) skreiv Marc Eshel:
> > Can you please consider adding this patch to lockd which is can be very
> > helpful for HA NFS.

> Why are you apparently inventing one system for NFSv2/v3 and a different
> one for NFSv4 here?

> The right way to do this is to have nfsd initiate NFSv4 state recovery,
> and have it call lockd back in order to notify it of the "reboot
> situation".

What I am trying to do is deny lock requests from all nodes in an HA
environment on nodes that are not part of the failure or takeover until
failover is complete and clients of the failed node have had a chance to
reclaim their locks. The rest of the clients are just temporarily blocked.
They will NOT have to reclaim locks and they should not be notified of a
reboot.
Can you say more on how NFSv4 state recovery and lockd be used in this
situation ?

Thanks, Marc.

> Cheers,
> Trond
> --
> Trond Myklebust <trond.myklebust@fys.uio.no>



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now. 
http://productguide.itmanagersjournal.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [patch] lockd control of grace period for HA NFS

[Trond Myklebust]

ty den 23.11.2004 Klokka 21:35 (-0800) skreiv Marc Eshel:

> What I am trying to do is deny lock requests from all nodes in an HA
> environment on nodes that are not part of the failure or takeover until
> failover is complete and clients of the failed node have had a chance to
> reclaim their locks. The rest of the clients are just temporarily blocked.
> They will NOT have to reclaim locks and they should not be notified of a
> reboot.

> Can you say more on how NFSv4 state recovery and lockd be used in this
> situation ?

You are missing the point. If you want to fail your NFS over from one
server to another, then *ALL* the state has to be transferred. That
means NFSv2/v3 locking state AND NFSv4 open owner+locks+delegation+...

Setting up an interface that allows you to transfer one but not the
other is just poor design.

Cheers,
  Trond

-- 
Trond Myklebust <trond.myklebust@fys.uio.no>



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now. 
http://productguide.itmanagersjournal.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [patch] lockd control of grace period for HA NFS

[Marc Eshel]

Trond Myklebust <trond.myklebust@fys.uio.no> wrote on 11/24/2004 08:21:44
AM:

> ty den 23.11.2004 Klokka 21:35 (-0800) skreiv Marc Eshel:

> > What I am trying to do is deny lock requests from all nodes in an HA
> > environment on nodes that are not part of the failure or takeover until
> > failover is complete and clients of the failed node have had a chance
to
> > reclaim their locks. The rest of the clients are just temporarily
blocked.
> > They will NOT have to reclaim locks and they should not be notified of
a
> > reboot.

> > Can you say more on how NFSv4 state recovery and lockd be used in this
> > situation ?

> You are missing the point. If you want to fail your NFS over from one
> server to another, then *ALL* the state has to be transferred. That
> means NFSv2/v3 locking state AND NFSv4 open owner+locks+delegation+...

In the solution I am considering, there is no transfer of state; the
clients are responsible for recovering lock (and other in case of v4) state
after IP takeover. The patch only allows grace period to be set through an
external proc interface. This enables other machines in the cluster that
must block lock activity until recovery is complete.

Yes, you are right that I did not handle blocking of locks that might come
from NFSv4 clients. I was just extending existing lockd proc interface
(/proc/fs/nfs/nlm_grace_period allows setting grace period) to
enable/disable grace with /proc/fs/nfs/nlm_grace.

Do you think that the existing NLM/lockd interfaces should be used control
NFSv4 grace period or do we need a new interface that will be common for
all NFS versions?

Marc.

> Setting up an interface that allows you to transfer one but not the
> other is just poor design.

> Cheers,
> Trond
>
> --
> Trond Myklebust <trond.myklebust@fys.uio.no>



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now. 
http://productguide.itmanagersjournal.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [patch] lockd control of grace period for HA NFS

[Trond Myklebust]

on den 24.11.2004 Klokka 15:49 (-0800) skreiv Marc Eshel:

> In the solution I am considering, there is no transfer of state; the
> clients are responsible for recovering lock (and other in case of v4) state
> after IP takeover. The patch only allows grace period to be set through an
> external proc interface. This enables other machines in the cluster that
> must block lock activity until recovery is complete.

The same model is valid for NFSv4 clients (unless there exists some
other method for directly transferring state between the servers).

> Do you think that the existing NLM/lockd interfaces should be used control
> NFSv4 grace period or do we need a new interface that will be common for
> all NFS versions?

So I'm an advocate of moving towards a common interface for grace period
management for the reasons I stated in the earlier mails.

Since the NFSv4 state model is by and large a superset of what lockd
has, I'd argue that we should move the lockd grace stuff into the NFSv4
interface rather than the opposite.

Cheers,
  Trond

-- 
Trond Myklebust <trond.myklebust@fys.uio.no>



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now. 
http://productguide.itmanagersjournal.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [patch] lockd control of grace period for HA NFS

[Marc Eshel]

Trond Myklebust wrote on 11/24/2004 04:15:22 PM:

> So I'm an advocate of moving towards a common interface for grace period
> management for the reasons I stated in the earlier mails.

> Since the NFSv4 state model is by and large a superset of what lockd
> has, I'd argue that we should move the lockd grace stuff into the NFSv4
> interface rather than the opposite.

Here is a new patch where NFSv3/v4 grace period can be controlled (on/off).
The option was added to NFSv4 which than controls lockd grace period. V4
and lockd are now synchronized as to when they go into grace period and
they use the same period which is v4's grace period.

The next step is to remove the lockd grace period code completely and have
all the code for grace period only in nfsd, but first I wanted to make sure
that this is acceptable to you.


--- fs/lockd/svc.c.orig 2004-11-22 16:59:13.000000000 -0800
+++ fs/lockd/svc.c      2004-11-29 14:53:45.302978384 -0800
@@ -45,6 +45,7 @@ static unsigned int           nlmsvc_users;
 static pid_t                 nlmsvc_pid;
 int                    nlmsvc_grace_period;
 unsigned long                nlmsvc_timeout;
+unsigned long                grace_period_expire;

 static DECLARE_MUTEX_LOCKED(lockd_start);
 static DECLARE_WAIT_QUEUE_HEAD(lockd_exit);
@@ -95,7 +96,6 @@ lockd(struct svc_rqst *rqstp)
 {
  struct svc_serv *serv = rqstp->rq_server;
  int       err = 0;
-     unsigned long grace_period_expire;

  /* Lock module and set up kernel thread */
  /* lockd_up is waiting for us to startup, so will
@@ -349,6 +349,21 @@ nlmsvc_dispatch(struct svc_rqst *rqstp,
  return 1;
 }

+void
+lockd_grace(int on, time_t period)
+{
+
+     dprintk("lockd: lockd_grace %s period %ld\n", on ? "on" : "off",
period);
+     if (on) {
+           if (period)
+                 nlm_grace_period = period;
+           grace_period_expire = set_grace_period();
+     }
+     else {
+           clear_grace_period();
+     }
+}
+
 /*
  * Sysctl parameters (same as module parameters, different interface).
  */
--- fs/lockd/lockd_syms.c.orig      2004-11-25 16:13:04.000000000 -0800
+++ fs/lockd/lockd_syms.c     2004-11-25 16:26:45.000000000 -0800
@@ -26,6 +26,7 @@
 /* Start/stop the daemon */
 EXPORT_SYMBOL(lockd_up);
 EXPORT_SYMBOL(lockd_down);
+EXPORT_SYMBOL(lockd_grace);

 /* NFS client entry */
 EXPORT_SYMBOL(nlmclnt_proc);
--- fs/nfsd/lockd.c.orig      2004-11-24 21:57:26.000000000 -0800
+++ fs/nfsd/lockd.c     2004-11-29 12:55:35.421804152 -0800
@@ -65,9 +65,16 @@ nlm_fclose(struct file *filp)
  mntput(filp->f_vfsmnt);
 }

+static void
+nlm_grace(int on, time_t period)
+{
+     lockd_grace(on, period);
+}
+
 struct nlmsvc_binding        nfsd_nlm_ops = {
  .fopen          = nlm_fopen,            /* open file for locking */
  .fclose         = nlm_fclose,           /* close file */
+     .grace            = nlm_grace,            /* start/stop grace period
 */
 };

 void
--- fs/nfsd/nfsctl.c.orig     2004-11-24 22:29:42.000000000 -0800
+++ fs/nfsd/nfsctl.c    2004-11-29 14:45:14.352654640 -0800
@@ -24,6 +24,7 @@
 #include <linux/pagemap.h>
 #include <linux/init.h>

+#include <linux/lockd/bind.h>
 #include <linux/nfs.h>
 #include <linux/nfsd_idmap.h>
 #include <linux/sunrpc/svc.h>
@@ -36,7 +37,7 @@
 #include <asm/uaccess.h>

 /*
- *   We have a single directory with 9 nodes in it.
+ *   We have a single directory with 13 nodes in it.
  */
 enum {
  NFSD_Root = 1,
@@ -51,6 +52,7 @@ enum {
  NFSD_Fh,
  NFSD_Threads,
  NFSD_Leasetime,
+     NFSD_Grace,
 };

 /*
@@ -66,6 +68,7 @@ static ssize_t write_getfs(struct file *
 static ssize_t write_filehandle(struct file *file, char *buf, size_t
 size);
 static ssize_t write_threads(struct file *file, char *buf, size_t size);
 static ssize_t write_leasetime(struct file *file, char *buf, size_t size);
+static ssize_t toggle_grace_period(struct file *file, char *buf, size_t
size);

 static ssize_t (*write_op[])(struct file *, char *, size_t) = {
  [NFSD_Svc] = write_svc,
@@ -78,6 +81,7 @@ static ssize_t (*write_op[])(struct file
  [NFSD_Fh] = write_filehandle,
  [NFSD_Threads] = write_threads,
  [NFSD_Leasetime] = write_leasetime,
+     [NFSD_Grace] = toggle_grace_period,
 };

 /* an argresp is stored in an allocated page and holds the
@@ -419,6 +423,34 @@ static ssize_t write_leasetime(struct fi
  return strlen(buf);
 }

+static ssize_t toggle_grace_period(struct file *file, char *buf, size_t
size)
+{
+     char *mesg = buf;
+     int rv;
+     int start;
+
+     if (size > 0) {
+           rv = get_int(&mesg, &start);
+           if (rv)
+                 return rv;
+           if (start < 0 || start > 1)
+                 return -EINVAL;
+
+           if (nlmsvc_ops && nlmsvc_ops->grace)
+                 /* use the same period for v3 and v4 */
+                 nlmsvc_ops->grace(start, nfs4_lease_time());
+           if (start) {
+                 nfs4_grace_start();
+           }
+           else {
+                 nfs4_grace_stop();
+           }
+     }
+     start = nfs4_in_grace();
+     sprintf(buf, "%d\n", start);
+     return strlen(buf);
+}
+
 /*----------------------------------------------------------------------------*/

 /*
  *   populating the filesystem.
@@ -440,6 +472,7 @@ static int nfsd_fill_super(struct super_
 #ifdef CONFIG_NFSD_V4
   [NFSD_Leasetime] = {"nfsv4leasetime", &transaction_ops,
   S_IWUSR|S_IRUSR},
 #endif
+           [NFSD_Grace] = {"grace", &transaction_ops, S_IWUSR|S_IRUSR},
   /* last one */ {""}
  };
  return simple_fill_super(sb, 0x6e667364, nfsd_files);
--- fs/nfsd/nfs4state.c.orig  2004-09-15 16:50:12.000000000 -0700
+++ fs/nfsd/nfs4state.c 2004-11-29 14:42:46.629112032 -0800
@@ -47,6 +47,7 @@
 #include <linux/nfs4.h>
 #include <linux/nfsd/state.h>
 #include <linux/nfsd/xdr4.h>
+#include <linux/lockd/bind.h>

 #define NFSDDBG_FACILITY                NFSDDBG_PROC

@@ -2659,6 +2660,9 @@ nfs4_state_init(void)
  if (grace_time)
   printk("NFSD: starting %ld-second grace period\n", grace_time);
  grace_end = boot_time + grace_time;
+     if (nlmsvc_ops && nlmsvc_ops->grace)
+           /* use the same period for v3 and v4 */
+           nlmsvc_ops->grace(1, grace_time);
  INIT_WORK(&laundromat_work,laundromat_main, NULL);
  schedule_delayed_work(&laundromat_work, NFSD_LEASE_TIME*HZ);
  nfs4_init = 1;
@@ -2671,6 +2675,18 @@ nfs4_in_grace(void)
 }

 void
+nfs4_grace_start(void)
+{
+     grace_end = get_seconds() + nfs4_lease_time();
+}
+
+void
+nfs4_grace_stop(void)
+{
+     grace_end = get_seconds();
+}
+
+void
 set_no_grace(void)
 {
  printk("NFSD: ERROR in reboot recovery.  State reclaims will fail.\n");
--- include/linux/lockd/bind.h.orig 2004-11-24 21:49:39.000000000 -0800
+++ include/linux/lockd/bind.h      2004-11-29 12:54:03.009852904 -0800
@@ -22,6 +22,7 @@ struct nlmsvc_binding {
       struct nfs_fh *,
       struct file *);
  void                  (*fclose)(struct file *);
+     void              (*grace)(int on_off, int period);
 };

 extern struct nlmsvc_binding *     nlmsvc_ops;
@@ -32,5 +33,6 @@ extern struct nlmsvc_binding *      nlmsvc_op
 extern int nlmclnt_proc(struct inode *, int, struct file_lock *);
 extern int lockd_up(void);
 extern void      lockd_down(void);
+extern void      lockd_grace(int on_off, time_t period);

 #endif /* LINUX_LOCKD_BIND_H */
--- include/linux/nfsd/nfsd.h.orig  2004-11-25 20:53:18.000000000 -0800
+++ include/linux/nfsd/nfsd.h 2004-11-25 21:03:29.000000000 -0800
@@ -135,11 +135,15 @@ void nfs4_state_init(void);
 void nfs4_state_shutdown(void);
 time_t nfs4_lease_time(void);
 void nfs4_reset_lease(time_t leasetime);
+void nfs4_grace_start(void);
+void nfs4_grace_stop(void);
 #else
 void static inline nfs4_state_init(void){}
 void static inline nfs4_state_shutdown(void){}
 time_t static inline nfs4_lease_time(void){return 0;}
 void static inline nfs4_reset_lease(time_t leasetime){}
+void nfs4_grace_start(void){}
+void nfs4_grace_stop(void){}
 #endif

 /*



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now. 
http://productguide.itmanagersjournal.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [patch] lockd control of grace period for HA NFS

[William A.(Andy) Adamson]

Trond Myklebust wrote on 11/24/2004 04:15:22 PM:

> So I'm an advocate of moving towards a common interface for grace period
> management for the reasons I stated in the earlier mails.

> Since the NFSv4 state model is by and large a superset of what lockd
> has, I'd argue that we should move the lockd grace stuff into the NFSv4
> interface rather than the opposite.


eshel@almaden.ibm.com said:
> Here is a new patch where NFSv3/v4 grace period can be controlled (on/
> off). The option was added to NFSv4 which than controls lockd grace
> period. V4 and lockd are now synchronized as to when they go into
> grace period and they use the same period which is v4's grace period.

The NFSv4 portion looks ok to me; I'm lockd ignorant :-)

To migrate a client from NFSv4 server1 to NFSv4 server2 both exporting a 
shared file system, the administrator will perform the following tasks

1) expire the client on NFSv4 server1 (remove the in-memory client state)
2) turn on (toggle) the grace period for all NFSv4 servers exporting the 
shared file system
3) change the mount on the client from NFSv4 server1 to NFSv4 server2.

While in the grace period, all NFSv4 servers will deny all non-recalim OPEN 
and LOCK requests. READ/WRITE/SETATTR(file size change) on existing or 
reclaimed OPEN/LOCK will succeed. So, when an NFSv4 client migrates from one 
NFSv4 server to another NFSv4 server (both exporting a shared file system), 
*all* non-reclaim OPEN/LOCK request from any client will stall with 
NFS4_ERR_GRACE.

I just completed a patch to allow READ/WRITE/SETATTR(file size change) while 
in the grace period for existing/reclaimed state.

I'm currently working on the stable storage portion of the NFSv4 server 
reclaim.

-->Andy






-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now. 
http://productguide.itmanagersjournal.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [patch] lockd control of grace period for HA NFS

[Marc Eshel]

William A.(Andy) Adamson wrote on 11/30/2004 07:55:07 AM:

> The NFSv4 portion looks ok to me; I'm lockd ignorant :-)

> To migrate a client from NFSv4 server1 to NFSv4 server2 both exporting a
> shared file system, the administrator will perform the following tasks

> 1) expire the client on NFSv4 server1 (remove the in-memory client state)
> 2) turn on (toggle) the grace period for all NFSv4 servers exporting the
> shared file system
> 3) change the mount on the client from NFSv4 server1 to NFSv4 server2.

I would do step 2 first to make sure that the client doesn't lose any locks
in the window between step 1 and 2.
I hope that when we are done the administrator will not have to do anything
on the client machines but instead the fs-location/moved will be used to
redirect the client from one server to another.

> While in the grace period, all NFSv4 servers will deny all non-recalim
OPEN
> and LOCK requests. READ/WRITE/SETATTR(file size change) on existing or
> reclaimed OPEN/LOCK will succeed. So, when an NFSv4 client migrates from
one
> NFSv4 server to another NFSv4 server (both exporting a shared file
system),
> *all* non-reclaim OPEN/LOCK request from any client will stall with
> NFS4_ERR_GRACE.

> I just completed a patch to allow READ/WRITE/SETATTR(file size change)
while
> in the grace period for existing/reclaimed state.

> I'm currently working on the stable storage portion of the NFSv4 server
> reclaim.

> -->Andy

>
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real users.
> Discover which products truly live up to the hype. Start reading now.
> http://productguide.itmanagersjournal.com/
> _______________________________________________
> NFS maillist  -  NFS@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nfs



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now. 
http://productguide.itmanagersjournal.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [patch] lockd control of grace period for HA NFS

[Calum Mackay]

Hi Marc, Andy,

Forgive my late joining of this thread, and apologies if I'm missing 
anything...

How do you intend to handle the stable storage issue? It seems to me 
that to comply with the RFC's requirement that stable storage be used 
when a grace period is offered, there needs to be a Cluster-wide, 
always-consistent, stable-storage, available to all nodes. I imagine 
that a correct implementation of this isn't a trivial exercise... :)

best regards,
calum.


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now. 
http://productguide.itmanagersjournal.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [patch] lockd control of grace period for HA NFS

[William A.(Andy) Adamson]

> Hi Marc, Andy,
> 
> Forgive my late joining of this thread, and apologies if I'm missing 
> anything...
> 
> How do you intend to handle the stable storage issue? It seems to me 
> that to comply with the RFC's requirement that stable storage be used 
> when a grace period is offered, there needs to be a Cluster-wide, 
> always-consistent, stable-storage, available to all nodes. I imagine 
> that a correct implementation of this isn't a trivial exercise... :)
> 
> best regards,
> calum.

i plan to use the cluster wide file system.

-->Andy

> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real users.
> Discover which products truly live up to the hype. Start reading now. 
> http://productguide.itmanagersjournal.com/
> _______________________________________________
> NFS maillist  -  NFS@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/nfs
> 




-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now. 
http://productguide.itmanagersjournal.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [patch] lockd control of grace period for HA NFS

[J. Bruce Fields]

On Tue, Nov 23, 2004 at 07:38:34PM -0800, Marc Eshel wrote:
> 'echo 1 > /porc/sys/fs/nfs/nlm_grace' will put lockd in grace period and
> 'echo 0 > /porc/sys/fs/nfs/nlm_grace' will take it out of grace period but
> it is not necessary to 'echo 0' since it will just come out of grace period
> when the period ends.

The "echo 0" is a little dangerous, isn't it?  Do we really need the
ability to end the grace period prematurely?--b.


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now. 
http://productguide.itmanagersjournal.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs

Re: [patch] lockd control of grace period for HA NFS

[Marc Eshel]

The "echo 0" was put in mainly for debugging and it is not critical at this
point. In a cluster environment the grace period might need to be set
higher than the minimum value so it is possible to go out of grace period
only when the last machine completed it required minimum grace period. It
might also be possible to fail one client from one machine to another for
load balancing in which case the cluster file system has the locks state
and can determine when all the locks got reclaimed. In this case the
reclaim is done to update the new machine local sate locks (posix locks).

Marc.

"J. Bruce Fields" <bfields@fieldses.org> wrote on 12/06/2004 07:04:28 AM:

> On Tue, Nov 23, 2004 at 07:38:34PM -0800, Marc Eshel wrote:
> > 'echo 1 > /proc/sys/fs/nfs/nlm_grace' will put lockd in grace period
and
> > 'echo 0 > /proc/sys/fs/nfs/nlm_grace' will take it out of grace period
but
> > it is not necessary to 'echo 0' since it will just come out of grace
period
> > when the period ends.

> The "echo 0" is a little dangerous, isn't it?  Do we really need the
> ability to end the grace period prematurely?--b.



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now. 
http://productguide.itmanagersjournal.com/
_______________________________________________
NFS maillist  -  NFS@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs