* NFSv4 Active-Active Cluster Queries [not found] ` <44BF6BE60200007F0000E3FF@lucius.provo.novell.com> @ 2006-07-20 6:11 ` Rajat Upadhyaya 2006-07-20 7:12 ` Neil Brown ` (2 more replies) 0 siblings, 3 replies; 5+ messages in thread From: Rajat Upadhyaya @ 2006-07-20 6:11 UTC (permalink / raw) To: nfsv4; +Cc: neilb, Olaf Kirch, nfs Hi, I have been working on setting up NFSv4 in cluster scenario. But when it comes to an Active-Active setup, it seems there are a few problems. In Active-Passive case, the /var/lib/nfs directory itself is moved to shared storage & on the nodes in the cluster, symlinks are created pointing to the location on shared storage. This isn't possible in case of Active-Active setup. In this case, we will need to use scripts to merge the information in /var/lib/nfs from the failed node with that on the running one and then do a re-export. 1) From the High Availability wiki (http://wiki.linux-ha.org), I got to this page - http://chilli.linuxmds.com/~mschilli/NFS/active-active-nfs.html where a method of setting up Active-Active NFS failover was described. Basically a set of scripts is used to duplicate the rmtab entries on the node that takes over the process during failover. Will this method hold in case of NFSv4 too? What other data would need to be duplicated? 2) Assuming there are 2 nodes in a cluster, a client mounts different exports from both the nodes. Then if one node goes down, the other node takes over the failed node's IP & re-exports its exports. In this case, if (1) holds, what happens to the rmtab entry? Is it possible to merge the 2 entries? 3) Locking issues - revocation & reacquiring of locks. 4) I can see that there is some HA callout code in nfs-utils (under support). Is this applicable for NFSv4 also? Is my understanding here correct? If so, are there any work-arounds/solutions for the issues? Also, are there any other problems which I have missed out? Regards, Rajat ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ NFS maillist - NFS@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nfs ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: NFSv4 Active-Active Cluster Queries 2006-07-20 6:11 ` NFSv4 Active-Active Cluster Queries Rajat Upadhyaya @ 2006-07-20 7:12 ` Neil Brown 2006-07-21 17:27 ` Wendy Cheng 2006-07-20 13:52 ` J. Bruce Fields 2006-07-26 8:58 ` Yatin Manerker 2 siblings, 1 reply; 5+ messages in thread From: Neil Brown @ 2006-07-20 7:12 UTC (permalink / raw) To: Rajat Upadhyaya; +Cc: nfs, nfsv4 On Thursday July 20, urajat@novell.com wrote: > Hi, > > I have been working on setting up NFSv4 in cluster scenario. But when > it comes to an Active-Active setup, it seems there are a few problems. > > In Active-Passive case, the /var/lib/nfs directory itself is moved to > shared storage & on the nodes in the cluster, symlinks are created > pointing to the location on shared storage. This isn't possible in case > of Active-Active setup. In this case, we will need to use scripts to > merge the information in /var/lib/nfs from the failed node with that on > the running one and then do a re-export. > > 1) From the High Availability wiki (http://wiki.linux-ha.org), I got to > this page - > http://chilli.linuxmds.com/~mschilli/NFS/active-active-nfs.html where > a method of setting up Active-Active NFS failover was described. > Basically a set of scripts is used to duplicate the rmtab entries on the > node that takes over the process during failover. Will this method hold > in case of NFSv4 too? What other data would need to be duplicated? rmtab is simply not an issue - you can ignore it. Provided /proc/fs/nfsd is mounted, the contents of rmtab are ignored. What is important is locking state. For NFSv4, that means the contents of /var/lib/nfs/v4recovery. I don't know exactly how this is handled. > > 3) Locking issues - revocation & reacquiring of locks. Wendy Cheng <wcheng@redhat.com> has been looking at what kernel/utils changes are needed to support this fully, though I think she is only looking at v3. > > 4) I can see that there is some HA callout code in nfs-utils (under > support). Is this applicable for NFSv4 also? No. Currently all v4 state is handled by the kernel, though that may change. NeilBrown ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: NFSv4 Active-Active Cluster Queries 2006-07-20 7:12 ` Neil Brown @ 2006-07-21 17:27 ` Wendy Cheng 0 siblings, 0 replies; 5+ messages in thread From: Wendy Cheng @ 2006-07-21 17:27 UTC (permalink / raw) To: Neil Brown; +Cc: Olaf Kirch, nfsv4, nfs Neil Brown wrote: >On Thursday July 20, urajat@novell.com wrote: > > >>Hi, >> >>I have been working on setting up NFSv4 in cluster scenario. But when >>it comes to an Active-Active setup, it seems there are a few problems. >> >> >> >>3) Locking issues - revocation & reacquiring of locks. >> >> > >Wendy Cheng <wcheng@redhat.com> has been looking at what kernel/utils >changes are needed to support this fully, though I think she is only >looking at v3. > > > The v3 patches are currently under function verification tests using RHCS cluser suite - so far so good (http://people.redhat.com/wcheng/Patches/NLM/). After documenting the procedure and restrictions, the formal submission (to nfs list) is scheduled at end of this month. Not sure what to do with V4 yet .. need to chat with CITI folks first. -- Wendy ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ NFS maillist - NFS@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nfs ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: NFSv4 Active-Active Cluster Queries 2006-07-20 6:11 ` NFSv4 Active-Active Cluster Queries Rajat Upadhyaya 2006-07-20 7:12 ` Neil Brown @ 2006-07-20 13:52 ` J. Bruce Fields 2006-07-26 8:58 ` Yatin Manerker 2 siblings, 0 replies; 5+ messages in thread From: J. Bruce Fields @ 2006-07-20 13:52 UTC (permalink / raw) To: Rajat Upadhyaya; +Cc: nfs, nfsv4 On Thu, Jul 20, 2006 at 12:11:26AM -0600, Rajat Upadhyaya wrote: > I have been working on setting up NFSv4 in cluster scenario. But when > it comes to an Active-Active setup, it seems there are a few problems. Yes. Even active-passive failover is problematic right now, mainly because the method we use for storing reboot recovery information (analagous to the state recorded by statd) is still in flux. We're aware of these problems and working on solving them. There are some rough (possibly out of date notes) here: http://wiki.linux-nfs.org/index.php/Cluster_Coherent_NFS_design but those are more of interest to someone working on design of a solution than to someone trying to set up and use any of this (which we don't recommend doing yet). --b. ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ NFS maillist - NFS@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nfs ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: NFSv4 Active-Active Cluster Queries 2006-07-20 6:11 ` NFSv4 Active-Active Cluster Queries Rajat Upadhyaya 2006-07-20 7:12 ` Neil Brown 2006-07-20 13:52 ` J. Bruce Fields @ 2006-07-26 8:58 ` Yatin Manerker 2 siblings, 0 replies; 5+ messages in thread From: Yatin Manerker @ 2006-07-26 8:58 UTC (permalink / raw) To: J. Bruce Fields, nfsv4; +Cc: nfs >Yes. Even active- passive failover is problematic right now, mainly > because the method we use for storing reboot recovery information > (analagous to the state recorded by statd) is still in flux. > We're aware of these problems and working on solving them. There are > some rough (possibly out of date notes) here: > http://wiki.linux-nfs.org/index.php/Cluster_Coherent_NFS_design > but those are more of interest to someone working on design of a > solution than to someone trying to set up and use any of this (which we > don't recommend doing yet). > -- b. Hi We have tested NFSv4 in Active/Passive HA clusters. I wanted to know if we have missed out anything test scenarios. I have not found any issues in the following test scenarios. Here are the test results Server: Suse Linux Enterprise server 10 Build : RC3 Kernel: 2.6.16.21 Hardware details: Server -> 32 bit hardware 2P - P4 2Gb RAM HA clusters : Active/Passive Client hardware: 1P, P3-512 MB RAM Following scenarios are tested and passed for Active/Passive HA clusters (iSCSI ) 1.Client connection during failover and failback scenarios. Tested with 40 clients. 2.Large file transfer from 4 clients. Single file size of 8GB from each client. Test was performed during failover and failback scenarios. 3.File listing : File listing was done from 40 clients. (A single directory having 0.9 million files). 4.Recovery test: Physical link breakage and Power down recovery when client is running test(connect athon) on NFS mounted volume. 5.Backward compatibility : NFSv3/v4 client connecting NFSv4/v3 server 6.Interop: Novell iFolder client data on NFSv4 mount point.NFSv4 client mounting NFSv4 exported file system and iFolder client storing data on the remote NFS file system 7.iozone and connect athon test executed for NFSv4 client in cluster environment Following scenarios are tested and passed for Active/Passive HA clusters (DRBD ) (for single client ) 1.Client connection during failover and failback scenarios. (V3 and V4 client) 2.Directory listing (v3 and v4 client) 3.Executed Connect athon test (Lock testing is done as part of this test) Regards, Yatin ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2006-07-26 8:58 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <44BCC5620200007F0000E2C6@lucius.provo.novell.com>
[not found] ` <44BCD1F80200007F0000E2CA@lucius.provo.novell.com>
[not found] ` <44BCE7BB0200007F0000E2E7@lucius.provo.novell.com>
[not found] ` <44BF54250200007F0000E3E0@lucius.provo.novell.com>
[not found] ` <44BF5EF80200007F0000E3E4@lucius.provo.novell.com>
[not found] ` <44BF5F580200007F0000E3E7@lucius.provo.novell.com>
[not found] ` <44BF5F8B0200007F0000E3EA@lucius.provo.novell.com>
[not found] ` <44BF607D0200007F0000E3ED@lucius.provo.novell.com>
[not found] ` <44BF6B4F0200007F0000E3F6@lucius.provo.novell.com>
[not found] ` <44BF6B870200007F0000E3F9@lucius.provo.novell.com>
[not found] ` <44BF6BA30200007F0000E3FC@lucius.provo.novell.com>
[not found] ` <44BF6BE60200007F0000E3FF@lucius.provo.novell.com>
2006-07-20 6:11 ` NFSv4 Active-Active Cluster Queries Rajat Upadhyaya
2006-07-20 7:12 ` Neil Brown
2006-07-21 17:27 ` Wendy Cheng
2006-07-20 13:52 ` J. Bruce Fields
2006-07-26 8:58 ` Yatin Manerker
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.