* Side effects of having NFSv4 mounted over udp
@ 2010-12-13 7:46 DENIEL Philippe
2010-12-13 19:04 ` Thomas Haynes
0 siblings, 1 reply; 6+ messages in thread
From: DENIEL Philippe @ 2010-12-13 7:46 UTC (permalink / raw)
To: linux-nfs
Hi,
I know NFSv4 is definitely a connection oriented protocol, but noting
prevent from using "mount -t nfs4 -o proto=udp <...>". Using TCP instead
of UDP could be of great interest, especially in a HPC context where
thousands of clients will operate at the same time. If a nfs mount point
is made over udp, what would be the side effects ? I tried to run the
connectathon on such a udp based mount point and I could find no errors.
Is NFSv4/UDP a suitable solution ? Or does it make no sense to use udp ?
Thanks in advance for your answer.
Regards
Philippe
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: Side effects of having NFSv4 mounted over udp 2010-12-13 7:46 Side effects of having NFSv4 mounted over udp DENIEL Philippe @ 2010-12-13 19:04 ` Thomas Haynes 2010-12-13 19:16 ` J. Bruce Fields 0 siblings, 1 reply; 6+ messages in thread From: Thomas Haynes @ 2010-12-13 19:04 UTC (permalink / raw) To: DENIEL Philippe; +Cc: linux-nfs On Dec 13, 2010, at 1:46 AM, DENIEL Philippe wrote: > Hi, > > I know NFSv4 is definitely a connection oriented protocol, but noting prevent from using "mount -t nfs4 -o proto=udp <...>". Using TCP instead of UDP could be of great interest, especially in a HPC context where thousands of clients will operate at the same time. If a nfs mount point is made over udp, what would be the side effects ? I tried to run the connectathon on such a udp based mount point and I could find no errors. Is NFSv4/UDP a suitable solution ? Or does it make no sense to use udp ? You might find that some implementations have UDP disabled and it might then make it hard to triage what is going on. I believe both NetApp's OnTap and Oracle's Solaris have it turned off. > > Thanks in advance for your answer. > > Regards > > Philippe > -- > To unsubscribe from this list: send the line "unsubscribe linux-nfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Side effects of having NFSv4 mounted over udp 2010-12-13 19:04 ` Thomas Haynes @ 2010-12-13 19:16 ` J. Bruce Fields 2010-12-13 19:47 ` Trond Myklebust 2010-12-14 7:35 ` DENIEL Philippe 0 siblings, 2 replies; 6+ messages in thread From: J. Bruce Fields @ 2010-12-13 19:16 UTC (permalink / raw) To: Thomas Haynes; +Cc: DENIEL Philippe, linux-nfs On Mon, Dec 13, 2010 at 01:04:28PM -0600, Thomas Haynes wrote: > > On Dec 13, 2010, at 1:46 AM, DENIEL Philippe wrote: > > > Hi, > > > > I know NFSv4 is definitely a connection oriented protocol, but noting prevent from using "mount -t nfs4 -o proto=udp <...>". Using TCP instead of UDP could be of great interest, especially in a HPC context where thousands of clients will operate at the same time. If a nfs mount point is made over udp, what would be the side effects ? I tried to run the connectathon on such a udp based mount point and I could find no errors. Is NFSv4/UDP a suitable solution ? Or does it make no sense to use udp ? > > > You might find that some implementations have UDP disabled and it might then make > it hard to triage what is going on. > > I believe both NetApp's OnTap and Oracle's Solaris have it turned off. The Linux server allows it, but I've been considering that a (low-priority) bug, so it wouldn't be safe to assume it will continue working. That aside, if you have a perfect network, NFSv4.0 at least will probably work. (Not 4.1 since backchannel setup will fail?) Are you really sure that you can't make tcp scale to thousands of clients? --b. > > > > > > Thanks in advance for your answer. > > > > Regards > > > > Philippe > > -- > > To unsubscribe from this list: send the line "unsubscribe linux-nfs" in > > the body of a message to majordomo@vger.kernel.org > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > -- > To unsubscribe from this list: send the line "unsubscribe linux-nfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Side effects of having NFSv4 mounted over udp 2010-12-13 19:16 ` J. Bruce Fields @ 2010-12-13 19:47 ` Trond Myklebust 2010-12-14 7:35 ` DENIEL Philippe 1 sibling, 0 replies; 6+ messages in thread From: Trond Myklebust @ 2010-12-13 19:47 UTC (permalink / raw) To: J. Bruce Fields; +Cc: Thomas Haynes, DENIEL Philippe, linux-nfs On Mon, 2010-12-13 at 14:16 -0500, J. Bruce Fields wrote: > On Mon, Dec 13, 2010 at 01:04:28PM -0600, Thomas Haynes wrote: > > > > On Dec 13, 2010, at 1:46 AM, DENIEL Philippe wrote: > > > > > Hi, > > > > > > I know NFSv4 is definitely a connection oriented protocol, but noting prevent from using "mount -t nfs4 -o proto=udp <...>". Using TCP instead of UDP could be of great interest, especially in a HPC context where thousands of clients will operate at the same time. If a nfs mount point is made over udp, what would be the side effects ? I tried to run the connectathon on such a udp based mount point and I could find no errors. Is NFSv4/UDP a suitable solution ? Or does it make no sense to use udp ? > > > > > > You might find that some implementations have UDP disabled and it might then make > > it hard to triage what is going on. > > > > I believe both NetApp's OnTap and Oracle's Solaris have it turned off. > > The Linux server allows it, but I've been considering that a > (low-priority) bug, so it wouldn't be safe to assume it will continue > working. > > That aside, if you have a perfect network, NFSv4.0 at least will > probably work. (Not 4.1 since backchannel setup will fail?) > > Are you really sure that you can't make tcp scale to thousands of > clients? Maybe SCTP would be the way to go then? We've often talked about it over the years, but nobody has yet (to my knowledge) written an ONC RPC transport for it. Cheers Trond -- Trond Myklebust Linux NFS client maintainer NetApp Trond.Myklebust@netapp.com www.netapp.com ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Side effects of having NFSv4 mounted over udp 2010-12-13 19:16 ` J. Bruce Fields 2010-12-13 19:47 ` Trond Myklebust @ 2010-12-14 7:35 ` DENIEL Philippe 2010-12-14 19:15 ` J. Bruce Fields 1 sibling, 1 reply; 6+ messages in thread From: DENIEL Philippe @ 2010-12-14 7:35 UTC (permalink / raw) To: J. Bruce Fields; +Cc: Thomas Haynes, linux-nfs Hi > The Linux server allows it, but I've been considering that a > (low-priority) bug, so it wouldn't be safe to assume it will continue > working. > I am mostly thinking at the client side using as well udp and tcp. Would the "NFSv4 client over UDP" behaves differently then the same client over TCP ? > That aside, if you have a perfect network, I can trust my network. It's not a WAN, it's located into a very massive cluster (it's kind of "internal LAN"). It's a very high throughput network (IB based) so I believe there are less "hardware based reason" to loose packets. > NFSv4.0 at least will > probably work. OK. > (Not 4.1 since backchannel setup will fail?) > What is erroneous in using UDP for NFSv4.1 backchannels ? > Are you really sure that you can't make tcp scale to thousands of > clients? > I am a bit afraid of a "No more file descriptors" effect. If I have one TCP socket per client and thousands of clients, I have less remaining fds for other purposes. Another point : UDP is a "cheap" protocol. I can have bunches of clients without overloading the server (a new client will almost cost nothing to the server, just the cost of a new clientid negotiation) . I was wondering if it could be reliable to use it for NFSv4 inside a large cluster. Philippe ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Side effects of having NFSv4 mounted over udp 2010-12-14 7:35 ` DENIEL Philippe @ 2010-12-14 19:15 ` J. Bruce Fields 0 siblings, 0 replies; 6+ messages in thread From: J. Bruce Fields @ 2010-12-14 19:15 UTC (permalink / raw) To: DENIEL Philippe; +Cc: Thomas Haynes, linux-nfs On Tue, Dec 14, 2010 at 08:35:14AM +0100, DENIEL Philippe wrote: > Hi > >The Linux server allows it, but I've been considering that a > >(low-priority) bug, so it wouldn't be safe to assume it will continue > >working. > I am mostly thinking at the client side using as well udp and tcp. > Would the "NFSv4 client over UDP" behaves differently then the same > client over TCP ? > > >That aside, if you have a perfect network, > I can trust my network. It's not a WAN, it's located into a very > massive cluster (it's kind of "internal LAN"). It's a very high > throughput network (IB based) so I believe there are less "hardware > based reason" to loose packets. > > >NFSv4.0 at least will > >probably work. > OK. > >(Not 4.1 since backchannel setup will fail?) > What is erroneous in using UDP for NFSv4.1 backchannels ? I assumed it would require some stronger notion of a "connection" than UDP could provide. But maybe it could be made to work somehow. > >Are you really sure that you can't make tcp scale to thousands of > >clients? > I am a bit afraid of a "No more file descriptors" effect. If I have > one TCP socket per client and thousands of clients, I have less > remaining fds for other purposes. Another point : UDP is a "cheap" > protocol. I can have bunches of clients without overloading the > server (a new client will almost cost nothing to the server, just > the cost of a new clientid negotiation) . Is the tcp state all that much more? > I was wondering if it > could be reliable to use it for NFSv4 inside a large cluster. Well, it'd require some spec changes in any case. --b. ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2010-12-14 19:15 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2010-12-13 7:46 Side effects of having NFSv4 mounted over udp DENIEL Philippe 2010-12-13 19:04 ` Thomas Haynes 2010-12-13 19:16 ` J. Bruce Fields 2010-12-13 19:47 ` Trond Myklebust 2010-12-14 7:35 ` DENIEL Philippe 2010-12-14 19:15 ` J. Bruce Fields
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).