Proxy

Proxy

[maillists0]

With NFS4's support for referrals and Kerberos, it seems like the
original reasons to prevent re-exporting of an NFS share might no
longer exist. With fs-proxy making its way into the mainline kernel
and things like cachefilesd, there are also very good reasons to allow
it. A proxy server with a persistent cache could give the ability to
robustly use shares across a WAN or do failover pairs with no need for
more complex replication. Speaking as an end-user, this would be very
desirable.

I see that others have implemented proxies with user-space NFS, which
seems reasonable but not optimal. What is the obstacle to allowing
re-exports with the standard nfs implentation? Is it possible at the
moment to patch a kernel to make this work? Anyone have experience
with it? Any input is appreciated.

Re: Proxy

[J. Bruce Fields]

On Mon, May 03, 2010 at 12:53:15PM -0400, maillists0@gmail.com wrote:
> With NFS4's support for referrals and Kerberos, it seems like the
> original reasons to prevent re-exporting of an NFS share might no
> longer exist. With fs-proxy making its way into the mainline kernel
> and things like cachefilesd, there are also very good reasons to allow
> it. A proxy server with a persistent cache could give the ability to
> robustly use shares across a WAN or do failover pairs with no need for
> more complex replication. Speaking as an end-user, this would be very
> desirable.
> 
> I see that others have implemented proxies with user-space NFS, which
> seems reasonable but not optimal. What is the obstacle to allowing
> re-exports with the standard nfs implentation? Is it possible at the
> moment to patch a kernel to make this work? Anyone have experience
> with it? Any input is appreciated.

It's probably possible, but some kernel hacking would be required.

Off the top of my head:

	- filehandles: you probably can't pass your server's filehandles
	  unchanged back to your client.  At a minimum you'd want to add
	  a header allowing you to distinguish filehandles for the
	  different filesystems you export.  What if you get a
	  filehandle from the server that's already at the protocol's
	  maximum size?  Are you going to try to maintain your own
	  persistent mapping of filehandles, and if so, is it possible
	  to do that with reasonable performance?
	- what do you do if your server takes a really long time to
	  answer a request?  Or stops responding completely?

--b.

Re: Proxy

[Trond Myklebust]

On Mon, 2010-05-03 at 14:56 -0400, J. Bruce Fields wrote: 
> On Mon, May 03, 2010 at 12:53:15PM -0400, maillists0@gmail.com wrote:
> > With NFS4's support for referrals and Kerberos, it seems like the
> > original reasons to prevent re-exporting of an NFS share might no
> > longer exist. With fs-proxy making its way into the mainline kernel
> > and things like cachefilesd, there are also very good reasons to allow
> > it. A proxy server with a persistent cache could give the ability to
> > robustly use shares across a WAN or do failover pairs with no need for
> > more complex replication. Speaking as an end-user, this would be very
> > desirable.
> > 
> > I see that others have implemented proxies with user-space NFS, which
> > seems reasonable but not optimal. What is the obstacle to allowing
> > re-exports with the standard nfs implentation? Is it possible at the
> > moment to patch a kernel to make this work? Anyone have experience
> > with it? Any input is appreciated.
> 
> It's probably possible, but some kernel hacking would be required.
> 
> Off the top of my head:
> 
> 	- filehandles: you probably can't pass your server's filehandles
> 	  unchanged back to your client.  At a minimum you'd want to add
> 	  a header allowing you to distinguish filehandles for the
> 	  different filesystems you export.  What if you get a
> 	  filehandle from the server that's already at the protocol's
> 	  maximum size?  Are you going to try to maintain your own
> 	  persistent mapping of filehandles, and if so, is it possible
> 	  to do that with reasonable performance?
> 	- what do you do if your server takes a really long time to
> 	  answer a request?  Or stops responding completely?

      * If you want to use Kerberos, then how do you proxy an RPCSEC_GSS
        session? 
      * How does the proxy server figure out the real server's export
        rules so that it can re-export them?

Re: Proxy

[maillists0]

On Mon, May 3, 2010 at 3:25 PM, Trond Myklebust
<trond.myklebust@fys.uio.no> wrote:
> On Mon, 2010-05-03 at 14:56 -0400, J. Bruce Fields wrote:
>> On Mon, May 03, 2010 at 12:53:15PM -0400, maillists0@gmail.com wrote:
>> > With NFS4's support for referrals and Kerberos, it seems like the
>> > original reasons to prevent re-exporting of an NFS share might no
>> > longer exist. With fs-proxy making its way into the mainline kernel
>> > and things like cachefilesd, there are also very good reasons to allow
>> > it. A proxy server with a persistent cache could give the ability to
>> > robustly use shares across a WAN or do failover pairs with no need for
>> > more complex replication. Speaking as an end-user, this would be very
>> > desirable.
>> >
>> > I see that others have implemented proxies with user-space NFS, which
>> > seems reasonable but not optimal. What is the obstacle to allowing
>> > re-exports with the standard nfs implentation? Is it possible at the
>> > moment to patch a kernel to make this work? Anyone have experience
>> > with it? Any input is appreciated.
>>
>> It's probably possible, but some kernel hacking would be required.
>>

Have a look at this old thing from 2006:
http://www.usenix.org/event/fast07/tech/full_papers/gulati/gulati_html/nache.html
. They claim to have implemented a proxy with only the tools I
mentioned above, along with their own modified version of nfs to allow
multi-hops.

I have a workload of lots of reads/almost no writes, and their
approach makes sense. It would be a great feature. Is something
missing from that paper that makes it unrealistic?

Re: Proxy

[Trond Myklebust]

On Mon, 2010-05-03 at 17:14 -0400, maillists0@gmail.com wrote: 
> On Mon, May 3, 2010 at 3:25 PM, Trond Myklebust
> <trond.myklebust@fys.uio.no> wrote:
> > On Mon, 2010-05-03 at 14:56 -0400, J. Bruce Fields wrote:
> >> On Mon, May 03, 2010 at 12:53:15PM -0400, maillists0@gmail.com wrote:
> >> > With NFS4's support for referrals and Kerberos, it seems like the
> >> > original reasons to prevent re-exporting of an NFS share might no
> >> > longer exist. With fs-proxy making its way into the mainline kernel
> >> > and things like cachefilesd, there are also very good reasons to allow
> >> > it. A proxy server with a persistent cache could give the ability to
> >> > robustly use shares across a WAN or do failover pairs with no need for
> >> > more complex replication. Speaking as an end-user, this would be very
> >> > desirable.
> >> >
> >> > I see that others have implemented proxies with user-space NFS, which
> >> > seems reasonable but not optimal. What is the obstacle to allowing
> >> > re-exports with the standard nfs implentation? Is it possible at the
> >> > moment to patch a kernel to make this work? Anyone have experience
> >> > with it? Any input is appreciated.
> >>
> >> It's probably possible, but some kernel hacking would be required.
> >>
> 
> Have a look at this old thing from 2006:
> http://www.usenix.org/event/fast07/tech/full_papers/gulati/gulati_html/nache.html
> . They claim to have implemented a proxy with only the tools I
> mentioned above, along with their own modified version of nfs to allow
> multi-hops.
> 
> I have a workload of lots of reads/almost no writes, and their
> approach makes sense. It would be a great feature. Is something
> missing from that paper that makes it unrealistic?

Possibly not for your workload, but none of the issues Bruce and I
raised appear to be addressed in that paper.

Furthermore, we do know several of the authors, and none of them have
ever approached us with a proposal to merge their implementation. I
therefore assume that it was written more as a proof of concept in
support of the paper, rather than something IBM is actually planning to
market.

Cheers
  Trond