why does each NFS mount have (at least) 2 rpc_clnt's ?

why does each NFS mount have (at least) 2 rpc_clnt's ?

[Jeff Layton]

I'm working on an issue in an older kernel where we see occasional
panics when trying to refresh credentials. Here's the bug in case
anyone is interested:

    https://bugzilla.redhat.com/show_bug.cgi?id=572870

...I think I understand the problem well enough now. The problem is
pretty complex, but the issue is that some operations are done using
credentials from a stateowner associated with a nfs_client, but using
the rpc_clnt in nfs_server->client. The two can have different
authtypes if there are a mix of mounts with different authtypes to the
same server. This problem seems to have been fixed in mainline with the
introduction of the auth_generic code.

It leaves me wondering though...what exactly is the reason for having
two rpc_clients per NFS mount? To clarify, I'm talking about these two,
which seem to be somewhat redundant:

nfs_server->client
nfs_server->nfs_client->cl_rpcclient

On mount, the nfs4_set_client calls nfs_get_client to search the list
of nfs_client structs until it finds one that matches the address, port,
etc of the NFS server. If one isn't found, the kernel creates one using
whatever authtype was requested for the mount.

Later, nfs_init_server_rpcclient looks at the rpc_clnt in the
nfs_client and copies it. If the auth pseudoflavor doesn't match
however, it creates a new rpc_auth for it.

What exactly is the point of having two rpc_clnt's? Why not just get
always use nfs_client->cl_rpcclient instead of nfs_server->client and
simply have nfs_get_client filter by authtype?

-- 
Jeff Layton <jlayton@redhat.com>
_______________________________________________
NFSv4 mailing list
NFSv4@linux-nfs.org
http://linux-nfs.org/cgi-bin/mailman/listinfo/nfsv4

Re: why does each NFS mount have (at least) 2 rpc_clnt's ?

[Trond Myklebust]

On Thu, 2010-04-01 at 08:46 -0400, Jeff Layton wrote: 
> I'm working on an issue in an older kernel where we see occasional
> panics when trying to refresh credentials. Here's the bug in case
> anyone is interested:
> 
>     https://bugzilla.redhat.com/show_bug.cgi?id=572870
> 
> ...I think I understand the problem well enough now. The problem is
> pretty complex, but the issue is that some operations are done using
> credentials from a stateowner associated with a nfs_client, but using
> the rpc_clnt in nfs_server->client. The two can have different
> authtypes if there are a mix of mounts with different authtypes to the
> same server. This problem seems to have been fixed in mainline with the
> introduction of the auth_generic code.
> 
> It leaves me wondering though...what exactly is the reason for having
> two rpc_clients per NFS mount? To clarify, I'm talking about these two,
> which seem to be somewhat redundant:
> 
> nfs_server->client
> nfs_server->nfs_client->cl_rpcclient
> 
> On mount, the nfs4_set_client calls nfs_get_client to search the list
> of nfs_client structs until it finds one that matches the address, port,
> etc of the NFS server. If one isn't found, the kernel creates one using
> whatever authtype was requested for the mount.
> 
> Later, nfs_init_server_rpcclient looks at the rpc_clnt in the
> nfs_client and copies it. If the auth pseudoflavor doesn't match
> however, it creates a new rpc_auth for it.
> 
> What exactly is the point of having two rpc_clnt's? Why not just get
> always use nfs_client->cl_rpcclient instead of nfs_server->client and
> simply have nfs_get_client filter by authtype?
> 

Look again at nfs_init_server_rpcclient(). The pseudoflavour is not the
only thing that is changed. We also change the soft flag and the timeout
properties of the server->client.

The point is that users sometimes want to specify per-mountpoint
transport properties, and so we try to give them that possibility, while
at the same time sharing sockets/rdma connections.

Cheers
  Trond

_______________________________________________
NFSv4 mailing list
NFSv4@linux-nfs.org
http://linux-nfs.org/cgi-bin/mailman/listinfo/nfsv4

Re: why does each NFS mount have (at least) 2 rpc_clnt's ?

[Ondrej Valousek]

Hi Trond,

Does it have anything to do with the situation when we want an the same 
filesystem to be mounted twice somewhere else with a different flags? 
(i.e. RO/RW for example)
If yes, I remember some discussion about it in the past (the above was 
not possible).
Thanks,

Ondrej
> Look again at nfs_init_server_rpcclient(). The pseudoflavour is not the
> only thing that is changed. We also change the soft flag and the timeout
> properties of the server->client.
>
> The point is that users sometimes want to specify per-mountpoint
> transport properties, and so we try to give them that possibility, while
> at the same time sharing sockets/rdma connections.
>
> Cheers
>   Trond
>
> _______________________________________________
> NFSv4 mailing list
> NFSv4@linux-nfs.org
> http://linux-nfs.org/cgi-bin/mailman/listinfo/nfsv4
>   

Re: why does each NFS mount have (at least) 2 rpc_clnt's ?

[Trond Myklebust]

On Thu, 2010-04-01 at 15:26 +0200, Ondrej Valousek wrote: 
> Hi Trond,
> 
> Does it have anything to do with the situation when we want an the same 
> filesystem to be mounted twice somewhere else with a different flags? 
> (i.e. RO/RW for example)
> If yes, I remember some discussion about it in the past (the above was 
> not possible).
> Thanks,

Yes. That's exactly what it is supposed to allow (and btw, mounting
filesystems both ro and rw should be possible now).

Cheers
  Trond

> Ondrej
> > Look again at nfs_init_server_rpcclient(). The pseudoflavour is not the
> > only thing that is changed. We also change the soft flag and the timeout
> > properties of the server->client.
> >
> > The point is that users sometimes want to specify per-mountpoint
> > transport properties, and so we try to give them that possibility, while
> > at the same time sharing sockets/rdma connections.
> >
> > Cheers
> >   Trond
> >
> > _______________________________________________
> > NFSv4 mailing list
> > NFSv4@linux-nfs.org
> > http://linux-nfs.org/cgi-bin/mailman/listinfo/nfsv4
> >   
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: why does each NFS mount have (at least) 2 rpc_clnt's ?

[Ondrej Valousek]

[-- Attachment #1: Type: text/html, Size: 651 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
NFSv4 mailing list
NFSv4@linux-nfs.org
http://linux-nfs.org/cgi-bin/mailman/listinfo/nfsv4

Re: why does each NFS mount have (at least) 2 rpc_clnt's ?

[Jeff Layton]

On Thu, 01 Apr 2010 09:17:49 -0400
Trond Myklebust <trond.myklebust@fys.uio.no> wrote:

> On Thu, 2010-04-01 at 08:46 -0400, Jeff Layton wrote: 
> > I'm working on an issue in an older kernel where we see occasional
> > panics when trying to refresh credentials. Here's the bug in case
> > anyone is interested:
> > 
> >     https://bugzilla.redhat.com/show_bug.cgi?id=572870
> > 
> > ...I think I understand the problem well enough now. The problem is
> > pretty complex, but the issue is that some operations are done using
> > credentials from a stateowner associated with a nfs_client, but using
> > the rpc_clnt in nfs_server->client. The two can have different
> > authtypes if there are a mix of mounts with different authtypes to the
> > same server. This problem seems to have been fixed in mainline with the
> > introduction of the auth_generic code.
> > 
> > It leaves me wondering though...what exactly is the reason for having
> > two rpc_clients per NFS mount? To clarify, I'm talking about these two,
> > which seem to be somewhat redundant:
> > 
> > nfs_server->client
> > nfs_server->nfs_client->cl_rpcclient
> > 
> > On mount, the nfs4_set_client calls nfs_get_client to search the list
> > of nfs_client structs until it finds one that matches the address, port,
> > etc of the NFS server. If one isn't found, the kernel creates one using
> > whatever authtype was requested for the mount.
> > 
> > Later, nfs_init_server_rpcclient looks at the rpc_clnt in the
> > nfs_client and copies it. If the auth pseudoflavor doesn't match
> > however, it creates a new rpc_auth for it.
> > 
> > What exactly is the point of having two rpc_clnt's? Why not just get
> > always use nfs_client->cl_rpcclient instead of nfs_server->client and
> > simply have nfs_get_client filter by authtype?
> > 
> 
> Look again at nfs_init_server_rpcclient(). The pseudoflavour is not the
> only thing that is changed. We also change the soft flag and the timeout
> properties of the server->client.
> 
> The point is that users sometimes want to specify per-mountpoint
> transport properties, and so we try to give them that possibility, while
> at the same time sharing sockets/rdma connections.
> 
> Cheers
>   Trond
> 

Ok, I think I sort of understand now. The nfs_client holds some global
state info that's common to all nfs mounts on the server so that info
has to be shared between NFS mounts. The nfs_server struct holds stuff
that's specific to a single superblock.

That said, I still am not certain I see why we want to have an rpc_clnt
that's shared between the mounts though clearly there's benefit to
sharing the xprt between them.

-- 
Jeff Layton <jlayton@redhat.com>
_______________________________________________
NFSv4 mailing list
NFSv4@linux-nfs.org
http://linux-nfs.org/cgi-bin/mailman/listinfo/nfsv4

Re: why does each NFS mount have (at least) 2 rpc_clnt's ?

[Trond Myklebust]

On Thu, 2010-04-01 at 13:26 -0400, Jeff Layton wrote: 
> That said, I still am not certain I see why we want to have an rpc_clnt
> that's shared between the mounts though clearly there's benefit to
> sharing the xprt between them.

Sharing the xprt is by far the main (and most important) benefit.
Without it, we would have the same mess with running out of privileged
port numbers that we had with the 2.4.x series.

Cheers
  Trond