NFS client virtualization plan

NFS client virtualization plan

[Stanislav Kinsbursky]

Hello.
I'm going to virtualize NFS in mainline and here is my plan for NFS client part.
Any suggestions or criticism are welcome.

==============================================================================

"NFS client" virtualization plan:

_RPC layer_:

1) Make rpcbind clients (rpcb_local_clnt and rpcb_local4_clnt) to be
per net namespace.

_NFS layer_:

1) Net namespace inheritance (current->nsproxy->net have to be used instead
of init_net).
Pointer to net namespace can be stored on nfs_client structure, which will
give easy access to proper net namespace during RPC transports creation.

2) Make Lockd kthread able to handle requests from different net namespaces.

3) Make NFS callbacks kthreads (nfs4_callback_svc and nfs4.1_callback_svc) able
to handle requests from different net namespaces.

4) Make NFS info lists (nfs_volume_list and nfs_client_list) to be per
net namespace in addition to global ones.

5) Make NFS proc variables and it's internal content variables to be per
net namespace in addition to global ones.

6) Make NFS sysctl variables and it's internal content variables to be per
net namespace in addition to global ones.



-- 
Best regards,
Stanislav Kinsbursky

Re: NFS client virtualization plan

[Stanislav Kinsbursky]

Added maintainer to mail recipients.
Trond, please, have a look.
Sorry for inconvenience.

16.08.2011 15:00, Stanislav Kinsbursky пишет:
> Hello.
> I'm going to virtualize NFS in mainline and here is my plan for NFS client part.
> Any suggestions or criticism are welcome.
>
> ==============================================================================
>
> "NFS client" virtualization plan:
>
> _RPC layer_:
>
> 1) Make rpcbind clients (rpcb_local_clnt and rpcb_local4_clnt) to be
> per net namespace.
>
> _NFS layer_:
>
> 1) Net namespace inheritance (current->nsproxy->net have to be used instead
> of init_net).
> Pointer to net namespace can be stored on nfs_client structure, which will
> give easy access to proper net namespace during RPC transports creation.
>
> 2) Make Lockd kthread able to handle requests from different net namespaces.
>
> 3) Make NFS callbacks kthreads (nfs4_callback_svc and nfs4.1_callback_svc) able
> to handle requests from different net namespaces.
>
> 4) Make NFS info lists (nfs_volume_list and nfs_client_list) to be per
> net namespace in addition to global ones.
>
> 5) Make NFS proc variables and it's internal content variables to be per
> net namespace in addition to global ones.
>
> 6) Make NFS sysctl variables and it's internal content variables to be per
> net namespace in addition to global ones.
>
>
>


-- 
Best regards,
Stanislav Kinsbursky

Re: NFS client virtualization plan

[Ben Greear]

On 08/16/2011 04:08 AM, Stanislav Kinsbursky wrote:
> Added maintainer to mail recipients.
> Trond, please, have a look.
> Sorry for inconvenience.

I have a set of patches that let NFS clients bind to a specific IP
address, but I haven't gotten any feedback in the month since
I posted them.

But, likely you will want to touch similar paths to what I touched.

Here's my patch series:

https://lkml.org/lkml/2011/7/15/382


You need some patches to mount.nfs
for full functionality, in case you want to try it out:

https://github.com/greearb/nfs-utils-ct

Thanks,
Ben


>
> 16.08.2011 15:00, Stanislav Kinsbursky пишет:
>> Hello.
>> I'm going to virtualize NFS in mainline and here is my plan for NFS client part.
>> Any suggestions or criticism are welcome.
>>
>> ==============================================================================
>>
>> "NFS client" virtualization plan:
>>
>> _RPC layer_:
>>
>> 1) Make rpcbind clients (rpcb_local_clnt and rpcb_local4_clnt) to be
>> per net namespace.
>>
>> _NFS layer_:
>>
>> 1) Net namespace inheritance (current->nsproxy->net have to be used instead
>> of init_net).
>> Pointer to net namespace can be stored on nfs_client structure, which will
>> give easy access to proper net namespace during RPC transports creation.
>>
>> 2) Make Lockd kthread able to handle requests from different net namespaces.
>>
>> 3) Make NFS callbacks kthreads (nfs4_callback_svc and nfs4.1_callback_svc) able
>> to handle requests from different net namespaces.
>>
>> 4) Make NFS info lists (nfs_volume_list and nfs_client_list) to be per
>> net namespace in addition to global ones.
>>
>> 5) Make NFS proc variables and it's internal content variables to be per
>> net namespace in addition to global ones.
>>
>> 6) Make NFS sysctl variables and it's internal content variables to be per
>> net namespace in addition to global ones.
>>
>>
>>
>
>


-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: NFS client virtualization plan

[Stanislav Kinsbursky]

Thanks. Will have a look.

16.08.2011 15:59, Ben Greear =D0=BF=D0=B8=D1=88=D0=B5=D1=82:
> On 08/16/2011 04:08 AM, Stanislav Kinsbursky wrote:
>> Added maintainer to mail recipients.
>> Trond, please, have a look.
>> Sorry for inconvenience.
>
> I have a set of patches that let NFS clients bind to a specific IP
> address, but I haven't gotten any feedback in the month since
> I posted them.
>
> But, likely you will want to touch similar paths to what I touched.
>
> Here's my patch series:
>
> https://lkml.org/lkml/2011/7/15/382
>
>
> You need some patches to mount.nfs
> for full functionality, in case you want to try it out:
>
> https://github.com/greearb/nfs-utils-ct
>
> Thanks,
> Ben
>
>
>>
>> 16.08.2011 15:00, Stanislav Kinsbursky =D0=BF=D0=B8=D1=88=D0=B5=D1=82=
:
>>> Hello.
>>> I'm going to virtualize NFS in mainline and here is my plan for NFS=
 client part.
>>> Any suggestions or criticism are welcome.
>>>
>>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D
>>>
>>> "NFS client" virtualization plan:
>>>
>>> _RPC layer_:
>>>
>>> 1) Make rpcbind clients (rpcb_local_clnt and rpcb_local4_clnt) to b=
e
>>> per net namespace.
>>>
>>> _NFS layer_:
>>>
>>> 1) Net namespace inheritance (current->nsproxy->net have to be used=
 instead
>>> of init_net).
>>> Pointer to net namespace can be stored on nfs_client structure, whi=
ch will
>>> give easy access to proper net namespace during RPC transports crea=
tion.
>>>
>>> 2) Make Lockd kthread able to handle requests from different net na=
mespaces.
>>>
>>> 3) Make NFS callbacks kthreads (nfs4_callback_svc and nfs4.1_callba=
ck_svc) able
>>> to handle requests from different net namespaces.
>>>
>>> 4) Make NFS info lists (nfs_volume_list and nfs_client_list) to be =
per
>>> net namespace in addition to global ones.
>>>
>>> 5) Make NFS proc variables and it's internal content variables to b=
e per
>>> net namespace in addition to global ones.
>>>
>>> 6) Make NFS sysctl variables and it's internal content variables to=
 be per
>>> net namespace in addition to global ones.
>>>
>>>
>>>
>>
>>
>
>


--=20
Best regards,
Stanislav Kinsbursky

Re: NFS client virtualization plan

[Chuck Lever]

On Aug 16, 2011, at 7:00 AM, Stanislav Kinsbursky wrote:

> Hello.
> I'm going to virtualize NFS in mainline and here is my plan for NFS client part.
> Any suggestions or criticism are welcome.

What are you going to do about statd?

> ==============================================================================
> 
> "NFS client" virtualization plan:
> 
> _RPC layer_:
> 
> 1) Make rpcbind clients (rpcb_local_clnt and rpcb_local4_clnt) to be
> per net namespace.
> 
> _NFS layer_:
> 
> 1) Net namespace inheritance (current->nsproxy->net have to be used instead
> of init_net).
> Pointer to net namespace can be stored on nfs_client structure, which will
> give easy access to proper net namespace during RPC transports creation.
> 
> 2) Make Lockd kthread able to handle requests from different net namespaces.
> 
> 3) Make NFS callbacks kthreads (nfs4_callback_svc and nfs4.1_callback_svc) able
> to handle requests from different net namespaces.
> 
> 4) Make NFS info lists (nfs_volume_list and nfs_client_list) to be per
> net namespace in addition to global ones.
> 
> 5) Make NFS proc variables and it's internal content variables to be per
> net namespace in addition to global ones.
> 
> 6) Make NFS sysctl variables and it's internal content variables to be per
> net namespace in addition to global ones.
> 
> 
> 
> -- 
> Best regards,
> Stanislav Kinsbursky
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
Chuck Lever
chuck[dot]lever[at]oracle[dot]com

Re: NFS client virtualization plan

[Stanislav Kinsbursky]

18.08.2011 19:54, Chuck Lever пишет:
>
> On Aug 16, 2011, at 7:00 AM, Stanislav Kinsbursky wrote:
>
>> Hello.
>> I'm going to virtualize NFS in mainline and here is my plan for NFS client part.
>> Any suggestions or criticism are welcome.
>
> What are you going to do about statd?
>

Do you mean user-space statd?
If so, what are the problems with it do you see?

>> ==============================================================================
>>
>> "NFS client" virtualization plan:
>>
>> _RPC layer_:
>>
>> 1) Make rpcbind clients (rpcb_local_clnt and rpcb_local4_clnt) to be
>> per net namespace.
>>
>> _NFS layer_:
>>
>> 1) Net namespace inheritance (current->nsproxy->net have to be used instead
>> of init_net).
>> Pointer to net namespace can be stored on nfs_client structure, which will
>> give easy access to proper net namespace during RPC transports creation.
>>
>> 2) Make Lockd kthread able to handle requests from different net namespaces.
>>
>> 3) Make NFS callbacks kthreads (nfs4_callback_svc and nfs4.1_callback_svc) able
>> to handle requests from different net namespaces.
>>
>> 4) Make NFS info lists (nfs_volume_list and nfs_client_list) to be per
>> net namespace in addition to global ones.
>>
>> 5) Make NFS proc variables and it's internal content variables to be per
>> net namespace in addition to global ones.
>>
>> 6) Make NFS sysctl variables and it's internal content variables to be per
>> net namespace in addition to global ones.
>>
>>
>>
>> --
>> Best regards,
>> Stanislav Kinsbursky
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>


-- 
Best regards,
Stanislav Kinsbursky

Re: NFS client virtualization plan

[Chuck Lever]

On Aug 19, 2011, at 5:46 AM, Stanislav Kinsbursky wrote:

> 18.08.2011 19:54, Chuck Lever пишет:
>> 
>> On Aug 16, 2011, at 7:00 AM, Stanislav Kinsbursky wrote:
>> 
>>> Hello.
>>> I'm going to virtualize NFS in mainline and here is my plan for NFS client part.
>>> Any suggestions or criticism are welcome.
>> 
>> What are you going to do about statd?
>> 
> 
> Do you mean user-space statd?
> If so, what are the problems with it do you see?

You go to the trouble of setting up multiple transports for rpcbind upcalls.  This suggests you plan to have an rpcbind in each net namespace.

On behalf of NFS, the kernel also performs idmapd, gssd, and statd (NSM) upcalls.  Do you intend to have separate instances of these daemons for each net namespace?

> 
>>> ==============================================================================
>>> 
>>> "NFS client" virtualization plan:
>>> 
>>> _RPC layer_:
>>> 
>>> 1) Make rpcbind clients (rpcb_local_clnt and rpcb_local4_clnt) to be
>>> per net namespace.
>>> 
>>> _NFS layer_:
>>> 
>>> 1) Net namespace inheritance (current->nsproxy->net have to be used instead
>>> of init_net).
>>> Pointer to net namespace can be stored on nfs_client structure, which will
>>> give easy access to proper net namespace during RPC transports creation.
>>> 
>>> 2) Make Lockd kthread able to handle requests from different net namespaces.
>>> 
>>> 3) Make NFS callbacks kthreads (nfs4_callback_svc and nfs4.1_callback_svc) able
>>> to handle requests from different net namespaces.
>>> 
>>> 4) Make NFS info lists (nfs_volume_list and nfs_client_list) to be per
>>> net namespace in addition to global ones.
>>> 
>>> 5) Make NFS proc variables and it's internal content variables to be per
>>> net namespace in addition to global ones.
>>> 
>>> 6) Make NFS sysctl variables and it's internal content variables to be per
>>> net namespace in addition to global ones.
>>> 
>>> 
>>> 
>>> --
>>> Best regards,
>>> Stanislav Kinsbursky
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> 
> 
> 
> -- 
> Best regards,
> Stanislav Kinsbursky
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
Chuck Lever
chuck[dot]lever[at]oracle[dot]com

Re: NFS client virtualization plan

[Stanislav Kinsbursky]

19.08.2011 20:12, Chuck Lever пишет:
>
> On Aug 19, 2011, at 5:46 AM, Stanislav Kinsbursky wrote:
>
>> 18.08.2011 19:54, Chuck Lever пишет:
>>>
>>> On Aug 16, 2011, at 7:00 AM, Stanislav Kinsbursky wrote:
>>>
>>>> Hello.
>>>> I'm going to virtualize NFS in mainline and here is my plan for NFS client part.
>>>> Any suggestions or criticism are welcome.
>>>
>>> What are you going to do about statd?
>>>
>>
>> Do you mean user-space statd?
>> If so, what are the problems with it do you see?
>
> You go to the trouble of setting up multiple transports for rpcbind upcalls.  This suggests you plan to have an rpcbind in each net namespace.
>
> On behalf of NFS, the kernel also performs idmapd, gssd, and statd (NSM) upcalls.  Do you intend to have separate instances of these daemons for each net namespace?
>

Actually, yes, I do. Thanks for notice.
Kernel NSM monitor routines will be virtualized.
For idmapd and gssd rpc_pipefs virtualization is required.
I wasn't going to virtualize rpc_pipefs for now - it was planned to be a part of NFS server virtualization, which is planned to be done after NFS client virtualization.

>>
>>>> ==============================================================================
>>>>
>>>> "NFS client" virtualization plan:
>>>>
>>>> _RPC layer_:
>>>>
>>>> 1) Make rpcbind clients (rpcb_local_clnt and rpcb_local4_clnt) to be
>>>> per net namespace.
>>>>
>>>> _NFS layer_:
>>>>
>>>> 1) Net namespace inheritance (current->nsproxy->net have to be used instead
>>>> of init_net).
>>>> Pointer to net namespace can be stored on nfs_client structure, which will
>>>> give easy access to proper net namespace during RPC transports creation.
>>>>
>>>> 2) Make Lockd kthread able to handle requests from different net namespaces.
>>>>
>>>> 3) Make NFS callbacks kthreads (nfs4_callback_svc and nfs4.1_callback_svc) able
>>>> to handle requests from different net namespaces.
>>>>
>>>> 4) Make NFS info lists (nfs_volume_list and nfs_client_list) to be per
>>>> net namespace in addition to global ones.
>>>>
>>>> 5) Make NFS proc variables and it's internal content variables to be per
>>>> net namespace in addition to global ones.
>>>>
>>>> 6) Make NFS sysctl variables and it's internal content variables to be per
>>>> net namespace in addition to global ones.
>>>>
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>> Stanislav Kinsbursky
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>
>>
>> --
>> Best regards,
>> Stanislav Kinsbursky
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>


-- 
Best regards,
Stanislav Kinsbursky

Re: NFS client virtualization plan

[J. Bruce Fields]

On Fri, Aug 19, 2011 at 09:21:16PM +0400, Stanislav Kinsbursky wrote:
> Actually, yes, I do. Thanks for notice.
> Kernel NSM monitor routines will be virtualized.
> For idmapd and gssd rpc_pipefs virtualization is required.
> I wasn't going to virtualize rpc_pipefs for now - it was planned to be a part of NFS server virtualization, which is planned to be done after NFS client virtualization.

You'll need rpc_pipefs for gssd and the old idmapd.

Also the new idmapper uses request_key.  I think there's a little
discussion about this in the archives.

--b.

Re: NFS client virtualization plan

[Stanislav Kinsbursky]

19.08.2011 23:32, J. Bruce Fields =D0=BF=D0=B8=D1=88=D0=B5=D1=82:
> On Fri, Aug 19, 2011 at 09:21:16PM +0400, Stanislav Kinsbursky wrote:
>> Actually, yes, I do. Thanks for notice.
>> Kernel NSM monitor routines will be virtualized.
>> For idmapd and gssd rpc_pipefs virtualization is required.
>> I wasn't going to virtualize rpc_pipefs for now - it was planned to =
be a part of NFS server virtualization, which is planned to be done aft=
er NFS client virtualization.
>
> You'll need rpc_pipefs for gssd and the old idmapd.
>
> Also the new idmapper uses request_key.  I think there's a little
> discussion about this in the archives.
>

Thanks, Bruce.
Nice to know, that you noticed this discussion.
I'll rpc_pipefs to the todo list for NFS client.

--=20
Best regards,
Stanislav Kinsbursky

Re: NFS client virtualization plan

[Jim Rees]

Stanislav Kinsbursky пишет:

  19.08.2011 23:32, J. Bruce Fields пишет:
  >On Fri, Aug 19, 2011 at 09:21:16PM +0400, Stanislav Kinsbursky wrote:
  >>Actually, yes, I do. Thanks for notice.
  >>Kernel NSM monitor routines will be virtualized.
  >>For idmapd and gssd rpc_pipefs virtualization is required.
  >>I wasn't going to virtualize rpc_pipefs for now - it was planned to be a part of NFS server virtualization, which is planned to be done after NFS client virtualization.
  >
  >You'll need rpc_pipefs for gssd and the old idmapd.
  >
  >Also the new idmapper uses request_key.  I think there's a little
  >discussion about this in the archives.
  >
  
  Thanks, Bruce.
  Nice to know, that you noticed this discussion.
  I'll rpc_pipefs to the todo list for NFS client.

As of 3.1, rpc_pipefs is also used by the block layout driver and blkmapd.