Re: Finding the namespace of a struct ib_device

[Ka-Cheong Poon <ka-cheong.poon@oracle.com>]

On 9/7/20 3:18 PM, Leon Romanovsky wrote:
> On Mon, Sep 07, 2020 at 11:33:38AM +0800, Ka-Cheong Poon wrote:
>> On 9/6/20 3:44 PM, Leon Romanovsky wrote:
>>> On Fri, Sep 04, 2020 at 10:02:10PM +0800, Ka-Cheong Poon wrote:
>>>> On 9/4/20 7:32 PM, Jason Gunthorpe wrote:
>>>>> On Fri, Sep 04, 2020 at 12:01:12PM +0800, Ka-Cheong Poon wrote:
>>>>>> On 9/4/20 1:39 AM, Jason Gunthorpe wrote:
>>>>>>> On Thu, Sep 03, 2020 at 10:02:01PM +0800, Ka-Cheong Poon wrote:
>>>>>>>> When a struct ib_client's add() function is called. is there a
>>>>>>>> supported method to find out the namespace of the passed in
>>>>>>>> struct ib_device?  There is rdma_dev_access_netns() but it does
>>>>>>>> not return the namespace.  It seems that it needs to have
>>>>>>>> something like the following.
>>>>>>>>
>>>>>>>> struct net *rdma_dev_to_netns(struct ib_device *ib_dev)
>>>>>>>> {
>>>>>>>>            return read_pnet(&ib_dev->coredev.rdma_net);
>>>>>>>> }
>>>>>>>>
>>>>>>>> Comments?
>>>>>>>
>>>>>>> I suppose, but why would something need this?
>>>>>>
>>>>>>
>>>>>> If the client needs to allocate stuff for the namespace
>>>>>> related to that device, it needs to know the namespace of
>>>>>> that device.  Then when that namespace is deleted, the
>>>>>> client can clean up those related stuff as the client's
>>>>>> namespace exit function can be called before the remove()
>>>>>> function is triggered in rdma_dev_exit_net().  Without
>>>>>> knowing the namespace of that device, coordination cannot
>>>>>> be done.
>>>>>
>>>>> Since each device can only be in one namespace, why would a client
>>>>> ever need to allocate at a level more granular than a device?
>>>>
>>>>
>>>> A client wants to have namespace specific info.  If the
>>>> device belongs to a namespace, it wants to associate those
>>>> info with that device.  When a namespace is deleted, the
>>>> info will need to be deleted.  You can consider the info
>>>> as associated with both a namespace and a device.
>>>
>>> Can you be more specific about which info you are talking about?
>>
>>
>> Actually, a lot of info can be both namespace and device specific.
>> For example, a client wants to have a different PD allocation policy
>> with a device when used in different namespaces.
>>
>>
>>> And what is the client that is net namespace-aware from one side,
>>> but from another separate data between them "manually"?
>>
>>
>> Could you please elaborate what is meant by "namespace aware from
>> one side but from another separate data between them manually"?
>> I understand what namespace aware means.  But it is not clear what
>> is meant by "separating data manually".  Do you mean having different
>> behavior in different namespaces?  If this is the case, there is
>> nothing special here.  An admin may choose to have different behavior
>> in different namespaces.  There is nothing manual going on in the
>> client code.
> 
> We are talking about net-namespaces, and as we wrote above, the ib_device
> that supports such namespace can exist only in a single one
> 
> The client that implemented such support can check its namespace while
> "client->add" is called. It should be equal to be seen by ib_device.
> 
> See:
>   rdma_dev_change_netns ->
>   	enable_device_and_get ->
> 		add_client_context ->
> 			client->add(device)


This is the original question.  How does the client's add() function
know the namespace of device?  What is your suggestion in finding
the net namespace of device at add() time?


> "Manual" means that client will store results of first client->add call
> (in init_net NS) and will use globally stored data for other NS, which
> is not netdev way to work with namespaces. The expectation that they are
> separated without shared data between.


It is not clear why client needs to use globally stored data for other
net namespaces.  When an RDMA device is moved from init_net to another net
namespace, the client's remove() function is called first, and then the
client's add() function is called.  If a client can know the net namespace
of a device when add()/remove() is called, it can use namespace specific
data storage.  It does not need to store namespace specific data in a
global store.  The original question is how to find out the net namespace
of a device.



-- 
K. Poon
ka-cheong.poon@oracle.com