Re: RDMA subsystem namespace related questions (was Re: Finding the namespace of a struct ib_device)

[Ka-Cheong Poon <ka-cheong.poon@oracle.com>]

On 10/5/20 10:25 PM, Jason Gunthorpe wrote:
> On Mon, Oct 05, 2020 at 09:57:47PM +0800, Ka-Cheong Poon wrote:
>>>> It is a kernel module.  Which FD are you referring to?  It is
>>>> unclear why a kernel module must associate itself with a user
>>>> space FD.  Is there a particular reason that rdma_create_id()
>>>> needs to behave differently than sock_create_kern() in this
>>>> regard?
>>>
>>> Somehow the kernel module has to be commanded to use this namespace,
>>> and generally I expect that command to be connected to FD.
>>
>>
>> It is an unnecessary restriction on what a kernel module
>> can do.  Is it a problem if a kernel module initiates its
>> own RDMA connection for doing various stuff in a namespace?
> 
> Yes, someone has to apply policy to authorize this. Kernel modules
> randomly running around using security objects is not OK.


The policy is to allow this.  It is not random stuff.
Can the RDMA subsystem support it?


> Kernel modules should not be doing networking unless commanded to by
> userspace.


It is still not clear why this is an issue with RDMA
connection, but not with general kernel socket.  It is
not random networking.  There is a purpose.


>> Any kernel module can initiate a TCP connection to do various
>> stuff without worrying about namespace deletion problem.  It
>> does not cause a problem AFAICT.  If the module needs to make
>> sure that the namespace does not go away, it can add its own
>> reference.  Is there a particular reason that RDMA subsystem
>> needs to behave differently?
> 
> We don't have those kinds of ULPs.


So if the reason of the current rdma_create_id() behavior
is that there is no such user, I am adding one.  It should
be clear that this difference between kernel socket and
rdma_create_id() causes a problem in namespace handling.


>> For scalability and namespace separation reasons as cma_wq is
>> single threaded.  For example, there can be many work to be done
>> in one namespace.  But this should not have an adverse effect on
>> other namespaces (as long as there are resources available).
> 
> This is a design issue of the cma_wq, it can be reworked to not need
> single threaded, nothing to do with namespaces


As mentioned, there are at least two parts.  The above is
on scalability.  There is also the namespace separation reason.
The goal is to make sure that processing of one namespace
should not have unwanted (positive nor negative) effect on
processing of other namespaces.  If the cma_wq is re-designed,
number of namespaces should be one input parameter on creating
how many threads and other resources allocation/scheduling.
One cma_wq per namespace is the simplest allocation.


-- 
K. Poon
ka-cheong.poon@oracle.com