Questions about RDMA subsystem shared mode for RoCE device with MLNX_OFED driver

Questions about RDMA subsystem shared mode for RoCE device with MLNX_OFED driver

[陈逸凡]

Questions:
1. Is RDMA shared mode supported for RoCE/iWARP devices? To be more
clearly, ibdev ant netdev required to be in the same net namespace or
not?
2. If the answer for first question is ‘YES’, but my test failed with
MLNX_OFED driver, it does check whether user can access the netdev of
the target gid attr, which means they(user and the netdev) should be
at the same namespace. Meanwhile the upstream code dose not have the
corresponding codes.


MLNX_OFED impl，form mlnx-ofa_kernel-23.10，compared to the upstream codes
---
@@ -1722,6 +1739,9 @@ static int ib_resolve_eth_dmac(struct ib_device *device,
 {
        int ret = 0;

+       if (!rdma_check_gid_user_access(ah_attr->grh.sgid_attr))
+               return -ENODEV;
+
        if (rdma_is_multicast_addr((struct in6_addr *)ah_attr->grh.dgid.raw)) {
                if (ipv6_addr_v4mapped((struct in6_addr
*)ah_attr->grh.dgid.raw)) {
                        __be32 addr = 0;
---

Its definition:
---
/**
 * rdma_check_gid_user_access - Check if user process can access
 * this GID entry or not.
 * @attr: Pointer to GID entry attribute
 *
 * rdma_check_gid_user_access() returns true if user process can access
 * this GID attribute otherwise returns false. This API should be called
 * from the userspace process context.
 */
bool rdma_check_gid_user_access(const struct ib_gid_attr *attr)
{
bool allow;
/*
 * For IB and iWarp, there is no netdevice associate with GID entry,
 * For RoCE consider the netdevice's net ns to validate against the
 * calling process.
 */
rcu_read_lock();
if (!attr->ndev ||
    (attr->ndev &&
     net_eq(dev_net(attr->ndev), current->nsproxy->net_ns)))
allow = true;
else
allow = false;
rcu_read_unlock();
return allow;
}
---

I think rdma_check_gid_user_access should be ignored while RDMA
subsystem configured as shared mode, It should works with exclusive
mode. Am i missing anything? Please tell me the background about why
MLNX_OFED driver perform the check if anyone knows.

Thanks!

Re: Questions about RDMA subsystem shared mode for RoCE device with MLNX_OFED driver

[Zhu Yanjun]

在 2024/1/19 20:31, 陈逸凡 写道:
> Questions:
> 1. Is RDMA shared mode supported for RoCE/iWARP devices? To be more
> clearly, ibdev ant netdev required to be in the same net namespace or
> not?

RoCE/RXE + the following commits can support ibdev in the net namespace.

https://patchwork.kernel.org/project/linux-rdma/cover/20230508075636.352138-1-yanjun.zhu@intel.com/

Current implementation of RXE does not support net namespace. We need 
the above commits to support net namespace.

IMO, the current implementation of iWARP does not support net namespace, 
too.

Zhu Yanjun

> 2. If the answer for first question is ‘YES’, but my test failed with
> MLNX_OFED driver, it does check whether user can access the netdev of
> the target gid attr, which means they(user and the netdev) should be
> at the same namespace. Meanwhile the upstream code dose not have the
> corresponding codes.
> 
> 
> MLNX_OFED impl，form mlnx-ofa_kernel-23.10，compared to the upstream codes
> ---
> @@ -1722,6 +1739,9 @@ static int ib_resolve_eth_dmac(struct ib_device *device,
>   {
>          int ret = 0;
> 
> +       if (!rdma_check_gid_user_access(ah_attr->grh.sgid_attr))
> +               return -ENODEV;
> +
>          if (rdma_is_multicast_addr((struct in6_addr *)ah_attr->grh.dgid.raw)) {
>                  if (ipv6_addr_v4mapped((struct in6_addr
> *)ah_attr->grh.dgid.raw)) {
>                          __be32 addr = 0;
> ---
> 
> Its definition:
> ---
> /**
>   * rdma_check_gid_user_access - Check if user process can access
>   * this GID entry or not.
>   * @attr: Pointer to GID entry attribute
>   *
>   * rdma_check_gid_user_access() returns true if user process can access
>   * this GID attribute otherwise returns false. This API should be called
>   * from the userspace process context.
>   */
> bool rdma_check_gid_user_access(const struct ib_gid_attr *attr)
> {
> bool allow;
> /*
>   * For IB and iWarp, there is no netdevice associate with GID entry,
>   * For RoCE consider the netdevice's net ns to validate against the
>   * calling process.
>   */
> rcu_read_lock();
> if (!attr->ndev ||
>      (attr->ndev &&
>       net_eq(dev_net(attr->ndev), current->nsproxy->net_ns)))
> allow = true;
> else
> allow = false;
> rcu_read_unlock();
> return allow;
> }
> ---
> 
> I think rdma_check_gid_user_access should be ignored while RDMA
> subsystem configured as shared mode, It should works with exclusive
> mode. Am i missing anything? Please tell me the background about why
> MLNX_OFED driver perform the check if anyone knows.
> 
> Thanks!

Re: Questions about RDMA subsystem shared mode for RoCE device with MLNX_OFED driver

[Zhu Yanjun]

在 2024/1/21 16:51, Zhu Yanjun 写道:
> 在 2024/1/19 20:31, 陈逸凡 写道:
>> Questions:
>> 1. Is RDMA shared mode supported for RoCE/iWARP devices? To be more
>> clearly, ibdev ant netdev required to be in the same net namespace or
>> not?
> 
> RoCE/RXE + the following commits can support ibdev in the net namespace.
> 
> https://patchwork.kernel.org/project/linux-rdma/cover/20230508075636.352138-1-yanjun.zhu@intel.com/
> 
> Current implementation of RXE does not support net namespace. We need 
> the above commits to support net namespace.
> 
> IMO, the current implementation of iWARP does not support net namespace, 
> too.

Sorry. I misunderstand what you mean in your mail. Please ignore this mail.

Zhu Yanjun

> 
> Zhu Yanjun
> 
>> 2. If the answer for first question is ‘YES’, but my test failed with
>> MLNX_OFED driver, it does check whether user can access the netdev of
>> the target gid attr, which means they(user and the netdev) should be
>> at the same namespace. Meanwhile the upstream code dose not have the
>> corresponding codes.
>>
>>
>> MLNX_OFED impl，form mlnx-ofa_kernel-23.10，compared to the upstream 
>> codes
>> ---
>> @@ -1722,6 +1739,9 @@ static int ib_resolve_eth_dmac(struct ib_device 
>> *device,
>>   {
>>          int ret = 0;
>>
>> +       if (!rdma_check_gid_user_access(ah_attr->grh.sgid_attr))
>> +               return -ENODEV;
>> +
>>          if (rdma_is_multicast_addr((struct in6_addr 
>> *)ah_attr->grh.dgid.raw)) {
>>                  if (ipv6_addr_v4mapped((struct in6_addr
>> *)ah_attr->grh.dgid.raw)) {
>>                          __be32 addr = 0;
>> ---
>>
>> Its definition:
>> ---
>> /**
>>   * rdma_check_gid_user_access - Check if user process can access
>>   * this GID entry or not.
>>   * @attr: Pointer to GID entry attribute
>>   *
>>   * rdma_check_gid_user_access() returns true if user process can access
>>   * this GID attribute otherwise returns false. This API should be called
>>   * from the userspace process context.
>>   */
>> bool rdma_check_gid_user_access(const struct ib_gid_attr *attr)
>> {
>> bool allow;
>> /*
>>   * For IB and iWarp, there is no netdevice associate with GID entry,
>>   * For RoCE consider the netdevice's net ns to validate against the
>>   * calling process.
>>   */
>> rcu_read_lock();
>> if (!attr->ndev ||
>>      (attr->ndev &&
>>       net_eq(dev_net(attr->ndev), current->nsproxy->net_ns)))
>> allow = true;
>> else
>> allow = false;
>> rcu_read_unlock();
>> return allow;
>> }
>> ---
>>
>> I think rdma_check_gid_user_access should be ignored while RDMA
>> subsystem configured as shared mode, It should works with exclusive
>> mode. Am i missing anything? Please tell me the background about why
>> MLNX_OFED driver perform the check if anyone knows.
>>
>> Thanks!
> 

Re: Questions about RDMA subsystem shared mode for RoCE device with MLNX_OFED driver

[Leon Romanovsky]

On Fri, Jan 19, 2024 at 08:31:00PM +0800, 陈逸凡 wrote:
> Questions:
> 1. Is RDMA shared mode supported for RoCE/iWARP devices? To be more
> clearly, ibdev ant netdev required to be in the same net namespace or
> not?
> 2. If the answer for first question is ‘YES’, but my test failed with
> MLNX_OFED driver, it does check whether user can access the netdev of
> the target gid attr, which means they(user and the netdev) should be
> at the same namespace. Meanwhile the upstream code dose not have the
> corresponding codes.

Please contact your support representative for MLNX_OFED related questions.
Linux RDMA mailing list is not correct place for it.

Thanks