rdma_cm resource over use

rdma_cm resource over use

[Eli Cohen]

Hi,
the rdma_cm library opens all availabe devices and keeps them open
even after binding to a specific device/port. This eats resources from
other devices that would otherwise be available to other applications.
Is there a way to avoid this? If not, maybe we should close all other
devices after binfing?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: rdma_cm resource over use

[Hefty, Sean]

> the rdma_cm library opens all availabe devices and keeps them open
> even after binding to a specific device/port. This eats resources from
> other devices that would otherwise be available to other applications.
> Is there a way to avoid this? If not, maybe we should close all other
> devices after binfing?

The librdmacm documents that all RDMA devices remain open while the librdmacm is loaded (see rdma_get_devices).  Also, a listen doesn't need to be bound to any specific device, but the connection request will be.  Personally, I really don't see this as a major use of resources.

- Sean


--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: rdma_cm resource over use

[Eli Cohen]

For ConnectX, mlx4 will consume a UAR page for each open of libibverbs
device file. On a 4K page architectures we usually have 1024 UAR
pages. On a 64K page size the situation becomes worth. Suppose you
have a system with more than one device. One of the devices is used by
rdam_cm and the other is not. You can easily exhaust all the
resources. We actually have this situation at a customer.

On Thu, Dec 9, 2010 at 8:30 PM, Hefty, Sean <sean.hefty-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org> wrote:
>> the rdma_cm library opens all availabe devices and keeps them open
>> even after binding to a specific device/port. This eats resources from
>> other devices that would otherwise be available to other applications.
>> Is there a way to avoid this? If not, maybe we should close all other
>> devices after binfing?
>
> The librdmacm documents that all RDMA devices remain open while the librdmacm is loaded (see rdma_get_devices).  Also, a listen doesn't need to be bound to any specific device, but the connection request will be.  Personally, I really don't see this as a major use of resources.
>
> - Sean
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html