Re: [PATCH 2/2] rdma/cm: allow user to specify IP to DGID mapping

Or Gerlitz
2009-10-08T23:13:48Z

On Thu, Oct 8, 2009 at 1:42 AM, Sean Hefty <sean.hefty-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org> wrote:
> My intent, which differs from Jason's, was to fully support the existing
> librdmacm interfaces as they are defined.

yes, I agree this is the way to go

> Implementation wise, if the user of the librdmacm calls rdma_resolve_addr with a
> src address, it's easy.  Without the src address, it's hard, but I may just be
> missing some easy interface for finding the src address.

note that dst can map to multiple src addresses, so you're just
looking for one of them... its doable, I will get you the details if
you still need them

>>> If a user sets the wrong address mapping or route, they should only affect themselves

>> I wasn't sure to follow this comment, can you elaborate a bit more?

> I meant that if some bogus app wants to specify an IP to GID mapping that's
> invalid, the incorrect mapping should only affect connections for that app.

yes, this makes sense and I believe the rdma-cm code is written such
that one bugus ID doesn't leak its defections to other IDs

> I can somewhat implement an ACM + librdmacm solution entirely in user space by
> layering the librdmacm over libibcm.  Because of the event reporting, it would be limited
> in how it could be use, and is unlikely to be something that would ever be supported.

yes, it would be limited and not really supportable, going that way
for research / experimentation and development is fine, just make sure
to never release that...

> Technically, rdma_resolve_addr could remain unchanged, in which case it will do
> everything it does today, which may include sending an ARP. This is the
> specific operation that I'd like to avoid.

again, apps (both user and kernel ones) do use rdma_resolve_addr and
we want them to keep doing so (I thought we agreed on that). For
staging you may develop the type II address resolution prototype on
top of libibcm but later rdma_resolve_addr would call IBACM and then
sync with the kernel.

Basically, can we agree that rdma_resolve_addr(src, dst, timeout) of
type II it would look like

if (src)
rdma_bind(src)
else
call_some_user_space_networking_api_to_convert_dst_to_netdev/src
next, now we have dev/pkey
- call ACM to resolve dst IP to GID and use dev/pkey for that
- sync the kernel rdma_cm with the resolution if needed for the
state-machine (hopefully its not a must at this point and can be done
when calling set_path).

Or