Re: [PATCH 2/2] rdma/cm: allow user to specify IP to DGID mapping

[Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>]

On Tue, Oct 20, 2009 at 01:05:15PM -0700, Sean Hefty wrote:
> >> I agree.  But we can still support AF_IB/PS_TCP by simply assigning
> >> the service ID correctly.  rdma_bind_addr only needs to fill in a
> >> service id if one is not given.  This should enable
> >
> >'one is not given' == 0 service ID?
> 
> yes
> 
> I'm guessing PS_IB would need to assign from the IB CM assigned range starting
> at 0x020::0 to avoid conflicts.

Something like that..

Ok, this seems sensible to me. So to recap the main differences:

Private data:
- AF_IB/PS_TCP - the kernel munges the private data to be compatible
  with AF_INET/PS_TCP, but otherwise is the same.
- AF_IB/PS_IB - the kernel doesn't touch the private data.

Service ID for bind or connect:
 - AF_IB/PS_TCP - the service ID passed in through the sockaddr_ib
   must be 0 or correctly formed as a IP RDMA CM service ID indicating
   a port number. This is true in the bind or connect case.
 - AF_IB/PS_IB - the service ID is just a service ID. No restrictions
   or alteration are done. The bind'd service ID is ignored on connect

0 Service ID on bind:
 - AF_IB/PS_TCP - a new random unused port is allocated and converted to a
   service ID
 - AF_IB/PS_IB - a new random unused service ID within the proper prefix is
   allocated (usefull!)

What about the service_mask feature of IB CM?

How are the IP source and dest IPs going to be picked in for PS_TCP
mode?

I guess user space does that and passes it through to the kernel?
Another bit of binary blob data from rdma_getaddrinfo I guess. I
suppose rdma_getaddrinfo could return an array of option blobs that
must be copied to the kernel to setup the connection, or something
like that.

> >What functional difference are you imagining compared to listening on
> >AF_INET/PS_TCP?
> 
> I hadn't thought about this, but making stuff up, I guess this could allow using
> the rdma_cm without ipoib running...?  From IB's perspective, PS_TCP is just a
> specific subset of the service ID range.

Plus the IP addies in the REQ private data.. 

I think it would be fine to EINVAL on AF_IB/PS_TCP listen().

> >> Even once PS_IB is added, we need to make sure that it doesn't collide with
> >> PS_TCP at the IB service ID level, or IB will be in the same situation that
> >> iWarp is in with space collisions.
> >
> >I thought there was already a single service ID registration list that was
> >shared between RDMA CM and IB CM?
> 
> Not really - the rdma cm constructs the service id, and the ib cm checks that
> it's usable, but not until listen is called.

Oh I see, well, a 'bind' api into the IB CM should take care of that? 

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html