Re: ibv_post_send/recv kernel path optimizations

[Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>]

On Wed, Dec 08, 2010 at 12:14:51PM +0000, Walukiewicz, Miroslaw wrote:
> Or, 
> 
> >I don't see why the ib uverbs flow (BTW - the data path has nothing to do with the 
> >rdma_cm, you're working with /dev/infiniband/uverbsX), can't be enhanced 
> >e.g to support shared-page which is allocated && mmaped from uverbs to 
> >user space and used in the same manner your implementation does.
> 
> The problem that I see is that the mmap is currently used for
> mapping of doorbell page in different drivers.
> 
> We can use it for mapping a page for transmit/receive operation when
> we are able to differentiate that we need to map Doorbell or our
> shared page.

There is the 64 bit file offset field to mmap which I think is
driver-specific. You could use 0 for the doorbell page, QPN*PAGE_SIZE
+ QPN_OFFSET for the per-QP page, etc..

> The second problem is that this rx/tx mmap should map the separate
> page per QP to avoid the unnecessary QP lookups so page identifier
> passed to mmap should be based on QP identifier.
> 
> I cannot find a specific code for /dev/infiniband/uverbsX. Is this
> device driver sharing the same functions like /dev/infiniband/rdmacm
> or it has own implementation.

It is in drivers/infiniband/core/uverbs*

For mmap the call is just routed to the driver's ib_dev mmap function,
so you can do whatever you want in your driver and match the
functionality in your userspace libibverbs driver library.

I think you should be able to implement your driver-specific
optimization within the uverbs framework - that would be best all
round.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html