Re: [PATCH 2/4] ib_core: implement XRC RCV qp's

[Jack Morgenstein <jackm-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>]

On Thursday 06 May 2010 01:56, Roland Dreier wrote:
>  > In my original implementation, the low-level driver was responsible for generating
>  > the events for all the processes.  To move this mechanism to the core would require
>  > a fairly extensive re-write.  I would need to introduce ib_core methods for create,
>  > reg, unreg, and destroy, since the uverbs layer operates per user process and does
>  > not track across multiple processes.  I was concerned that modifications of this
>  > magnitude would introduce instability in XRC, and would require a significant QA cycle
> 
> It seems to me that it should, if anything, be easier to track all these
> lists at the uverbs layer.  We already have a global xrcd structure for
> each xrcd that we could easily stick a list of rcv QPs in (and keep the
> list of rcv QP registrations per rcv QP). 
> 
Roland,
While examining your proposed solution, I ran into the following difficulty.
In order to avoid the reg/unreg verbs in the low-level driver, I need to move
event handling for XRC RCV qp's to the core layer.

Currently, the low-level driver maintains, for each XRC RCV qp a list of user contexts
which are registered to that QP -- and generates an event for each of those contexts.
(this is so that each of the contexts may unregister, thus allowing the QP to be
destroyed).

To move this to the core requires that I maintain a global list of all xrc_rcv QPs,
and per QP, a list of contexts registered to it -- regardless of which xrc domain the
xrc rcv QP is attached to (I do not have xrc domain information in the XRC RCV qp event).

Thus, I do not see how we can reasonably use the global xrcd structure for this purpose.
(I don't see searching all xrc domains for the QP as a reasonable solution).

I am therefore back at my proposal above to introduce ib_core methods: ib_reg_xrc_rcv_qp
and ib_unreg_xrc_rcv_qp (using the reg for create as well, and the unreg for destroy as well,
as you suggested in a subsequent mail).  That way, I can track xrc rcv QPs, and keep a list
per QP of process files to signal an event to.

I have an initial implementation of this which I will clean up and send you for review (actually, an
implementation which has the ib_create_xrc_rcv_qp/ib_destroy_xrc_rcv_qp, and which does
not need the low-level driver implementations of reg/unreg.  I started off with this
implementation, and discontinued it when I noticed how different it was from the original
XRC RCV implementation... but saved it just in case).

Comments?
-Jack
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html