Re: [PATCH 2/4] ib_core: implement XRC RCV qp's

[Jack Morgenstein <jackm-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>]

On Monday 10 May 2010 13:01, Jack Morgenstein wrote:
> I have an initial implementation of this which I will clean up and send you for review (actually, an
> implementation which has the ib_create_xrc_rcv_qp/ib_destroy_xrc_rcv_qp, and which does
> not need the low-level driver implementations of reg/unreg.  I started off with this
> implementation, and discontinued it when I noticed how different it was from the original
> XRC RCV implementation... but saved it just in case).
> 
I remember now why I discontinued developing this implementation.  There is no provision for core-layer-only
verbs in the core driver.

To implement ib_reg_xrc_rcv_qp/ib_unreg_xrc_rcv_qp (when I still had the create and destroy verbs),
I needed to declare these in the uverbs_cmd_mask in the low-level driver.

(from procedure mlx4_ib_add, in file hw/mlx4/main.c:)
       	ibdev->ib_dev.uverbs_cmd_mask |=
			(1ull << IB_USER_VERBS_CMD_CREATE_XRC_SRQ)	|
			(1ull << IB_USER_VERBS_CMD_OPEN_XRCD)	|
			(1ull << IB_USER_VERBS_CMD_CLOSE_XRCD)	|
			(1ull << IB_USER_VERBS_CMD_CREATE_XRC_RCV_QP)	|
			(1ull << IB_USER_VERBS_CMD_MODIFY_XRC_RCV_QP)	|
			(1ull << IB_USER_VERBS_CMD_QUERY_XRC_RCV_QP)	|
			(1ull << IB_USER_VERBS_CMD_DESTROY_XRC_RCV_QP)  |
==>			(1ull << IB_USER_VERBS_CMD_REG_XRC_RCV_QP)	|
==>			(1ull << IB_USER_VERBS_CMD_UNREG_XRC_RCV_QP);

with use of this mask in ib_uverbs_write:
	if (!(file->device->ib_dev->uverbs_cmd_mask & (1ull << hdr.command)))
		return -ENOSYS;

In order to fix this layering violation,I would have needed to do fairly ugly things in
ib_register_device -- to test if, say, the IB_USER_VERBS_CMD_CREATE_XRC_RCV_QP bit is
set -- and if yes, to also add bits at the core layer for IB_USER_VERBS_CMD_REG_XRC_RCV_QP and
IB_USER_VERBS_CMD_UNREG_XRC_RCV_QP. Ugh.

However, with your suggestion of combining the create/reg and the destroy/unreg into a single
verb, this difficulty goes away, and the low-level driver can still declare those verbs for its task of
creating and destroying the XRC RCV QPs.

I will continue with the reg/unreg implementation.

-Jack
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html