Different signatures to ibv_open_xrcd - branch issue?

Different signatures to ibv_open_xrcd - branch issue?

[Shlomo Pongratz]

Hi Sean,

In libibverbs/include/infiniband/verbs.h the API "ibv_open_xrcd" is 
defined with the following signature;
static inline struct ibv_xrcd *
ibv_open_xrcd(struct ibv_context *context, struct ibv_xrcd_init_attr 
*xrcd_init_attr)
And libibverbs/examples/xsrq_pingpong.c the call is O.K. however in 
librdmacm/examples/rdma_xserver.c I see a different call sequence i.e. 
"attr.ext.xrc.xrcd = ibv_open_xrcd(srq_id->verbs, -1, 0);"
My git branch for libivvers is based on origin/xrc2 and for librdmacm is 
based on origin/xrc.

Is any of my branches wrong?

best regards,

S.P.




--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: Different signatures to ibv_open_xrcd - branch issue?

[Hefty, Sean]

> In libibverbs/include/infiniband/verbs.h the API "ibv_open_xrcd" is
> defined with the following signature;
> static inline struct ibv_xrcd *
> ibv_open_xrcd(struct ibv_context *context, struct ibv_xrcd_init_attr
> *xrcd_init_attr)
> And libibverbs/examples/xsrq_pingpong.c the call is O.K. however in
> librdmacm/examples/rdma_xserver.c I see a different call sequence i.e.
> "attr.ext.xrc.xrcd = ibv_open_xrcd(srq_id->verbs, -1, 0);"
> My git branch for libivvers is based on origin/xrc2 and for librdmacm is
> based on origin/xrc.
> 
> Is any of my branches wrong?

The librdmacm changes are against an older version of the changes.  The libibverbs has the correct API, which incorporates the extension mechanism that's been discussed.  Once libibverbs has been accepted, the librdmacm can be updated.  (The librdmacm code is surrounded by a #define, which disables it from the build.)

- Sean
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Different signatures to ibv_open_xrcd - branch issue?

[Shlomo Pongratz]

On 3/25/2013 6:35 PM, Hefty, Sean wrote:
>> In libibverbs/include/infiniband/verbs.h the API "ibv_open_xrcd" is
>> defined with the following signature;
>> static inline struct ibv_xrcd *
>> ibv_open_xrcd(struct ibv_context *context, struct ibv_xrcd_init_attr
>> *xrcd_init_attr)
>> And libibverbs/examples/xsrq_pingpong.c the call is O.K. however in
>> librdmacm/examples/rdma_xserver.c I see a different call sequence i.e.
>> "attr.ext.xrc.xrcd = ibv_open_xrcd(srq_id->verbs, -1, 0);"
>> My git branch for libivvers is based on origin/xrc2 and for librdmacm is
>> based on origin/xrc.
>>
>> Is any of my branches wrong?
> The librdmacm changes are against an older version of the changes.  The libibverbs has the correct API, which incorporates the extension mechanism that's been discussed.  Once libibverbs has been accepted, the librdmacm can be updated.  (The librdmacm code is surrounded by a #define, which disables it from the build.)
>
> - Sean
I understand that currently I can't do RDMA using XRC, is that the case?
How complicated are the required modifications needed to make librdmacm 
complies to the libibverbs changes?



--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: Different signatures to ibv_open_xrcd - branch issue?

[Hefty, Sean]

> I understand that currently I can't do RDMA using XRC, is that the case?
> How complicated are the required modifications needed to make librdmacm
> complies to the libibverbs changes?

The changes needed for the librdmacm to support XRC aren't huge.  The complication is getting XRC merged with libibverbs.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Different signatures to ibv_open_xrcd - branch issue?

[Shlomo Pongratz]

On 3/26/2013 5:56 PM, Hefty, Sean wrote:
>> I understand that currently I can't do RDMA using XRC, is that the case?
>> How complicated are the required modifications needed to make librdmacm
>> complies to the libibverbs changes?
> The changes needed for the librdmacm to support XRC aren't huge.  The complication is getting XRC merged with libibverbs.
I think I missed something here, can I use your branch xrc2 of 
libibverbs for testing purpose, or there are still things to merge there?
And for librdmacm, I'm looking forward to try it with a real application 
the moment you'll have a branch witch is compatible with libibverbs.

S.P.

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: Different signatures to ibv_open_xrcd - branch issue?

[Hefty, Sean]

> I think I missed something here, can I use your branch xrc2 of
> libibverbs for testing purpose, or there are still things to merge there?

The xrc2 branch is usable.  I'm about 10-15 minutes away from posting v5 of those patches.

> And for librdmacm, I'm looking forward to try it with a real application
> the moment you'll have a branch witch is compatible with libibverbs.

The librdmacm needs to be updated to match the latest libibverbs.  If the libibverbs changes are ever merged upstream, I'll update the librdmacm at that point.

Until then, the only sample app for XRC is ibv_xsrq_pingpong.

- Sean
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Different signatures to ibv_open_xrcd - branch issue?

[Dominique Martinet]

Hi,

Hefty, Sean wrote on Tue, Mar 26, 2013 :
> > I think I missed something here, can I use your branch xrc2 of
> > libibverbs for testing purpose, or there are still things to merge there?
> 
> The xrc2 branch is usable.  I'm about 10-15 minutes away from posting v5 of those patches.

Sorry for hijacking this thread, but I was actually running into some
troubles with libibverbs-1.1.4 and a recent kernel, using
ibv_open_xrc_domain... Which is basically failing with an invalid
argument when writing to the uverb0 char device. 

So I guess the "commands" (IB_USER_VERBS_CMD_OPEN_XRCD vs
IB_USER_VERBS_CMD_OPEN_XRC_DOMAIN) changed since 2.6.32 and I would need
to use this libibverbs branch for now? (doesn't look like there is any
xrcd at all in 1.1.6 anyway)
What I don't get is that the abi_version did not change at all, if the
commands changed, shouldn't the kernel abi version be updated as well?
(None of the four abi_version I could find (ib_verbs, ib_mad, rdma_cm,
uverbs0 in the device) changed at all)

This looks like it's impossible to get the same library to fully work
for multiple kernel versions, while all the other basic features do work
quite well independantly.



Regards,
-- 
Dominique Martinet
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: Different signatures to ibv_open_xrcd - branch issue?

[Hefty, Sean]

> So I guess the "commands" (IB_USER_VERBS_CMD_OPEN_XRCD vs
> IB_USER_VERBS_CMD_OPEN_XRC_DOMAIN) changed since 2.6.32 and I would need
> to use this libibverbs branch for now? (doesn't look like there is any
> xrcd at all in 1.1.6 anyway)
> What I don't get is that the abi_version did not change at all, if the
> commands changed, shouldn't the kernel abi version be updated as well?
> (None of the four abi_version I could find (ib_verbs, ib_mad, rdma_cm,
> uverbs0 in the device) changed at all)

The *upstream* kernel ABI has not changed.  OFED provided kernel patches that modified previous kernels.  The ABI that OFED patched in differed from what was finally merged upstream.  I expect that the upstream libibverbs will work against all upstream kernels, with full binary compatibility for existing applications.  Your binary compatibility support with OFED will vary.

- Sean