Simplified iWARP Consumer Library

Simplified iWARP Consumer Library

[Philip Frey]

[-- Attachment #1: Type: text/plain, Size: 1022 bytes --]

Dear all,

as announced last week (on general-ZwoEplunGu1OwGhvXhtEPSCwEArCW2h5@public.gmane.org), I have developed a 
shared library to simplify iWARP/RDMA application development on Linux in the 
course of my PhD. The library, called libiwarp, is based on libibverbs and 
librdmacm - it is essentially a set of wrappers to hide some of the complexity.

Upon request, I hereby post the source code of the library (libiwarp) as well as 
a sample application that demonstrates the usage (libiwarp_sample_app). After 
having extracted the tarball, change to the libiwarp directory where you will 
find a README file explaining the installation and giving an overview of the 
library.

I would be very interested in your feedback! Also, if you have any questions, 
please do not hesitate to contact me.

Best regards,
  Philip

---
  Philip Frey
  Systems Group, Dept. of Computer Science
  Swiss Federal Institute of Technology (ETH), Zurich
  CH-8092 Zurich/Switzerland
  philip.frey-gy3b+zu4XSAfv37vnLkPlQ@public.gmane.org

[-- Attachment #2: libiwarp.tar.bz2 --]
[-- Type: application/octet-stream, Size: 31565 bytes --]

Re: Simplified iWARP Consumer Library

[Or Gerlitz]

Philip Frey wrote:
> as announced last week (on general-ZwoEplunGu1OwGhvXhtEPSCwEArCW2h5@public.gmane.org) [...] I 
> would be very interested in your feedback!
The general list isn't functional since last fall, as of such, your 
announcement wasn't seen by any of the non directly CCed recipients... 
can you please resend it here (linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org)?

Or.

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Simplified iWARP Consumer Library

[Philip Frey]

Dear all,

I was made aware of the fact that the general-ZwoEplunGu1OwGhvXhtEPSCwEArCW2h5@public.gmane.org mailinglist 
was deprecated and that my initial post about the iWARP library did not reach 
you. (The general list is still linked on openfabrics.org without a notice of it 
being deprecated.)

Please, find below a copy of the original message.

"Dear all,

throughout the course of my PhD, I have written a number of user applications 
based on the OFED verbs. After some time, I realized that many programming tasks 
are repetitive (particularly the connection management) and started to wrap them.

This eventually led to a slim, shared library called libiwarp which includes all 
the basic primitives of RDMA-based communication, implemented as wrappers around 
libibverbs. Throughout various research projects, the library has turned out to 
be quite useful especially for people who are not familiar with all the details 
of the OFED verbs (like master students that only have 6 months to complete 
their projects or people that simply want to play around with iWARP and run a 
few simple tests).

We have shown, by experiment, that the overhead induced by the wrappers is 
negligible, yet it simplifies writing RDMA-based applications significantly. 
Furthermore, as the library does not replace the OFED verbs, it does not impose 
any restrictions with regard to the original functionality. Whenever there is 
something missing in the libiwarp, you can always use the interface provided by 
the OFED verbs.

As I am now in the process of finishing my work, I was wondering whether there 
is an interest in the OpenFabrics community for such a library which provides a 
simplified interface for the OFED verbs. If there is, I would be glad to share 
my source code with you.

Many thanks for your consideration and best regards,
  Philip"

Or Gerlitz wrote:
> Philip Frey wrote:
>> as announced last week (on general-ZwoEplunGu1OwGhvXhtEPSCwEArCW2h5@public.gmane.org) [...] I 
>> would be very interested in your feedback!
> The general list isn't functional since last fall, as of such, your 
> announcement wasn't seen by any of the non directly CCed recipients... 
> can you please resend it here (linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org)?
> 
> Or.
> 
> -- 
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: Simplified iWARP Consumer Library

[Sean Hefty]

>as announced last week (on general-ZwoEplunGu1OwGhvXhtEPSCwEArCW2h5@public.gmane.org), I have developed a
>shared library to simplify iWARP/RDMA application development on Linux in the
>course of my PhD. The library, called libiwarp, is based on libibverbs and
>librdmacm - it is essentially a set of wrappers to hide some of the complexity.

I briefly toured through the code, and it seems to offer these main features:

1. Joins the rdma_cm_id and qp together into a single construct.
2. Supports synchronous rdma_cm operation.
3. Abstracts some initialization, like opening the device, creating PDs, etc.
4. Simplifies some verbs operations.

Would it be possible to submit patches against the librdmacm to integrate some
of these features?

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Simplified iWARP Consumer Library

[Philip Frey]

I'll have to look into that. So far, the library was intended as an add-on
to the existing code with the option to fall back to the original API if
need be.

Which of the features would you want to see in librdmacm? I guess the
integration of the cm_id and qp as well as the simpler (synchronous) connection
establishment and teardown functionality would fit. However, all cm events
will thereafter be handled within the library rather than being presented to the
user. Does that make sense?

Sean Hefty wrote:
>> as announced last week (on general-ZwoEplunGu1OwGhvXhtEPSCwEArCW2h5@public.gmane.org), I have developed a
>> shared library to simplify iWARP/RDMA application development on Linux in the
>> course of my PhD. The library, called libiwarp, is based on libibverbs and
>> librdmacm - it is essentially a set of wrappers to hide some of the complexity.
> 
> I briefly toured through the code, and it seems to offer these main features:
> 
> 1. Joins the rdma_cm_id and qp together into a single construct.
> 2. Supports synchronous rdma_cm operation.
> 3. Abstracts some initialization, like opening the device, creating PDs, etc.
> 4. Simplifies some verbs operations.
> 
> Would it be possible to submit patches against the librdmacm to integrate some
> of these features?
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: Simplified iWARP Consumer Library

[Sean Hefty]

>I'll have to look into that. So far, the library was intended as an add-on
>to the existing code with the option to fall back to the original API if
>need be.

I think we keep this concept.

>Which of the features would you want to see in librdmacm? I guess the
>integration of the cm_id and qp as well as the simpler (synchronous) connection
>establishment and teardown functionality would fit. However, all cm events
>will thereafter be handled within the library rather than being presented to
>the user. Does that make sense?

I wasn't thinking about replacing existing calls, but rather adding new
functions.  I would start with just trying to integrate one feature, probably
the simpler connection model for the active side, then look to expand from
there.

- Sean

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: Simplified iWARP Consumer Library

[Sean Hefty]

>>Which of the features would you want to see in librdmacm? I guess the
>>integration of the cm_id and qp as well as the simpler (synchronous)
>connection
>>establishment and teardown functionality would fit. However, all cm events
>>will thereafter be handled within the library rather than being presented to
>>the user. Does that make sense?

I have some ideas for simplifying connection establishment as part of my changes
to support native IB addresses through the rdma_cm interface.  This includes
adding new calls to the librdmacm: rdma_getaddrinfo and
rdma_create_id_and_optional_qp_and_bind_addr_and_set_route.
I'll pick a different name for the latter. :)  Combined, I think these calls
could allow a user to more easily connect, while still providing low level
control for anyone who wants it.

I haven't looked at the details for this, but we may be able to support
synchronous operation by allowing the user to provide a NULL rdma_event_channel
when creating the rdma_cm_id.  We can take advantage of rdma_migrate_id to
implement this, or even toggle between synchronous and asynchronous mode.  The
hope is that the librdmacm can remain threadless.

If you can give me a month or so to finish my current patches, I'll take a look
at this as part of my work.

- Sean

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Simplified iWARP Consumer Library

[philip.frey-gy3b+zu4XSAfv37vnLkPlQ]

Sounds good to me. I'll wait with the integration of my library routines as they are designed around a CM thread.

Looking forward to your update.

 Phil
------Original Message------
From: Sean Hefty
Sender: linux-rdma-owner@vger.kernel.org
To: Hefty, Sean
To: phil@frey.ws
Cc: rdreier@cisco.com
Cc: Bernard Metzler
Cc: linux-rdma@vger.kernel.org
Subject: RE: Simplified iWARP Consumer Library
Sent: Mar 11, 2010 23:49

>>Which of the features would you want to see in librdmacm? I guess the
>>integration of the cm_id and qp as well as the simpler (synchronous)
>connection
>>establishment and teardown functionality would fit. However, all cm events
>>will thereafter be handled within the library rather than being presented to
>>the user. Does that make sense?

I have some ideas for simplifying connection establishment as part of my changes
to support native IB addresses through the rdma_cm interface.  This includes
adding new calls to the librdmacm: rdma_getaddrinfo and
rdma_create_id_and_optional_qp_and_bind_addr_and_set_route.
I'll pick a different name for the latter. :)  Combined, I think these calls
could allow a user to more easily connect, while still providing low level
control for anyone who wants it.

I haven't looked at the details for this, but we may be able to support
synchronous operation by allowing the user to provide a NULL rdma_event_channel
when creating the rdma_cm_id.  We can take advantage of rdma_migrate_id to
implement this, or even toggle between synchronous and asynchronous mode.  The
hope is that the librdmacm can remain threadless.

If you can give me a month or so to finish my current patches, I'll take a look
at this as part of my work.

- Sean

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html