Software iWARP stack

Software iWARP stack

[Bernard Metzler]

We at IBM Zurich Laboratory are developing a full-software
iWARP stack to run within the OpenFabrics verbs framework.
Today we open sourced that work to be available at
http://gitorious.org/softiwarp. It consists of a kernel
module (siw) and a user level library (libsiw).

The Linux kernel module implements the iWARP protocol suite
(MPA/DDP/RDMAP, IETF-RFC 5044/5041/5040) completely in software.
Targeted for integration with OpenFabrics kernel interfaces,
it appears as a kernel module in the drivers/infiniband/hw
subdirectory of the Linux kernel. siw exports the OFA RDMA
verbs interface, currently useable only for user level
applications. The code assumes that connection management
is done via the OpenFabrics connection manager.
siw runs on top of TCP kernel sockets, which are used in
non-blocking mode. Current development work is done in Linux
kernel version 2.6.34-rc1.

libsiw implements user level access to the siw modules.
It attaches with the OpenFabrics user level verbs (libibvers)
and the OpenFabrics rdma connection manager (librdmacm).
It is tested to run in version 1.4.1 of OpenFabrics.


We belief that a software only implementation of the
iWARP protocol is beneficial for the wider acceptance
of the RDMA communication semantics for many reasons,
including:
o It allows low end systems to peer with RDMA hardware).
  equipped systems. This, conversely, makes the use of).
  RDMA hardware in loaded systems like servers a more
  interesting option.
o It eases the migration of existing non-RDMA solutions.
  to the RDMA model both at application and at network
  stack level.
o We expect that kernel clients like NSF and iSER can
  benefit from the existence of a software kernel RDMA
  stack.
o While potentially not running at the full speed of an.
  offloaded RDMA stack, the asynchronous and one-sided
  communication model of RDMA itself may give the application
  an advantage over running on plain sockets.
o The explicit communication buffer ownership management of
  RDMA enables the consistent implementation of user data copy
  avoidance techniques without hardware support.
o Last not least, softiwarp offers the application programmer
  access to a well-documented asynchronous communication interface
  w/o the need to install potentially expensive hardware.
  Asynchronous I/O may become a viable option.


We are making our work available under Dual License (BSD/Gnu) -
see the Copyright statement within the source code files for details.
Since the siw kernel module implements the iWARP protocol
suite of the IETF, we refer to the IPR page of the IETF
(http://www.ietf.org/ipr/).

We explicitely declare our work as 'work-in-progress'. We make
that work open source when its functionality and stability
still has some limitations (see IMPLEMENTATION.txt in the
infiniband/hw/softiwarp subdirectory). Some of the functional
limitations are not the result of lack of time and labor:
they are intentionally - we would like to discuss with the
Community how to architect it the most appropriate way.

We are completely open to any improvements of the code base
and in fact hope to help establishing a community contributing
to the improvement and completion of a software based RDMA
stack.

Thanks, Bernard.

--
Bernard Metzler, PhD
IBM Zurich Research Laboratory
Säumerstrasse 4
CH-8803 Rüschlikon/Switzerland
bmt-OA+xvbQnYDHMbYB6QlFGEg@public.gmane.org
phone +41-44-7248605

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Software iWARP stack

[Roland Dreier]

 > We at IBM Zurich Laboratory are developing a full-software
 > iWARP stack to run within the OpenFabrics verbs framework.
 > Today we open sourced that work to be available at
 > http://gitorious.org/softiwarp. It consists of a kernel
 > module (siw) and a user level library (libsiw).

Great, congratulations on finally releasing this.

I will take a look at this, and I hope we can move this towards the
mainline kernel.  With the current backlog of things to review/merge,
unfortunately I don't think 2.6.35 is realistic; so let's set a goal of
getting this into 2.6.36.

One thing you could do that would make this a little easier to test and
review would be to have the kernel driver tree be based on the main
kernel -- ie a git tree based on Linus's kernel tree with exactly the
patches required to add softiwarp support committed.  It may seem crazy
to carry the full kernel tree with your relatively small amount of code,
but git makes this really cheap (I can clone your tree with "--reference
<my-main-kernel-tree>" and only get one copy of all the objects in
common), and it makes it much easier to build and test.

Also I would suggest posting your announcement to netdev-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
too, to get review comments from the networking guys.  If you can split
your code up into a digestible series of patches for review by email,
that would help too (post to both netdev@ and linux-rmda@).  Having the
git tree is useful for following development, testing, building etc, and
having the patches by email is useful for reading and reviewing the
code -- it makes sending comments back much easier.

 - Roland
-- 
Roland Dreier  <rolandd-FYB4Gu1CFyUAvxtiuMwx3w@public.gmane.org>
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/index.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html