[Announce] rxe dev tree available (soft RDMAoE)

[Announce] rxe dev tree available (soft RDMAoE)

[frank zago]

Hello,

The development tree for a soft RDMA transport over Ethernet driver 
(rxe) is available in the OFA git repository. This is a work in progress 
but has enough functionality for people interested in looking at it to 
be worth sharing. Going forward changes to the driver will be maintained 
here.

To get a copy of the driver type

   git clone git://git.openfabrics.org/~fzago/rxe

This work is being supported by Mellanox and System Fabric Works.

Regards,
   Frank
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Announce] rxe dev tree available (soft RDMAoE)

[Jason Gunthorpe]

On Wed, Dec 16, 2009 at 03:06:37PM -0600, frank zago wrote:
> Hello,
>
> The development tree for a soft RDMA transport over Ethernet driver  
> (rxe) is available in the OFA git repository. This is a work in progress  
> but has enough functionality for people interested in looking at it to  
> be worth sharing. Going forward changes to the driver will be maintained  
> here.

Is this going to be turned into a proper kernel patch submission?

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [Announce] rxe dev tree available (soft RDMAoE)

[Robert Pearson]

Jason,

Yes, but later. For now it is still being actively developed and is changing
a lot. Once it settles down we plan to convert to patches and push up
stream.

Bob Pearson

-----Original Message-----
From: Jason Gunthorpe [mailto:jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org] 
Sent: Wednesday, December 16, 2009 3:38 PM
To: frank zago
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; Bob Pearson; John Groves
Subject: Re: [Announce] rxe dev tree available (soft RDMAoE)

On Wed, Dec 16, 2009 at 03:06:37PM -0600, frank zago wrote:
> Hello,
>
> The development tree for a soft RDMA transport over Ethernet driver  
> (rxe) is available in the OFA git repository. This is a work in progress  
> but has enough functionality for people interested in looking at it to  
> be worth sharing. Going forward changes to the driver will be maintained  
> here.

Is this going to be turned into a proper kernel patch submission?

Jason


--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [Announce] rxe dev tree available (soft RDMAoE)

[Sean Hefty]

>Yes, but later. For now it is still being actively developed and is changing
>a lot. Once it settles down we plan to convert to patches and push up
>stream.

Is there any commonality that can be shared between rxe and the qlogic driver?

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [Announce] rxe dev tree available (soft RDMAoE)

[Robert Pearson]

Hi Sean,

Probably someday. We looked at the ipath driver before writing rxe but the
qib driver came out much later and we haven't really spent any time looking
at it. Currently rxe is written to be standalone and doesn't have an obvious
place to split into pieces. Some of the header files could maybe go into
core. (For example rxe_hdr.h and rxe_opcode.[ch] overlap
include/rdma/ib_pack.h).

The hard parts of the driver are the state machines and they are fairly
specific.

Bob

-----Original Message-----
From: Sean Hefty [mailto:sean.hefty-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org] 
Sent: Wednesday, December 16, 2009 4:20 PM
To: 'Robert Pearson'; 'Jason Gunthorpe'; 'frank zago'
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; 'John Groves'
Subject: RE: [Announce] rxe dev tree available (soft RDMAoE)

>Yes, but later. For now it is still being actively developed and is
changing
>a lot. Once it settles down we plan to convert to patches and push up
>stream.

Is there any commonality that can be shared between rxe and the qlogic
driver?



--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Announce] rxe dev tree available (soft RDMAoE)

[Roland Dreier]

 > Is there any commonality that can be shared between rxe and the qlogic driver?

Interesting to note that according to ohcount, rxe is 9591 non-blank
non-comment lines of kernel code, while qib is 36957 lines.  So the
complexity of IB transport is a small part of the complexity of qib.

 - R.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Announce] rxe dev tree available (soft RDMAoE)

[Dave Olson]

On Wed, 16 Dec 2009, Roland Dreier wrote:

| 
|  > Is there any commonality that can be shared between rxe and the qlogic driver?
| 
| Interesting to note that according to ohcount, rxe is 9591 non-blank
| non-comment lines of kernel code, while qib is 36957 lines.  So the
| complexity of IB transport is a small part of the complexity of qib.

Yep.  And a good chunk of the complexity of the qib driver (and ipath
before it) was in dealing with the serdes and it's related issues
(no firmware on the chip to do it, so it's all done in the driver.).

Dave Olson
dave.olson-h88ZbnxC6KDQT0dZR+AlfA@public.gmane.org
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [Announce] rxe dev tree available (soft RDMAoE)

[Robert Pearson]

The ipath driver, which I am more familiar with, replicated the transport
for RC, UC and UD which also caused some code duplication. We tried to
combine them all together in rxe.

Bob Pearson

-----Original Message-----
From: Roland Dreier [mailto:rdreier-FYB4Gu1CFyUAvxtiuMwx3w@public.gmane.org] 
Sent: Wednesday, December 16, 2009 5:53 PM
To: Sean Hefty
Cc: 'Robert Pearson'; 'Jason Gunthorpe'; 'frank zago';
linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; 'John Groves'
Subject: Re: [Announce] rxe dev tree available (soft RDMAoE)


 > Is there any commonality that can be shared between rxe and the qlogic
driver?

Interesting to note that according to ohcount, rxe is 9591 non-blank
non-comment lines of kernel code, while qib is 36957 lines.  So the
complexity of IB transport is a small part of the complexity of qib.

 - R.


--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Announce] rxe dev tree available (soft RDMAoE)

[Roland Dreier]

 > Yes, but later. For now it is still being actively developed and is changing
 > a lot. Once it settles down we plan to convert to patches and push up
 > stream.

I think you would probably be better off developing the kernel driver as
a branch in an up-to-date kernel tree, and then worrying about backports
later.  You could get rid of some of the redundant infrastructure like
dump_hex() and crc32 (too bad IB didn't choose the CRC32C polynomial, so
you can't use the crypto API and get Intel CRC32 instruction
acceleration for free).

 - R.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [Announce] rxe dev tree available (soft RDMAoE)

[Robert Pearson]

I agree about the polynomial. That would have been nice. As currently
proposed the embedding of the IB transport in Ethernet frames preserves the
IB ICRC as part of the transport and trades the VCRC for Ethernet's CRC32
over the whole frame. This means that the packet is effectively covered by
CRC32 twice except for the 802 headers and the fields that are excluded from
the ICRC. CRC32C should give better protection.

We have thought that we could try CRC32C first and if that works then use
that, otherwise fall back to CRC32 which would allow rxe to use CRC32C when
talking to itself and get a speedup on new Intel CPUs.

The crc32 code we have is based on some suggestions from Intel and seems to
be much faster than the 'standard' code in the kernel and we are/were
planning to merge the crc computation with the data copy. It is actually
less explicitly optimized than the code in the kernel but when we use the
loop unrolling directive in the compiler we get better overall results. That
core loop gives us about 5-6GB/sec for the copy/crc vs about 500MB/sec for
the 'standard' crc + copy.

Our plan is to strip out all the trace and dump code when the driver is
stable. We will keep the warn and error messages.

Bob

-----Original Message-----
From: Roland Dreier [mailto:rdreier-FYB4Gu1CFyUAvxtiuMwx3w@public.gmane.org] 
Sent: Thursday, December 17, 2009 3:26 PM
To: Robert Pearson
Cc: 'Jason Gunthorpe'; 'frank zago'; linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; 'John
Groves'
Subject: Re: [Announce] rxe dev tree available (soft RDMAoE)


 > Yes, but later. For now it is still being actively developed and is
changing
 > a lot. Once it settles down we plan to convert to patches and push up
 > stream.

I think you would probably be better off developing the kernel driver as
a branch in an up-to-date kernel tree, and then worrying about backports
later.  You could get rid of some of the redundant infrastructure like
dump_hex() and crc32 (too bad IB didn't choose the CRC32C polynomial, so
you can't use the crypto API and get Intel CRC32 instruction
acceleration for free).

 - R.


--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Announce] rxe dev tree available (soft RDMAoE)

[Roland Dreier]

 > We have thought that we could try CRC32C first and if that works then use
 > that, otherwise fall back to CRC32 which would allow rxe to use CRC32C when
 > talking to itself and get a speedup on new Intel CPUs.

How would you negotiate that?  Is this being worked in the definition of
the IBoE spec?  It seems in a final standard, having something stronger
than two CRC32s and something that can handle jumbo frames would be
worth it.

 > The crc32 code we have is based on some suggestions from Intel and seems to
 > be much faster than the 'standard' code in the kernel and we are/were
 > planning to merge the crc computation with the data copy. It is actually
 > less explicitly optimized than the code in the kernel but when we use the
 > loop unrolling directive in the compiler we get better overall results. That
 > core loop gives us about 5-6GB/sec for the copy/crc vs about 500MB/sec for
 > the 'standard' crc + copy.

Interesting ... can the upstream kernel code be improved similarly?

 > Our plan is to strip out all the trace and dump code when the driver is
 > stable. We will keep the warn and error messages.

If you develop upstream and use the ftrace tracepoints stuff then the
overhead of (runtime) disabled trace points is nearly zero (one 5-byte
NOP, so a cycle or two).  So you can keep them forever (which is
realistically how long you're going to be debugging this :)

 - R.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Announce] rxe dev tree available (soft RDMAoE)

[Jason Gunthorpe]

On Thu, Dec 17, 2009 at 03:43:56PM -0600, Robert Pearson wrote:
> I agree about the polynomial. That would have been nice. As currently
> proposed the embedding of the IB transport in Ethernet frames preserves the
> IB ICRC as part of the transport and trades the VCRC for Ethernet's CRC32
> over the whole frame. This means that the packet is effectively covered by
> CRC32 twice except for the 802 headers and the fields that are excluded from
> the ICRC. CRC32C should give better protection.

The ICRC isn't even necessary from a technical sense for DCE. The
underlying reasons for the ICRC/VCRC split are not really present for
ethernet. Considering RDMAoE isn't interoperable with existing IB anyhow
and DCE can't do routing, the best course would be to just get rid of
it entirely.

IMHO, a smart thing would be to figure out how to exchange ICRC
capability during the connection setup and turn it off if both sides
support it. With the ICRC removed the rxe send side should be able to
be completely 0 copy...

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [Announce] rxe dev tree available (soft RDMAoE)

[Robert Pearson]

Oops. I should have said ICRC not VCRC. There is no VCRC for rdmaoe.
The ICRC is, however, computed exactly the same as for an IB packet. It is
bit for bit identical.

-----Original Message-----
From: Robert Pearson [mailto:rpearson-klaOcWyJdxkshyMvu7JE4pqQE7yCjDx5@public.gmane.org] 
Sent: Thursday, December 17, 2009 5:00 PM
To: 'Jason Gunthorpe'
Cc: 'Roland Dreier'; 'frank zago'; 'linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org'; 'John
Groves'
Subject: RE: [Announce] rxe dev tree available (soft RDMAoE)

Of course we need to interoperate with ConnectX-en and it, as far as I know,
only knows how to compute the VCRC as though the packet was going to get
sent to IB including a phantom 12 byte LRH that is filled with 1's without
any way to turn it off. It is also a requirement that UD work well too.
Instead of connection setup if we could get a capability bit in a port info
record we could handle the general case.

Bob

-----Original Message-----
From: Jason Gunthorpe [mailto:jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org] 
Sent: Thursday, December 17, 2009 4:05 PM
To: Robert Pearson
Cc: 'Roland Dreier'; 'frank zago'; linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; 'John Groves'
Subject: Re: [Announce] rxe dev tree available (soft RDMAoE)

On Thu, Dec 17, 2009 at 03:43:56PM -0600, Robert Pearson wrote:
> I agree about the polynomial. That would have been nice. As currently
> proposed the embedding of the IB transport in Ethernet frames preserves
the
> IB ICRC as part of the transport and trades the VCRC for Ethernet's CRC32
> over the whole frame. This means that the packet is effectively covered by
> CRC32 twice except for the 802 headers and the fields that are excluded
from
> the ICRC. CRC32C should give better protection.

The ICRC isn't even necessary from a technical sense for DCE. The
underlying reasons for the ICRC/VCRC split are not really present for
ethernet. Considering RDMAoE isn't interoperable with existing IB anyhow
and DCE can't do routing, the best course would be to just get rid of
it entirely.

IMHO, a smart thing would be to figure out how to exchange ICRC
capability during the connection setup and turn it off if both sides
support it. With the ICRC removed the rxe send side should be able to
be completely 0 copy...

Jason


--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [Announce] rxe dev tree available (soft RDMAoE)

[Robert Pearson]

Of course we need to interoperate with ConnectX-en and it, as far as I know,
only knows how to compute the VCRC as though the packet was going to get
sent to IB including a phantom 12 byte LRH that is filled with 1's without
any way to turn it off. It is also a requirement that UD work well too.
Instead of connection setup if we could get a capability bit in a port info
record we could handle the general case.

Bob

-----Original Message-----
From: Jason Gunthorpe [mailto:jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org] 
Sent: Thursday, December 17, 2009 4:05 PM
To: Robert Pearson
Cc: 'Roland Dreier'; 'frank zago'; linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; 'John Groves'
Subject: Re: [Announce] rxe dev tree available (soft RDMAoE)

On Thu, Dec 17, 2009 at 03:43:56PM -0600, Robert Pearson wrote:
> I agree about the polynomial. That would have been nice. As currently
> proposed the embedding of the IB transport in Ethernet frames preserves
the
> IB ICRC as part of the transport and trades the VCRC for Ethernet's CRC32
> over the whole frame. This means that the packet is effectively covered by
> CRC32 twice except for the 802 headers and the fields that are excluded
from
> the ICRC. CRC32C should give better protection.

The ICRC isn't even necessary from a technical sense for DCE. The
underlying reasons for the ICRC/VCRC split are not really present for
ethernet. Considering RDMAoE isn't interoperable with existing IB anyhow
and DCE can't do routing, the best course would be to just get rid of
it entirely.

IMHO, a smart thing would be to figure out how to exchange ICRC
capability during the connection setup and turn it off if both sides
support it. With the ICRC removed the rxe send side should be able to
be completely 0 copy...

Jason


--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Announce] rxe dev tree available (soft RDMAoE)

[Jason Gunthorpe]

On Thu, Dec 17, 2009 at 05:00:26PM -0600, Robert Pearson wrote:
> Of course we need to interoperate with ConnectX-en and it, as far as I know,
> only knows how to compute the VCRC as though the packet was going to get
> sent to IB including a phantom 12 byte LRH that is filled with 1's without
> any way to turn it off. It is also a requirement that UD work well too.
> Instead of connection setup if we could get a capability bit in a port info
> record we could handle the general case.

There is no IB management layer (so no port info record) for RDMAoE..

Another good choice might be to not use IPv6 ND and include a
capabilities struct of some sort in a RDMAoE specific ND packet, and,
say, define GRH.nextHeader = 0x2B as meaning 'BTH with no ICRC'

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Announce] rxe dev tree available (soft RDMAoE)

[Roland Dreier]

 > Of course we need to interoperate with ConnectX-en and it, as far as I know,
 > only knows how to compute the VCRC as though the packet was going to get
 > sent to IB including a phantom 12 byte LRH that is filled with 1's without
 > any way to turn it off. It is also a requirement that UD work well too.
 > Instead of connection setup if we could get a capability bit in a port info
 > record we could handle the general case.

How do you get port info records in an IBoE fabric?  There's no SA to
query, right?

 - R.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Announce] rxe dev tree available (soft RDMAoE)

[Roland Dreier]

 > The ICRC isn't even necessary from a technical sense for DCE. The
 > underlying reasons for the ICRC/VCRC split are not really present for
 > ethernet. Considering RDMAoE isn't interoperable with existing IB anyhow
 > and DCE can't do routing, the best course would be to just get rid of
 > it entirely.

Not sure I agree... it would be entirely possible and sensible to create
an IBoE router (and if the IBoE GRH used global scope addresses, pretty
much the same as routing IPv6), and I think you would want end-to-end
ICRC protection in that case (since the router would be regenerating the
ethernet CRC).  Even in the ethernet case there are probably realistic
cases of adding a VLAN tag or something that require the CRC to be
recalculated by a switch -- and some switches probably cut corners and
just recalculate the CRC of all packets on egress, just to avoid
conditional cases.

 - R.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [Announce] rxe dev tree available (soft RDMAoE)

[Robert Pearson]

Forgot to reply to one other point. For the most part we have been
developing in vanilla recent kernel.org trees when we are testing rxe<->rxe.
To talk to ConnectX we need to use the OFED-rdmaoe environment and then we
have to play with symvers. If support for ConnectX-en with rdmaoe were
upstream then we would only use those kernels.

-----Original Message-----
From: Roland Dreier [mailto:rdreier-FYB4Gu1CFyUAvxtiuMwx3w@public.gmane.org] 
Sent: Thursday, December 17, 2009 3:26 PM
To: Robert Pearson
Cc: 'Jason Gunthorpe'; 'frank zago'; linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; 'John
Groves'
Subject: Re: [Announce] rxe dev tree available (soft RDMAoE)


 > Yes, but later. For now it is still being actively developed and is
changing
 > a lot. Once it settles down we plan to convert to patches and push up
 > stream.

I think you would probably be better off developing the kernel driver as
a branch in an up-to-date kernel tree, and then worrying about backports
later.  You could get rid of some of the redundant infrastructure like
dump_hex() and crc32 (too bad IB didn't choose the CRC32C polynomial, so
you can't use the crypto API and get Intel CRC32 instruction
acceleration for free).

 - R.


--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [Announce] rxe dev tree available (soft RDMAoE)

[Woodruff, Robert J]

What is soft RDMA over ethernet, does this mean it is RDMAoE that will run over
any ethernet interface that is installed in the system ?

woody
 

-----Original Message-----
From: linux-rdma-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org [mailto:linux-rdma-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org] On Behalf Of frank zago
Sent: Wednesday, December 16, 2009 1:07 PM
To: linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Cc: Bob Pearson; John Groves
Subject: [Announce] rxe dev tree available (soft RDMAoE)

Hello,

The development tree for a soft RDMA transport over Ethernet driver 
(rxe) is available in the OFA git repository. This is a work in progress 
but has enough functionality for people interested in looking at it to 
be worth sharing. Going forward changes to the driver will be maintained 
here.

To get a copy of the driver type

   git clone git://git.openfabrics.org/~fzago/rxe

This work is being supported by Mellanox and System Fabric Works.

Regards,
   Frank
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [Announce] rxe dev tree available (soft RDMAoE)

[fzago-klaOcWyJdxkshyMvu7JE4pqQE7yCjDx5]

---- "Woodruff wrote: 
> What is soft RDMA over ethernet, does this mean it is RDMAoE that will run over
> any ethernet interface that is installed in the system ?
> 

That's correct. Currently for every ethX interface present, it creates a corresponding rxeX device. These rxeX devices behaves the same way than regular IB devices.

Frank.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html