Re: [RFC][PATCH 1/1] cxgb3i: cxgb3 iSCSI initiator

[Mike Christie <michaelc@cs.wisc.edu>]

Divy Le Ray wrote:
> On Thursday 31 July 2008 05:51:59 pm Divy Le Ray wrote:
>> On Wednesday 30 July 2008 02:35:51 pm Roland Dreier wrote:
>>>  > * From a networking standpoint, our main concern becomes how this
>>>  > interacts with the networking stack.  In particular, I'm concerned
>>>  > based on reading the source that this driver uses "TCP port stealing"
>>>  > rather than using a totally separate MAC address (and IP).
>>>  >
>>>  > Stealing a TCP port on an IP/interface already assigned is a common
>>>  > solution in this space, but also a flawed one.  Precisely because the
>>>  > kernel and applications are unaware of this "special, magic TCP port"
>>>  > you open the potential for application problems that are very
>>>  > difficult for an admin to diagnose based on observed behavior.
>>>
>>> That's true, but using a separate MAC and IP opens up a bunch of other
>>> operational problems.  I don't think the right answer for iSCSI offload
>>> is clear yet.
>>>
>>>  - R.
>> Hi Jeff,
>>
>> We've considered the approach of having a separate IP/MAC addresses to
>> manage iSCSI connections. In such a context, the stack would have to be
>> unaware of this iSCSI specific IP address. The iSCSI driver would then have
>> to implement at least its own ARP reply mechanism. DHCP too would have to
>> be managed separately. Most network setting/monitoring tools would also be
>> unavailable.
>>
>> The open-iscsi initiator is not a huge consumer of TCP connections,
>> allocating a TCP port from the stack would be reasonable in terms of
>> resources in this context. It is however unclear if it is an acceptable
>> approach.
>>
>> Our current implementation was designed to be the most tolerable one
>> within the constraints - real or expected - aforementioned.
>>
> 
> Hi Jeff,
> 
> Mike Christie will not merge this code until he has an explicit 
> acknowledgement from netdev.
> 
> As you mentioned, the port stealing approach we've taken has its issues.
> We consequently analyzed your suggestion to use a different IP/MAC address for 
> iSCSI and it raises other tough issues (separate ARP and DHCP management, 
> unavailability of common networking tools).

If the iscsi tools could not have to deal with networking issues that 
are already handled by other networking tools it would great for the 
iscsi users so they do not have to learn new tools. Maybe we could 
somehow hook into the existing network tools so they support these iscsi 
hbas as well as normal NICs. Would it be possible to have the iscsi hbas 
export the necessary network interfaces so that existing network tools 
can manage them?

If it comes down to it and your port stealing implementation is not 
acceptable like how broadcom's was not, I will be ok with doing some 
special iscsi network tools. Or instead of special iscsi tools, is there 
something that the RDMA/iWarp guys are using that we can share?