How to monitor traffic with IBoE/RoCEE?

How to monitor traffic with IBoE/RoCEE?

[Pradeep Satyanarayana]

With IBoE/RoCEE, the traditional SM in IB clusters is not needed. Most of the current
IB tools rely on the SM and PM to get packet and error statistics and so on. These 
won't be applicable with IBoE/RoCEE. netstat will have no value since the kernel 
has been bypassed. So, how does one monitor traffic in such a cluster?

The possibilities that I can think of are to get information from the switches or if there 
are some special tools to get information from the adapter ports themselves. Are such tools
available with ConnectX adapters? How would one get cluster-wide traffic information when
such a cluster is deployed?

Thanks
Pradeep

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [ewg] How to monitor traffic with IBoE/RoCEE?

[Woodruff, Robert J]

Pradeep wrote,

>With IBoE/RoCEE, the traditional SM in IB clusters is not needed. Most of the current
>IB tools rely on the SM and PM to get packet and error statistics and so on. These 
>won't be applicable with IBoE/RoCEE. netstat will have no value since the kernel 
>has been bypassed. So, how does one monitor traffic in such a cluster?

Some managed Ethernet switches have the ability to log into them and get information
on packets transmited/received on each port and errors that are detected. However,
these are typically proprietary. I am not sure about the Mellanox card. 

So I guess the answer is that you manage RoCCE and iWarp clusters just like
you would any Ethernet cluster, with existing management tools that are available
for managing Ethernet.

woody


--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [ewg] How to monitor traffic with IBoE/RoCEE?

[Tom Ammon]

A widely-supported method for collecting statistics from ethernet 
switching devices is SNMP. As Woody said, you wouldn't manage your 
ethernet network any differently than you do now. In addition many many 
tools have been written to make use of SNMP counters, so you won't have 
a hard time finding tools, both commercial and open source.

Tom

On 03/26/2010 11:29 AM, Woodruff, Robert J wrote:
> Pradeep wrote,
>
>    
>> With IBoE/RoCEE, the traditional SM in IB clusters is not needed. Most of the current
>> IB tools rely on the SM and PM to get packet and error statistics and so on. These
>> won't be applicable with IBoE/RoCEE. netstat will have no value since the kernel
>> has been bypassed. So, how does one monitor traffic in such a cluster?
>>      
> Some managed Ethernet switches have the ability to log into them and get information
> on packets transmited/received on each port and errors that are detected. However,
> these are typically proprietary. I am not sure about the Mellanox card.
>
> So I guess the answer is that you manage RoCCE and iWarp clusters just like
> you would any Ethernet cluster, with existing management tools that are available
> for managing Ethernet.
>
> woody
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>    

-- 
--------------------------------------------------------------------
Tom Ammon
Network Engineer
Office: 801.587.0976
Mobile: 801.674.9273

Center for High Performance Computing
University of Utah
http://www.chpc.utah.edu

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: How to monitor traffic with IBoE/RoCEE?

[Pradeep Satyanarayana]

Tom Ammon wrote:
> A widely-supported method for collecting statistics from ethernet
> switching devices is SNMP. As Woody said, you wouldn't manage your
> ethernet network any differently than you do now. In addition many many
> tools have been written to make use of SNMP counters, so you won't have
> a hard time finding tools, both commercial and open source.
> 
> Tom
> 
> On 03/26/2010 11:29 AM, Woodruff, Robert J wrote:
>> Pradeep wrote,
>>
>>   
>>> With IBoE/RoCEE, the traditional SM in IB clusters is not needed.
>>> Most of the current
>>> IB tools rely on the SM and PM to get packet and error statistics and
>>> so on. These
>>> won't be applicable with IBoE/RoCEE. netstat will have no value since
>>> the kernel
>>> has been bypassed. So, how does one monitor traffic in such a cluster?
>>>      
>> Some managed Ethernet switches have the ability to log into them and
>> get information
>> on packets transmited/received on each port and errors that are
>> detected. However,
>> these are typically proprietary. I am not sure about the Mellanox card.
>>
>> So I guess the answer is that you manage RoCCE and iWarp clusters just
>> like
>> you would any Ethernet cluster, with existing management tools that
>> are available
>> for managing Ethernet.
>>
>> woody

True, I suspected that would be the case. However, that means we are using
a different management model than with IB clusters. Hence, I was curious if
Mellanox had any tools to get say some hardware counters without going to a
different management model.

Thanks
Pradeep