RE: rsockets and fork

Hefty, Sean
2012-08-16T23:40:42Z

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 2905 bytes --]

> This test is using Mellanox 10Gb RoCEE with MTU set to 9000
>
> Server is started using
> # ldr netserver -D
>
> 2 clients are started in 2 windows as follows.
>
> # ldr netperf -v2 -c -C -H 192.168.0.22 -l10
> MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.0.22 (192.168.0.22) port 0
> AF_INET
> Recv Send Send Utilization Service Demand
> Socket Socket Message Elapsed Send Recv Send Recv
> Size Size Size Time Throughput local remote local remote
> bytes bytes bytes secs. 10^6bits/s % S % S us/KB us/KB
>
> 87380 65536 65536 10.00 9679.80 2.58 2.59 0.349 0.351
>
> Alignment Offset Bytes Bytes Sends Bytes Recvs
> Local Remote Local Remote Xfered Per Per
> Send Recv Send Recv Send (avg) Recv (avg)
> 8 8 0 0 12101877760 65536.00 184660 21967.51 550899
>
> Maximum
> Segment
> Size (bytes)
> 4096
>
> # ldr netperf -v2 -c -C -H 192.168.0.22 -l10
> MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.0.22 (192.168.0.22) port 0
> AF_INET
> Recv Send Send Utilization Service Demand
> Socket Socket Message Elapsed Send Recv Send Recv
> Size Size Size Time Throughput local remote local remote
> bytes bytes bytes secs. 10^6bits/s % S % S us/KB us/KB
>
> 87380 65536 65536 10.00 30.69 2.27 2.78 96.892 118.616
>
> Alignment Offset Bytes Bytes Sends Bytes Recvs
> Local Remote Local Remote Xfered Per Per
> Send Recv Send Recv Send (avg) Recv (avg)
> 8 8 0 0 38469632 65536.00 587 21360.15 1801
>
> Maximum
> Segment
> Size (bytes)
> 4096

I don't have RoCE installed. With IB, I haven't been able to see this problem after dozens of attempts. The performance isn't divided equally, but I usually see between 3-10 Gbps out of each connection. I'm running with some additional patches, but I don't see where those would affect any hang.

During development, I've seen issues where specific transfer patterns just happen to fall on boundary conditions that result in hangs or slowness. Hopefully that's not the case because those are a pain to identify.

Is this something that you would be able to use a debugger on? Dumping the contents of the struct rsocket * would be useful in troubleshooting if we're waiting on credits, buffer space, the app, or something else.

- Sean
N‹§²æìr¸›yúèšØb²X¬¶Ç§vØ^–)Þº{.nÇ+‰·¥Š{±­ÙšŠ{ayº\x1dÊ‡Ú™ë,j\a­¢f£¢·hš‹»öì\x17/oSc¾™Ú³9˜uÀ¦æå‰È&jw¨®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿïêäz¹Þ–Šàþf£¢·hšˆ§~ˆmš