public inbox for linux-rdma@vger.kernel.org
 help / color / mirror / Atom feed
* Why our soft-RoCE throughput is quite low compared with TCP
@ 2019-11-15 13:26 QWang
  2019-11-15 16:07 ` Leon Romanovsky
  0 siblings, 1 reply; 10+ messages in thread
From: QWang @ 2019-11-15 13:26 UTC (permalink / raw)
  To: linux-rdma

Dear experts on RDMA,
       We are sorry to disturb you. Because of a project, we need to 
integrate soft-RoCE in our system. However ,we are very confused by our 
soft-RoCE throughput results, which are quite low compared with TCP 
throughput. The throughput of soft-RoCE in our tests measured by 
ib_send_bw and ib_read_bw is only 2 Gbps (the net link bandwidth is 100 
Gbps and the two Xeon E5 servers with Mellanox ConnectX-4 cards are 
connected via back-to-back, the OS is ubuntu16.04 with kernel 
4.15.0-041500-generic). The throughput of hard-RoCE and TCP are normal, 
which are 100 Gbps and 20 Gbps, respectively. But in the figure 6 in the 
attached paper "A Performance Comparison of Container Networking 
Alternatives", the throughput of soft-RoCE can be up to 23 Gbps.  In our 
tests, we get the open-source soft-RoCE from github in 
https://github.com/linux-rdma. Do you know how can we get such high 
bandwidth? Do we need to configure some OS system settings?
       We find that in 2017, someone finds the same problem and he posts 
all his detailed results on 
https://bugzilla.kernel.org/show_bug.cgi?id=190951   . But it remains 
unsolved. His results are nearly the same with our's. For simplicity,  
we do not post our results in this email. You can get very detailed 
information in the web page listed above.
       We are very confused by our results. We will very appreciate it 
if we can receive your early reply. Best wishes,
Wang Qi


^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2019-11-19  7:39 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2019-11-15 13:26 Why our soft-RoCE throughput is quite low compared with TCP QWang
2019-11-15 16:07 ` Leon Romanovsky
2019-11-18  6:38   ` [question]Why " wangqi
2019-11-18  9:49     ` Leon Romanovsky
2019-11-18 10:13       ` wangqi
2019-11-18 12:28         ` Leon Romanovsky
2019-11-18 12:56           ` wangqi
2019-11-18 14:41             ` Leon Romanovsky
2019-11-19  0:22               ` wangqi
2019-11-19  7:38                 ` Leon Romanovsky

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox