Re: SMC-R throughput drops for specific message sizes

[Wenjia Zhang <wenjia@linux.ibm.com>]

On 01.12.23 14:33, Nikolaou Alexandros (SO/PAF1-Mb) wrote:
> Dear SMC Maintainers and Contributors,
>   
> I would like to bring to your attention some observations made during recent experiments conducted with the SMC-R on the following system:
>   
> - SMC version: 3E92E1460DA96BE2B2DDC2F (version 1)
> - Kernel: both v5.4 and v5.15
> - Ubuntu: 20.04
> - Benchmarking tool: ‘qperf’
> - The SMC-R vs TCP/IP throughput has been measured using the ‘qperf’ tool for various message sizes ranging from 0 to 8 MB.
>   
> 
> Based on the attached plot, you may notice that the SMC-R throughput drops substantially according to a periodic regular pattern.
> 
> A few configurations have been checked, but the pattern described above persists:
>   
> - All the min/default/max values of ‘net.ipv4.tcp_rmem’ and ‘net.ipv4.tcp_wmem’ have been utilized.
> - Experiments with the Linux kernel v5.4 and v5.15 have been conducted.
> - The MTU size has been changed from 1500 up to 9000.
> - Several message sizes have been tried.
>   
> I was wondering whether you could help me understand a reason behind these drops, please? This behavior is not observed with the TCP/IP stack. Any insights or guidance would be highly appreciated.
>   
> Thank you very much in advance.
>   
> Mit freundlichen Grüßen / Best regards
> 
> Alexandros Nikolaou
> 
> Bosch Service Solutions Magdeburg GmbH | Otto-von-Guericke-Str. 13 | 39104 Magdeburg | GERMANY | [www.boschservicesolutions.com]www.boschservicesolutions.com
> Alexandros.Nikolaou@de.bosch.com
> 
> Sitz: Magdeburg, Registergericht: Amtsgericht Stendal, HRB 24039
> Geschäftsführung: Robert Mulatz, Daniel Meyer

Hi Nikolaou,

I'd like to help you. However, I need more details on your environment, 
setup, and test scenarios. e.g. which buffersize are you using and on 
which message sizes exactly you got the drops? How many connections you 
are using? etc. If possible, could you show me the output of the qperf 
tests with your commands and the output of following commands:
smcr info
smc_rnics -a
smcr -d stats

Thanks,
Wenjia Zhang