RE: Linux SCTP performance question

[David Laight <David.Laight@ACULAB.COM>]

From: o.evistel@free.fr
> Sent: 18 July 2022 15:10
> Hi David
> 
> I try to perform a performance test with a configuration consisting of a M3UA-ASP VM (client) served
> by 4 M3UA-SGP VMs (servers).
> All the VMs are RHEL8.4.
> The TCAP traffic is initiated by the ASP (10K DATA-Chunk/sec) and evenly distributed over the 4 SGPs
> which reflect the received traffic to the ASP.
> Within a random interval (15 min to several hours) the ASP experiences an error when calling
> sctp_sendv() with ERRNO="Resource Temporarily Unavailable".

That's looks like a non-blocking send hitting the socket write buffer limit.
Almost certainly very transient.
Possibly cause by dropped packets or the receiving being overloaded.

> As I don't know what causes theese errors I am wondering if this is not a performance issue.
> 
> Referring to your performance test:
> - Does 40K reflects/sec mean 40K Tx DATA-Chunk/sec and 40K Rx DATA-Chunk/sec ?

Yes.
It will be a 'double-reflect' test (both ends reflect) with a smallish
number of packets in the loop - enough to get maximum throughput
without packets being discarded.

> - Did you apply specific kernel parameters tuning (rmem, wmem, ...) ?

No.

> - Can you share the test program you were using ?

It won't help you.
It uses our proprietary protocol over TCP to pass TCAP messages into
a driver (mostly SCCP, ISUP and MTP3) which uses kernel sockets to do
the actual send/recv.

> 
> Thank you in advance for your help and support.
> 
> Regards
> Omar AIT AMRANE
> ========================================================================================
> 
> 
> ----- Mail original -----
> De: "David Laight" <David.Laight@ACULAB.COM>
> À: "o" <o.evistel@free.fr>, "Marcelo Ricardo Leitner" <marcelo.leitner@gmail.com>
> Cc: "linux-sctp" <linux-sctp@vger.kernel.org>
> Envoyé: Samedi 16 Juillet 2022 14:13:31
> Objet: RE: Linux SCTP performance question
> 
> From: o.evistel@free.fr
> > Sent: 13 July 2022 13:59
> >
> > I am using linux-sctp as transport for SIGTRAN M3UA on RHEL 8.4 and I am
> > using sctp_sendmsg() and sctp_recvmsg() to send/receive.
> > I would like to know if the use of sctp_sendv() and sctp_recvv() enhances performances ?
> 
> Every copy to/from a user buffer has a small (but measurable) cost.
> So anything with an iov[] array is a bit slower than an equivalent
> call that only has a single buffer.
> This is measurable when comparing sendmsg() and sendto() on (say)
> a UDP socket.
> OTOH it is all probably noise unless you are trying to send/receive
> 'silly numbers' of messages.
> 
> Reducing the number of system calls may help.
> But the cost of recvmmsg() checking for a second message is
> significantly greater than using epoll().
> So unless you actually expect lots of messages it probably
> isn't worth using.
> 
> Assuming you have disabled Nagle, then the biggest performance
> gain will be from setting MSG_MORE when you know you have another
> message to send.
> Without that pretty much every M3UA message ends up in its own
> ethernet packet.
> 
> You don't mention why you think you have a performance issue in SCTP.
> I think we got 40000 reflects/sec from a tcap application over M3UA
> and SCTP last time I measured out stack.
> 
> 	David
> 
> -
> Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
> Registration No: 1397386 (Wales)

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)