From: Traiano Welcome <traiano@gmail.com>
To: David Laight <David.Laight@aculab.com>
Cc: "linux-sctp@vger.kernel.org" <linux-sctp@vger.kernel.org>,
"netdev@vger.kernel.org" <netdev@vger.kernel.org>
Subject: Re: Kernel Performance Tuning for High Volume SCTP traffic
Date: Sat, 14 Oct 2017 22:29:53 +0800 [thread overview]
Message-ID: <CAFKCRVKScoF+AkBtsgGiFogN3KQP704v8Us9-OABy2BS-w4EVA@mail.gmail.com> (raw)
In-Reply-To: <063D6719AE5E284EB5DD2968C1650D6DD00950DD@AcuExch.aculab.com>
I've upped the value of the following sctp and udp related parameters,
in the hope that this would help:
sysctl -w net.core.rmem_max=900000000
sysctl -w net.core.wmem_max=900000000
sysctl -w net.sctp.sctp_mem="2100000000 2100000000 2100000000"
sysctl -w net.sctp.sctp_rmem="2100000000 2100000000 2100000000"
sysctl -w net.sctp.sctp_wmem="2100000000 2100000000 2100000000"
sysctl -w net.ipv4.udp_mem="5000000000 5000000000 5000000000"
sysctl -w net.ipv4.udp_mem="10000000000 10000000000 10000000000"
However, I'm still seeing rapidly incrementing rx discards reported on the NIC:
:~# ethtool -S ens4f1 | egrep -i rx_discards
[0]: rx_discards: 6390805462
[1]: rx_discards: 6659315919
[2]: rx_discards: 6542570026
[3]: rx_discards: 6431513008
[4]: rx_discards: 6436779078
[5]: rx_discards: 6665897051
[6]: rx_discards: 6167985560
[7]: rx_discards: 11340068788
rx_discards: 56634934892
Despite the fact that I've set the NIC ring buffer on the Netextreme
interface to he maximum:
:~# ethtool -g ens4f0
Ring parameters for ens4f0:
Pre-set maximums:
RX: 4078
RX Mini: 0
RX Jumbo: 0
TX: 4078
Current hardware settings:
RX: 4078
RX Mini: 0
RX Jumbo: 0
TX: 4078
I see no ip errors at the physical interface:
ethtool -S ens4f0 | egrep phy_ip_err_discard| tail -1
rx_phy_ip_err_discards: 0
Could anyone suggest alternative approaches I might take to optimising
the system's handling of SCTP traffic?
On Sat, Oct 14, 2017 at 12:35 AM, David Laight <David.Laight@aculab.com> wrote:
> From: Traiano Welcome
>> Sent: 13 October 2017 17:04
>> On Fri, Oct 13, 2017 at 11:56 PM, David Laight <David.Laight@aculab.com> wrote:
>> > From: Traiano Welcome
>> >
>> > (copied to netdev)
>> >> Sent: 13 October 2017 07:16
>> >> To: linux-sctp@vger.kernel.org
>> >> Subject: Kernel Performance Tuning for High Volume SCTP traffic
>> >>
>> >> Hi List
>> >>
>> >> I'm running a linux server processing high volumes of SCTP traffic and
>> >> am seeing large numbers of packet overruns (ifconfig output).
>> >
>> > I'd guess that overruns indicate that the ethernet MAC is failing to
>> > copy the receive frames into kernel memory.
>> > It is probably running out of receive buffers, but might be
>> > suffering from a lack of bus bandwidth.
>> > MAC drivers usually discard receive frames if they can't get
>> > a replacement buffer - so you shouldn't run out of rx buffers.
>> >
>> > This means the errors are probably below SCTP - so changing SCTP parameters
>> > is unlikely to help.
>>
>> Does this mean that tuning UDP performance could help ? Or do you mean
>> hardware (NIC) performance could be the issue?
>
> I'd certainly check UDP performance.
>
> David
>
next prev parent reply other threads:[~2017-10-14 14:29 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <CAFKCRVL-e03Cn35-Vvtb8+NSAz3Tk2Hje_Hg5K9HfqgokEBPgg@mail.gmail.com>
2017-10-13 15:56 ` Kernel Performance Tuning for High Volume SCTP traffic David Laight
2017-10-13 16:03 ` Traiano Welcome
2017-10-13 16:35 ` David Laight
2017-10-14 14:29 ` Traiano Welcome [this message]
2017-10-16 13:34 ` Neil Horman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAFKCRVKScoF+AkBtsgGiFogN3KQP704v8Us9-OABy2BS-w4EVA@mail.gmail.com \
--to=traiano@gmail.com \
--cc=David.Laight@aculab.com \
--cc=linux-sctp@vger.kernel.org \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).