netdev.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Traiano Welcome <traiano@gmail.com>
To: David Laight <David.Laight@aculab.com>
Cc: "linux-sctp@vger.kernel.org" <linux-sctp@vger.kernel.org>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>
Subject: Re: Kernel Performance Tuning for High Volume SCTP traffic
Date: Sat, 14 Oct 2017 22:29:53 +0800	[thread overview]
Message-ID: <CAFKCRVKScoF+AkBtsgGiFogN3KQP704v8Us9-OABy2BS-w4EVA@mail.gmail.com> (raw)
In-Reply-To: <063D6719AE5E284EB5DD2968C1650D6DD00950DD@AcuExch.aculab.com>

I've upped the value of the following sctp and udp related parameters,
in the hope that this would help:

sysctl -w net.core.rmem_max=900000000
sysctl -w net.core.wmem_max=900000000

sysctl -w net.sctp.sctp_mem="2100000000 2100000000 2100000000"
sysctl -w net.sctp.sctp_rmem="2100000000 2100000000 2100000000"
sysctl -w net.sctp.sctp_wmem="2100000000 2100000000 2100000000"

sysctl -w net.ipv4.udp_mem="5000000000 5000000000 5000000000"
sysctl -w net.ipv4.udp_mem="10000000000 10000000000 10000000000"

However, I'm still seeing rapidly incrementing rx discards reported on the NIC:

:~# ethtool -S ens4f1 | egrep -i rx_discards
     [0]: rx_discards: 6390805462
     [1]: rx_discards: 6659315919
     [2]: rx_discards: 6542570026
     [3]: rx_discards: 6431513008
     [4]: rx_discards: 6436779078
     [5]: rx_discards: 6665897051
     [6]: rx_discards: 6167985560
     [7]: rx_discards: 11340068788
     rx_discards: 56634934892

Despite the fact that I've set the NIC ring buffer on the Netextreme
interface to he maximum:

:~# ethtool -g ens4f0
Ring parameters for ens4f0:
Pre-set maximums:
RX:             4078
RX Mini:        0
RX Jumbo:       0
TX:             4078
Current hardware settings:
RX:             4078
RX Mini:        0
RX Jumbo:       0
TX:             4078

I see no ip errors at the physical interface:

ethtool -S ens4f0 | egrep phy_ip_err_discard| tail -1
     rx_phy_ip_err_discards: 0


Could anyone suggest alternative approaches I might take to optimising
the system's handling of SCTP traffic?



On Sat, Oct 14, 2017 at 12:35 AM, David Laight <David.Laight@aculab.com> wrote:
> From: Traiano Welcome
>> Sent: 13 October 2017 17:04
>> On Fri, Oct 13, 2017 at 11:56 PM, David Laight <David.Laight@aculab.com> wrote:
>> > From: Traiano Welcome
>> >
>> > (copied to netdev)
>> >> Sent: 13 October 2017 07:16
>> >> To: linux-sctp@vger.kernel.org
>> >> Subject: Kernel Performance Tuning for High Volume SCTP traffic
>> >>
>> >> Hi List
>> >>
>> >> I'm running a linux server processing high volumes of SCTP traffic and
>> >> am seeing large numbers of packet overruns (ifconfig output).
>> >
>> > I'd guess that overruns indicate that the ethernet MAC is failing to
>> > copy the receive frames into kernel memory.
>> > It is probably running out of receive buffers, but might be
>> > suffering from a lack of bus bandwidth.
>> > MAC drivers usually discard receive frames if they can't get
>> > a replacement buffer - so you shouldn't run out of rx buffers.
>> >
>> > This means the errors are probably below SCTP - so changing SCTP parameters
>> > is unlikely to help.
>>
>> Does this mean that tuning UDP performance could help ? Or do you mean
>> hardware (NIC) performance could be the issue?
>
> I'd certainly check UDP performance.
>
>         David
>

  reply	other threads:[~2017-10-14 14:29 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <CAFKCRVL-e03Cn35-Vvtb8+NSAz3Tk2Hje_Hg5K9HfqgokEBPgg@mail.gmail.com>
2017-10-13 15:56 ` Kernel Performance Tuning for High Volume SCTP traffic David Laight
2017-10-13 16:03   ` Traiano Welcome
2017-10-13 16:35     ` David Laight
2017-10-14 14:29       ` Traiano Welcome [this message]
2017-10-16 13:34         ` Neil Horman

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAFKCRVKScoF+AkBtsgGiFogN3KQP704v8Us9-OABy2BS-w4EVA@mail.gmail.com \
    --to=traiano@gmail.com \
    --cc=David.Laight@aculab.com \
    --cc=linux-sctp@vger.kernel.org \
    --cc=netdev@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).