All of lore.kernel.org
 help / color / mirror / Atom feed
From: Aaron Lu <aaron.lu@intel.com>
To: lkp@lists.01.org
Subject: Re: [sctp] a6c2f79287: netperf.Throughput_Mbps -37.2% regression
Date: Fri, 05 Aug 2016 11:31:09 +0800	[thread overview]
Message-ID: <20160805033109.GA28539@aaronlu.sh.intel.com> (raw)
In-Reply-To: <CADvbK_fA=cLGz94Gd1BhR7JCHY7nzjsj7KNUfgV8d-+HqknDmA@mail.gmail.com>

[-- Attachment #1: Type: text/plain, Size: 2661 bytes --]

On Thu, Jul 28, 2016 at 03:01:36PM +0800, Xin Long wrote:
> On Wed, Jul 27, 2016 at 9:54 AM, kernel test robot
> <xiaolong.ye@intel.com> wrote:
> >
> > FYI, we noticed a -37.2% regression of netperf.Throughput_Mbps due to commit:
> >
> > commit a6c2f792873aff332a4689717c3cd6104f46684c ("sctp: implement prsctp TTL policy")
> > https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git master
> >
> > in testcase: netperf
> > on test machine: 4 threads Ivy Bridge with 8G memory
> > with following parameters:
> >
> >         ip: ipv4
> >         runtime: 300s
> >         nr_threads: 200%
> >         cluster: cs-localhost
> >         send_size: 10K
> >         test: SCTP_STREAM_MANY
> >         cpufreq_governor: performance
> >
> >
> >
> > Disclaimer:
> > Results have been estimated based on internal Intel analysis and are provided
> > for informational purposes only. Any difference in system hardware or software
> > design or configuration may affect actual performance.
> >
> It doesn't make much sense to me. the codes I added cannot be
> triggered without enable any pr policies. and I also did the tests in

It seems these pr policies has to be turned on by user space, i.e.
netperf in this case?

I checked netperf's source code, it doesn't seem set any option
related to SCTP PR POLICY but I'm new to network code so I could be
wrong or missing something.

> my local environment,  the result looks normal to me compare to
> prior version.

Can you share your number?
We run netperf like this:
netperf -4 -t SCTP_STREAM_MANY -c -C -l 300 -- -m 10K -H 127.0.0.1
The full log of the run is attached for your reference.

> 
> Recently the sctp performance is not stable,  as during these patches,
> netperf cannot get the result, but return ENOTCONN. which may
> also affect the testing. anyway we've fixed the -ENOTCONN issue
> already in the latest version.

I tested commit 96b585267f55, which is Linus' git tree HEAD on 08/03, I
guess the fix you mentioned should already be in there? But
unfortunately, the throughput of netperf is still at low number(we did
the test 5 times):
$ cat */netperf.json
{
  "netperf.Throughput_Mbps": [
    2470.6974999999998
  ]
}{
  "netperf.Throughput_Mbps": [
    2486.7675
  ]
}{
  "netperf.Throughput_Mbps": [
    2478.945
  ]
}{
  "netperf.Throughput_Mbps": [
    2429.465
  ]
}{
  "netperf.Throughput_Mbps": [
    2476.9150000000004
  ]

Considering what you have said that the patch shouldn't make a
difference, the performance drop is really confusing. Any idea what
could be the cause? Thanks.

Regards,
Aaron

[-- Attachment #2: netperf.ksh --]
[-- Type: text/plain, Size: 4640 bytes --]

2016-08-04 16:12:43 netperf -4 -t SCTP_STREAM_MANY -c -C -l 300 -- -m 10K -H 127.0.0.1 &
2016-08-04 16:12:43 netperf -4 -t SCTP_STREAM_MANY -c -C -l 300 -- -m 10K -H 127.0.0.1 &
2016-08-04 16:12:43 netperf -4 -t SCTP_STREAM_MANY -c -C -l 300 -- -m 10K -H 127.0.0.1 &
2016-08-04 16:12:43 netperf -4 -t SCTP_STREAM_MANY -c -C -l 300 -- -m 10K -H 127.0.0.1 &
2016-08-04 16:12:43 netperf -4 -t SCTP_STREAM_MANY -c -C -l 300 -- -m 10K -H 127.0.0.1 &
2016-08-04 16:12:43 netperf -4 -t SCTP_STREAM_MANY -c -C -l 300 -- -m 10K -H 127.0.0.1 &
2016-08-04 16:12:43 netperf -4 -t SCTP_STREAM_MANY -c -C -l 300 -- -m 10K -H 127.0.0.1 &
2016-08-04 16:12:43 netperf -4 -t SCTP_STREAM_MANY -c -C -l 300 -- -m 10K -H 127.0.0.1 &
SCTP 1-TO-MANY STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 127.0.0.1 () port 0 AF_INET : demo
Recv   Send    Send                          Utilization       Service Demand
Socket Socket  Message  Elapsed              Send     Recv     Send    Recv
Size   Size    Size     Time     Throughput  local    remote   local   remote
bytes  bytes   bytes    secs.    10^6bits/s  % S      % S      us/KB   us/KB

212992 212992  10240    300.00     2373.19   51.14    51.14    7.061   7.061 
SCTP 1-TO-MANY STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 127.0.0.1 () port 0 AF_INET : demo
Recv   Send    Send                          Utilization       Service Demand
Socket Socket  Message  Elapsed              Send     Recv     Send    Recv
Size   Size    Size     Time     Throughput  local    remote   local   remote
bytes  bytes   bytes    secs.    10^6bits/s  % S      % S      us/KB   us/KB

212992 212992  10240    300.00     2374.59   51.14    51.11    7.057   7.053 
SCTP 1-TO-MANY STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 127.0.0.1 () port 0 AF_INET : demo
Recv   Send    Send                          Utilization       Service Demand
Socket Socket  Message  Elapsed              Send     Recv     Send    Recv
Size   Size    Size     Time     Throughput  local    remote   local   remote
bytes  bytes   bytes    secs.    10^6bits/s  % S      % S      us/KB   us/KB

212992 212992  10240    300.00     2633.32   51.14    51.11    6.364   6.360 
SCTP 1-TO-MANY STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 127.0.0.1 () port 0 AF_INET : demo
Recv   Send    Send                          Utilization       Service Demand
Socket Socket  Message  Elapsed              Send     Recv     Send    Recv
Size   Size    Size     Time     Throughput  local    remote   local   remote
bytes  bytes   bytes    secs.    10^6bits/s  % S      % S      us/KB   us/KB

212992 212992  10240    300.00     2429.70   51.14    51.11    6.897   6.893 
SCTP 1-TO-MANY STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 127.0.0.1 () port 0 AF_INET : demo
Recv   Send    Send                          Utilization       Service Demand
Socket Socket  Message  Elapsed              Send     Recv     Send    Recv
Size   Size    Size     Time     Throughput  local    remote   local   remote
bytes  bytes   bytes    secs.    10^6bits/s  % S      % S      us/KB   us/KB

212992 212992  10240    300.00     2397.80   51.14    51.11    6.989   6.985 
SCTP 1-TO-MANY STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 127.0.0.1 () port 0 AF_INET : demo
Recv   Send    Send                          Utilization       Service Demand
Socket Socket  Message  Elapsed              Send     Recv     Send    Recv
Size   Size    Size     Time     Throughput  local    remote   local   remote
bytes  bytes   bytes    secs.    10^6bits/s  % S      % S      us/KB   us/KB

212992 212992  10240    300.00     2610.31   51.14    51.11    6.420   6.416 
SCTP 1-TO-MANY STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 127.0.0.1 () port 0 AF_INET : demo
Recv   Send    Send                          Utilization       Service Demand
Socket Socket  Message  Elapsed              Send     Recv     Send    Recv
Size   Size    Size     Time     Throughput  local    remote   local   remote
bytes  bytes   bytes    secs.    10^6bits/s  % S      % S      us/KB   us/KB

212992 212992  10240    300.00     2377.36   51.14    51.11    7.049   7.045 
SCTP 1-TO-MANY STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 127.0.0.1 () port 0 AF_INET : demo
Recv   Send    Send                          Utilization       Service Demand
Socket Socket  Message  Elapsed              Send     Recv     Send    Recv
Size   Size    Size     Time     Throughput  local    remote   local   remote
bytes  bytes   bytes    secs.    10^6bits/s  % S      % S      us/KB   us/KB

212992 212992  10240    300.00     2569.31   51.14    51.11    6.523   6.518 

WARNING: multiple messages have this Message-ID (diff)
From: Aaron Lu <aaron.lu@intel.com>
To: Xin Long <lucien.xin@gmail.com>
Cc: kernel test robot <xiaolong.ye@intel.com>,
	Stephen Rothwell <sfr@canb.auug.org.au>,
	lkp@01.org, "David S. Miller" <davem@davemloft.net>,
	LKML <linux-kernel@vger.kernel.org>
Subject: Re: [LKP] [lkp] [sctp] a6c2f79287: netperf.Throughput_Mbps -37.2% regression
Date: Fri, 5 Aug 2016 11:31:09 +0800	[thread overview]
Message-ID: <20160805033109.GA28539@aaronlu.sh.intel.com> (raw)
In-Reply-To: <CADvbK_fA=cLGz94Gd1BhR7JCHY7nzjsj7KNUfgV8d-+HqknDmA@mail.gmail.com>

[-- Attachment #1: Type: text/plain, Size: 2577 bytes --]

On Thu, Jul 28, 2016 at 03:01:36PM +0800, Xin Long wrote:
> On Wed, Jul 27, 2016 at 9:54 AM, kernel test robot
> <xiaolong.ye@intel.com> wrote:
> >
> > FYI, we noticed a -37.2% regression of netperf.Throughput_Mbps due to commit:
> >
> > commit a6c2f792873aff332a4689717c3cd6104f46684c ("sctp: implement prsctp TTL policy")
> > https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git master
> >
> > in testcase: netperf
> > on test machine: 4 threads Ivy Bridge with 8G memory
> > with following parameters:
> >
> >         ip: ipv4
> >         runtime: 300s
> >         nr_threads: 200%
> >         cluster: cs-localhost
> >         send_size: 10K
> >         test: SCTP_STREAM_MANY
> >         cpufreq_governor: performance
> >
> >
> >
> > Disclaimer:
> > Results have been estimated based on internal Intel analysis and are provided
> > for informational purposes only. Any difference in system hardware or software
> > design or configuration may affect actual performance.
> >
> It doesn't make much sense to me. the codes I added cannot be
> triggered without enable any pr policies. and I also did the tests in

It seems these pr policies has to be turned on by user space, i.e.
netperf in this case?

I checked netperf's source code, it doesn't seem set any option
related to SCTP PR POLICY but I'm new to network code so I could be
wrong or missing something.

> my local environment,  the result looks normal to me compare to
> prior version.

Can you share your number?
We run netperf like this:
netperf -4 -t SCTP_STREAM_MANY -c -C -l 300 -- -m 10K -H 127.0.0.1
The full log of the run is attached for your reference.

> 
> Recently the sctp performance is not stable,  as during these patches,
> netperf cannot get the result, but return ENOTCONN. which may
> also affect the testing. anyway we've fixed the -ENOTCONN issue
> already in the latest version.

I tested commit 96b585267f55, which is Linus' git tree HEAD on 08/03, I
guess the fix you mentioned should already be in there? But
unfortunately, the throughput of netperf is still at low number(we did
the test 5 times):
$ cat */netperf.json
{
  "netperf.Throughput_Mbps": [
    2470.6974999999998
  ]
}{
  "netperf.Throughput_Mbps": [
    2486.7675
  ]
}{
  "netperf.Throughput_Mbps": [
    2478.945
  ]
}{
  "netperf.Throughput_Mbps": [
    2429.465
  ]
}{
  "netperf.Throughput_Mbps": [
    2476.9150000000004
  ]

Considering what you have said that the patch shouldn't make a
difference, the performance drop is really confusing. Any idea what
could be the cause? Thanks.

Regards,
Aaron

[-- Attachment #2: netperf --]
[-- Type: text/plain, Size: 4640 bytes --]

2016-08-04 16:12:43 netperf -4 -t SCTP_STREAM_MANY -c -C -l 300 -- -m 10K -H 127.0.0.1 &
2016-08-04 16:12:43 netperf -4 -t SCTP_STREAM_MANY -c -C -l 300 -- -m 10K -H 127.0.0.1 &
2016-08-04 16:12:43 netperf -4 -t SCTP_STREAM_MANY -c -C -l 300 -- -m 10K -H 127.0.0.1 &
2016-08-04 16:12:43 netperf -4 -t SCTP_STREAM_MANY -c -C -l 300 -- -m 10K -H 127.0.0.1 &
2016-08-04 16:12:43 netperf -4 -t SCTP_STREAM_MANY -c -C -l 300 -- -m 10K -H 127.0.0.1 &
2016-08-04 16:12:43 netperf -4 -t SCTP_STREAM_MANY -c -C -l 300 -- -m 10K -H 127.0.0.1 &
2016-08-04 16:12:43 netperf -4 -t SCTP_STREAM_MANY -c -C -l 300 -- -m 10K -H 127.0.0.1 &
2016-08-04 16:12:43 netperf -4 -t SCTP_STREAM_MANY -c -C -l 300 -- -m 10K -H 127.0.0.1 &
SCTP 1-TO-MANY STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 127.0.0.1 () port 0 AF_INET : demo
Recv   Send    Send                          Utilization       Service Demand
Socket Socket  Message  Elapsed              Send     Recv     Send    Recv
Size   Size    Size     Time     Throughput  local    remote   local   remote
bytes  bytes   bytes    secs.    10^6bits/s  % S      % S      us/KB   us/KB

212992 212992  10240    300.00     2373.19   51.14    51.14    7.061   7.061 
SCTP 1-TO-MANY STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 127.0.0.1 () port 0 AF_INET : demo
Recv   Send    Send                          Utilization       Service Demand
Socket Socket  Message  Elapsed              Send     Recv     Send    Recv
Size   Size    Size     Time     Throughput  local    remote   local   remote
bytes  bytes   bytes    secs.    10^6bits/s  % S      % S      us/KB   us/KB

212992 212992  10240    300.00     2374.59   51.14    51.11    7.057   7.053 
SCTP 1-TO-MANY STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 127.0.0.1 () port 0 AF_INET : demo
Recv   Send    Send                          Utilization       Service Demand
Socket Socket  Message  Elapsed              Send     Recv     Send    Recv
Size   Size    Size     Time     Throughput  local    remote   local   remote
bytes  bytes   bytes    secs.    10^6bits/s  % S      % S      us/KB   us/KB

212992 212992  10240    300.00     2633.32   51.14    51.11    6.364   6.360 
SCTP 1-TO-MANY STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 127.0.0.1 () port 0 AF_INET : demo
Recv   Send    Send                          Utilization       Service Demand
Socket Socket  Message  Elapsed              Send     Recv     Send    Recv
Size   Size    Size     Time     Throughput  local    remote   local   remote
bytes  bytes   bytes    secs.    10^6bits/s  % S      % S      us/KB   us/KB

212992 212992  10240    300.00     2429.70   51.14    51.11    6.897   6.893 
SCTP 1-TO-MANY STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 127.0.0.1 () port 0 AF_INET : demo
Recv   Send    Send                          Utilization       Service Demand
Socket Socket  Message  Elapsed              Send     Recv     Send    Recv
Size   Size    Size     Time     Throughput  local    remote   local   remote
bytes  bytes   bytes    secs.    10^6bits/s  % S      % S      us/KB   us/KB

212992 212992  10240    300.00     2397.80   51.14    51.11    6.989   6.985 
SCTP 1-TO-MANY STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 127.0.0.1 () port 0 AF_INET : demo
Recv   Send    Send                          Utilization       Service Demand
Socket Socket  Message  Elapsed              Send     Recv     Send    Recv
Size   Size    Size     Time     Throughput  local    remote   local   remote
bytes  bytes   bytes    secs.    10^6bits/s  % S      % S      us/KB   us/KB

212992 212992  10240    300.00     2610.31   51.14    51.11    6.420   6.416 
SCTP 1-TO-MANY STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 127.0.0.1 () port 0 AF_INET : demo
Recv   Send    Send                          Utilization       Service Demand
Socket Socket  Message  Elapsed              Send     Recv     Send    Recv
Size   Size    Size     Time     Throughput  local    remote   local   remote
bytes  bytes   bytes    secs.    10^6bits/s  % S      % S      us/KB   us/KB

212992 212992  10240    300.00     2377.36   51.14    51.11    7.049   7.045 
SCTP 1-TO-MANY STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 127.0.0.1 () port 0 AF_INET : demo
Recv   Send    Send                          Utilization       Service Demand
Socket Socket  Message  Elapsed              Send     Recv     Send    Recv
Size   Size    Size     Time     Throughput  local    remote   local   remote
bytes  bytes   bytes    secs.    10^6bits/s  % S      % S      us/KB   us/KB

212992 212992  10240    300.00     2569.31   51.14    51.11    6.523   6.518 

  reply	other threads:[~2016-08-05  3:31 UTC|newest]

Thread overview: 72+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-07-27  1:54 [sctp] a6c2f79287: netperf.Throughput_Mbps -37.2% regression kernel test robot
2016-07-27  1:54 ` [lkp] " kernel test robot
2016-07-28  7:01 ` Xin Long
2016-07-28  7:01   ` [lkp] " Xin Long
2016-08-05  3:31   ` Aaron Lu [this message]
2016-08-05  3:31     ` [LKP] " Aaron Lu
2016-08-05 11:53     ` Xin Long
2016-08-05 11:53       ` [LKP] [lkp] " Xin Long
2016-08-08  2:10       ` Aaron Lu
2016-08-08  2:10         ` [LKP] [lkp] " Aaron Lu
2016-08-16  2:38         ` Aaron Lu
2016-08-16  2:38           ` [LKP] [lkp] " Aaron Lu
2016-08-16  8:02         ` Xin Long
2016-08-16  8:02           ` [LKP] [lkp] " Xin Long
2016-08-16  8:30           ` Aaron Lu
2016-08-16  8:30             ` [LKP] [lkp] " Aaron Lu
2016-08-16  8:51           ` Aaron Lu
2016-08-16  8:51             ` [LKP] [lkp] " Aaron Lu
2016-08-16  9:56             ` Xin Long
2016-08-16  9:56               ` [LKP] [lkp] " Xin Long
2016-08-17  5:04               ` Aaron Lu
2016-08-17  5:04                 ` [LKP] [lkp] " Aaron Lu
2016-08-17  5:34                 ` Xin Long
2016-08-17  5:34                   ` [LKP] [lkp] " Xin Long
2016-08-17  5:34                 ` Aaron Lu
2016-08-17  5:34                   ` [LKP] [lkp] " Aaron Lu
2016-08-17  5:41                   ` Xin Long
2016-08-17  5:41                     ` [LKP] [lkp] " Xin Long
2016-08-17  6:14                     ` Aaron Lu
2016-08-17  6:14                       ` [LKP] [lkp] " Aaron Lu
2016-08-17  6:37                       ` Aaron Lu
2016-08-17  6:37                         ` [LKP] [lkp] " Aaron Lu
2016-08-17  6:42                         ` Aaron Lu
2016-08-17  6:42                           ` [LKP] [lkp] " Aaron Lu
2016-08-17  7:35                         ` Xin Long
2016-08-17  7:35                           ` [LKP] [lkp] " Xin Long
2016-08-17  7:42                           ` Aaron Lu
2016-08-17  7:42                             ` [LKP] [lkp] " Aaron Lu
2016-08-17  7:53                             ` Aaron Lu
2016-08-17  7:53                               ` [LKP] [lkp] " Aaron Lu
2016-08-17  8:02                             ` Xin Long
2016-08-17  8:02                               ` [LKP] [lkp] " Xin Long
2016-08-17  8:48                               ` Aaron Lu
2016-08-17  8:48                                 ` [LKP] [lkp] " Aaron Lu
2016-08-17  8:58                                 ` Xin Long
2016-08-17  8:58                                   ` [LKP] [lkp] " Xin Long
2016-08-17  9:20                                   ` Aaron Lu
2016-08-17  9:20                                     ` [LKP] [lkp] " Aaron Lu
2016-08-17 18:06                                     ` Xin Long
2016-08-17 18:06                                       ` [LKP] [lkp] " Xin Long
2016-08-18  3:21                                       ` Aaron Lu
2016-08-18  3:21                                         ` [LKP] [lkp] " Aaron Lu
2016-08-18 12:45                                         ` Xin Long
2016-08-18 12:45                                           ` [LKP] [lkp] " Xin Long
2016-08-19  5:29                                           ` Aaron Lu
2016-08-19  5:29                                             ` [LKP] [lkp] " Aaron Lu
2016-08-19  7:19                                             ` Marcelo Ricardo Leitner
2016-08-19  7:19                                               ` [LKP] [lkp] " Marcelo Ricardo Leitner
2016-08-19  7:24                                               ` Aaron Lu
2016-08-19  7:24                                                 ` [LKP] [lkp] " Aaron Lu
2016-08-22 21:44                                                 ` Marcelo Ricardo Leitner
2016-08-22 21:44                                                   ` [LKP] [lkp] " Marcelo Ricardo Leitner
2016-08-23  9:19                                                   ` Aaron Lu
2016-08-23  9:19                                                     ` [LKP] [lkp] " Aaron Lu
2016-09-30  7:05                                                   ` Aaron Lu
2016-09-30  7:05                                                     ` [LKP] [lkp] " Aaron Lu
2016-10-03  2:32                                                     ` Xin Long
2016-10-03  2:32                                                       ` [LKP] [lkp] " Xin Long
2016-10-09  7:41                                                       ` Aaron Lu
2016-10-09  7:41                                                         ` [LKP] [lkp] " Aaron Lu
2016-08-16 18:34             ` Xin Long
2016-08-16 18:34               ` [LKP] [lkp] " Xin Long

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20160805033109.GA28539@aaronlu.sh.intel.com \
    --to=aaron.lu@intel.com \
    --cc=lkp@lists.01.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.