From: Greg KH <gregkh@linuxfoundation.org>
To: lkp@lists.01.org
Subject: Re: increased vmap_area_lock contentions on "n_tty: Move buffers into n_tty_data"
Date: Thu, 12 Sep 2013 20:44:13 -0700 [thread overview]
Message-ID: <20130913034413.GA26166@kroah.com> (raw)
In-Reply-To: <20130913033803.GB8704@localhost>
[-- Attachment #1: Type: text/plain, Size: 4262 bytes --]
On Fri, Sep 13, 2013 at 11:38:04AM +0800, Fengguang Wu wrote:
> On Thu, Sep 12, 2013 at 08:17:00PM -0700, Greg KH wrote:
> > On Fri, Sep 13, 2013 at 08:51:33AM +0800, Fengguang Wu wrote:
> > > Hi Peter,
> > >
> > > FYI, we noticed much increased vmap_area_lock contentions since this
> > > commit:
> >
> > What does that mean? What is happening, are we allocating/removing more
> > memory now?
>
> // leave this question to Peter and Tejun. :)
>
> > What type of load were you running that showed this problem?
>
> The increased contentions and lock hold/wait time showed up in a
> number of test cases.
>
> The second email has the details, and this section of data is most
> illustrating.
>
> 8cb06c983822103da1cf 20bafb3d23d108bc0a89
> ------------------------ ------------------------
> 3470.31 +1631.0% 60070.49 fat/micro/dd-write/1HDD-cfq-ext4-10dd
> 3339.57 +1676.9% 59340.71 fat/micro/dd-write/1HDD-cfq-ext4-1dd
> 2848.79 +1489.1% 45269.54 lkp-a04/micro/netperf/120s-200%-TCP_CRR
> 3563.01 +1515.2% 57548.50 lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
> 2678.64 +1346.0% 38733.80 lkp-a04/micro/netperf/120s-200%-TCP_RR
> 2839.84 +1450.2% 44022.08 lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
> 3417.37 +1571.4% 57116.34 lkp-a04/micro/netperf/120s-200%-TCP_STREAM
> 2558.59 +1450.8% 39677.58 lkp-a04/micro/netperf/120s-200%-UDP_RR
> 3737.24 +1558.0% 61963.62 lkp-a04/micro/netperf/120s-200%-UDP_STREAM
> 20219.50 +1488.7% 321218.02 lkp-a06/crypto/tcrypt/2s-200-204
> 21017.17 +1457.1% 327257.41 lkp-a06/crypto/tcrypt/2s-205-210
> 22109.84 +1240.3% 296346.33 lkp-a06/crypto/tcrypt/2s-401-417
> 17909.60 +1467.3% 280693.71 lkp-a06/micro/dbench/100%
> 489739.50 +978.5% 5281916.05 lkp-ne04/micro/aim7/shell_rtns_1
> 1601675.63 +906.7% 16123642.52 lkp-snb01/micro/aim7/exec_test
> 12105.00 +2453.6% 309110.42 nhm-white/micro/aim7/dbase
> 822461.02 +1585.0% 13858430.62 nhm-white/micro/aim7/exec_test
> 9858.11 +2715.9% 277595.41 nhm-white/micro/aim7/fork_test
> 3452.91 +1685.5% 61650.74 nhm-white/micro/aim7/fserver
> 300.14 +2621.5% 8168.53 nhm-white/micro/aim7/misc_rtns_1
> 345479.21 +1624.5% 5957828.25 nhm-white/micro/aim7/shell_rtns_1
> 2694.48 +1974.4% 55894.19 nhm-white/sysbench/oltp/100%-600-1000000
> 4415.67 +1202.2% 57501.52 nhm8/micro/dbench/100%
> 2284.65 +1505.2% 36672.75 snb-drag/crypto/tcrypt/2s-200-204
> 2446.02 +1537.1% 40042.87 snb-drag/crypto/tcrypt/2s-205-210
> 2484.11 +1599.6% 42219.71 snb-drag/crypto/tcrypt/2s-500-504
> 2118.55 +1155.8% 26604.99 vpx/crypto/tcrypt/2s-200-204
> 2713.48 +1198.5% 35234.77 vpx/crypto/tcrypt/2s-205-210
> 2711.31 +973.8% 29114.07 vpx/crypto/tcrypt/2s-301-319
> 2369.23 +940.3% 24648.12 vpx/crypto/tcrypt/2s-401-417
> 2620.64 +1428.7% 40060.71 vpx/crypto/tcrypt/2s-500-504
> 1713.98 +1624.3% 29553.72 vpx/crypto/tcrypt/2s-505-509
> 3423353.12 +1184.9% 43985148.08 TOTAL lock_stat.vmap_area_lock.holdtime-total
>
> The format of report is
>
> parent commit commit
> ------------------------ ------------------------
> number %change number testbox/testcase/test-params
> ...
> ...
> ...
> total number %change total number TOTAL perf-metric-name being compared
That's a lot of slowdowns, especially for such a simple patch.
Peter, any ideas?
WARNING: multiple messages have this Message-ID (diff)
From: Greg KH <gregkh@linuxfoundation.org>
To: Fengguang Wu <fengguang.wu@intel.com>
Cc: Peter Hurley <peter@hurleysoftware.com>,
LKML <linux-kernel@vger.kernel.org>,
lkp@01.org, Tejun Heo <tj@kernel.org>
Subject: Re: increased vmap_area_lock contentions on "n_tty: Move buffers into n_tty_data"
Date: Thu, 12 Sep 2013 20:44:13 -0700 [thread overview]
Message-ID: <20130913034413.GA26166@kroah.com> (raw)
In-Reply-To: <20130913033803.GB8704@localhost>
On Fri, Sep 13, 2013 at 11:38:04AM +0800, Fengguang Wu wrote:
> On Thu, Sep 12, 2013 at 08:17:00PM -0700, Greg KH wrote:
> > On Fri, Sep 13, 2013 at 08:51:33AM +0800, Fengguang Wu wrote:
> > > Hi Peter,
> > >
> > > FYI, we noticed much increased vmap_area_lock contentions since this
> > > commit:
> >
> > What does that mean? What is happening, are we allocating/removing more
> > memory now?
>
> // leave this question to Peter and Tejun. :)
>
> > What type of load were you running that showed this problem?
>
> The increased contentions and lock hold/wait time showed up in a
> number of test cases.
>
> The second email has the details, and this section of data is most
> illustrating.
>
> 8cb06c983822103da1cf 20bafb3d23d108bc0a89
> ------------------------ ------------------------
> 3470.31 +1631.0% 60070.49 fat/micro/dd-write/1HDD-cfq-ext4-10dd
> 3339.57 +1676.9% 59340.71 fat/micro/dd-write/1HDD-cfq-ext4-1dd
> 2848.79 +1489.1% 45269.54 lkp-a04/micro/netperf/120s-200%-TCP_CRR
> 3563.01 +1515.2% 57548.50 lkp-a04/micro/netperf/120s-200%-TCP_MAERTS
> 2678.64 +1346.0% 38733.80 lkp-a04/micro/netperf/120s-200%-TCP_RR
> 2839.84 +1450.2% 44022.08 lkp-a04/micro/netperf/120s-200%-TCP_SENDFILE
> 3417.37 +1571.4% 57116.34 lkp-a04/micro/netperf/120s-200%-TCP_STREAM
> 2558.59 +1450.8% 39677.58 lkp-a04/micro/netperf/120s-200%-UDP_RR
> 3737.24 +1558.0% 61963.62 lkp-a04/micro/netperf/120s-200%-UDP_STREAM
> 20219.50 +1488.7% 321218.02 lkp-a06/crypto/tcrypt/2s-200-204
> 21017.17 +1457.1% 327257.41 lkp-a06/crypto/tcrypt/2s-205-210
> 22109.84 +1240.3% 296346.33 lkp-a06/crypto/tcrypt/2s-401-417
> 17909.60 +1467.3% 280693.71 lkp-a06/micro/dbench/100%
> 489739.50 +978.5% 5281916.05 lkp-ne04/micro/aim7/shell_rtns_1
> 1601675.63 +906.7% 16123642.52 lkp-snb01/micro/aim7/exec_test
> 12105.00 +2453.6% 309110.42 nhm-white/micro/aim7/dbase
> 822461.02 +1585.0% 13858430.62 nhm-white/micro/aim7/exec_test
> 9858.11 +2715.9% 277595.41 nhm-white/micro/aim7/fork_test
> 3452.91 +1685.5% 61650.74 nhm-white/micro/aim7/fserver
> 300.14 +2621.5% 8168.53 nhm-white/micro/aim7/misc_rtns_1
> 345479.21 +1624.5% 5957828.25 nhm-white/micro/aim7/shell_rtns_1
> 2694.48 +1974.4% 55894.19 nhm-white/sysbench/oltp/100%-600-1000000
> 4415.67 +1202.2% 57501.52 nhm8/micro/dbench/100%
> 2284.65 +1505.2% 36672.75 snb-drag/crypto/tcrypt/2s-200-204
> 2446.02 +1537.1% 40042.87 snb-drag/crypto/tcrypt/2s-205-210
> 2484.11 +1599.6% 42219.71 snb-drag/crypto/tcrypt/2s-500-504
> 2118.55 +1155.8% 26604.99 vpx/crypto/tcrypt/2s-200-204
> 2713.48 +1198.5% 35234.77 vpx/crypto/tcrypt/2s-205-210
> 2711.31 +973.8% 29114.07 vpx/crypto/tcrypt/2s-301-319
> 2369.23 +940.3% 24648.12 vpx/crypto/tcrypt/2s-401-417
> 2620.64 +1428.7% 40060.71 vpx/crypto/tcrypt/2s-500-504
> 1713.98 +1624.3% 29553.72 vpx/crypto/tcrypt/2s-505-509
> 3423353.12 +1184.9% 43985148.08 TOTAL lock_stat.vmap_area_lock.holdtime-total
>
> The format of report is
>
> parent commit commit
> ------------------------ ------------------------
> number %change number testbox/testcase/test-params
> ...
> ...
> ...
> total number %change total number TOTAL perf-metric-name being compared
That's a lot of slowdowns, especially for such a simple patch.
Peter, any ideas?
next prev parent reply other threads:[~2013-09-13 3:44 UTC|newest]
Thread overview: 58+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-09-13 0:51 increased vmap_area_lock contentions on "n_tty: Move buffers into n_tty_data" Fengguang Wu
2013-09-13 0:51 ` Fengguang Wu
2013-09-13 1:09 ` Fengguang Wu
2013-09-13 1:09 ` Fengguang Wu
2013-09-17 15:34 ` Peter Hurley
2013-09-17 15:34 ` Peter Hurley
2013-09-17 23:22 ` Fengguang Wu
2013-09-17 23:22 ` Fengguang Wu
2013-09-18 0:22 ` Peter Hurley
2013-09-18 0:22 ` Peter Hurley
2013-09-25 9:04 ` Lin Ming
2013-09-25 9:04 ` Lin Ming
2013-09-25 11:30 ` Peter Hurley
2013-09-25 11:30 ` Peter Hurley
2013-09-25 14:53 ` Lin Ming
2013-09-25 14:53 ` Lin Ming
2013-09-25 16:02 ` Lin Ming
2013-09-25 16:02 ` Lin Ming
2013-09-26 3:20 ` Andi Kleen
2013-09-26 3:20 ` Andi Kleen
2013-09-26 11:52 ` Peter Hurley
2013-09-26 11:52 ` Peter Hurley
2013-09-26 15:32 ` Andi Kleen
2013-09-26 15:32 ` Andi Kleen
2013-09-26 17:22 ` Peter Hurley
2013-09-26 17:22 ` Peter Hurley
2013-09-26 7:33 ` Andrew Morton
2013-09-26 7:33 ` Andrew Morton
2013-09-26 11:31 ` Peter Hurley
2013-09-26 11:31 ` Peter Hurley
2013-09-26 15:04 ` Greg KH
2013-09-26 15:04 ` Greg KH
2013-09-26 17:35 ` Peter Hurley
2013-09-26 17:35 ` Peter Hurley
2013-09-26 18:05 ` Andrew Morton
2013-09-26 18:05 ` Andrew Morton
2013-09-26 21:42 ` Peter Hurley
2013-09-26 21:42 ` Peter Hurley
2013-09-26 21:58 ` Andrew Morton
2013-09-26 21:58 ` Andrew Morton
2013-09-26 22:21 ` Peter Hurley
2013-09-26 22:21 ` Peter Hurley
2013-09-18 0:49 ` Peter Hurley
2013-09-18 0:49 ` Peter Hurley
2013-09-13 3:17 ` Greg KH
2013-09-13 3:17 ` Greg KH
2013-09-13 3:38 ` Fengguang Wu
2013-09-13 3:38 ` Fengguang Wu
2013-09-13 3:44 ` Greg KH [this message]
2013-09-13 3:44 ` Greg KH
2013-09-13 9:55 ` Peter Hurley
2013-09-13 9:55 ` Peter Hurley
2013-09-13 12:34 ` Greg KH
2013-09-13 12:34 ` Greg KH
2013-09-17 2:42 ` Peter Hurley
2013-09-17 2:42 ` Peter Hurley
2013-09-17 2:56 ` Fengguang Wu
2013-09-17 2:56 ` Fengguang Wu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20130913034413.GA26166@kroah.com \
--to=gregkh@linuxfoundation.org \
--cc=lkp@lists.01.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.