From: Peter Hurley <peter@hurleysoftware.com>
To: lkp@lists.01.org
Subject: Re: increased vmap_area_lock contentions on "n_tty: Move buffers into n_tty_data"
Date: Tue, 17 Sep 2013 20:49:51 -0400 [thread overview]
Message-ID: <5238F8AF.4050309@hurleysoftware.com> (raw)
In-Reply-To: <20130913010936.GA1291@localhost>
[-- Attachment #1: Type: text/plain, Size: 2007 bytes --]
On 09/12/2013 09:09 PM, Fengguang Wu wrote:
> On Fri, Sep 13, 2013 at 08:51:33AM +0800, Fengguang Wu wrote:
>> Hi Peter,
>>
>> FYI, we noticed much increased vmap_area_lock contentions since this
>> commit:
>>
>> commit 20bafb3d23d108bc0a896eb8b7c1501f4f649b77
>> Author: Peter Hurley <peter@hurleysoftware.com>
>> Date: Sat Jun 15 10:21:19 2013 -0400
>>
>> n_tty: Move buffers into n_tty_data
>>
>> Reduce pointer reloading and improve locality-of-reference;
>> allocate read_buf and echo_buf within struct n_tty_data.
>
> Here are some comparison between this commit [o] with its parent commit [*].
[...]
> 8cb06c983822103da1cf 20bafb3d23d108bc0a89
> ------------------------ ------------------------
> 976.67 +108.3% 2034.67 lkp-a04/micro/netperf/120s-200%-TCP_STREAM
> 8971.36 +11.4% 9997.05 nhm-white/micro/aim7/exec_test
> 9948.03 +20.9% 12031.72 TOTAL slabinfo.kmalloc-128.active_objs
>
> 8cb06c983822103da1cf 20bafb3d23d108bc0a89
> ------------------------ ------------------------
> 976.67 +108.3% 2034.67 lkp-a04/micro/netperf/120s-200%-TCP_STREAM
> 9127.64 +11.4% 10164.15 nhm-white/micro/aim7/exec_test
> 10104.31 +20.7% 12198.82 TOTAL slabinfo.kmalloc-128.num_objs
The dramatic increase in 128-byte kmalloc blocks is from vmalloc overhead
with associated with each allocation. On a x86_64, struct vmap_area is 104 bytes,
rounded to 128, allocated with every vmalloc allocation. This is approx 1% overhead
(which seems high to me).
The reason this is still visible after the test completes is the vmap area is
lazily reclaimed (see mm/vmalloc.c:__purge_vmap_area_lazy()).
1% memory overhead coupled with the unwanted vmap_area_lock contention (even though it is
test-induced) -- I might revert this anyway.
Regards,
Peter Hurley
WARNING: multiple messages have this Message-ID (diff)
From: Peter Hurley <peter@hurleysoftware.com>
To: Fengguang Wu <fengguang.wu@intel.com>
Cc: Greg KH <gregkh@linuxfoundation.org>,
LKML <linux-kernel@vger.kernel.org>,
lkp@01.org, Tejun Heo <tj@kernel.org>
Subject: Re: increased vmap_area_lock contentions on "n_tty: Move buffers into n_tty_data"
Date: Tue, 17 Sep 2013 20:49:51 -0400 [thread overview]
Message-ID: <5238F8AF.4050309@hurleysoftware.com> (raw)
In-Reply-To: <20130913010936.GA1291@localhost>
On 09/12/2013 09:09 PM, Fengguang Wu wrote:
> On Fri, Sep 13, 2013 at 08:51:33AM +0800, Fengguang Wu wrote:
>> Hi Peter,
>>
>> FYI, we noticed much increased vmap_area_lock contentions since this
>> commit:
>>
>> commit 20bafb3d23d108bc0a896eb8b7c1501f4f649b77
>> Author: Peter Hurley <peter@hurleysoftware.com>
>> Date: Sat Jun 15 10:21:19 2013 -0400
>>
>> n_tty: Move buffers into n_tty_data
>>
>> Reduce pointer reloading and improve locality-of-reference;
>> allocate read_buf and echo_buf within struct n_tty_data.
>
> Here are some comparison between this commit [o] with its parent commit [*].
[...]
> 8cb06c983822103da1cf 20bafb3d23d108bc0a89
> ------------------------ ------------------------
> 976.67 +108.3% 2034.67 lkp-a04/micro/netperf/120s-200%-TCP_STREAM
> 8971.36 +11.4% 9997.05 nhm-white/micro/aim7/exec_test
> 9948.03 +20.9% 12031.72 TOTAL slabinfo.kmalloc-128.active_objs
>
> 8cb06c983822103da1cf 20bafb3d23d108bc0a89
> ------------------------ ------------------------
> 976.67 +108.3% 2034.67 lkp-a04/micro/netperf/120s-200%-TCP_STREAM
> 9127.64 +11.4% 10164.15 nhm-white/micro/aim7/exec_test
> 10104.31 +20.7% 12198.82 TOTAL slabinfo.kmalloc-128.num_objs
The dramatic increase in 128-byte kmalloc blocks is from vmalloc overhead
with associated with each allocation. On a x86_64, struct vmap_area is 104 bytes,
rounded to 128, allocated with every vmalloc allocation. This is approx 1% overhead
(which seems high to me).
The reason this is still visible after the test completes is the vmap area is
lazily reclaimed (see mm/vmalloc.c:__purge_vmap_area_lazy()).
1% memory overhead coupled with the unwanted vmap_area_lock contention (even though it is
test-induced) -- I might revert this anyway.
Regards,
Peter Hurley
next prev parent reply other threads:[~2013-09-18 0:49 UTC|newest]
Thread overview: 58+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-09-13 0:51 increased vmap_area_lock contentions on "n_tty: Move buffers into n_tty_data" Fengguang Wu
2013-09-13 0:51 ` Fengguang Wu
2013-09-13 1:09 ` Fengguang Wu
2013-09-13 1:09 ` Fengguang Wu
2013-09-17 15:34 ` Peter Hurley
2013-09-17 15:34 ` Peter Hurley
2013-09-17 23:22 ` Fengguang Wu
2013-09-17 23:22 ` Fengguang Wu
2013-09-18 0:22 ` Peter Hurley
2013-09-18 0:22 ` Peter Hurley
2013-09-25 9:04 ` Lin Ming
2013-09-25 9:04 ` Lin Ming
2013-09-25 11:30 ` Peter Hurley
2013-09-25 11:30 ` Peter Hurley
2013-09-25 14:53 ` Lin Ming
2013-09-25 14:53 ` Lin Ming
2013-09-25 16:02 ` Lin Ming
2013-09-25 16:02 ` Lin Ming
2013-09-26 3:20 ` Andi Kleen
2013-09-26 3:20 ` Andi Kleen
2013-09-26 11:52 ` Peter Hurley
2013-09-26 11:52 ` Peter Hurley
2013-09-26 15:32 ` Andi Kleen
2013-09-26 15:32 ` Andi Kleen
2013-09-26 17:22 ` Peter Hurley
2013-09-26 17:22 ` Peter Hurley
2013-09-26 7:33 ` Andrew Morton
2013-09-26 7:33 ` Andrew Morton
2013-09-26 11:31 ` Peter Hurley
2013-09-26 11:31 ` Peter Hurley
2013-09-26 15:04 ` Greg KH
2013-09-26 15:04 ` Greg KH
2013-09-26 17:35 ` Peter Hurley
2013-09-26 17:35 ` Peter Hurley
2013-09-26 18:05 ` Andrew Morton
2013-09-26 18:05 ` Andrew Morton
2013-09-26 21:42 ` Peter Hurley
2013-09-26 21:42 ` Peter Hurley
2013-09-26 21:58 ` Andrew Morton
2013-09-26 21:58 ` Andrew Morton
2013-09-26 22:21 ` Peter Hurley
2013-09-26 22:21 ` Peter Hurley
2013-09-18 0:49 ` Peter Hurley [this message]
2013-09-18 0:49 ` Peter Hurley
2013-09-13 3:17 ` Greg KH
2013-09-13 3:17 ` Greg KH
2013-09-13 3:38 ` Fengguang Wu
2013-09-13 3:38 ` Fengguang Wu
2013-09-13 3:44 ` Greg KH
2013-09-13 3:44 ` Greg KH
2013-09-13 9:55 ` Peter Hurley
2013-09-13 9:55 ` Peter Hurley
2013-09-13 12:34 ` Greg KH
2013-09-13 12:34 ` Greg KH
2013-09-17 2:42 ` Peter Hurley
2013-09-17 2:42 ` Peter Hurley
2013-09-17 2:56 ` Fengguang Wu
2013-09-17 2:56 ` Fengguang Wu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5238F8AF.4050309@hurleysoftware.com \
--to=peter@hurleysoftware.com \
--cc=lkp@lists.01.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.