From: Tim Bird <tim.bird@am.sony.com>
To: Eric Dumazet <eric.dumazet@gmail.com>
Cc: Ezequiel Garcia <elezegarcia@gmail.com>,
David Rientjes <rientjes@google.com>,
Andi Kleen <andi@firstfloor.org>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
"linux-mm@kvack.org" <linux-mm@kvack.org>,
"celinux-dev@lists.celinuxforum.org"
<celinux-dev@lists.celinuxforum.org>
Subject: Re: [Q] Default SLAB allocator
Date: Wed, 17 Oct 2012 11:45:23 -0700 [thread overview]
Message-ID: <507EFCC3.1050304@am.sony.com> (raw)
In-Reply-To: <1350414968.3954.1427.camel@edumazet-glaptop>
On 10/16/2012 12:16 PM, Eric Dumazet wrote:
> On Tue, 2012-10-16 at 15:27 -0300, Ezequiel Garcia wrote:
>
>> Yes, we have some numbers:
>>
>> http://elinux.org/Kernel_dynamic_memory_analysis#Kmalloc_objects
>>
>> Are they too informal? I can add some details...
>>
>> They've been measured on a **very** minimal setup, almost every option
>> is stripped out, except from initramfs, sysfs, and trace.
>>
>> On this scenario, strings allocated for file names and directories
>> created by sysfs
>> are quite noticeable, being 4-16 bytes, and produce a lot of fragmentation from
>> that 32 byte cache at SLAB.
>>
>> Is an option to enable small caches on SLUB and SLAB worth it?
>
> Random small web server :
>
> # free
> total used free shared buffers cached
> Mem: 7884536 5412572 2471964 0 155440 1803340
> -/+ buffers/cache: 3453792 4430744
> Swap: 2438140 51164 2386976
8G is a small web server? The RAM budget for Linux on one of
Sony's cameras was 10M. We're not merely not in the same ballpark -
you're in a ballpark and I'm trimming bonsai trees... :-)
> # grep Slab /proc/meminfo
> Slab: 351592 kB
>
> # egrep "kmalloc-32|kmalloc-16|kmalloc-8" /proc/slabinfo
> kmalloc-32 11332 12544 32 128 1 : tunables 0 0 0 : slabdata 98 98 0
> kmalloc-16 5888 5888 16 256 1 : tunables 0 0 0 : slabdata 23 23 0
> kmalloc-8 76563 82432 8 512 1 : tunables 0 0 0 : slabdata 161 161 0
>
> Really, some waste on these small objects is pure noise on SMP hosts.
In this example, it appears that if all kmalloc-8's were pushed into 32-byte slabs,
we'd lose about 1.8 meg due to pure slab overhead. This would not be noise
on my system.
> (Waste on bigger objects is probably more important by orders of magnitude)
Maybe.
I need to run some measurements on systems that are more similar to what
we're deploying in products. I'll see if I can share them.
-- Tim
=============================
Tim Bird
Architecture Group Chair, CE Workgroup of the Linux Foundation
Senior Staff Engineer, Sony Network Entertainment
=============================
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
WARNING: multiple messages have this Message-ID (diff)
From: Tim Bird <tim.bird@am.sony.com>
To: Eric Dumazet <eric.dumazet@gmail.com>
Cc: Ezequiel Garcia <elezegarcia@gmail.com>,
David Rientjes <rientjes@google.com>,
Andi Kleen <andi@firstfloor.org>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
"linux-mm@kvack.org" <linux-mm@kvack.org>,
"celinux-dev@lists.celinuxforum.org"
<celinux-dev@lists.celinuxforum.org>
Subject: Re: [Q] Default SLAB allocator
Date: Wed, 17 Oct 2012 11:45:23 -0700 [thread overview]
Message-ID: <507EFCC3.1050304@am.sony.com> (raw)
In-Reply-To: <1350414968.3954.1427.camel@edumazet-glaptop>
On 10/16/2012 12:16 PM, Eric Dumazet wrote:
> On Tue, 2012-10-16 at 15:27 -0300, Ezequiel Garcia wrote:
>
>> Yes, we have some numbers:
>>
>> http://elinux.org/Kernel_dynamic_memory_analysis#Kmalloc_objects
>>
>> Are they too informal? I can add some details...
>>
>> They've been measured on a **very** minimal setup, almost every option
>> is stripped out, except from initramfs, sysfs, and trace.
>>
>> On this scenario, strings allocated for file names and directories
>> created by sysfs
>> are quite noticeable, being 4-16 bytes, and produce a lot of fragmentation from
>> that 32 byte cache at SLAB.
>>
>> Is an option to enable small caches on SLUB and SLAB worth it?
>
> Random small web server :
>
> # free
> total used free shared buffers cached
> Mem: 7884536 5412572 2471964 0 155440 1803340
> -/+ buffers/cache: 3453792 4430744
> Swap: 2438140 51164 2386976
8G is a small web server? The RAM budget for Linux on one of
Sony's cameras was 10M. We're not merely not in the same ballpark -
you're in a ballpark and I'm trimming bonsai trees... :-)
> # grep Slab /proc/meminfo
> Slab: 351592 kB
>
> # egrep "kmalloc-32|kmalloc-16|kmalloc-8" /proc/slabinfo
> kmalloc-32 11332 12544 32 128 1 : tunables 0 0 0 : slabdata 98 98 0
> kmalloc-16 5888 5888 16 256 1 : tunables 0 0 0 : slabdata 23 23 0
> kmalloc-8 76563 82432 8 512 1 : tunables 0 0 0 : slabdata 161 161 0
>
> Really, some waste on these small objects is pure noise on SMP hosts.
In this example, it appears that if all kmalloc-8's were pushed into 32-byte slabs,
we'd lose about 1.8 meg due to pure slab overhead. This would not be noise
on my system.
> (Waste on bigger objects is probably more important by orders of magnitude)
Maybe.
I need to run some measurements on systems that are more similar to what
we're deploying in products. I'll see if I can share them.
-- Tim
=============================
Tim Bird
Architecture Group Chair, CE Workgroup of the Linux Foundation
Senior Staff Engineer, Sony Network Entertainment
=============================
next prev parent reply other threads:[~2012-10-17 18:41 UTC|newest]
Thread overview: 66+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-10-11 14:19 [Q] Default SLAB allocator Ezequiel Garcia
2012-10-11 14:19 ` Ezequiel Garcia
2012-10-11 22:42 ` Andi Kleen
2012-10-11 22:42 ` Andi Kleen
2012-10-11 22:59 ` David Rientjes
2012-10-11 22:59 ` David Rientjes
2012-10-11 23:10 ` Andi Kleen
2012-10-11 23:10 ` Andi Kleen
2012-10-12 12:07 ` Ezequiel Garcia
2012-10-12 12:07 ` Ezequiel Garcia
2012-10-13 9:54 ` David Rientjes
2012-10-13 9:54 ` David Rientjes
2012-10-13 12:44 ` Ezequiel Garcia
2012-10-13 12:44 ` Ezequiel Garcia
2012-10-16 0:46 ` David Rientjes
2012-10-16 0:46 ` David Rientjes
2012-10-16 12:35 ` Ezequiel Garcia
2012-10-16 12:35 ` Ezequiel Garcia
2012-10-16 12:56 ` Eric Dumazet
2012-10-16 12:56 ` Eric Dumazet
2012-10-16 18:07 ` Tim Bird
2012-10-16 18:07 ` Tim Bird
2012-10-16 18:27 ` Ezequiel Garcia
2012-10-16 18:27 ` Ezequiel Garcia
2012-10-16 18:44 ` Tim Bird
2012-10-16 18:44 ` Tim Bird
2012-10-16 18:49 ` Ezequiel Garcia
2012-10-16 18:49 ` Ezequiel Garcia
2012-10-16 19:16 ` Eric Dumazet
2012-10-16 19:16 ` Eric Dumazet
2012-10-17 18:45 ` Tim Bird [this message]
2012-10-17 18:45 ` Tim Bird
2012-10-17 19:13 ` Eric Dumazet
2012-10-17 19:13 ` Eric Dumazet
2012-10-17 19:20 ` Shentino
2012-10-17 19:20 ` Shentino
2012-10-17 20:33 ` Tim Bird
2012-10-17 20:33 ` Tim Bird
2012-10-18 0:46 ` Shentino
2012-10-18 0:46 ` Shentino
2012-10-17 20:58 ` Tim Bird
2012-10-17 20:58 ` Tim Bird
2012-10-17 21:05 ` Ezequiel Garcia
2012-10-17 21:05 ` Ezequiel Garcia
2012-10-16 18:36 ` Ezequiel Garcia
2012-10-16 18:36 ` Ezequiel Garcia
2012-10-16 18:54 ` Christoph Lameter
2012-10-16 18:54 ` Christoph Lameter
2012-10-13 9:51 ` David Rientjes
2012-10-13 9:51 ` David Rientjes
2012-10-13 15:10 ` Eric Dumazet
2012-10-13 15:10 ` Eric Dumazet
2012-10-16 1:28 ` JoonSoo Kim
2012-10-16 1:28 ` JoonSoo Kim
2012-10-16 7:23 ` Eric Dumazet
2012-10-16 7:23 ` Eric Dumazet
2012-10-19 0:03 ` JoonSoo Kim
2012-10-19 0:03 ` JoonSoo Kim
2012-10-19 7:01 ` Eric Dumazet
2012-10-19 7:01 ` Eric Dumazet
2012-10-16 0:45 ` David Rientjes
2012-10-16 0:45 ` David Rientjes
2012-10-16 18:53 ` Christoph Lameter
2012-10-16 18:53 ` Christoph Lameter
2012-10-16 19:02 ` Christoph Lameter
2012-10-16 19:02 ` Christoph Lameter
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=507EFCC3.1050304@am.sony.com \
--to=tim.bird@am.sony.com \
--cc=andi@firstfloor.org \
--cc=celinux-dev@lists.celinuxforum.org \
--cc=elezegarcia@gmail.com \
--cc=eric.dumazet@gmail.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=rientjes@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.