Kmem_cache handling in linux-2.6.2x kernel

Kmem_cache handling in linux-2.6.2x kernel

[KokHow.Teh]

Hi list;
	I have a question about kmem_cache implemented in Linux-2.6.2x
kernel. I have an application that allocates and free 64KByte chunks of
memory (32-byte aligned) quite often. Therefore, I create a lookaside
cache for that purpose and use kmem_cache_alloc(), kmem_cache_free() to
allocate and free the caches. The application works very well in this
model. However, my concern here is if kmem_cache_free() does return the
cache to the system-wide pool so that it could be used by other
applications when need arises; when system is low in memory resources,
for instance. This is a question about the internal workings of the
memory management system of the Linux-2.6.2x kernel as to how efficient
it manages this lookasie caches. The concern is valid because if this
lookaside cache is not managed well, i.e, it is not returned to the
system-wide free memory pools to be used by other applications, this
will penalize the performace and throughput of the whole system due to
the dynamic behaviour of the utilization of system memory resources. For
example, other applications might be swapping in and out of the harddisk
and if the kmem_cache_free()'ed memory objects could be used by these
applications, it will help in this case to reduce the number of swaps
that happen, thereby freeing the CPU and/or DMA from doing the swapping
to do other critical tasks.

	On the other hand, if the caches are returned to the system-wide
free memory pool, what are the advantages of using kmem_cache_t compared
to the conventional kmalloc()/kfree()?

	Any insight and advice is appreciated.

Regards,
KH

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Kmem_cache handling in linux-2.6.2x kernel

[Christoph Lameter]

KokHow.Teh@infineon.com wrote:
> Hi list;
> 	I have a question about kmem_cache implemented in Linux-2.6.2x
> kernel. I have an application that allocates and free 64KByte chunks of
> memory (32-byte aligned) quite often. Therefore, I create a lookaside
> cache for that purpose and use kmem_cache_alloc(), kmem_cache_free() to
> allocate and free the caches. The application works very well in this
> model. However, my concern here is if kmem_cache_free() does return the
> cache to the system-wide pool so that it could be used by other
> applications when need arises; when system is low in memory resources,
> for instance. This is a question about the internal workings of the
> memory management system of the Linux-2.6.2x kernel as to how efficient
> it manages this lookasie caches. The concern is valid because if this
> lookaside cache is not managed well, i.e, it is not returned to the
> system-wide free memory pools to be used by other applications, this
> will penalize the performace and throughput of the whole system due to
> the dynamic behaviour of the utilization of system memory resources. For
> example, other applications might be swapping in and out of the harddisk
> and if the kmem_cache_free()'ed memory objects could be used by these
> applications, it will help in this case to reduce the number of swaps
> that happen, thereby freeing the CPU and/or DMA from doing the swapping
> to do other critical tasks.
> 
> 	On the other hand, if the caches are returned to the system-wide
> free memory pool, what are the advantages of using kmem_cache_t compared
> to the conventional kmalloc()/kfree()?
> 
> 	Any insight and advice is appreciated.

Any kmem_cache allocation and frees larger than PAGE_SIZE are converted to page allocator allocs and frees. Thus you are allocating and freeing directly from the general pool.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Kmem_cache handling in linux-2.6.2x kernel

[KokHow.Teh]

Hi list;
	I have a question about kmem_cache implemented in Linux-2.6.2x
kernel. I have an application that allocates and free 64KByte chunks of
memory (32-byte aligned) quite often. Therefore, I create a lookaside
cache for that purpose and use kmem_cache_alloc(), kmem_cache_free() to
allocate and free the caches. The application works very well in this
model. However, my concern here is if kmem_cache_free() does return the
cache to the system-wide pool so that it could be used by other
applications when need arises; when system is low in memory resources,
for instance. This is a question about the internal workings of the
memory management system of the Linux-2.6.2x kernel as to how efficient
it manages this lookasie caches. The concern is valid because if this
lookaside cache is not managed well, i.e, it is not returned to the
system-wide free memory pools to be used by other applications, this
will penalize the performace and throughput of the whole system due to
the dynamic behaviour of the utilization of system memory resources. For
example, other applications might be swapping in and out of the harddisk
and if the kmem_cache_free()'ed memory objects could be used by these
applications, it will help in this case to reduce the number of swaps
that happen, thereby freeing the CPU and/or DMA from doing the swapping
to do other critical tasks.

	On the other hand, if the caches are returned to the system-wide
free memory pool, what are the advantages of using kmem_cache_t compared
to the conventional kmalloc()/kfree()?

	Any insight and advice is appreciated.

Regards,
KH

Re: Kmem_cache handling in linux-2.6.2x kernel

[Pekka Enberg]

Hi,

On Wed, Jul 9, 2008 at 8:06 AM,  <KokHow.Teh@infineon.com> wrote:
>        I have a question about kmem_cache implemented in Linux-2.6.2x
> kernel. I have an application that allocates and free 64KByte chunks of
> memory (32-byte aligned) quite often. Therefore, I create a lookaside
> cache for that purpose and use kmem_cache_alloc(), kmem_cache_free() to
> allocate and free the caches. The application works very well in this
> model. However, my concern here is if kmem_cache_free() does return the
> cache to the system-wide pool so that it could be used by other
> applications when need arises; when system is low in memory resources,
> for instance. This is a question about the internal workings of the
> memory management system of the Linux-2.6.2x kernel as to how efficient
> it manages this lookasie caches. The concern is valid because if this
> lookaside cache is not managed well, i.e, it is not returned to the
> system-wide free memory pools to be used by other applications, this
> will penalize the performace and throughput of the whole system due to
> the dynamic behaviour of the utilization of system memory resources. For
> example, other applications might be swapping in and out of the harddisk
> and if the kmem_cache_free()'ed memory objects could be used by these
> applications, it will help in this case to reduce the number of swaps
> that happen, thereby freeing the CPU and/or DMA from doing the swapping
> to do other critical tasks.

I'm not sure I understand the question. The pages allocated for a
particular cache are given back to the page allocator whenever all
objects of a slab are freed. In addition, SLUB does slab merging so
the same cache can be transparently used by other kmem_cache_alloc()
callers. So there really are no reservation guarantees for a cache in
OOM conditions.

On Wed, Jul 9, 2008 at 8:06 AM,  <KokHow.Teh@infineon.com> wrote:
>        On the other hand, if the caches are returned to the system-wide
> free memory pool, what are the advantages of using kmem_cache_t compared
> to the conventional kmalloc()/kfree()?

The main advantages for using kmem_cache_create() are that (1) you
control the alignment and (2) you get tighter packing for the objects
(less internal fragmentation).

RE: Kmem_cache handling in linux-2.6.2x kernel

[KokHow.Teh]

Hi;

>On Wed, Jul 9, 2008 at 8:06 AM,  <KokHow.Teh@infineon.com> wrote:
>>        I have a question about kmem_cache implemented in Linux-2.6.2x

>> kernel. I have an application that allocates and free 64KByte chunks 
>> of memory (32-byte aligned) quite often. Therefore, I create a 
>> lookaside cache for that purpose and use kmem_cache_alloc(), 
>> kmem_cache_free() to allocate and free the caches. The application 
>> works very well in this model. However, my concern here is if 
>> kmem_cache_free() does return the cache to the system-wide pool so 
>> that it could be used by other applications when need arises; when 
>> system is low in memory resources, for instance. This is a question 
>> about the internal workings of the memory management system of the 
>> Linux-2.6.2x kernel as to how efficient it manages this lookasie 
>> caches. The concern is valid because if this lookaside cache is not 
>> managed well, i.e, it is not returned to the system-wide free memory 
>> pools to be used by other applications, this will penalize the 
>> performace and throughput of the whole system due to the dynamic 
>> behaviour of the utilization of system memory resources. For example,

>> other applications might be swapping in and out of the harddisk and
if 
>> the kmem_cache_free()'ed memory objects could be used by these 
>> applications, it will help in this case to reduce the number of swaps

>> that happen, thereby freeing the CPU and/or DMA from doing the
swapping to do other critical tasks.

>I'm not sure I understand the question. The pages allocated for a
particular cache are given back to the page allocator whenever all
objects of a slab are freed. 

In my applications, only part of total number of objects of the
kmem_cache are freed. So my question is what happen to these "freed"
objects? How are these "freed" objects managed by the linux-MM? Would
they be reused by other kmem_cache_alloc() and/or kmalloc()?

>In addition, SLUB does slab merging so the same cache can be
transparently used by other kmem_cache_alloc() callers. So there really
are no reservation guarantees for a cache in OOM conditions.

(1) SLUB is not available in 2.6.20 kernel which I am using for my
products. In this case, is there similar mechamisms in place to
faciliate "page-sharing" amongst the kmem_cache_alloc() callers?

(2) Does this "page-sharing" happen for kmalloc() callers?

On Wed, Jul 9, 2008 at 8:06 AM,  <KokHow.Teh@infineon.com> wrote:
>>        On the other hand, if the caches are returned to the 
>> system-wide free memory pool, what are the advantages of using 
>> kmem_cache_t compared to the conventional kmalloc()/kfree()?

>The main advantages for using kmem_cache_create() are that (1) you
control the alignment and (2) you get tighter packing for the objects
(less internal fragmentation).

Thanks.

Regards,
KH

Re: Kmem_cache handling in linux-2.6.2x kernel

[Pekka Enberg]

Hi,

On Wed, Jul 9, 2008 at 1:30 PM,  <KokHow.Teh@infineon.com> wrote:
> In my applications, only part of total number of objects of the
> kmem_cache are freed. So my question is what happen to these "freed"
> objects? How are these "freed" objects managed by the linux-MM? Would
> they be reused by other kmem_cache_alloc() and/or kmalloc()?

The free'd objects will be returned to the cache and are, of course,
reused by later kmem_cache_alloc() and kmalloc() calls. Note that with
_SLAB_, you never have cache sharing, so an object free'd by
kmem_cache_free() is only available for a kmem_cache_alloc() of the
same cache.

At some point in time, I wrote:
>>In addition, SLUB does slab merging so the same cache can be
>> transparently used by other kmem_cache_alloc() callers. So there really
>> are no reservation guarantees for a cache in OOM conditions.

On Wed, Jul 9, 2008 at 1:30 PM,  <KokHow.Teh@infineon.com> wrote:
> (1) SLUB is not available in 2.6.20 kernel which I am using for my
> products. In this case, is there similar mechamisms in place to
> faciliate "page-sharing" amongst the kmem_cache_alloc() callers?

No.

On Wed, Jul 9, 2008 at 1:30 PM,  <KokHow.Teh@infineon.com> wrote:
> (2) Does this "page-sharing" happen for kmalloc() callers?

The caches for kmalloc() are set up first, so a kmalloc() never dips
into a cache created by kmem_cache_create(). But a kmem_cache_alloc()
can dip into a kmalloc cache. Look at the create_kmalloc_cache() calls
in kmem_cache_init() and the find_mergeable() call in
kmem_cache_create() in mm/slub.c for details.

Btw, you mentioned that you're allocating 64 KBs. So with SLUB, if you
use _kmalloc()_ the request will be passed through to the page
allocator directly (see kmalloc_large()).

RE: Kmem_cache handling in linux-2.6.2x kernel

[KokHow.Teh]

Hi; 

>On Wed, Jul 9, 2008 at 1:30 PM,  <KokHow.Teh@infineon.com> wrote:
>> In my applications, only part of total number of objects of the 
>> kmem_cache are freed. So my question is what happen to these "freed"
>> objects? How are these "freed" objects managed by the linux-MM? Would

>> they be reused by other kmem_cache_alloc() and/or kmalloc()?

>The free'd objects will be returned to the cache and are, of course,
reused by later kmem_cache_alloc() and kmalloc() calls. 
>Note that with _SLAB_, you never have cache sharing, so an object
free'd by kmem_cache_free() is only available for a kmem_cache_alloc()
of the same cache.

So with _SLAB_, kmem_cache_free() objects are only reusuable by
kmem_cache_alloc() of the same cache.

Are kmem_cache_free() objects reusable by _all_ kmalloc() calls? (*)

>At some point in time, I wrote:
>>>In addition, SLUB does slab merging so the same cache can be  
>>>transparently used by other kmem_cache_alloc() callers. So there 
>>>really  are no reservation guarantees for a cache in OOM conditions.

>On Wed, Jul 9, 2008 at 1:30 PM,  <KokHow.Teh@infineon.com> wrote:
>> (1) SLUB is not available in 2.6.20 kernel which I am using for my 
>> products. In this case, is there similar mechamisms in place to 
>> faciliate "page-sharing" amongst the kmem_cache_alloc() callers?

>No.

>On Wed, Jul 9, 2008 at 1:30 PM,  <KokHow.Teh@infineon.com> wrote:
>> (2) Does this "page-sharing" happen for kmalloc() callers?

>The caches for kmalloc() are set up first, so a kmalloc() never dips
into a cache created by kmem_cache_create(). 

This conflicts with (*) above.

>But a kmem_cache_alloc() can dip into a kmalloc cache. Look at the
create_kmalloc_cache() calls in kmem_cache_init() and the
find_mergeable() call in
>kmem_cache_create() in mm/slub.c for details.

>Btw, you mentioned that you're allocating 64 KBs. So with SLUB, if you
use _kmalloc()_ the request will be passed through to the page allocator
directly (see kmalloc_large()).

What's your point here? How would this compare to using the kmem_cache?

Thanks.

Regards,
KH

Kmem_cache handling in linux-2.6.2x kernel.

[KokHow.Teh]

Hi list;
	I have a question about kmem_cache implemented in Linux-2.6.2x
kernel. I have an application that allocates and free 64KByte chunks of
memory (32-byte aligned) quite often. Therefore, I create a lookaside
cache for that purpose and use kmem_cache_alloc(), kmem_cache_free() to
allocate and free the caches. The application works very well in this
model. However, my concern here is if kmem_cache_free() does return the
cache to the system-wide pool so that it could be used by other
applications when need arises; when system is low in memory resources,
for instance. This is a question about the internal workings of the
memory management system of the Linux-2.6.2x kernel as to how efficient
it manages this lookasie caches. The concern is valid because if this
lookaside cache is not managed well, i.e, it is not returned to the
system-wide pool of free memory pools to be used by other applications,
this will penalize the performace and throughput of the whole system due
to the dynamic behaviour of the utilization of system memory resources.
For example, other applications might be swapping in and out of the
harddisk and if the kmem_cache_free()'ed memory objects could be used by
these applications, it will help in this case to reduce the number of
swaps that happen, thereby freeing the CPU and/or DMA from doing the
swapping to do other critical tasks.

	Any insight and advice is appreciated.

Regards,
KH

Kmem_cache handling in linux-2.6.2x kernel.

[KokHow.Teh]

Hi list;
	I have a question about kmem_cache implemented in Linux-2.6.2x
kernel. I have an application that allocates and free 64KByte chunks of
memory (32-byte aligned) quite often. Therefore, I create a lookaside
cache for that purpose and use kmem_cache_alloc(), kmem_cache_free() to
allocate and free the caches. The application works very well in this
model. However, my concern here is if kmem_cache_free() does return the
cache to the system-wide pool so that it could be used by other
applications when need arises; when system is low in memory resources,
for instance. This is a question about the internal workings of the
memory management system of the Linux-2.6.2x kernel as to how efficient
it manages this lookasie caches. The concern is valid because if this
lookaside cache is not managed well, i.e, it is not returned to the
system-wide pool of free memory pools to be used by other applications,
this will penalize the performace and throughput of the whole system due
to the dynamic behaviour of the utilization of system memory resources.
For example, other applications might be swapping in and out of the
harddisk and if the kmem_cache_free()'ed memory objects could be used by
these applications, it will help in this case to reduce the number of
swaps that happen, thereby freeing the CPU and/or DMA from doing the
swapping to do other critical tasks.

	Any insight and advice is appreciated.

Regards,
KH