From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from psmtp.com (na3sys010amx178.postini.com [74.125.245.178]) by kanga.kvack.org (Postfix) with SMTP id 3729B8D0001 for ; Mon, 24 Dec 2012 20:19:02 -0500 (EST) Received: by mail-vc0-f199.google.com with SMTP id p16so11013563vcq.10 for ; Mon, 24 Dec 2012 17:18:20 -0800 (PST) MIME-Version: 1.0 In-Reply-To: References: Date: Mon, 24 Dec 2012 17:18:20 -0800 Message-ID: Subject: Re: BUG: slub creates kmalloc slabs with refcount=0 From: Paul Hargrove Content-Type: multipart/alternative; boundary=20cf307c9d64db9afa04d1a316af Sender: owner-linux-mm@kvack.org List-ID: To: linux-mm@kvack.org --20cf307c9d64db9afa04d1a316af Content-Type: text/plain; charset=ISO-8859-1 Just to be clear the "BUG...." line I quoted contains an error of my own. It should say "kmalloc-8" twice rather than having "kmalloc-96" toward the end. This is the result of a cut-and-paste error (and brain already on vacation). The module ALSO has a larger struct the cache for which gets merged with the kmalloc-96 cache. I grabbed the first BUG line from the dmesg output and replaced "96" with "8" when composing the email, but missed the second occurrence. Merry Christmas, -Paul On Mon, Dec 24, 2012 at 4:55 PM, Paul Hargrove wrote: > > I have a 3.7.1 kernel on x86-86 > It is configured with > CONFIG_SLUB=y > CONFIG_SLUB_DEBUG=y > > I have an out-of-tree module calling KMEM_CACHE for an 8-byte struct: > cr_pdata_cachep = KMEM_CACHE(cr_pdata_s,0); > if (!cr_pdata_cachep) goto no_pdata_cachep; > printk(KERN_ERR "@ refcount = %d name = '%s'\n", > cr_pdata_cachep->refcount, cr_pdata_cachep->name); > > The output of the printk, below, shows that the request has been merged > with the built-in 8-byte kmalloc pool, BUT the resulting refcount is 1, > rather than 2 (or more): > @ refcount = 1 name = 'kmalloc-8' > > This results in a very unhappy kernel when the module calls > kmem_cache_destroy(cr_pdata_cachep); > at rmmod time, resulting is messages like > BUG kmalloc-8 (Tainted: G O): Objects remaining in > kmalloc-96 on kmem_cache_close() > > A quick look through mm/slub.c appears to confirm my suspicion that > "s->refcount" is never incremented for the built-in kmalloc-* caches. > However, I leave it to the experts to determine where the increment > belongs. > > FWIW: I am currently passing SLAB_POISON for the flags argument to > KMEM_CACHE() as a work-around (it prevents merging and, if I understand > correctly, has no overhead in a non-debug build). > > -Paul > > -- > Paul H. Hargrove PHHargrove@lbl.gov > Future Technologies Group > Computer and Data Sciences Department Tel: +1-510-495-2352 > Lawrence Berkeley National Laboratory Fax: +1-510-486-6900 > -- Paul H. Hargrove PHHargrove@lbl.gov Future Technologies Group Computer and Data Sciences Department Tel: +1-510-495-2352 Lawrence Berkeley National Laboratory Fax: +1-510-486-6900 --20cf307c9d64db9afa04d1a316af Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Just to be clear the "BUG...." line I quoted con= tains an error of my own.

It should say "kmalloc-8" = twice rather than having "kmalloc-96" toward the end. =A0This is = the result of a cut-and-paste error (and brain already on vacation). =A0The= module ALSO has a larger struct the cache for which gets merged with the k= malloc-96 cache. =A0I grabbed the first BUG line from the dmesg output and = replaced "96" with "8" when composing the email, but mi= ssed the second=A0occurrence.

Merry Christmas,
-Paul


= On Mon, Dec 24, 2012 at 4:55 PM, Paul Hargrove <phhargrove@lbl.gov>= ; wrote:

I have a 3.7= .1 kernel on x86-86
It is configured with
=A0 CONFIG_SLUB=3Dy=
=A0 CONFIG_SLUB_DEBUG=3Dy

I have an out-of-tr= ee module calling KMEM_CACHE for an 8-byte struct:
=A0 =A0 =A0 =A0 cr_pdata_cachep =3D KMEM_CACHE(cr_pdata_s,0);
=A0 =A0 =A0 =A0 if (!cr_pdata_cachep) goto no_pdata_cachep;
=A0 =A0 =A0 =A0 printk(KERN_ERR "@ refcount =3D %d name =3D '%s&= #39;\n", cr_pdata_cachep->refcount, cr_pdata_cachep->name);=A0

The output of the printk, below, shows that the request= has been merged with the built-in 8-byte kmalloc pool, BUT the resulting r= efcount is 1, rather than 2 (or more): =A0
=A0 =A0 @ refcou= nt =3D 1 name =3D 'kmalloc-8'

This results in a very unhappy kernel when t= he module calls
=A0=A0 =A0kmem_cache_destroy(cr_pdata_cachep);
at rmmod time, resulting is messages like
=A0 =A0=A0BUG kmalloc-8 (Tainted: G =A0 =A0 =A0 =A0 =A0 O): Objects remaini= ng in kmalloc-96 on kmem_cache_close()

A quick loo= k through mm/slub.c appears to confirm my=A0suspicion=A0that "s->re= fcount" is never incremented for the built-in kmalloc-* caches. =A0How= ever, I leave it to the experts to determine where the increment belongs.

FWIW: I am currently passing SLAB_POISON for the flags = argument to KMEM_CACHE() as a work-around (it prevents merging and, if I un= derstand correctly, has no overhead in a non-debug build).

-Paul

--
Paul H. Hargrove =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0PHHa= rgrove@lbl.gov
Future Technologies Group
Computer and Data Sciences Departm= ent =A0 =A0 Tel: +1-510-495-2352
Lawrence Berkeley National L= aboratory =A0 =A0 Fax: +1-510-486-6900



--
Paul H. Hargrove =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0 =A0 =A0 =A0PHHargrove@lbl.gov
Future Technologies Group
Computer and Data Sciences Departm= ent =A0 =A0 Tel: +1-510-495-2352
Lawrence Berkeley National Labor= atory =A0 =A0 Fax: +1-510-486-6900
--20cf307c9d64db9afa04d1a316af-- -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org