Re: 3.14.65: Memory leak when slub

All of lore.kernel.org
 help / color / mirror / Atom feed

* Re: 3.14.65: Memory leak when slub_debug is enabled
       [not found] <CAA4-JFLOmeYrWOEO_d2ALPgf0cWhC_fv1Gisz5fyH3uY1ogV1g@mail.gmail.com>
@ 2016-03-29 23:35 ` Christoph Lameter
  2016-03-30  8:58   ` Jesper Dangaard Brouer
  1 sibling, 0 replies; 4+ messages in thread
From: Christoph Lameter @ 2016-03-29 23:35 UTC (permalink / raw)
  To: Ajay Patel; +Cc: linux-kernel, penberg, brouer, rientjes, iamjoonsoo.kim

On Tue, 29 Mar 2016, Ajay Patel wrote:

> We have custom board with Marvell Armada dual core ARMV7.
> The driver uses buffers from kmalloc-8192 slab heavily.
> When slub_debug is enabled, the kmalloc-8192 active slabs are
> increasing. The slub stats shows  cmpxchg_double_fail and objects_partial
> are increasing too. Eventually system panics on oom.

Hmmm... I thought we fall back to pass through to the page allocator for
order 1 requests? Why is it going through the regular allocator paths?

> Following patch fixes the issue.

Wonder how that could be? Does the __cmpxchg_double work correctly on ARM?

> Has anybody encountered this issue?
> Is this right fix?

Looks like something is screwing around with the page flags because an
order 1 page is a compound page? Can you ensure that order 1 allocs are
using page allocator fallback. See kmalloc_large().

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: 3.14.65: Memory leak when slub_debug is enabled
       [not found] <CAA4-JFLOmeYrWOEO_d2ALPgf0cWhC_fv1Gisz5fyH3uY1ogV1g@mail.gmail.com>
@ 2016-03-30  8:58   ` Jesper Dangaard Brouer
  2016-03-30  8:58   ` Jesper Dangaard Brouer
  1 sibling, 0 replies; 4+ messages in thread
From: Jesper Dangaard Brouer @ 2016-03-30  8:58 UTC (permalink / raw)
  To: Ajay Patel
  Cc: linux-kernel, cl, penberg, rientjes, iamjoonsoo.kim, brouer,
	linux-mm


Hi Ajay,

Could you please provide info on kernel .config settings via commands:

 grep HAVE_ALIGNED_STRUCT_PAGE .config
 grep CONFIG_HAVE_CMPXCHG_DOUBLE .config

You can try to further debug your problem by defining SLUB_DEBUG_CMPXCHG
manually in mm/slub.c to get some verbose output on the cmpxchg failures.

Is the "Marvell Armada dual core ARMV7" a 32-bit CPU?

--Jesper

On Tue, 29 Mar 2016 15:32:26 -0700 Ajay Patel <patela@gmail.com> wrote:

> We have custom board with Marvell Armada dual core ARMV7.
> The driver uses buffers from kmalloc-8192 slab heavily.
> When slub_debug is enabled, the kmalloc-8192 active slabs are
> increasing. The slub stats shows  cmpxchg_double_fail and objects_partial
> are increasing too. Eventually system panics on oom.
> 
> Following patch fixes the issue.
> Has anybody encountered this issue?
> Is this right fix?
> 
> I am not in mailing list please cc me.
> 
> Thanks
> Ajay
> 
> 
> --- slub.c.orig Tue Mar 29 11:54:42 2016
> +++ slub.c      Tue Mar 29 15:08:30 2016
> @@ -1562,9 +1562,12 @@
>         void *freelist;
>         unsigned long counters;
>         struct page new;
> +       int retry_count = 0;
> +#define RETRY_COUNT 10
> 
>         lockdep_assert_held(&n->list_lock);
> 
> +again:
>         /*
>          * Zap the freelist and set the frozen bit.
>          * The old freelist is the list of objects for the
> @@ -1587,8 +1590,13 @@
>         if (!__cmpxchg_double_slab(s, page,
>                         freelist, counters,
>                         new.freelist, new.counters,
> -                       "acquire_slab"))
> +                       "acquire_slab")) {
> +               if (retry_count++ < RETRY_COUNT) {
> +                       new.frozen = 0;
> +                       goto again;
> +               }
>                 return NULL;
> +       }
> 
>         remove_partial(n, page);
>         WARN_ON(!freelist);



-- 
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Principal Kernel Engineer at Red Hat
  Author of http://www.iptv-analyzer.org
  LinkedIn: http://www.linkedin.com/in/brouer

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: 3.14.65: Memory leak when slub_debug is enabled
@ 2016-03-30  8:58   ` Jesper Dangaard Brouer
  0 siblings, 0 replies; 4+ messages in thread
From: Jesper Dangaard Brouer @ 2016-03-30  8:58 UTC (permalink / raw)
  To: Ajay Patel
  Cc: linux-kernel, cl, penberg, rientjes, iamjoonsoo.kim, brouer,
	linux-mm


Hi Ajay,

Could you please provide info on kernel .config settings via commands:

 grep HAVE_ALIGNED_STRUCT_PAGE .config
 grep CONFIG_HAVE_CMPXCHG_DOUBLE .config

You can try to further debug your problem by defining SLUB_DEBUG_CMPXCHG
manually in mm/slub.c to get some verbose output on the cmpxchg failures.

Is the "Marvell Armada dual core ARMV7" a 32-bit CPU?

--Jesper

On Tue, 29 Mar 2016 15:32:26 -0700 Ajay Patel <patela@gmail.com> wrote:

> We have custom board with Marvell Armada dual core ARMV7.
> The driver uses buffers from kmalloc-8192 slab heavily.
> When slub_debug is enabled, the kmalloc-8192 active slabs are
> increasing. The slub stats shows  cmpxchg_double_fail and objects_partial
> are increasing too. Eventually system panics on oom.
> 
> Following patch fixes the issue.
> Has anybody encountered this issue?
> Is this right fix?
> 
> I am not in mailing list please cc me.
> 
> Thanks
> Ajay
> 
> 
> --- slub.c.orig Tue Mar 29 11:54:42 2016
> +++ slub.c      Tue Mar 29 15:08:30 2016
> @@ -1562,9 +1562,12 @@
>         void *freelist;
>         unsigned long counters;
>         struct page new;
> +       int retry_count = 0;
> +#define RETRY_COUNT 10
> 
>         lockdep_assert_held(&n->list_lock);
> 
> +again:
>         /*
>          * Zap the freelist and set the frozen bit.
>          * The old freelist is the list of objects for the
> @@ -1587,8 +1590,13 @@
>         if (!__cmpxchg_double_slab(s, page,
>                         freelist, counters,
>                         new.freelist, new.counters,
> -                       "acquire_slab"))
> +                       "acquire_slab")) {
> +               if (retry_count++ < RETRY_COUNT) {
> +                       new.frozen = 0;
> +                       goto again;
> +               }
>                 return NULL;
> +       }
> 
>         remove_partial(n, page);
>         WARN_ON(!freelist);



-- 
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Principal Kernel Engineer at Red Hat
  Author of http://www.iptv-analyzer.org
  LinkedIn: http://www.linkedin.com/in/brouer

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: 3.14.65: Memory leak when slub_debug is enabled
  2016-03-30  8:58   ` Jesper Dangaard Brouer
  (?)
@ 2016-03-30 20:50   ` Ajay Patel
  -1 siblings, 0 replies; 4+ messages in thread
From: Ajay Patel @ 2016-03-30 20:50 UTC (permalink / raw)
  To: Jesper Dangaard Brouer
  Cc: linux-kernel, cl, penberg, rientjes, iamjoonsoo.kim, linux-mm

[-- Attachment #1: Type: text/plain, Size: 18753 bytes --]

Jesper/Christop

Thanks for getting back.

This is Marvel Armada dual core ARMV7 and it is 32bit CPU.

The problem is NOT seen if maxcpus=1 used in command line.

The HAVE_ALIGNED_STRUCT_PAGE and CONFIG_HAVE_CMPXCHG_DOUBLE is NOT defined
for the board where problem is seen.

The problem is profoundly seen on kmalloc-8192 slab.
The slab size and object size is displayed below.
Also the object partial is growing and active slabs are growing.

It seems one core is trying to allocate the buffers while other core is
freeing the buffer and causing this.
I have to add some debug to confirm the theory.

I also turned the SLUB_DEBUG_CMPXCHG and it is flooding the console.
Some of those messages pasted below.

Let me know if you need more info.

Thanks
Ajay


================== CPU info ============================
:/# cat /proc/cpuinfo
processor       : 0
model name      : ARMv7 Processor rev 1 (v7l)
Features        : swp half thumb fastmult vfp edsp thumbee neon vfpv3 tls
vfpd32
CPU implementer : 0x41
CPU architecture: 7
CPU variant     : 0x4
CPU part        : 0xc09
CPU revision    : 1

processor       : 1
model name      : ARMv7 Processor rev 1 (v7l)
Features        : swp half thumb fastmult vfp edsp thumbee neon vfpv3 tls
vfpd32
CPU implementer : 0x41
CPU architecture: 7
CPU variant     : 0x4
CPU part        : 0xc09
CPU revision    : 1

Hardware        : Marvell Armada 380/385/390/398

:/#



#:/sys/kernel/slab/kmalloc-8192# cat slab_size
8384
#:/sys/kernel/slab/kmalloc-8192# cat object_size
8192
=====================Initial state of counters  =====================
:/sys/kernel/slab/kmalloc-8192# cat objects_partial
2
:/sys/kernel/slab/kmalloc-8192# cat /proc/slabinfo | grep 8192
kmalloc-8192       32786  32790   8384    3    8 : tunables    0 0    0 :
slabdata  10930  10930      0

===================== counters after some time ============
#:/sys/kernel/slab/kmalloc-8192# cat /proc/slabinfo | grep 8192
kmalloc-8192       32789  44712   8384    3    8 : tunables    0 0    0 :
slabdata  14904  14904      0

#:/sys/kernel/slab/kmalloc-8192# cat objects_partial
15006



======================= Debug messages ======================

[03/30/2016 13:19:21.9400] acquire_slab kmalloc-8192: cmpxchg double redo
__slab_free kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:22.0700] __slab_free skbuff_head_cache: cmpxchg double
redo acquire_slab names_cache: cmpxchg double redo
[03/30/2016 13:19:22.2100] __slab_free names_cache: cmpxchg double redo
unfreezing slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:22.3400] acquire_slab skbuff_head_cache: cmpxchg double
redo acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:22.5000] __slab_free filp: cmpxchg double redo unfreezing
slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:22.6300] acquire_slab skbuff_head_cache: cmpxchg double
redo acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:22.7900] acquire_slab skbuff_head_cache: cmpxchg double
redo __slab_free kmalloc-2048: cmpxchg double redo
[03/30/2016 13:19:22.9500] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:23.0900] acquire_slab skbuff_head_cache: cmpxchg double
redo __slab_free kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:23.2300] __slab_free kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:23.3600] acquire_slab kmalloc-8192: cmpxchg double redo
__slab_free kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:23.4900] acquire_slab skbuff_head_cache: cmpxchg double
redo __slab_free skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:23.6500] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:23.7900] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:23.9200] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:24.0500] __slab_free skbuff_head_cache: cmpxchg double
redo acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:24.2300] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:24.3600] acquire_slab kmalloc-8192: cmpxchg double redo
unfreezing slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:24.5000] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:24.6300] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:24.7600] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:24.9000] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:25.0300] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:25.1700] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:25.3000] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:25.4300] acquire_slab kmalloc-8192: cmpxchg double redo
unfreezing slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:25.5600] unfreezing slab kmalloc-8192: cmpxchg double
redo unfreezing slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:25.6900] acquire_slab skbuff_head_cache: cmpxchg double
redo unfreezing slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:25.8300] unfreezing slab kmalloc-8192: cmpxchg double
redo unfreezing slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:25.9600] __slab_free kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:26.0900] unfreezing slab kmalloc-8192: cmpxchg double
redo unfreezing slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:26.2300] unfreezing slab kmalloc-8192: cmpxchg double
redo unfreezing slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:26.3600] __slab_free kmalloc-8192: cmpxchg double redo
__slab_free kmalloc-256: cmpxchg double redo
[03/30/2016 13:19:26.5000] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:26.6900] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:26.8200] acquire_slab kmalloc-8192: cmpxchg double redo
__slab_free kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:26.9400] acquire_slab kmalloc-8192: cmpxchg double redo
__slab_free kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:27.0700] unfreezing slab skbuff_head_cache: cmpxchg
double redo acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:27.2100] __slab_free kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:27.3400] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:27.4800] acquire_slab skbuff_head_cache: cmpxchg double
redo __slab_free kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:27.6100] acquire_slab kmalloc-8192: cmpxchg double redo
__slab_free kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:27.7400] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:27.8700] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:28.0000] __slab_free kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:28.1300] __slab_free kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:28.2600] acquire_slab kmalloc-4096: cmpxchg double redo
__slab_free kmalloc-4096: cmpxchg double redo
[03/30/2016 13:19:28.4300] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:28.5700] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:28.7000] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:28.8400] acquire_slab kmalloc-8192: cmpxchg double redo
__slab_free kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:28.9700] __slab_free kmalloc-8192: cmpxchg double redo
__slab_free kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:29.0900] __slab_free kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:29.2300] acquire_slab skbuff_head_cache: cmpxchg double
redo unfreezing slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:29.3600] unfreezing slab kmalloc-8192: cmpxchg double
redo acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:29.5000] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:29.6300] acquire_slab skbuff_head_cache: cmpxchg double
redo acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:29.7700] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:29.9000] __slab_free kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:30.0300] __slab_free kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:30.1700] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:30.3000] acquire_slab skbuff_head_cache: cmpxchg double
redo acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:30.4400] __slab_free kmalloc-8192: cmpxchg double redo
__slab_free kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:30.5700] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:30.7000] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:30.8300] __slab_free skbuff_head_cache: cmpxchg double
redo acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:30.9800] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:31.1100] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:31.2400] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:31.3700] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:31.5000] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:31.6300] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:31.7700] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:31.9000] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:32.0300] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:32.1600] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:32.2900] acquire_slab kmalloc-8192: cmpxchg double redo
__slab_free kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:32.4300] __slab_free kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:32.5600] acquire_slab skbuff_head_cache: cmpxchg double
redo acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:32.7000] __slab_free kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:32.8300] acquire_slab skbuff_head_cache: cmpxchg double
redo acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:32.9700] acquire_slab skbuff_head_cache: cmpxchg double
redo acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:33.1200] __slab_free skbuff_head_cache: cmpxchg double
redo acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:33.2600] acquire_slab skbuff_head_cache: cmpxchg double
redo __slab_free kmalloc-64: cmpxchg double redo
[03/30/2016 13:19:33.5900] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:33.7300] acquire_slab kmalloc-8192: cmpxchg double redo
unfreezing slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:33.8700] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:34.0000] acquire_slab kmalloc-8192: cmpxchg double redo
__slab_free skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:34.2300] acquire_slab kmalloc-8192: cmpxchg double redo
unfreezing slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:34.3700] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:34.5000] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:34.6300] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:34.7600] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:34.9000] acquire_slab kmalloc-8192: cmpxchg double redo
__slab_free kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:35.0300] acquire_slab skbuff_head_cache: cmpxchg double
redo acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:35.1700] acquire_slab skbuff_head_cache: cmpxchg double
redo unfreezing slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:35.3000] __slab_free kmalloc-8192: cmpxchg double redo
unfreezing slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:35.4400] unfreezing slab kmalloc-8192: cmpxchg double
redo acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:35.5700] __slab_free kmalloc-8192: cmpxchg double redo
unfreezing slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:35.7000] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:35.8400] __slab_free kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:35.9600] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:36.0900] unfreezing slab kmalloc-8192: cmpxchg double
redo acquire_slab kmalloc-4096: cmpxchg double redo
[03/30/2016 13:19:36.2300] __slab_free kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:36.3500] __slab_free kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:36.4900] acquire_slab skbuff_head_cache: cmpxchg double
redo acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:36.6200] __slab_free kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:36.7500] acquire_slab kmalloc-8192: cmpxchg double redo
__slab_free kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:36.8800] __slab_free skbuff_head_cache: cmpxchg double
redo acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:37.0200] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:37.1500] acquire_slab kmalloc-8192: cmpxchg double redo
unfreezing slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:37.2900] acquire_slab kmalloc-8192: cmpxchg double redo
acquire_slab skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:37.4200] acquire_slab skbuff_head_cache: cmpxchg double
redo acquire_slab kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:37.5500] acquire_slab kmalloc-8192: cmpxchg double redo
__slab_free skbuff_head_cache: cmpxchg double redo
[03/30/2016 13:19:37.7500] acquire_slab kmalloc-8192: cmpxchg double redo
__slab_free kmalloc-8192: cmpxchg double redo
[03/30/2016 13:19:37.8900] acquire_slab kmalloc-8192: cmpxchg double redo ac





On Wed, Mar 30, 2016 at 1:58 AM, Jesper Dangaard Brouer <brouer@redhat.com>
wrote:

>
> Hi Ajay,
>
> Could you please provide info on kernel .config settings via commands:
>
>  grep HAVE_ALIGNED_STRUCT_PAGE .config
>  grep CONFIG_HAVE_CMPXCHG_DOUBLE .config
>
> You can try to further debug your problem by defining SLUB_DEBUG_CMPXCHG
> manually in mm/slub.c to get some verbose output on the cmpxchg failures.
>
> Is the "Marvell Armada dual core ARMV7" a 32-bit CPU?
>
> --Jesper
>
> On Tue, 29 Mar 2016 15:32:26 -0700 Ajay Patel <patela@gmail.com> wrote:
>
> > We have custom board with Marvell Armada dual core ARMV7.
> > The driver uses buffers from kmalloc-8192 slab heavily.
> > When slub_debug is enabled, the kmalloc-8192 active slabs are
> > increasing. The slub stats shows  cmpxchg_double_fail and objects_partial
> > are increasing too. Eventually system panics on oom.
> >
> > Following patch fixes the issue.
> > Has anybody encountered this issue?
> > Is this right fix?
> >
> > I am not in mailing list please cc me.
> >
> > Thanks
> > Ajay
> >
> >
> > --- slub.c.orig Tue Mar 29 11:54:42 2016
> > +++ slub.c      Tue Mar 29 15:08:30 2016
> > @@ -1562,9 +1562,12 @@
> >         void *freelist;
> >         unsigned long counters;
> >         struct page new;
> > +       int retry_count = 0;
> > +#define RETRY_COUNT 10
> >
> >         lockdep_assert_held(&n->list_lock);
> >
> > +again:
> >         /*
> >          * Zap the freelist and set the frozen bit.
> >          * The old freelist is the list of objects for the
> > @@ -1587,8 +1590,13 @@
> >         if (!__cmpxchg_double_slab(s, page,
> >                         freelist, counters,
> >                         new.freelist, new.counters,
> > -                       "acquire_slab"))
> > +                       "acquire_slab")) {
> > +               if (retry_count++ < RETRY_COUNT) {
> > +                       new.frozen = 0;
> > +                       goto again;
> > +               }
> >                 return NULL;
> > +       }
> >
> >         remove_partial(n, page);
> >         WARN_ON(!freelist);
>
>
>
> --
> Best regards,
>   Jesper Dangaard Brouer
>   MSc.CS, Principal Kernel Engineer at Red Hat
>   Author of http://www.iptv-analyzer.org
>   LinkedIn: http://www.linkedin.com/in/brouer
>

[-- Attachment #2: Type: text/html, Size: 21198 bytes --]

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2016-03-30 20:50 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
     [not found] <CAA4-JFLOmeYrWOEO_d2ALPgf0cWhC_fv1Gisz5fyH3uY1ogV1g@mail.gmail.com>
2016-03-29 23:35 ` 3.14.65: Memory leak when slub_debug is enabled Christoph Lameter
2016-03-30  8:58 ` Jesper Dangaard Brouer
2016-03-30  8:58   ` Jesper Dangaard Brouer
2016-03-30 20:50   ` Ajay Patel

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.