From: "Holger Hoffstätte" <holger.hoffstaette@googlemail.com>
To: linux-kernel@vger.kernel.org
Subject: Re: Kernel 4.1.6 Panic due to slab corruption
Date: Mon, 7 Sep 2015 10:37:07 +0000 (UTC) [thread overview]
Message-ID: <pan.2015.09.07.10.37.07@googlemail.com> (raw)
In-Reply-To: 55ED4DAD.7080701@kyup.com
On Mon, 07 Sep 2015 11:41:17 +0300, Nikolay Borisov wrote:
> Hello,
>
> On one of our servers I've observed the a kernel pannic
> happening with the following backtrace:
>
> [654405.527070] BUG: unable to handle kernel paging request at 0000000000028001
> [654405.527076] IP: [<ffffffff81182a59>] kmem_cache_alloc_node+0x99/0x1e0
> [654405.527085] PGD 14bef58067 PUD 2ab358067 PMD 0
Interesting! I can't offer much help but had a similar panic just the other day
for no apparent reason while running a bunch of compiles. First time I've seen
this with 4.1.6:
Sep 5 20:42:02 ragnarok kernel: BUG: unable to handle kernel paging request at ffff8800e789b740
Sep 5 20:42:02 ragnarok kernel: IP: [<ffffffff8115bd4d>] kmem_cache_alloc+0x6d/0x160
Sep 5 20:42:02 ragnarok kernel: PGD 1aa2067 PUD 61f7fd067 PMD 0
Sep 5 20:42:02 ragnarok kernel: Oops: 0000 [#1] SMP
Sep 5 20:42:02 ragnarok kernel: Modules linked in: auth_rpcgss oid_registry nfsv4 nfs lockd grace fscache sunrpc autofs4 sch_fq_codel snd_hda_codec_realtek x86_pkg_temp_thermal coretemp snd_hda_codec_generic crc32_pclmul crc32c_intel aesni_intel radeon aes_x86_64 glue_helper snd_hda_codec_hdmi lrw gf128mul ablk_helper cryptd i2c_algo_bit snd_usb_audio uvcvideo snd_hda_intel drm_kms_helper snd_hda_controller snd_hwdep videobuf2_vmalloc snd_usbmidi_lib videobuf2_memops snd_hda_codec videobuf2_core snd_rawmidi i2c_i801 ttm snd_hda_core v4l2_common snd_seq_device videodev snd_pcm usbhid drm snd_timer r8169 snd i2c_core mii soundcore parport_pc parport
Sep 5 20:42:02 ragnarok kernel: CPU: 0 PID: 32755 Comm: sh Not tainted 4.1.6 #1
Sep 5 20:42:02 ragnarok kernel: Hardware name: Gigabyte Technology Co., Ltd. P67-DS3-B3/P67-DS3-B3, BIOS F1 05/06/2011
Sep 5 20:42:02 ragnarok kernel: task: ffff880569712e20 ti: ffff8804e4d90000 task.ti: ffff8804e4d90000
Sep 5 20:42:02 ragnarok kernel: RIP: 0010:[<ffffffff8115bd4d>] [<ffffffff8115bd4d>] kmem_cache_alloc+0x6d/0x160
Sep 5 20:42:02 ragnarok kernel: RSP: 0018:ffff8804e4d93d88 EFLAGS: 00010282
Sep 5 20:42:02 ragnarok kernel: RAX: 0000000000000000 RBX: ffff8805e7eacce0 RCX: 000000000001f7e8
Sep 5 20:42:02 ragnarok kernel: RDX: 000000000001f7e7 RSI: 00000000000000d0 RDI: 0000000000018c70
Sep 5 20:42:02 ragnarok kernel: RBP: ffff8804e4d93dc8 R08: ffff88061f418c70 R09: 0000000000000000
Sep 5 20:42:02 ragnarok kernel: R10: ffffffff81748318 R11: ffffea00139bb500 R12: 00000000000000d0
Sep 5 20:42:02 ragnarok kernel: R13: ffff880606890600 R14: ffffffff8100d039 R15: ffff8800e789b740
Sep 5 20:42:02 ragnarok kernel: FS: 00007f9c1d2f2700(0000) GS:ffff88061f400000(0000) knlGS:0000000000000000
Sep 5 20:42:02 ragnarok kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Sep 5 20:42:02 ragnarok kernel: CR2: ffff8800e789b740 CR3: 00000005f68ce000 CR4: 00000000000406f0
Sep 5 20:42:02 ragnarok kernel: Stack:
Sep 5 20:42:02 ragnarok kernel: 0000000000000000 ffff88061f7e6c00 0000000000000002 ffff8805e7eacce0
Sep 5 20:42:02 ragnarok kernel: ffff880569712e20 0000000001200011 ffff8805e7eacce0 ffff880569712e20
Sep 5 20:42:02 ragnarok kernel: ffff8804e4d93de8 ffffffff8100d039 0000000000000000 00007f9c1d2f29d0
Sep 5 20:42:02 ragnarok kernel: Call Trace:
Sep 5 20:42:02 ragnarok kernel: [<ffffffff8100d039>] arch_dup_task_struct+0x69/0x170
Sep 5 20:42:02 ragnarok kernel: [<ffffffff8104de8f>] copy_process.part.8+0x14f/0x1760
Sep 5 20:42:02 ragnarok kernel: [<ffffffff8113909f>] ? handle_mm_fault+0xd0f/0x13a0
Sep 5 20:42:02 ragnarok kernel: [<ffffffff81171c14>] ? get_empty_filp+0xd4/0x1c0
Sep 5 20:42:02 ragnarok kernel: [<ffffffff8105b63f>] ? recalc_sigpending+0x1f/0x60
Sep 5 20:42:02 ragnarok kernel: [<ffffffff8104f657>] do_fork+0xd7/0x370
Sep 5 20:42:02 ragnarok kernel: [<ffffffff8105ed07>] ? sigprocmask+0x57/0x90
Sep 5 20:42:02 ragnarok kernel: [<ffffffff8104f976>] SyS_clone+0x16/0x20
Sep 5 20:42:02 ragnarok kernel: [<ffffffff81571d17>] system_call_fastpath+0x12/0x6a
Sep 5 20:42:02 ragnarok kernel: Code: 65 4c 03 05 ee e3 ea 7e 49 83 78 10 00 4d 8b 38 0f 84 b0 00 00 00 4d 85 ff 0f 84 a7 00 00 00 49 63 45 20 48 8d 4a 01 49 8b 7d 00 <49> 8b 1c 07 4c 89 f8 65 48 0f c7 0f 0f 94 c0 84 c0 74 b9 49 63
Sep 5 20:42:02 ragnarok kernel: RIP [<ffffffff8115bd4d>] kmem_cache_alloc+0x6d/0x160
Sep 5 20:42:02 ragnarok kernel: RSP <ffff8804e4d93d88>
Sep 5 20:42:02 ragnarok kernel: CR2: ffff8800e789b740
Sep 5 20:42:02 ragnarok kernel: ---[ end trace e4478715791f5752 ]---
Sep 5 20:42:02 ragnarok kernel: BUG: unable to handle kernel paging request at ffff8800e789b740
Sep 5 20:42:02 ragnarok kernel: IP: [<ffffffff8115bd4d>] kmem_cache_alloc+0x6d/0x160
Sep 5 20:42:02 ragnarok kernel: PGD 1aa2067 PUD 61f7fd067 PMD 0
Sep 5 20:42:02 ragnarok kernel: Oops: 0000 [#2] SMP
Sep 5 20:42:02 ragnarok kernel: Modules linked in: auth_rpcgss oid_registry nfsv4 nfs lockd grace fscache sunrpc autofs4 sch_fq_codel snd_hda_codec_realtek x86_pkg_temp_thermal coretemp snd_hda_codec_generic crc32_pclmul crc32c_intel aesni_intel radeon aes_x86_64 glue_helper snd_hda_codec_hdmi lrw gf128mul ablk_helper cryptd i2c_algo_bit snd_usb_audio uvcvideo snd_hda_intel drm_kms_helper snd_hda_controller snd_hwdep videobuf2_vmalloc snd_usbmidi_lib videobuf2_memops snd_hda_codec videobuf2_core snd_rawmidi i2c_i801 ttm snd_hda_core v4l2_common snd_seq_device videodev snd_pcm usbhid drm snd_timer r8169 snd i2c_core mii soundcore parport_pc parport
Sep 5 20:42:02 ragnarok kernel: CPU: 0 PID: 32550 Comm: sh Tainted: G D 4.1.6 #1
Sep 5 20:42:02 ragnarok kernel: Hardware name: Gigabyte Technology Co., Ltd. P67-DS3-B3/P67-DS3-B3, BIOS F1 05/06/2011
Sep 5 20:42:02 ragnarok kernel: task: ffff880602cd1ec0 ti: ffff8805b26ac000 task.ti: ffff8805b26ac000
Sep 5 20:42:02 ragnarok kernel: RIP: 0010:[<ffffffff8115bd4d>] [<ffffffff8115bd4d>] kmem_cache_alloc+0x6d/0x160
Sep 5 20:42:02 ragnarok kernel: RSP: 0018:ffff8805b26afd88 EFLAGS: 00010282
Sep 5 20:42:02 ragnarok kernel: RAX: 0000000000000000 RBX: ffff8805e7ea8f60 RCX: 000000000001f7e8
Sep 5 20:42:02 ragnarok kernel: RDX: 000000000001f7e7 RSI: 00000000000000d0 RDI: 0000000000018c70
Sep 5 20:42:02 ragnarok kernel: RBP: ffff8805b26afdc8 R08: ffff88061f418c70 R09: 0000000000000000
Sep 5 20:42:02 ragnarok kernel: R10: ffffffff81748318 R11: ffffea0015a2ec00 R12: 00000000000000d0
Sep 5 20:42:02 ragnarok kernel: R13: ffff880606890600 R14: ffffffff8100d039 R15: ffff8800e789b740
Sep 5 20:42:02 ragnarok kernel: FS: 00007f9c1d2f2700(0000) GS:ffff88061f400000(0000) knlGS:0000000000000000
Sep 5 20:42:02 ragnarok kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Sep 5 20:42:02 ragnarok kernel: CR2: ffff8800e789b740 CR3: 00000005e3b76000 CR4: 00000000000406f0
Sep 5 20:42:02 ragnarok kernel: Stack:
Sep 5 20:42:02 ragnarok kernel: 0000000000000000 ffff88061f7e6c00 0000000000000002 ffff8805e7ea8f60
Sep 5 20:42:02 ragnarok kernel: ffff880602cd1ec0 0000000001200011 ffff8805e7ea8f60 ffff880602cd1ec0
Sep 5 20:42:02 ragnarok kernel: ffff8805b26afde8 ffffffff8100d039 0000000000000000 00007f9c1d2f29d0
Sep 5 20:42:02 ragnarok kernel: Call Trace:
Sep 5 20:42:02 ragnarok kernel: [<ffffffff8100d039>] arch_dup_task_struct+0x69/0x170
Sep 5 20:42:02 ragnarok kernel: [<ffffffff8104de8f>] copy_process.part.8+0x14f/0x1760
Sep 5 20:42:02 ragnarok kernel: [<ffffffff8126a936>] ? security_file_alloc+0x16/0x20
Sep 5 20:42:02 ragnarok kernel: [<ffffffff81171c14>] ? get_empty_filp+0xd4/0x1c0
Sep 5 20:42:02 ragnarok kernel: [<ffffffff81185966>] ? __d_instantiate+0x96/0xf0
Sep 5 20:42:02 ragnarok kernel: [<ffffffff812c6b1a>] ? find_next_zero_bit+0x1a/0x30
Sep 5 20:42:02 ragnarok kernel: [<ffffffff8105b63f>] ? recalc_sigpending+0x1f/0x60
Sep 5 20:42:02 ragnarok kernel: [<ffffffff8104f657>] do_fork+0xd7/0x370
Sep 5 20:42:02 ragnarok kernel: [<ffffffff8105ed07>] ? sigprocmask+0x57/0x90
Sep 5 20:42:02 ragnarok kernel: [<ffffffff8104f976>] SyS_clone+0x16/0x20
Sep 5 20:42:02 ragnarok kernel: [<ffffffff81571d17>] system_call_fastpath+0x12/0x6a
Sep 5 20:42:02 ragnarok kernel: Code: 65 4c 03 05 ee e3 ea 7e 49 83 78 10 00 4d 8b 38 0f 84 b0 00 00 00 4d 85 ff 0f 84 a7 00 00 00 49 63 45 20 48 8d 4a 01 49 8b 7d 00 <49> 8b 1c 07 4c 89 f8 65 48 0f c7 0f 0f 94 c0 84 c0 74 b9 49 63
Sep 5 20:42:02 ragnarok kernel: RIP [<ffffffff8115bd4d>] kmem_cache_alloc+0x6d/0x160
Sep 5 20:42:02 ragnarok kernel: RSP <ffff8805b26afd88>
Sep 5 20:42:02 ragnarok kernel: CR2: ffff8800e789b740
Sep 5 20:42:02 ragnarok kernel: ---[ end trace e4478715791f5753 ]---
..etc.
I also have all of
CONFIG_SLUB_DEBUG=y
CONFIG_SLUB=y
CONFIG_SLUB_CPU_PARTIAL=y
set.
Hope this helps somewhat.
Holger
next prev parent reply other threads:[~2015-09-07 10:37 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-09-07 8:41 Kernel 4.1.6 Panic due to slab corruption Nikolay Borisov
2015-09-07 10:37 ` Holger Hoffstätte [this message]
2015-09-07 11:30 ` Nikolay Borisov
2015-09-07 11:49 ` Holger Hoffstätte
2015-09-07 11:58 ` Holger Hoffstätte
2015-09-07 11:14 ` Nikolay Borisov
2015-09-07 11:40 ` Fwd: " Nikolay Borisov
2015-09-08 13:58 ` Christoph Lameter
2015-09-08 14:06 ` Nikolay Borisov
2015-09-08 14:27 ` Christoph Lameter
2015-09-08 14:41 ` Nikolay Borisov
2015-09-08 15:15 ` Christoph Lameter
2015-09-09 11:40 ` Nikolay Borisov
2015-09-09 14:01 ` Christoph Lameter
2015-09-09 16:26 ` Vlastimil Babka
2015-09-09 17:58 ` Christoph Lameter
2015-09-10 6:12 ` Nikolay Borisov
2015-09-10 16:30 ` Christoph Lameter
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=pan.2015.09.07.10.37.07@googlemail.com \
--to=holger.hoffstaette@googlemail.com \
--cc=linux-kernel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.