public inbox for linux-kernel@vger.kernel.org
 help / color / mirror / Atom feed
From: Ingo Molnar <mingo@elte.hu>
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Pekka Enberg <penberg@cs.helsinki.fi>,
	Christoph Lameter <clameter@sgi.com>,
	linux-kernel@vger.kernel.org, Mel Gorman <mel@csn.ul.ie>,
	Nick Piggin <npiggin@suse.de>,
	Andrew Morton <akpm@linux-foundation.org>,
	"Rafael J. Wysocki" <rjw@sisk.pl>,
	Yinghai.Lu@sun.com, apw@shadowen.org,
	KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>,
	Arjan van de Ven <arjan@infradead.org>
Subject: Re: [patch] mm: sparsemem memory_present() memory corruption fix
Date: Wed, 16 Apr 2008 17:03:53 +0200	[thread overview]
Message-ID: <20080416150353.GA26740@elte.hu> (raw)
In-Reply-To: <20080416000356.GA24737@elte.hu>


* Ingo Molnar <mingo@elte.hu> wrote:

> ps. anyone who can correctly guess the method with which i found the
>     exact place that corrupted memory will get a free beer next time 
>     we meet :-)

the method was to notice that the slub_debug_slabs SLUB variable got 
corrupted from an expected value of 0 to a value of 0x1.

Then i added a simple brute-force function-tracer hook (in sched-devel) 
that checked when slub_debug_slabs went from 0 to 1, and which then 
printed a backtrace.

Since under CONFIG_FTRACE=y every kernel function calls this callback, 
it triggered immediately after the value got corrupted:

[    0.000000] console [earlyser0] enabled
[    0.000000] BUG: slub_debug_slabs: 00000001
[    0.000000] Pid: 0, comm: swapper Not tainted 2.6.25-rc9-sched-devel.git-x86-latest.git #982
[    0.000000]  [<c0177fba>] print_slub_debug_slabs+0x3a/0x40
[    0.000000]  [<c01050f7>] trace+0x8/0x11
[    0.000000]  [<c0cc929e>] ? mtrr_bp_init+0xe/0x320
[    0.000000]  [<c01050f7>] ? trace+0x8/0x11
[    0.000000]  [<c0cd7369>] ? memory_present+0x9/0x50
[    0.000000]  [<c0cc7a09>] ? find_max_pfn+0x99/0xb0
[    0.000000]  [<c0cc6af7>] setup_arch+0x217/0x470
[    0.000000]  [<c012c59b>] ? printk+0x1b/0x20
[    0.000000]  [<c0cc2b46>] start_kernel+0x96/0x3f0
[    0.000000]  [<c0cc22fd>] i386_start_kernel+0xd/0x10
[    0.000000]  =======================
[    0.000000] x86: PAT support disabled.

and the backtrace had all the guilty parties on stack - memory_present() 
[which was just called] and find_max_pfn()/setup_arch() - thanks to the 
new fuzzy "?" backtrace entries we print out in v2.6.25.

(i could also have printed out the current ftrace buffer as well, 
showing the history of all recent function calls that the kernel 
executed.)

	Ingo

  parent reply	other threads:[~2008-04-16 15:04 UTC|newest]

Thread overview: 95+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2008-04-11  7:41 [bug] mm/slab.c boot crash in -git, "kernel BUG at mm/slab.c:2103!" Ingo Molnar
2008-04-11  8:21 ` Pekka Enberg
2008-04-11  8:50   ` Pekka Enberg
2008-04-11  8:54     ` Ingo Molnar
2008-04-11  9:05       ` Pekka Enberg
2008-04-11  9:08         ` Pekka Enberg
2008-04-11  9:11           ` Pekka Enberg
2008-04-11  9:24             ` Ingo Molnar
2008-04-11 10:34               ` Nick Piggin
2008-04-11 19:28               ` Christoph Lameter
2008-04-12 10:38                 ` Christoph Lameter
2008-04-12 17:22                   ` Yinghai Lu
2008-04-15  5:43                 ` Ingo Molnar
2008-04-15  9:36               ` Mel Gorman
2008-04-15 10:03                 ` Ingo Molnar
2008-04-15  6:25             ` [bug] SLUB + mm/slab.c boot crash in -rc9 Ingo Molnar
2008-04-15  6:41               ` Pekka Enberg
2008-04-15  7:08                 ` Ingo Molnar
2008-04-15  8:31                   ` Yinghai Lu
2008-04-15  8:46                     ` Ingo Molnar
2008-04-15  9:11                   ` Ingo Molnar
2008-04-15 16:02               ` Linus Torvalds
2008-04-15 16:15                 ` Ingo Molnar
2008-04-15 17:23                   ` Linus Torvalds
2008-04-15 19:35                     ` Ingo Molnar
2008-04-15 19:41                       ` Ingo Molnar
2008-04-15 19:39                     ` Christoph Lameter
2008-04-15 19:54                       ` Ingo Molnar
2008-04-15 20:03                         ` Christoph Lameter
2008-04-15 20:17                           ` Ingo Molnar
2008-04-15 20:28                             ` Ingo Molnar
2008-04-15 20:34                               ` Ingo Molnar
2008-04-15 20:42                                 ` Ingo Molnar
2008-04-15 20:50                                   ` Christoph Lameter
2008-04-15 20:58                                     ` Ingo Molnar
2008-04-15 21:08                                       ` Christoph Lameter
2008-04-15 21:16                                         ` Mike Travis
2008-04-15 21:19                                         ` Ingo Molnar
2008-04-15 21:21                                           ` Christoph Lameter
2008-04-15 21:23                                             ` Ingo Molnar
2008-04-15 21:24                                               ` Christoph Lameter
2008-04-15 21:28                                                 ` Ingo Molnar
2008-04-15 21:33                                                   ` Christoph Lameter
2008-04-15 21:43                                                   ` Mike Travis
2008-04-15 22:07                                                   ` Ingo Molnar
2008-04-15 21:27                                           ` Mike Travis
2008-04-15 20:34                             ` Pekka Enberg
2008-04-15 20:40                               ` Ingo Molnar
2008-04-15 21:06                                 ` Linus Torvalds
2008-04-15 21:13                                   ` Ingo Molnar
2008-04-15 21:24                                     ` Ingo Molnar
2008-04-15 21:42                                       ` Christoph Lameter
2008-04-15 21:55                                         ` Ingo Molnar
2008-04-15 22:06                                           ` Christoph Lameter
2008-04-15 22:13                                             ` Ingo Molnar
2008-04-15 22:27                                               ` Christoph Lameter
2008-04-15 22:32                                                 ` Ingo Molnar
2008-04-15 23:22                                                 ` Christoph Lameter
2008-04-15 23:27                                                   ` Ingo Molnar
2008-04-15 23:32                                                     ` Christoph Lameter
2008-04-16  0:04                                                     ` Christoph Lameter
2008-04-15 23:18                                             ` Yinghai Lu
2008-04-16  0:03                                   ` [patch] mm: sparsemem memory_present() memory corruption fix Ingo Molnar
2008-04-16  0:10                                     ` Christoph Lameter
2008-04-16  0:18                                     ` Ingo Molnar
2008-04-16  0:32                                       ` Yinghai Lu
2008-04-16  0:44                                         ` Ingo Molnar
2008-04-16  0:46                                           ` Christoph Lameter
2008-04-16  0:52                                             ` Ingo Molnar
2008-04-16  1:17                                               ` Ingo Molnar
2008-04-16  1:30                                                 ` Yinghai Lu
2008-04-16  2:00                                                   ` Yinghai Lu
2008-04-16  2:20                                                     ` KAMEZAWA Hiroyuki
2008-04-16  0:56                                           ` Yinghai Lu
2008-04-16  1:02                                             ` Ingo Molnar
2008-04-16  1:17                                               ` Yinghai Lu
2008-04-16  0:19                                     ` Christoph Lameter
2008-04-16  0:33                                       ` Yinghai Lu
2008-04-16  0:36                                       ` Ingo Molnar
2008-04-16  0:34                                     ` Ingo Molnar
2008-04-16  0:40                                       ` Ingo Molnar
2008-04-16  0:45                                         ` Christoph Lameter
2008-04-16  0:52                                           ` Ingo Molnar
2008-04-16  1:14                                         ` Ingo Molnar
2008-04-16  2:45                                       ` Linus Torvalds
2008-04-16  1:48                                     ` KAMEZAWA Hiroyuki
2008-04-16 14:05                                     ` Mel Gorman
2008-04-16 15:03                                     ` Ingo Molnar [this message]
2008-04-15 20:54                             ` [bug] SLUB + mm/slab.c boot crash in -rc9 Christoph Lameter
2008-04-15 20:58                               ` Ingo Molnar
2008-04-15 21:08                                 ` Ingo Molnar
2008-04-15 20:23                   ` Ingo Molnar
2008-04-11 19:26           ` [bug] mm/slab.c boot crash in -git, "kernel BUG at mm/slab.c:2103!" Christoph Lameter
2008-04-11 19:25         ` Christoph Lameter
2008-04-15  5:49           ` Ingo Molnar

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20080416150353.GA26740@elte.hu \
    --to=mingo@elte.hu \
    --cc=Yinghai.Lu@sun.com \
    --cc=akpm@linux-foundation.org \
    --cc=apw@shadowen.org \
    --cc=arjan@infradead.org \
    --cc=clameter@sgi.com \
    --cc=kamezawa.hiroyu@jp.fujitsu.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mel@csn.ul.ie \
    --cc=npiggin@suse.de \
    --cc=penberg@cs.helsinki.fi \
    --cc=rjw@sisk.pl \
    --cc=torvalds@linux-foundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox