All of lore.kernel.org
 help / color / mirror / Atom feed
From: Josh Poimboeuf <jpoimboe@redhat.com>
To: lkp@lists.01.org
Subject: Re: [lockdep] b09be676e0 BUG: unable to handle kernel NULL pointer dereference at 000001f2
Date: Wed, 04 Oct 2017 16:06:24 -0500	[thread overview]
Message-ID: <20171004210624.yyi64y2o5cvbr3d3@treble> (raw)
In-Reply-To: <201710040644.GJE89219.FOHMFSFOVtOQLJ@I-love.SAKURA.ne.jp>

[-- Attachment #1: Type: text/plain, Size: 2976 bytes --]

On Wed, Oct 04, 2017 at 06:44:50AM +0900, Tetsuo Handa wrote:
> Josh Poimboeuf wrote:
> > On Tue, Oct 03, 2017 at 11:28:15AM -0500, Josh Poimboeuf wrote:
> > > There are two bugs:
> > > 
> > > 1) Somebody -- presumably lockdep -- is corrupting the stack.  Need the
> > >    lockdep people to look at that.
> > > 
> > > 2) The 32-bit FP unwinder isn't handling the corrupt stack very well,
> > >    It's blindly dereferencing untrusted data:
> > > 
> > > 	/* Is the next frame pointer an encoded pointer to pt_regs? */
> > > 	regs = decode_frame_pointer(next_bp);
> > > 	if (regs) {
> > > 		frame = (unsigned long *)regs;
> > > 		len = regs_size(regs);
> > > 		state->got_irq = true;
> > > 
> > >   On 32-bit, regs_size() dereferences the regs pointer before we know it
> > >   points to a valid stack.  I'll fix that, along with the other unwinder
> > >   improvements I discussed with Linus.
> > 
> > Tetsuo and/or Fengguang,
> > 
> > Would you mind testing with this patch?  It should at least prevent the
> > unwinder panic and should hopefully print a useful unwinder dump
> > instead.
> > 
> Here are two outputs.

Thanks, both outputs were helpful.

This is another unwinder-related issue, unrelated to lockdep this time.

I compiled the same kernel with a similar version of GCC.  It turns out
that GCC *does* create unaligned stacks with frame pointers enabled:

  c124a388 <acpi_rs_move_data>:
  c124a388:       55                      push   %ebp
  c124a389:       89 e5                   mov    %esp,%ebp
  c124a38b:       57                      push   %edi
  c124a38c:       56                      push   %esi
  c124a38d:       89 d6                   mov    %edx,%esi
  c124a38f:       53                      push   %ebx
  c124a390:       31 db                   xor    %ebx,%ebx
  c124a392:       83 ec 03                sub    $0x3,%esp
  ...
  c124a3e3:       83 c4 03                add    $0x3,%esp
  c124a3e6:       5b                      pop    %ebx
  c124a3e7:       5e                      pop    %esi
  c124a3e8:       5f                      pop    %edi
  c124a3e9:       5d                      pop    %ebp
  c124a3ea:       c3                      ret

This was a leaf function.  For no apparent reason, GCC 4.8 decided to
subtract 3 from the stack pointer in the prologue.

Then in the middle of the function, it got an interrupt.  On 64-bit,
interrupts always align the stack, but on 32-bit they don't.  So the
pt_regs were stored at an unaligned address (0xf60bbb25).

The frame pointer encoding logic assumes an aligned stack pointer, so it
didn't work as expected.

  .macro ENCODE_FRAME_POINTER
  #ifdef CONFIG_FRAME_POINTER
  	mov %esp, %ebp
  	orl $0x1, %ebp
  #endif
  .endm

That effectively cleared the LSB of the encoded pt_regs address,
confusing the unwinder.

So on 32-bit, maybe the encoding should clear the MSB instead of setting
the LSB.

-- 
Josh

WARNING: multiple messages have this Message-ID (diff)
From: Josh Poimboeuf <jpoimboe@redhat.com>
To: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Cc: fengguang.wu@intel.com, byungchul.park@lge.com, mingo@kernel.org,
	peterz@infradead.org, linux-kernel@vger.kernel.org, lkp@01.org,
	torvalds@linux-foundation.org, bp@alien8.de, x86@kernel.org,
	hpa@zytor.com, tglx@linutronix.de
Subject: Re: [lockdep] b09be676e0 BUG: unable to handle kernel NULL pointer dereference at 000001f2
Date: Wed, 4 Oct 2017 16:06:24 -0500	[thread overview]
Message-ID: <20171004210624.yyi64y2o5cvbr3d3@treble> (raw)
In-Reply-To: <201710040644.GJE89219.FOHMFSFOVtOQLJ@I-love.SAKURA.ne.jp>

On Wed, Oct 04, 2017 at 06:44:50AM +0900, Tetsuo Handa wrote:
> Josh Poimboeuf wrote:
> > On Tue, Oct 03, 2017 at 11:28:15AM -0500, Josh Poimboeuf wrote:
> > > There are two bugs:
> > > 
> > > 1) Somebody -- presumably lockdep -- is corrupting the stack.  Need the
> > >    lockdep people to look at that.
> > > 
> > > 2) The 32-bit FP unwinder isn't handling the corrupt stack very well,
> > >    It's blindly dereferencing untrusted data:
> > > 
> > > 	/* Is the next frame pointer an encoded pointer to pt_regs? */
> > > 	regs = decode_frame_pointer(next_bp);
> > > 	if (regs) {
> > > 		frame = (unsigned long *)regs;
> > > 		len = regs_size(regs);
> > > 		state->got_irq = true;
> > > 
> > >   On 32-bit, regs_size() dereferences the regs pointer before we know it
> > >   points to a valid stack.  I'll fix that, along with the other unwinder
> > >   improvements I discussed with Linus.
> > 
> > Tetsuo and/or Fengguang,
> > 
> > Would you mind testing with this patch?  It should at least prevent the
> > unwinder panic and should hopefully print a useful unwinder dump
> > instead.
> > 
> Here are two outputs.

Thanks, both outputs were helpful.

This is another unwinder-related issue, unrelated to lockdep this time.

I compiled the same kernel with a similar version of GCC.  It turns out
that GCC *does* create unaligned stacks with frame pointers enabled:

  c124a388 <acpi_rs_move_data>:
  c124a388:       55                      push   %ebp
  c124a389:       89 e5                   mov    %esp,%ebp
  c124a38b:       57                      push   %edi
  c124a38c:       56                      push   %esi
  c124a38d:       89 d6                   mov    %edx,%esi
  c124a38f:       53                      push   %ebx
  c124a390:       31 db                   xor    %ebx,%ebx
  c124a392:       83 ec 03                sub    $0x3,%esp
  ...
  c124a3e3:       83 c4 03                add    $0x3,%esp
  c124a3e6:       5b                      pop    %ebx
  c124a3e7:       5e                      pop    %esi
  c124a3e8:       5f                      pop    %edi
  c124a3e9:       5d                      pop    %ebp
  c124a3ea:       c3                      ret

This was a leaf function.  For no apparent reason, GCC 4.8 decided to
subtract 3 from the stack pointer in the prologue.

Then in the middle of the function, it got an interrupt.  On 64-bit,
interrupts always align the stack, but on 32-bit they don't.  So the
pt_regs were stored at an unaligned address (0xf60bbb25).

The frame pointer encoding logic assumes an aligned stack pointer, so it
didn't work as expected.

  .macro ENCODE_FRAME_POINTER
  #ifdef CONFIG_FRAME_POINTER
  	mov %esp, %ebp
  	orl $0x1, %ebp
  #endif
  .endm

That effectively cleared the LSB of the encoded pt_regs address,
confusing the unwinder.

So on 32-bit, maybe the encoding should clear the MSB instead of setting
the LSB.

-- 
Josh

  reply	other threads:[~2017-10-04 21:06 UTC|newest]

Thread overview: 102+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-10-03 14:06 [lockdep] b09be676e0 BUG: unable to handle kernel NULL pointer dereference at 000001f2 Fengguang Wu
2017-10-03 14:06 ` Fengguang Wu
2017-10-03 14:31 ` Josh Poimboeuf
2017-10-03 14:31   ` Josh Poimboeuf
2017-10-03 14:41   ` Josh Poimboeuf
2017-10-03 14:41     ` Josh Poimboeuf
2017-10-03 15:05     ` Josh Poimboeuf
2017-10-03 15:05       ` Josh Poimboeuf
2017-10-03 16:28       ` Josh Poimboeuf
2017-10-03 16:28         ` Josh Poimboeuf
2017-10-03 17:34         ` Josh Poimboeuf
2017-10-03 17:34           ` Josh Poimboeuf
2017-10-03 21:44           ` Tetsuo Handa
2017-10-03 21:44             ` Tetsuo Handa
2017-10-04 21:06             ` Josh Poimboeuf [this message]
2017-10-04 21:06               ` Josh Poimboeuf
2017-10-04 21:30               ` Linus Torvalds
2017-10-04 21:30                 ` Linus Torvalds
2017-10-04 22:15                 ` Josh Poimboeuf
2017-10-04 22:15                   ` Josh Poimboeuf
2017-10-04 22:40             ` Josh Poimboeuf
2017-10-04 22:40               ` Josh Poimboeuf
2017-10-05 11:02               ` Tetsuo Handa
2017-10-05 11:02                 ` Tetsuo Handa
2017-10-05 13:57                 ` Josh Poimboeuf
2017-10-05 13:57                   ` Josh Poimboeuf
2017-10-04  8:34       ` Peter Zijlstra
2017-10-04  8:34         ` Peter Zijlstra
2017-10-10  5:57         ` Byungchul Park
2017-10-10  5:57           ` Byungchul Park
2017-10-03 16:54 ` Linus Torvalds
2017-10-03 16:54   ` Linus Torvalds
2017-10-03 16:57   ` Linus Torvalds
2017-10-03 16:57     ` Linus Torvalds
2017-10-10  5:48     ` Byungchul Park
2017-10-10  5:48       ` Byungchul Park
2017-10-10 16:22       ` Linus Torvalds
2017-10-10 16:22         ` Linus Torvalds
2017-10-10 16:56         ` Linus Torvalds
2017-10-10 16:56           ` Linus Torvalds
2017-10-10 18:14           ` Peter Zijlstra
2017-10-10 18:14             ` Peter Zijlstra
2017-10-10 18:38             ` Linus Torvalds
2017-10-10 18:38               ` Linus Torvalds
2017-10-11  1:14             ` Byungchul Park
2017-10-11  1:14               ` Byungchul Park
2017-10-11  2:36           ` Byungchul Park
2017-10-11  2:36             ` Byungchul Park
2017-10-11  0:56         ` Byungchul Park
2017-10-11  0:56           ` Byungchul Park
2017-10-11  1:02           ` Byungchul Park
2017-10-11  1:02             ` Byungchul Park
2017-10-12  1:15           ` Byungchul Park
2017-10-12  1:15             ` Byungchul Park
2017-10-03 17:18   ` Ingo Molnar
2017-10-03 17:18     ` Ingo Molnar
2017-10-04  9:20     ` Peter Zijlstra
2017-10-04  9:20       ` Peter Zijlstra
2017-10-04 10:31       ` Ingo Molnar
2017-10-04 10:31         ` Ingo Molnar
2017-10-04 14:15       ` Josh Poimboeuf
2017-10-04 14:15         ` Josh Poimboeuf
2017-10-10  5:30     ` Byungchul Park
2017-10-10  5:30       ` Byungchul Park
2017-10-05 13:01   ` Josh Poimboeuf
2017-10-05 13:01     ` Josh Poimboeuf
2017-10-05 14:54     ` Josh Poimboeuf
2017-10-05 14:54       ` Josh Poimboeuf
2017-10-09 10:50       ` Peter Zijlstra
2017-10-09 10:50         ` Peter Zijlstra
2017-10-09 12:21         ` Fengguang Wu
2017-10-09 12:21           ` Fengguang Wu
2017-10-09 12:54           ` Peter Zijlstra
2017-10-09 12:54             ` Peter Zijlstra
2017-10-09 12:59             ` Fengguang Wu
2017-10-09 12:59               ` Fengguang Wu
2017-10-09 13:03             ` Josh Poimboeuf
2017-10-09 13:03               ` Josh Poimboeuf
2017-10-09 12:55           ` Fengguang Wu
2017-10-09 12:55             ` Fengguang Wu
2017-10-09 13:26             ` Josh Poimboeuf
2017-10-09 13:26               ` Josh Poimboeuf
2017-10-09 14:17               ` Fengguang Wu
2017-10-09 14:17                 ` Fengguang Wu
2017-10-09 15:28                 ` Peter Zijlstra
2017-10-09 15:28                   ` Peter Zijlstra
2017-10-09 15:41                   ` Fengguang Wu
2017-10-09 15:41                     ` Fengguang Wu
2017-10-09 15:44                     ` Peter Zijlstra
2017-10-09 15:44                       ` Peter Zijlstra
2017-10-09 15:47                       ` Fengguang Wu
2017-10-09 15:47                         ` Fengguang Wu
2017-10-10  5:08   ` Byungchul Park
2017-10-10  5:08     ` Byungchul Park
2017-10-12  8:47 ` Peter Zijlstra
2017-10-12  8:47   ` Peter Zijlstra
2017-10-12  9:21   ` Fengguang Wu
2017-10-12  9:21     ` Fengguang Wu
2017-10-12  9:28     ` Fengguang Wu
2017-10-12  9:28       ` Fengguang Wu
2017-10-12 11:45       ` Peter Zijlstra
2017-10-12 11:45         ` Peter Zijlstra

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20171004210624.yyi64y2o5cvbr3d3@treble \
    --to=jpoimboe@redhat.com \
    --cc=lkp@lists.01.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.