From: Josh Poimboeuf <jpoimboe@redhat.com>
To: lkp@lists.01.org
Subject: Re: [lockdep] b09be676e0 BUG: unable to handle kernel NULL pointer dereference at 000001f2
Date: Wed, 04 Oct 2017 16:06:24 -0500 [thread overview]
Message-ID: <20171004210624.yyi64y2o5cvbr3d3@treble> (raw)
In-Reply-To: <201710040644.GJE89219.FOHMFSFOVtOQLJ@I-love.SAKURA.ne.jp>
[-- Attachment #1: Type: text/plain, Size: 2976 bytes --]
On Wed, Oct 04, 2017 at 06:44:50AM +0900, Tetsuo Handa wrote:
> Josh Poimboeuf wrote:
> > On Tue, Oct 03, 2017 at 11:28:15AM -0500, Josh Poimboeuf wrote:
> > > There are two bugs:
> > >
> > > 1) Somebody -- presumably lockdep -- is corrupting the stack. Need the
> > > lockdep people to look at that.
> > >
> > > 2) The 32-bit FP unwinder isn't handling the corrupt stack very well,
> > > It's blindly dereferencing untrusted data:
> > >
> > > /* Is the next frame pointer an encoded pointer to pt_regs? */
> > > regs = decode_frame_pointer(next_bp);
> > > if (regs) {
> > > frame = (unsigned long *)regs;
> > > len = regs_size(regs);
> > > state->got_irq = true;
> > >
> > > On 32-bit, regs_size() dereferences the regs pointer before we know it
> > > points to a valid stack. I'll fix that, along with the other unwinder
> > > improvements I discussed with Linus.
> >
> > Tetsuo and/or Fengguang,
> >
> > Would you mind testing with this patch? It should at least prevent the
> > unwinder panic and should hopefully print a useful unwinder dump
> > instead.
> >
> Here are two outputs.
Thanks, both outputs were helpful.
This is another unwinder-related issue, unrelated to lockdep this time.
I compiled the same kernel with a similar version of GCC. It turns out
that GCC *does* create unaligned stacks with frame pointers enabled:
c124a388 <acpi_rs_move_data>:
c124a388: 55 push %ebp
c124a389: 89 e5 mov %esp,%ebp
c124a38b: 57 push %edi
c124a38c: 56 push %esi
c124a38d: 89 d6 mov %edx,%esi
c124a38f: 53 push %ebx
c124a390: 31 db xor %ebx,%ebx
c124a392: 83 ec 03 sub $0x3,%esp
...
c124a3e3: 83 c4 03 add $0x3,%esp
c124a3e6: 5b pop %ebx
c124a3e7: 5e pop %esi
c124a3e8: 5f pop %edi
c124a3e9: 5d pop %ebp
c124a3ea: c3 ret
This was a leaf function. For no apparent reason, GCC 4.8 decided to
subtract 3 from the stack pointer in the prologue.
Then in the middle of the function, it got an interrupt. On 64-bit,
interrupts always align the stack, but on 32-bit they don't. So the
pt_regs were stored at an unaligned address (0xf60bbb25).
The frame pointer encoding logic assumes an aligned stack pointer, so it
didn't work as expected.
.macro ENCODE_FRAME_POINTER
#ifdef CONFIG_FRAME_POINTER
mov %esp, %ebp
orl $0x1, %ebp
#endif
.endm
That effectively cleared the LSB of the encoded pt_regs address,
confusing the unwinder.
So on 32-bit, maybe the encoding should clear the MSB instead of setting
the LSB.
--
Josh
WARNING: multiple messages have this Message-ID (diff)
From: Josh Poimboeuf <jpoimboe@redhat.com>
To: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Cc: fengguang.wu@intel.com, byungchul.park@lge.com, mingo@kernel.org,
peterz@infradead.org, linux-kernel@vger.kernel.org, lkp@01.org,
torvalds@linux-foundation.org, bp@alien8.de, x86@kernel.org,
hpa@zytor.com, tglx@linutronix.de
Subject: Re: [lockdep] b09be676e0 BUG: unable to handle kernel NULL pointer dereference at 000001f2
Date: Wed, 4 Oct 2017 16:06:24 -0500 [thread overview]
Message-ID: <20171004210624.yyi64y2o5cvbr3d3@treble> (raw)
In-Reply-To: <201710040644.GJE89219.FOHMFSFOVtOQLJ@I-love.SAKURA.ne.jp>
On Wed, Oct 04, 2017 at 06:44:50AM +0900, Tetsuo Handa wrote:
> Josh Poimboeuf wrote:
> > On Tue, Oct 03, 2017 at 11:28:15AM -0500, Josh Poimboeuf wrote:
> > > There are two bugs:
> > >
> > > 1) Somebody -- presumably lockdep -- is corrupting the stack. Need the
> > > lockdep people to look at that.
> > >
> > > 2) The 32-bit FP unwinder isn't handling the corrupt stack very well,
> > > It's blindly dereferencing untrusted data:
> > >
> > > /* Is the next frame pointer an encoded pointer to pt_regs? */
> > > regs = decode_frame_pointer(next_bp);
> > > if (regs) {
> > > frame = (unsigned long *)regs;
> > > len = regs_size(regs);
> > > state->got_irq = true;
> > >
> > > On 32-bit, regs_size() dereferences the regs pointer before we know it
> > > points to a valid stack. I'll fix that, along with the other unwinder
> > > improvements I discussed with Linus.
> >
> > Tetsuo and/or Fengguang,
> >
> > Would you mind testing with this patch? It should at least prevent the
> > unwinder panic and should hopefully print a useful unwinder dump
> > instead.
> >
> Here are two outputs.
Thanks, both outputs were helpful.
This is another unwinder-related issue, unrelated to lockdep this time.
I compiled the same kernel with a similar version of GCC. It turns out
that GCC *does* create unaligned stacks with frame pointers enabled:
c124a388 <acpi_rs_move_data>:
c124a388: 55 push %ebp
c124a389: 89 e5 mov %esp,%ebp
c124a38b: 57 push %edi
c124a38c: 56 push %esi
c124a38d: 89 d6 mov %edx,%esi
c124a38f: 53 push %ebx
c124a390: 31 db xor %ebx,%ebx
c124a392: 83 ec 03 sub $0x3,%esp
...
c124a3e3: 83 c4 03 add $0x3,%esp
c124a3e6: 5b pop %ebx
c124a3e7: 5e pop %esi
c124a3e8: 5f pop %edi
c124a3e9: 5d pop %ebp
c124a3ea: c3 ret
This was a leaf function. For no apparent reason, GCC 4.8 decided to
subtract 3 from the stack pointer in the prologue.
Then in the middle of the function, it got an interrupt. On 64-bit,
interrupts always align the stack, but on 32-bit they don't. So the
pt_regs were stored at an unaligned address (0xf60bbb25).
The frame pointer encoding logic assumes an aligned stack pointer, so it
didn't work as expected.
.macro ENCODE_FRAME_POINTER
#ifdef CONFIG_FRAME_POINTER
mov %esp, %ebp
orl $0x1, %ebp
#endif
.endm
That effectively cleared the LSB of the encoded pt_regs address,
confusing the unwinder.
So on 32-bit, maybe the encoding should clear the MSB instead of setting
the LSB.
--
Josh
next prev parent reply other threads:[~2017-10-04 21:06 UTC|newest]
Thread overview: 102+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-10-03 14:06 [lockdep] b09be676e0 BUG: unable to handle kernel NULL pointer dereference at 000001f2 Fengguang Wu
2017-10-03 14:06 ` Fengguang Wu
2017-10-03 14:31 ` Josh Poimboeuf
2017-10-03 14:31 ` Josh Poimboeuf
2017-10-03 14:41 ` Josh Poimboeuf
2017-10-03 14:41 ` Josh Poimboeuf
2017-10-03 15:05 ` Josh Poimboeuf
2017-10-03 15:05 ` Josh Poimboeuf
2017-10-03 16:28 ` Josh Poimboeuf
2017-10-03 16:28 ` Josh Poimboeuf
2017-10-03 17:34 ` Josh Poimboeuf
2017-10-03 17:34 ` Josh Poimboeuf
2017-10-03 21:44 ` Tetsuo Handa
2017-10-03 21:44 ` Tetsuo Handa
2017-10-04 21:06 ` Josh Poimboeuf [this message]
2017-10-04 21:06 ` Josh Poimboeuf
2017-10-04 21:30 ` Linus Torvalds
2017-10-04 21:30 ` Linus Torvalds
2017-10-04 22:15 ` Josh Poimboeuf
2017-10-04 22:15 ` Josh Poimboeuf
2017-10-04 22:40 ` Josh Poimboeuf
2017-10-04 22:40 ` Josh Poimboeuf
2017-10-05 11:02 ` Tetsuo Handa
2017-10-05 11:02 ` Tetsuo Handa
2017-10-05 13:57 ` Josh Poimboeuf
2017-10-05 13:57 ` Josh Poimboeuf
2017-10-04 8:34 ` Peter Zijlstra
2017-10-04 8:34 ` Peter Zijlstra
2017-10-10 5:57 ` Byungchul Park
2017-10-10 5:57 ` Byungchul Park
2017-10-03 16:54 ` Linus Torvalds
2017-10-03 16:54 ` Linus Torvalds
2017-10-03 16:57 ` Linus Torvalds
2017-10-03 16:57 ` Linus Torvalds
2017-10-10 5:48 ` Byungchul Park
2017-10-10 5:48 ` Byungchul Park
2017-10-10 16:22 ` Linus Torvalds
2017-10-10 16:22 ` Linus Torvalds
2017-10-10 16:56 ` Linus Torvalds
2017-10-10 16:56 ` Linus Torvalds
2017-10-10 18:14 ` Peter Zijlstra
2017-10-10 18:14 ` Peter Zijlstra
2017-10-10 18:38 ` Linus Torvalds
2017-10-10 18:38 ` Linus Torvalds
2017-10-11 1:14 ` Byungchul Park
2017-10-11 1:14 ` Byungchul Park
2017-10-11 2:36 ` Byungchul Park
2017-10-11 2:36 ` Byungchul Park
2017-10-11 0:56 ` Byungchul Park
2017-10-11 0:56 ` Byungchul Park
2017-10-11 1:02 ` Byungchul Park
2017-10-11 1:02 ` Byungchul Park
2017-10-12 1:15 ` Byungchul Park
2017-10-12 1:15 ` Byungchul Park
2017-10-03 17:18 ` Ingo Molnar
2017-10-03 17:18 ` Ingo Molnar
2017-10-04 9:20 ` Peter Zijlstra
2017-10-04 9:20 ` Peter Zijlstra
2017-10-04 10:31 ` Ingo Molnar
2017-10-04 10:31 ` Ingo Molnar
2017-10-04 14:15 ` Josh Poimboeuf
2017-10-04 14:15 ` Josh Poimboeuf
2017-10-10 5:30 ` Byungchul Park
2017-10-10 5:30 ` Byungchul Park
2017-10-05 13:01 ` Josh Poimboeuf
2017-10-05 13:01 ` Josh Poimboeuf
2017-10-05 14:54 ` Josh Poimboeuf
2017-10-05 14:54 ` Josh Poimboeuf
2017-10-09 10:50 ` Peter Zijlstra
2017-10-09 10:50 ` Peter Zijlstra
2017-10-09 12:21 ` Fengguang Wu
2017-10-09 12:21 ` Fengguang Wu
2017-10-09 12:54 ` Peter Zijlstra
2017-10-09 12:54 ` Peter Zijlstra
2017-10-09 12:59 ` Fengguang Wu
2017-10-09 12:59 ` Fengguang Wu
2017-10-09 13:03 ` Josh Poimboeuf
2017-10-09 13:03 ` Josh Poimboeuf
2017-10-09 12:55 ` Fengguang Wu
2017-10-09 12:55 ` Fengguang Wu
2017-10-09 13:26 ` Josh Poimboeuf
2017-10-09 13:26 ` Josh Poimboeuf
2017-10-09 14:17 ` Fengguang Wu
2017-10-09 14:17 ` Fengguang Wu
2017-10-09 15:28 ` Peter Zijlstra
2017-10-09 15:28 ` Peter Zijlstra
2017-10-09 15:41 ` Fengguang Wu
2017-10-09 15:41 ` Fengguang Wu
2017-10-09 15:44 ` Peter Zijlstra
2017-10-09 15:44 ` Peter Zijlstra
2017-10-09 15:47 ` Fengguang Wu
2017-10-09 15:47 ` Fengguang Wu
2017-10-10 5:08 ` Byungchul Park
2017-10-10 5:08 ` Byungchul Park
2017-10-12 8:47 ` Peter Zijlstra
2017-10-12 8:47 ` Peter Zijlstra
2017-10-12 9:21 ` Fengguang Wu
2017-10-12 9:21 ` Fengguang Wu
2017-10-12 9:28 ` Fengguang Wu
2017-10-12 9:28 ` Fengguang Wu
2017-10-12 11:45 ` Peter Zijlstra
2017-10-12 11:45 ` Peter Zijlstra
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20171004210624.yyi64y2o5cvbr3d3@treble \
--to=jpoimboe@redhat.com \
--cc=lkp@lists.01.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.