From: "Michael S. Tsirkin" <mst@redhat.com>
To: Jean-Philippe Menil <jean-philippe.menil@univ-nantes.fr>
Cc: netdev@vger.kernel.org, kvm@vger.kernel.org,
virtualization@lists.linux-foundation.org
Subject: Re: Bug inkvm_set_irq
Date: Fri, 4 Mar 2011 11:35:00 +0200 [thread overview]
Message-ID: <20110304093500.GA31844@redhat.com> (raw)
In-Reply-To: <4D70AF3B.5080007@univ-nantes.fr>
On Fri, Mar 04, 2011 at 10:22:03AM +0100, Jean-Philippe Menil wrote:
> Le 03/03/2011 16:55, Michael S. Tsirkin a écrit :
> >On Thu, Mar 03, 2011 at 04:26:11PM +0100, Jean-Philippe Menil wrote:
> >>Le 03/03/2011 15:47, Michael S. Tsirkin a écrit :
> >>>On Tue, Mar 01, 2011 at 03:39:12PM +0100, Jean-Philippe Menil wrote:
> >>>>so this time the bug is:
> >>>>
> >>>>[17882.612303] BUG: unable to handle kernel paging request at
> >>>>0000000000002458
> >>>>[17882.612342] IP: [<ffffffffa03898a0>] kvm_set_irq+0x30/0x140 [kvm]
> >>>>
> >>>>markup_oops give me this:
> >>>>
> >>>>root@ayrshire:~# cat bug-0103.txt | perl markup_oops.pl -m
> >>>>/lib/modules/2.6.37.2-dsiun-110105+/kernel/arch/x86/kvm/kvm.ko
> >>>>/boot/vmlinuz-2.6.37.2-dsiun-110105+
> >>>>vmaoffset = 18446744072102621184 ffffffffa0389871: 48 89 e5 mov
> >>>>%rsp,%rbp
> >>>> ffffffffa0389874: 41 57 push %r15
> >>>> ffffffffa0389876: 41 89 cf mov %ecx,%r15d | %r15
> >>>>=> 1 %ecx = 1
> >>>> ffffffffa0389879: 41 56 push %r14 | %r14
> >>>>=> ffffffffa038aad0
> >>>> ffffffffa038987b: 41 55 push %r13
> >>>> ffffffffa038987d: 49 89 fd mov %rdi,%r13 | %edi
> >>>>= 0 %r13 => 0
> >>>> ffffffffa0389880: 41 54 push %r12 | %r12 => 0
> >>>> ffffffffa0389882: 53 push %rbx
> >>>> ffffffffa0389883: 89 d3 mov %edx,%ebx | %ebx => 1a
> >>>> ffffffffa0389885: 48 81 ec a8 00 00 00 sub $0xa8,%rsp
> >>>> ffffffffa038988c: 8b 15 00 00 00 00 mov 0x0(%rip),%edx
> >>>># ffffffffa0389892<kvm_set_irq+0x22>
> >>>> ffffffffa0389892: 89 b5 3c ff ff ff mov %esi,-0xc4(%rbp) |
> >>>>%esi = 0
> >>>> ffffffffa0389898: 85 d2 test %edx,%edx | %edx => 0
> >>>> ffffffffa038989a: 0f 85 d5 00 00 00 jne ffffffffa0389975
> >>>><kvm_set_irq+0x105>
> >>>>*ffffffffa03898a0: 49 8b 85 58 24 00 00 mov 0x2458(%r13),%rax |
> >>>>%eax = 0 %r13 = 0<--- faulting instruction
> >>>> ffffffffa03898a7: 3b 98 28 01 00 00 cmp 0x128(%rax),%ebx
> >>>> ffffffffa03898ad: 73 61 jae ffffffffa0389910
> >>>><kvm_set_irq+0xa0>
> >>>> ffffffffa03898af: 89 db mov %ebx,%ebx
> >>>> ffffffffa03898b1: 48 8b 84 d8 30 01 00 mov 0x130(%rax,%rbx,8),%rax
> >>>> ffffffffa03898b8: 00
> >>>> ffffffffa03898b9: 48 85 c0 test %rax,%rax
> >>>> ffffffffa03898bc: 74 52 je ffffffffa0389910
> >>>><kvm_set_irq+0xa0>
> >>>> ffffffffa03898be: 48 8d 95 40 ff ff ff lea -0xc0(%rbp),%rdx
> >>>> ffffffffa03898c5: 31 db xor %ebx,%ebx
> >>>> ffffffffa03898c7: 48 8b 08 mov (%rax),%rcx
> >>>> ffffffffa03898ca: 83 c3 01 add $0x1,%ebx
> >>>> ffffffffa03898cd: 0f 18 09 prefetcht0 (%rcx)
> >>>> ffffffffa03898d0: 48 8b 48 e0 mov -0x20(%rax),%rcx
> >>>> ffffffffa03898d4: 48 89 0a mov %rcx,(%rdx)
> >>>> ffffffffa03898d7: 48 8b 48 e8 mov -0x18(%rax),%rcx
> >>>> ffffffffa03898db: 48 89 4a 08 mov %rcx,0x8(%rdx)
> >>>> ffffffffa03898df: 48 8b 48 f0 mov -0x10(%rax),%rcx
> >>>> ffffffffa03898e3: 48 89 4a 10 mov %rcx,0x10(%rdx)
> >>>> ffffffffa03898e7: 48 8b 48 f8 mov -0x8(%rax),%rcx
> >>>> ffffffffa03898eb: 48 89 4a 18 mov %rcx,0x18(%rdx)
> >>>>
> >>>>wich correspond to offset 68a0 (from objdump):
> >>>>
> >>>>kvm_set_irq():
> >>>>/usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:161
> >>>> 68a0: 49 8b 85 58 24 00 00 mov 0x2458(%r13),%rax
> >>>>/usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:162
> >>>> 68a7: 3b 98 28 01 00 00 cmp 0x128(%rax),%ebx
> >>>>
> >>>>root@ayrshire:~# addr2line -e
> >>>>/lib/modules/2.6.37.2-dsiun-110105+/kernel/arch/x86/kvm/kvm.ko
> >>>>0x68a0
> >>>>/usr/src/GIT/linux-2.6-stable/arch/x86/kvm/../../../virt/kvm/irq_comm.c:161
> >>>>
> >>>>So here kvm->irq_routing is null.
> >>>>
> >>>>How can it be?
> >>>>
> >>>>Regards.
> >>>Not null, this seems to be invalid.
> >>>I suspect use after free where the kvm pointer is
> >>>pointing at some random memory. Use after free?
> >>>Could you please try enabling a slab debugger,
> >>>recompile and rerun the test?
> >>>
> >>Hi,
> >>
> >>I'm not sure to activate the right thing.
> >>Is that what you want?
> >>
> >>CONFIG_SLAB=y
> >>CONFIG_SLABINFO=y
> >>CONFIG_DEBUG_SLAB=y
> >>CONFIG_DEBUG_SLAB_LEAK=y
> >>
> >>Regards.
> >Yes, maybe disable SLAB_LEAK.
> >
> >>--
> >>Jean-Philippe Menil - Pôle réseau Service IRTS
> >>DSI Université de Nantes
> >>jean-philippe.menil@univ-nantes.fr
> >>Tel : 02.53.48.49.27 - Fax : 02.53.48.49.09
> >--
> >To unsubscribe from this list: send the line "unsubscribe kvm" in
> >the body of a message to majordomo@vger.kernel.org
> >More majordomo info at http://vger.kernel.org/majordomo-info.html
> Hi,
>
> so this time, here is what markup_oops says:
>
> root@ayrshire:~# cat oops-0403.txt | perl markup_oops.pl -m
> /lib/modules/2.6.37.2.999-dsiun-110105+/kernel/arch/x86/kvm/kvm.ko
> /boot/vmlinuz-2.6.37.2.999-dsiun-110105+
> vmaoffset = 18446744072102948864 ffffffffa03d9811: 48 89 e5
> mov %rsp,%rbp
> ffffffffa03d9814: 41 57 push %r15
> ffffffffa03d9816: 41 89 cf mov %ecx,%r15d |
> %r15 => 1 %ecx = 1
> ffffffffa03d9819: 41 56 push %r14 |
> %r14 => ffffffffa03daa50
> ffffffffa03d981b: 41 55 push %r13
> ffffffffa03d981d: 49 89 fd mov %rdi,%r13 |
> %edi = 6b6b6b6b6b6b6b6b %r13 => 6b6b6b6b6b6b6b6b
> ffffffffa03d9820: 41 54 push %r12 |
> %r12 => 6b6b6b6b6b6b6b6b
> ffffffffa03d9822: 53 push %rbx
> ffffffffa03d9823: 89 d3 mov %edx,%ebx |
> %ebx => 6b6b6b6b
> ffffffffa03d9825: 48 81 ec a8 00 00 00 sub $0xa8,%rsp
> ffffffffa03d982c: 8b 15 00 00 00 00 mov 0x0(%rip),%edx
> # ffffffffa03d9832 <kvm_set_irq+0x22>
> ffffffffa03d9832: 89 b5 3c ff ff ff mov
> %esi,-0xc4(%rbp) | %esi = 0
> ffffffffa03d9838: 85 d2 test %edx,%edx |
> %edx => 0
> ffffffffa03d983a: 0f 85 d5 00 00 00 jne
> ffffffffa03d9915 <kvm_set_irq+0x105>
> *ffffffffa03d9840: 49 8b 85 58 24 00 00 mov
> 0x2458(%r13),%rax | %eax = 0 %r13 = 6b6b6b6b6b6b6b6b <--- faulting
> instruction
> ffffffffa03d9847: 3b 98 28 01 00 00 cmp 0x128(%rax),%ebx
> ffffffffa03d984d: 73 61 jae
> ffffffffa03d98b0 <kvm_set_irq+0xa0>
> ffffffffa03d984f: 89 db mov %ebx,%ebx
> ffffffffa03d9851: 48 8b 84 d8 30 01 00 mov
> 0x130(%rax,%rbx,8),%rax
> ffffffffa03d9858: 00
> ffffffffa03d9859: 48 85 c0 test %rax,%rax
> ffffffffa03d985c: 74 52 je
> ffffffffa03d98b0 <kvm_set_irq+0xa0>
> ffffffffa03d985e: 48 8d 95 40 ff ff ff lea -0xc0(%rbp),%rdx
> ffffffffa03d9865: 31 db xor %ebx,%ebx
> ffffffffa03d9867: 48 8b 08 mov (%rax),%rcx
> ffffffffa03d986a: 83 c3 01 add $0x1,%ebx
> ffffffffa03d986d: 0f 18 09 prefetcht0 (%rcx)
> ffffffffa03d9870: 48 8b 48 e0 mov -0x20(%rax),%rcx
> ffffffffa03d9874: 48 89 0a mov %rcx,(%rdx)
> ffffffffa03d9877: 48 8b 48 e8 mov -0x18(%rax),%rcx
> ffffffffa03d987b: 48 89 4a 08 mov %rcx,0x8(%rdx)
> ffffffffa03d987f: 48 8b 48 f0 mov -0x10(%rax),%rcx
> ffffffffa03d9883: 48 89 4a 10 mov %rcx,0x10(%rdx)
> ffffffffa03d9887: 48 8b 48 f8 mov -0x8(%rax),%rcx
> ffffffffa03d988b: 48 89 4a 18 mov %rcx,0x18(%rdx)
>
>
> Is that you wanted, the "6b6b6b6b6b6b6b6b" ?
>
> Regards.
Yes, excellent. So now we can detect the problem by comparing
kvm with 6b6b6b6b6b6b6b6b, and print out stuff to understand where
this comes from.
I will prepare such a debugging patch.
For that, could you please tell me which kernel version, exactly, are
you using?
--
MST
next prev parent reply other threads:[~2011-03-04 9:35 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-02-25 9:07 Bug inkvm_set_irq Jean-Philippe Menil
2011-02-25 10:36 ` Gleb Natapov
2011-02-27 17:00 ` Michael S. Tsirkin
2011-02-28 8:56 ` Jean-Philippe Menil
2011-02-28 10:11 ` Michael S. Tsirkin
2011-02-28 10:40 ` Jean-Philippe Menil
2011-02-28 11:39 ` Michael S. Tsirkin
2011-02-28 15:13 ` Jean-Philippe Menil
2011-02-28 22:34 ` Jean-Philippe Menil
2011-03-01 7:03 ` Michael S. Tsirkin
2011-03-01 14:39 ` Jean-Philippe Menil
2011-03-03 14:47 ` Michael S. Tsirkin
2011-03-03 15:26 ` Jean-Philippe Menil
2011-03-03 15:55 ` Michael S. Tsirkin
2011-03-04 9:22 ` Jean-Philippe Menil
2011-03-04 9:35 ` Michael S. Tsirkin [this message]
2011-03-04 9:39 ` Jean-Philippe Menil
2011-03-08 11:13 ` Michael S. Tsirkin
2011-03-09 12:28 ` Jean-Philippe Menil
2011-03-09 13:00 ` Michael S. Tsirkin
2011-03-09 13:12 ` Jean-Philippe Menil
2011-03-09 13:59 ` Michael S. Tsirkin
2011-03-10 8:42 ` Jean-Philippe Menil
2011-03-15 14:32 ` Michael S. Tsirkin
2011-03-15 14:58 ` Jean-Philippe Menil
2011-03-17 8:00 ` Jean-Philippe Menil
2011-03-17 8:21 ` Michael S. Tsirkin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20110304093500.GA31844@redhat.com \
--to=mst@redhat.com \
--cc=jean-philippe.menil@univ-nantes.fr \
--cc=kvm@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=virtualization@lists.linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).