BUG: using smp_processor_id() in preemptible

BUG: using smp_processor_id() in preemptible

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 2502 bytes --]

Hi,

I'm trying to run a test environment in kvm (because uml doesn't have
lockdep), and am running into the following problems:

1) I get the $subject warning a lot, when starting kvm:
[85763.262707] BUG: using smp_processor_id() in preemptible [00000000] code: kvm/13877
[85763.262719] caller is kvm_write_guest_time+0x40/0x220 [kvm]
[85763.262722] Pid: 13877, comm: kvm Not tainted 2.6.30-wl-26837-g0ee651a-dirty #54
[85763.262725] Call Trace:
[85763.262729]  [<ffffffff8041d482>] debug_smp_processor_id+0xf2/0x100
[85763.262741]  [<ffffffffa0331390>] kvm_write_guest_time+0x40/0x220 [kvm]
[85763.262753]  [<ffffffffa0331890>] vcpu_enter_guest+0x320/0x580 [kvm]
[85763.262780]  [<ffffffffa03347f4>] __vcpu_run+0x74/0x2f0 [kvm]
[85763.262792]  [<ffffffffa033571f>] kvm_arch_vcpu_ioctl_run+0x8f/0x200 [kvm]
[85763.262804]  [<ffffffffa0329b48>] kvm_vcpu_ioctl+0x4b8/0x900 [kvm]
[85763.262816]  [<ffffffff802f5216>] vfs_ioctl+0x36/0xb0
[85763.262819]  [<ffffffff802f55f9>] do_vfs_ioctl+0x89/0x320
[85763.262826]  [<ffffffff802f58df>] sys_ioctl+0x4f/0x80
[85763.262830]  [<ffffffff8020b6fb>] system_call_fastpath+0x16/0x1b

That kernel version is wireless-testing, which is currently based on
v2.6.30, and the -dirty is for some wireless patches I did.

2) The second problem is that it doesn't actually work. I use this
command line:
kvm -kernel arch/x86_64/boot/bzImage \
	-hda ../uml/Ubuntu-IntrepidIbex-amd64-root_fs \
	-append "root=/dev/hda console=ttyS0" -curses

and the system hangs after
              Plex86/Bochs VGABios (PCI) current-cvs 12 Jun 2009
              This VGA/VBE Bios is released under the GNU LGPL

              Please visit :
               . http://bochs.sourceforge.net
               . http://www.nongnu.org/vgabios

              cirrus-compatible VGA is detected

              QEMU BIOS - build: 06/12/09
              $Revision: 1.182 $ $Date: 2007/08/01 17:09:51 $
              Options: apmbios pcibios eltorito rombios32

              ata0 master: QEMU HARDDISK ATA-7 Hard-Disk (1024 MBytes)
              ata1 master: QEMU DVD-ROM ATAPI-4 CD-Rom/DVD-Rom

              Press F12 for boot menu.


              Decompressing Linux... Parsing ELF... done.
              Booting the kernel.

The guest kernel is the same as the host, but with somewhat different
config options.

The strange thing here is that the exact same command line, with
qemu-system-x86_64 instead of kvm works perfectly.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: BUG: using smp_processor_id() in preemptible

[Avi Kivity]

On 06/24/2009 05:15 PM, Johannes Berg wrote:
> Hi,
>
> I'm trying to run a test environment in kvm (because uml doesn't have
> lockdep), and am running into the following problems:
>
> 1) I get the $subject warning a lot, when starting kvm:
> [85763.262707] BUG: using smp_processor_id() in preemptible [00000000] code: kvm/13877
> [85763.262719] caller is kvm_write_guest_time+0x40/0x220 [kvm]
> [85763.262722] Pid: 13877, comm: kvm Not tainted 2.6.30-wl-26837-g0ee651a-dirty #54
> [85763.262725] Call Trace:
> [85763.262729]  [<ffffffff8041d482>] debug_smp_processor_id+0xf2/0x100
> [85763.262741]  [<ffffffffa0331390>] kvm_write_guest_time+0x40/0x220 [kvm]
> [85763.262753]  [<ffffffffa0331890>] vcpu_enter_guest+0x320/0x580 [kvm]
> [85763.262780]  [<ffffffffa03347f4>] __vcpu_run+0x74/0x2f0 [kvm]
> [85763.262792]  [<ffffffffa033571f>] kvm_arch_vcpu_ioctl_run+0x8f/0x200 [kvm]
> [85763.262804]  [<ffffffffa0329b48>] kvm_vcpu_ioctl+0x4b8/0x900 [kvm]
> [85763.262816]  [<ffffffff802f5216>] vfs_ioctl+0x36/0xb0
> [85763.262819]  [<ffffffff802f55f9>] do_vfs_ioctl+0x89/0x320
> [85763.262826]  [<ffffffff802f58df>] sys_ioctl+0x4f/0x80
> [85763.262830]  [<ffffffff8020b6fb>] system_call_fastpath+0x16/0x1b
>
>    

ISTR this was fixed...

> That kernel version is wireless-testing, which is currently based on
> v2.6.30, and the -dirty is for some wireless patches I did.
>    

Please post the output of 'git merge-base wireless-testing 
origin/master' so I can know what tree to look at.

> 2) The second problem is that it doesn't actually work. I use this
> command line:
> kvm -kernel arch/x86_64/boot/bzImage \
> 	-hda ../uml/Ubuntu-IntrepidIbex-amd64-root_fs \
> 	-append "root=/dev/hda console=ttyS0" -curses
>
> and the system hangs after
>                Plex86/Bochs VGABios (PCI) current-cvs 12 Jun 2009
>                This VGA/VBE Bios is released under the GNU LGPL
>
>                Please visit :
>                 . http://bochs.sourceforge.net
>                 . http://www.nongnu.org/vgabios
>
>                cirrus-compatible VGA is detected
>
>                QEMU BIOS - build: 06/12/09
>                $Revision: 1.182 $ $Date: 2007/08/01 17:09:51 $
>                Options: apmbios pcibios eltorito rombios32
>
>                ata0 master: QEMU HARDDISK ATA-7 Hard-Disk (1024 MBytes)
>                ata1 master: QEMU DVD-ROM ATAPI-4 CD-Rom/DVD-Rom
>
>                Press F12 for boot menu.
>
>
>                Decompressing Linux... Parsing ELF... done.
>                Booting the kernel.
>    

Does it hang or switch to some graphics mode?  What happens if you drop 
curses?

You can see where it hangs using the monitor 'info registers' and 'x/30i 
$eip' commands.

> The guest kernel is the same as the host, but with somewhat different
> config options.
>
> The strange thing here is that the exact same command line, with
> qemu-system-x86_64 instead of kvm works perfectly.
>    

That's probably a qemu without kvm support.

-- 
error compiling committee.c: too many arguments to function

Re: BUG: using smp_processor_id() in preemptible

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 13458 bytes --]

On Sun, 2009-06-28 at 17:11 +0300, Avi Kivity wrote:

> > 1) I get the $subject warning a lot, when starting kvm:
> > [85763.262707] BUG: using smp_processor_id() in preemptible [00000000] code: kvm/13877
> > [85763.262719] caller is kvm_write_guest_time+0x40/0x220 [kvm]
> > [85763.262722] Pid: 13877, comm: kvm Not tainted 2.6.30-wl-26837-g0ee651a-dirty #54
> > [85763.262725] Call Trace:
> > [85763.262729]  [<ffffffff8041d482>] debug_smp_processor_id+0xf2/0x100
> > [85763.262741]  [<ffffffffa0331390>] kvm_write_guest_time+0x40/0x220 [kvm]
> > [85763.262753]  [<ffffffffa0331890>] vcpu_enter_guest+0x320/0x580 [kvm]
> > [85763.262780]  [<ffffffffa03347f4>] __vcpu_run+0x74/0x2f0 [kvm]
> > [85763.262792]  [<ffffffffa033571f>] kvm_arch_vcpu_ioctl_run+0x8f/0x200 [kvm]
> > [85763.262804]  [<ffffffffa0329b48>] kvm_vcpu_ioctl+0x4b8/0x900 [kvm]
> > [85763.262816]  [<ffffffff802f5216>] vfs_ioctl+0x36/0xb0
> > [85763.262819]  [<ffffffff802f55f9>] do_vfs_ioctl+0x89/0x320
> > [85763.262826]  [<ffffffff802f58df>] sys_ioctl+0x4f/0x80
> > [85763.262830]  [<ffffffff8020b6fb>] system_call_fastpath+0x16/0x1b
> >
> >    
> 
> ISTR this was fixed...
> 
> > That kernel version is wireless-testing, which is currently based on
> > v2.6.30, and the -dirty is for some wireless patches I did.
> >    
> 
> Please post the output of 'git merge-base wireless-testing 
> origin/master' so I can know what tree to look at.

ITYM

$ git merge-base wireless-testing/master linux-2.6/master
07a2039b8eb0af4ff464efd3dfd95de5c02648c6
$ git describe 07a2039b8eb0af4ff464efd3dfd95de5c02648c6
v2.6.30

since my personal 'origin' branch is something completely different.

> > 2) The second problem is that it doesn't actually work. I use this
> > command line:
> > kvm -kernel arch/x86_64/boot/bzImage \
> > 	-hda ../uml/Ubuntu-IntrepidIbex-amd64-root_fs \
> > 	-append "root=/dev/hda console=ttyS0" -curses
> >
> > and the system hangs after
> >                Plex86/Bochs VGABios (PCI) current-cvs 12 Jun 2009
> >                This VGA/VBE Bios is released under the GNU LGPL
> >
> >                Please visit :
> >                 . http://bochs.sourceforge.net
> >                 . http://www.nongnu.org/vgabios
> >
> >                cirrus-compatible VGA is detected
> >
> >                QEMU BIOS - build: 06/12/09
> >                $Revision: 1.182 $ $Date: 2007/08/01 17:09:51 $
> >                Options: apmbios pcibios eltorito rombios32
> >
> >                ata0 master: QEMU HARDDISK ATA-7 Hard-Disk (1024 MBytes)
> >                ata1 master: QEMU DVD-ROM ATAPI-4 CD-Rom/DVD-Rom
> >
> >                Press F12 for boot menu.
> >
> >
> >                Decompressing Linux... Parsing ELF... done.
> >                Booting the kernel.
> >    
> 
> Does it hang or switch to some graphics mode?  What happens if you drop 
> curses?

Same, I just used curses to copy/paste the messages I get.

> You can see where it hangs using the monitor 'info registers' and 'x/30i 
> $eip' commands.

not much luck since it doesn't hang at a specific instruction:
(qemu) info registers
RAX=0000000000000001 RBX=0000000000000000 RCX=0000000001062560 RDX=0000000000000000
RSI=0000000000000001 RDI=0000000000000001 RBP=ffffffff80a6dd98 RSP=ffffffff80a6dd98
R8 =0000000000000001 R9 =0000000000000000 R10=0000000000000001 R11=0000000000000001
R12=000000011544e510 R13=0000000000000010 R14=0000000000000b8e R15=ffff8800001fee00
RIP=ffffffff803d5360 RFL=00000202 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0018 0000000000000000 ffffffff 00c09300
CS =0010 0000000000000000 ffffffff 00a09b00
SS =0018 0000000000000000 ffffffff 00c09300
DS =0018 0000000000000000 ffffffff 00c09300
FS =0000 0000000000000000 ffffffff 00000000
GS =0000 ffff880006200000 ffffffff 00000000
LDT=0000 0000000000000000 ffffffff 00000000
TR =0040 ffff8800063d0a40 00002087 00008b00
GDT=     ffff880006204000 0000007f
IDT=     ffffffff80ca5000 00000fff
CR0=8005003b CR2=0000000000000000 CR3=0000000000201000 CR4=000006a0
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 
DR6=00000000ffff0ff0 DR7=0000000000000400
FCW=037f FSW=0000 [ST=0] FTW=00 MXCSR=00000000
FPR0=0000000000000000 0000 FPR1=0000000000000000 0000
FPR2=0000000000000000 0000 FPR3=0000000000000000 0000
FPR4=0000000000000000 0000 FPR5=0000000000000000 0000
FPR6=0000000000000000 0000 FPR7=0000000000000000 0000
XMM00=00000000000000000000000000000000 XMM01=00000000000000000000000000000000
XMM02=00000000000000000000000000000000 XMM03=00000000000000000000000000000000
XMM04=00000000000000000000000000000000 XMM05=00000000000000000000000000000000
XMM06=00000000000000000000000000000000 XMM07=00000000000000000000000000000000
XMM08=00000000000000000000000000000000 XMM09=00000000000000000000000000000000
XMM10=00000000000000000000000000000000 XMM11=00000000000000000000000000000000
XMM12=00000000000000000000000000000000 XMM13=00000000000000000000000000000000
XMM14=00000000000000000000000000000000 XMM15=00000000000000000000000000000000
(qemu) x/30i $eip
0xffffffff8028fc60:  push   %rbp
0xffffffff8028fc61:  mov    $0x1cebe8,%rax
0xffffffff8028fc68:  mov    %rsp,%rbp
0xffffffff8028fc6b:  mov    %gs:0xc8a0,%rdx
0xffffffff8028fc74:  movq   $0x0,(%rax,%rdx,1)
0xffffffff8028fc7c:  leaveq 
0xffffffff8028fc7d:  retq   
0xffffffff8028fc7e:  xchg   %ax,%ax
0xffffffff8028fc80:  push   %rbp
0xffffffff8028fc81:  mov    %rsp,%rbp
0xffffffff8028fc84:  sub    $0x10,%rsp
0xffffffff8028fc88:  mov    %rbx,(%rsp)
0xffffffff8028fc8c:  mov    %r12,0x8(%rsp)
0xffffffff8028fc91:  mov    $0x1cebe8,%rbx
0xffffffff8028fc98:  mov    %gs:0xc8a0,%r12
0xffffffff8028fca1:  mov    %gs:0xc8a8,%edi
0xffffffff8028fca9:  callq  0xffffffff80268ef0
0xffffffff8028fcae:  shr    $0x1e,%rax
0xffffffff8028fcb2:  mov    %rax,(%r12,%rbx,1)
0xffffffff8028fcb6:  mov    (%rsp),%rbx
0xffffffff8028fcba:  mov    0x8(%rsp),%r12
0xffffffff8028fcbf:  leaveq 
0xffffffff8028fcc0:  retq   
0xffffffff8028fcc1:  nopw   %cs:0x0(%rax,%rax,1)
0xffffffff8028fcd0:  push   %rbp
0xffffffff8028fcd1:  mov    $0x1,%esi
0xffffffff8028fcd6:  mov    %rsp,%rbp
0xffffffff8028fcd9:  push   %rbx
0xffffffff8028fcda:  lea    -0x20(%rbp),%rdx
0xffffffff8028fcde:  sub    $0x18,%rsp
(qemu) info registers
RAX=ffffffffffffffff RBX=0000000000000000 RCX=0000000001062560 RDX=0000000000000000
RSI=0000000000000001 RDI=0000000000000001 RBP=ffffffff80a6de98 RSP=ffffffff80a6ddb8
R8 =0000000000000001 R9 =0000000000000000 R10=0000000000000001 R11=0000000000000001
R12=0000000125ac5486 R13=0000000000000010 R14=0000000000000b8e R15=ffff8800001fee00
RIP=ffffffff805da6be RFL=00000296 [--S-AP-] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0018 0000000000000000 ffffffff 00c09300
CS =0010 0000000000000000 ffffffff 00a09b00
SS =0018 0000000000000000 ffffffff 00c09300
DS =0018 0000000000000000 ffffffff 00c09300
FS =0000 0000000000000000 ffffffff 00000000
GS =0000 ffff880006200000 ffffffff 00000000
LDT=0000 0000000000000000 ffffffff 00000000
TR =0040 ffff8800063d0a40 00002087 00008b00
GDT=     ffff880006204000 0000007f
IDT=     ffffffff80ca5000 00000fff
CR0=8005003b CR2=0000000000000000 CR3=0000000000201000 CR4=000006a0
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 
DR6=00000000ffff0ff0 DR7=0000000000000400
FCW=037f FSW=0000 [ST=0] FTW=00 MXCSR=00000000
FPR0=0000000000000000 0000 FPR1=0000000000000000 0000
FPR2=0000000000000000 0000 FPR3=0000000000000000 0000
FPR4=0000000000000000 0000 FPR5=0000000000000000 0000
FPR6=0000000000000000 0000 FPR7=0000000000000000 0000
XMM00=00000000000000000000000000000000 XMM01=00000000000000000000000000000000
XMM02=00000000000000000000000000000000 XMM03=00000000000000000000000000000000
XMM04=00000000000000000000000000000000 XMM05=00000000000000000000000000000000
XMM06=00000000000000000000000000000000 XMM07=00000000000000000000000000000000
XMM08=00000000000000000000000000000000 XMM09=00000000000000000000000000000000
XMM10=00000000000000000000000000000000 XMM11=00000000000000000000000000000000
XMM12=00000000000000000000000000000000 XMM13=00000000000000000000000000000000
XMM14=00000000000000000000000000000000 XMM15=00000000000000000000000000000000
(qemu) x/30i $eip
0xffffffff80249ce6:  leaveq 
0xffffffff80249ce7:  retq   
0xffffffff80249ce8:  nopl   0x0(%rax,%rax,1)
0xffffffff80249cf0:  push   %rbp
0xffffffff80249cf1:  mov    0xa93668(%rip),%rax        # 0xffffffff80cdd360
0xffffffff80249cf8:  mov    %rsp,%rbp
0xffffffff80249cfb:  leaveq 
0xffffffff80249cfc:  retq   
0xffffffff80249cfd:  nopl   (%rax)
0xffffffff80249d00:  push   %rbp
0xffffffff80249d01:  xor    %eax,%eax
0xffffffff80249d03:  mov    %rsp,%rbp
0xffffffff80249d06:  cmpl   $0x0,0xa9365f(%rip)        # 0xffffffff80cdd36c
0xffffffff80249d0d:  leaveq 
0xffffffff80249d0e:  sete   %al
0xffffffff80249d11:  retq   
0xffffffff80249d12:  nopw   %cs:0x0(%rax,%rax,1)
0xffffffff80249d20:  push   %rbp
0xffffffff80249d21:  mov    0xa93648(%rip),%rax        # 0xffffffff80cdd370
0xffffffff80249d28:  mov    %rsp,%rbp
0xffffffff80249d2b:  test   %rax,%rax
0xffffffff80249d2e:  je     0xffffffff80249d40
0xffffffff80249d30:  inc    %rax
0xffffffff80249d33:  mov    %rax,0xa93636(%rip)        # 0xffffffff80cdd370
0xffffffff80249d3a:  xor    %eax,%eax
0xffffffff80249d3c:  leaveq 
0xffffffff80249d3d:  retq   
0xffffffff80249d3e:  xchg   %ax,%ax
0xffffffff80249d40:  mov    $0x8,%esi
0xffffffff80249d45:  mov    $0xffffffff80cdd370,%rdi
(qemu) info registers
RAX=0000000000000000 RBX=0000000000000000 RCX=0000000001062560 RDX=0000000000000000
RSI=0000000000000001 RDI=0000000000418958 RBP=ffffffff80a6dda8 RSP=ffffffff80a6dda8
R8 =0000000000000001 R9 =0000000000000000 R10=0000000000000001 R11=0000000000000001
R12=000000013473bc5c R13=0000000000000010 R14=0000000000000b8e R15=ffff8800001fee00
RIP=ffffffff803d53da RFL=00000246 [---Z-P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0018 0000000000000000 ffffffff 00c09300
CS =0010 0000000000000000 ffffffff 00a09b00
SS =0018 0000000000000000 ffffffff 00c09300
DS =0018 0000000000000000 ffffffff 00c09300
FS =0000 0000000000000000 ffffffff 00000000
GS =0000 ffff880006200000 ffffffff 00000000
LDT=0000 0000000000000000 ffffffff 00000000
TR =0040 ffff8800063d0a40 00002087 00008b00
GDT=     ffff880006204000 0000007f
IDT=     ffffffff80ca5000 00000fff
CR0=8005003b CR2=0000000000000000 CR3=0000000000201000 CR4=000006a0
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 
DR6=00000000ffff0ff0 DR7=0000000000000400
FCW=037f FSW=0000 [ST=0] FTW=00 MXCSR=00000000
FPR0=0000000000000000 0000 FPR1=0000000000000000 0000
FPR2=0000000000000000 0000 FPR3=0000000000000000 0000
FPR4=0000000000000000 0000 FPR5=0000000000000000 0000
FPR6=0000000000000000 0000 FPR7=0000000000000000 0000
XMM00=00000000000000000000000000000000 XMM01=00000000000000000000000000000000
XMM02=00000000000000000000000000000000 XMM03=00000000000000000000000000000000
XMM04=00000000000000000000000000000000 XMM05=00000000000000000000000000000000
XMM06=00000000000000000000000000000000 XMM07=00000000000000000000000000000000
XMM08=00000000000000000000000000000000 XMM09=00000000000000000000000000000000
XMM10=00000000000000000000000000000000 XMM11=00000000000000000000000000000000
XMM12=00000000000000000000000000000000 XMM13=00000000000000000000000000000000
XMM14=00000000000000000000000000000000 XMM15=00000000000000000000000000000000
(qemu) x/30i $eip
0xffffffff805da6be:  callq  0xffffffff8028fc60
0xffffffff805da6c3:  mov    %r12,%rdi
0xffffffff805da6c6:  callq  *0x702854(%rip)        # 0xffffffff80cdcf20
0xffffffff805da6cc:  mov    $0x418958,%edi
0xffffffff805da6d1:  mov    %rax,%rbx
0xffffffff805da6d4:  callq  0xffffffff803d53a0
0xffffffff805da6d9:  lea    0x1(%r12,%rbx,1),%r12
0xffffffff805da6de:  jmp    0xffffffff805da6be
0xffffffff805da6e0:  movq   $0xffffffff80249ce0,0x702835(%rip)        # 0xffffffff80cdcf20
0xffffffff805da6eb:  jmp    0xffffffff805da6a5
0xffffffff805da6ed:  xor    %eax,%eax
0xffffffff805da6ef:  mov    $0xffffffff806d0063,%rdi
0xffffffff805da6f6:  callq  0xffffffff805da747
0xffffffff805da6fb:  imul   $0x3e8,0x702823(%rip),%eax        # 0xffffffff80cdcf28
0xffffffff805da705:  test   %eax,%eax
0xffffffff805da707:  jle    0xffffffff805da73d
0xffffffff805da709:  xor    %r12d,%r12d
0xffffffff805da70c:  callq  0xffffffff80227860
0xffffffff805da711:  mov    %r12,%rdi
0xffffffff805da714:  callq  *0x702806(%rip)        # 0xffffffff80cdcf20
0xffffffff805da71a:  mov    $0x418958,%edi
0xffffffff805da71f:  mov    %rax,%rbx
0xffffffff805da722:  callq  0xffffffff803d53a0
0xffffffff805da727:  lea    0x1(%r12,%rbx,1),%r12
0xffffffff805da72c:  imul   $0x3e8,0x7027f2(%rip),%eax        # 0xffffffff80cdcf28
0xffffffff805da736:  cltq   
0xffffffff805da738:  cmp    %r12,%rax
0xffffffff805da73b:  jg     0xffffffff805da70c
0xffffffff805da73d:  callq  0xffffffff8025cd30
0xffffffff805da742:  jmpq   0xffffffff805da6af



> > The guest kernel is the same as the host, but with somewhat different
> > config options.
> >
> > The strange thing here is that the exact same command line, with
> > qemu-system-x86_64 instead of kvm works perfectly.
> >    
> 
> That's probably a qemu without kvm support.

Yes, I know that, I just used that to verify the guest kernel is ok.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: BUG: using smp_processor_id() in preemptible

[Avi Kivity]

On 06/29/2009 11:32 AM, Johannes Berg wrote:
> On Sun, 2009-06-28 at 17:11 +0300, Avi Kivity wrote:
>
>    
>>> 1) I get the $subject warning a lot, when starting kvm:
>>> [85763.262707] BUG: using smp_processor_id() in preemptible [00000000] code: kvm/13877
>>> [85763.262719] caller is kvm_write_guest_time+0x40/0x220 [kvm]
>>> [85763.262722] Pid: 13877, comm: kvm Not tainted 2.6.30-wl-26837-g0ee651a-dirty #54
>>> [85763.262725] Call Trace:
>>> [85763.262729]  [<ffffffff8041d482>] debug_smp_processor_id+0xf2/0x100
>>> [85763.262741]  [<ffffffffa0331390>] kvm_write_guest_time+0x40/0x220 [kvm]
>>> [85763.262753]  [<ffffffffa0331890>] vcpu_enter_guest+0x320/0x580 [kvm]
>>> [85763.262780]  [<ffffffffa03347f4>] __vcpu_run+0x74/0x2f0 [kvm]
>>> [85763.262792]  [<ffffffffa033571f>] kvm_arch_vcpu_ioctl_run+0x8f/0x200 [kvm]
>>> [85763.262804]  [<ffffffffa0329b48>] kvm_vcpu_ioctl+0x4b8/0x900 [kvm]
>>> [85763.262816]  [<ffffffff802f5216>] vfs_ioctl+0x36/0xb0
>>> [85763.262819]  [<ffffffff802f55f9>] do_vfs_ioctl+0x89/0x320
>>> [85763.262826]  [<ffffffff802f58df>] sys_ioctl+0x4f/0x80
>>> [85763.262830]  [<ffffffff8020b6fb>] system_call_fastpath+0x16/0x1b
>>>
>>>
>>>        
>> ISTR this was fixed...
>>
>>      
>>> That kernel version is wireless-testing, which is currently based on
>>> v2.6.30, and the -dirty is for some wireless patches I did.
>>>
>>>        
>> Please post the output of 'git merge-base wireless-testing
>> origin/master' so I can know what tree to look at.
>>      
>
> ITYM
>
> $ git merge-base wireless-testing/master linux-2.6/master
> 07a2039b8eb0af4ff464efd3dfd95de5c02648c6
> $ git describe 07a2039b8eb0af4ff464efd3dfd95de5c02648c6
> v2.6.30
>
> since my personal 'origin' branch is something completely different.
>
>    

Yes.  It was fixed in mainline by 2dea4c84bc.  I'll prepare something 
for -stable.


>> You can see where it hangs using the monitor 'info registers' and 'x/30i
>> $eip' commands.
>>      
>
> not much luck since it doesn't hang at a specific instruction:
>    

You can try mapping these with gdb (in fact, you can have gdb connect to 
qemu and do source level debugging).

-- 
error compiling committee.c: too many arguments to function

Re: BUG: using smp_processor_id() in preemptible

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 2224 bytes --]

On Mon, 2009-06-29 at 12:08 +0300, Avi Kivity wrote:

> >> You can see where it hangs using the monitor 'info registers' and 'x/30i
> >> $eip' commands.
> >>      
> >
> > not much luck since it doesn't hang at a specific instruction:
> >    
> 
> You can try mapping these with gdb (in fact, you can have gdb connect to 
> qemu and do source level debugging).

It's actually panic'ed because it ran out of memory, but didn't print
that to my vga/console/curses/...

(gdb) bt full
#0  touch_softlockup_watchdog ()
    at /home/johannes/sys/wireless-testing/kernel/softlockup.c:77
No locals.
#1  0xffffffff805d6373 in panic (fmt=0x0)
    at /home/johannes/sys/wireless-testing/kernel/panic.c:134
        args = {{gp_offset = 8, fp_offset = 48, overflow_arg_area = 
    0xffffffff80a69ea8, reg_save_area = 0xffffffff80a69dd8}}
        i = 1810713685
        buf = "Out of memory", '\0' <repeats 1010 times>
#2  0xffffffff80a889c4 in ___alloc_bootmem (size=512, 
    align=<value optimized out>, goal=<value optimized out>, 
    limit=<value optimized out>)
    at /home/johannes/sys/wireless-testing/mm/bootmem.c:607
        mem = <value optimized out>
#3  0xffffffff80a88a49 in __alloc_bootmem (size=1, align=1, goal=0)
    at /home/johannes/sys/wireless-testing/mm/bootmem.c:627
No locals.
#4  0xffffffff80a8daa3 in alloc_bootmem_cpumask_var (mask=0xffffffff8097dda8)
    at /home/johannes/sys/wireless-testing/lib/cpumask.c:161
No locals.
#5  0xffffffff80a87804 in early_irq_init ()
    at /home/johannes/sys/wireless-testing/include/linux/irq.h:442
        i = 2978
#6  0xffffffff80a71bda in start_kernel ()
    at /home/johannes/sys/wireless-testing/init/main.c:604
        command_line = 0xffffffff80a9ed00 "root=/dev/hda"
#7  0xffffffff80a71299 in x86_64_start_reservations (
    real_mode_data=<value optimized out>)
    at /home/johannes/sys/wireless-testing/arch/x86/kernel/head64.c:123
No locals.
#8  0xffffffff80a713be in x86_64_start_kernel (real_mode_data=
    0x13050 <Address 0x13050 out of bounds>)
    at /home/johannes/sys/wireless-testing/arch/x86/kernel/head64.c:94
No locals.
#9  0x0000000000000000 in ?? ()
No symbol table info available.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: BUG: using smp_processor_id() in preemptible

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 710 bytes --]

On Mon, 2009-06-29 at 11:54 +0200, Johannes Berg wrote:
> On Mon, 2009-06-29 at 12:08 +0300, Avi Kivity wrote:
> 
> > >> You can see where it hangs using the monitor 'info registers' and 'x/30i
> > >> $eip' commands.
> > >>      
> > >
> > > not much luck since it doesn't hang at a specific instruction:
> > >    
> > 
> > You can try mapping these with gdb (in fact, you can have gdb connect to 
> > qemu and do source level debugging).
> 
> It's actually panic'ed because it ran out of memory, but didn't print
> that to my vga/console/curses/...

And if I add -m 512 it works just fine. Seems the default is too small
for me... But why would qemu/kvm have different defaults?

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: BUG: using smp_processor_id() in preemptible

[Avi Kivity]

On 06/29/2009 12:57 PM, Johannes Berg wrote:
>> It's actually panic'ed because it ran out of memory, but didn't print
>> that to my vga/console/curses/...
>>      
>
> And if I add -m 512 it works just fine. Seems the default is too small
> for me... But why would qemu/kvm have different defaults?
>    

Both are 128 MB.  Strange.  And certainly Linux should boot in 128 MB.

-- 
error compiling committee.c: too many arguments to function

Re: BUG: using smp_processor_id() in preemptible

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 1168 bytes --]

On Mon, 2009-06-29 at 13:00 +0300, Avi Kivity wrote:
> On 06/29/2009 12:57 PM, Johannes Berg wrote:
> >> It's actually panic'ed because it ran out of memory, but didn't print
> >> that to my vga/console/curses/...
> >>      
> >
> > And if I add -m 512 it works just fine. Seems the default is too small
> > for me... But why would qemu/kvm have different defaults?
> >    
> 
> Both are 128 MB.  Strange.  And certainly Linux should boot in 128 MB.

If I boot with -m 512 I get this:
root@(none):~# free
             total       used       free     shared    buffers     cached
Mem:        378608      25196     353412          0       1012       8692
-/+ buffers/cache:      15492     363116
Swap:            0          0          0


so something is definitely eating a lot of memory (missing about 142M).
Same guest kernel/fs in qemu gets:
freeroot@(none):~# free
             total       used       free     shared    buffers     cached
Mem:        407080      23236     383844          0       1056       8692
-/+ buffers/cache:      13488     393592
Swap:            0          0          0

so it's missing about 114M.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: BUG: using smp_processor_id() in preemptible

[Avi Kivity]

On 06/29/2009 01:06 PM, Johannes Berg wrote:
>>> And if I add -m 512 it works just fine. Seems the default is too small
>>> for me... But why would qemu/kvm have different defaults?
>>>
>>>        
>> Both are 128 MB.  Strange.  And certainly Linux should boot in 128 MB.
>>      
>
> If I boot with -m 512 I get this:
> root@(none):~# free
>               total       used       free     shared    buffers     cached
> Mem:        378608      25196     353412          0       1012       8692
> -/+ buffers/cache:      15492     363116
> Swap:            0          0          0
>
>
> so something is definitely eating a lot of memory (missing about 142M).
> Same guest kernel/fs in qemu gets:
> freeroot@(none):~# free
>               total       used       free     shared    buffers     cached
> Mem:        407080      23236     383844          0       1056       8692
> -/+ buffers/cache:      13488     393592
> Swap:            0          0          0
>
> so it's missing about 114M.
>    

I get 500MB with 2.6.31-rc1+.  What does dmesg say about the e820 map?

-- 
error compiling committee.c: too many arguments to function

Re: BUG: using smp_processor_id() in preemptible

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 1524 bytes --]

On Mon, 2009-06-29 at 13:16 +0300, Avi Kivity wrote:

> > If I boot with -m 512 I get this:
> > root@(none):~# free
> >               total       used       free     shared    buffers     cached
> > Mem:        378608      25196     353412          0       1012       8692
> > -/+ buffers/cache:      15492     363116
> > Swap:            0          0          0
> >
> >
> > so something is definitely eating a lot of memory (missing about 142M).
> > Same guest kernel/fs in qemu gets:
> > freeroot@(none):~# free
> >               total       used       free     shared    buffers     cached
> > Mem:        407080      23236     383844          0       1056       8692
> > -/+ buffers/cache:      13488     393592
> > Swap:            0          0          0
> >
> > so it's missing about 114M.
> >    
> 
> I get 500MB with 2.6.31-rc1+.  

So even my qemu is missing a lot more than you are. But I guess my
kernel might also be a lot larger.

> What does dmesg say about the e820 map?

[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  BIOS-e820: 0000000000000000 - 000000000009f000 (usable)
[    0.000000]  BIOS-e820: 000000000009f000 - 00000000000a0000 (reserved)
[    0.000000]  BIOS-e820: 00000000000e8000 - 0000000000100000 (reserved)
[    0.000000]  BIOS-e820: 0000000000100000 - 000000000fff0000 (usable)
[    0.000000]  BIOS-e820: 000000000fff0000 - 0000000010000000 (ACPI data)
[    0.000000]  BIOS-e820: 00000000fffbc000 - 0000000100000000 (reserved)

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: BUG: using smp_processor_id() in preemptible

[Avi Kivity]

On 06/29/2009 01:18 PM, Johannes Berg wrote:
>>> If I boot with -m 512 I get this:
>>> root@(none):~# free
>>>                total       used       free     shared    buffers     cached
>>> Mem:        378608      25196     353412          0       1012       8692
>>> -/+ buffers/cache:      15492     363116
>>> Swap:            0          0          0
>>>
>>>
>>> so something is definitely eating a lot of memory (missing about 142M).
>>> Same guest kernel/fs in qemu gets:
>>> freeroot@(none):~# free
>>>                total       used       free     shared    buffers     cached
>>> Mem:        407080      23236     383844          0       1056       8692
>>> -/+ buffers/cache:      13488     393592
>>> Swap:            0          0          0
>>>
>>> so it's missing about 114M.
>>>
>>>        
>> I get 500MB with 2.6.31-rc1+.
>>      
>
> So even my qemu is missing a lot more than you are. But I guess my
> kernel might also be a lot larger.
>    

Aha.  Maybe paravirt patching allocates a lot of memory?  Otherwise 
there should be no difference between qemu-kvm and qemu.

Do you really have a 100MB kernel?

>> What does dmesg say about the e820 map?
>>      
>
> [    0.000000] BIOS-provided physical RAM map:
> [    0.000000]  BIOS-e820: 0000000000000000 - 000000000009f000 (usable)
> [    0.000000]  BIOS-e820: 000000000009f000 - 00000000000a0000 (reserved)
> [    0.000000]  BIOS-e820: 00000000000e8000 - 0000000000100000 (reserved)
> [    0.000000]  BIOS-e820: 0000000000100000 - 000000000fff0000 (usable)
> [    0.000000]  BIOS-e820: 000000000fff0000 - 0000000010000000 (ACPI data)
> [    0.000000]  BIOS-e820: 00000000fffbc000 - 0000000100000000 (reserved)
>    

That's only 256MB (0xfff0000).

-- 
error compiling committee.c: too many arguments to function

Re: BUG: using smp_processor_id() in preemptible

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 1871 bytes --]

On Mon, 2009-06-29 at 13:25 +0300, Avi Kivity wrote:

> > So even my qemu is missing a lot more than you are. But I guess my
> > kernel might also be a lot larger.
> >    
> 
> Aha.  Maybe paravirt patching allocates a lot of memory?  Otherwise 
> there should be no difference between qemu-kvm and qemu.

No idea.

> Do you really have a 100MB kernel?

Hmm. Maybe?

   text	   data	    bss		    dec		    hex	filename
5635486	5482828	84287488	95405802	5afc6ea	vmlinux
-rwxr-xr-x 1 johannes johannes 86242879 2009-06-29 11:31 vmlinux
-rwxr-xr-x 1 johannes johannes 83M 2009-06-29 11:31 vmlinux

But that's with complete debug info.

> >> What does dmesg say about the e820 map?
> >>      
> >
> > [    0.000000] BIOS-provided physical RAM map:
> > [    0.000000]  BIOS-e820: 0000000000000000 - 000000000009f000 (usable)
> > [    0.000000]  BIOS-e820: 000000000009f000 - 00000000000a0000 (reserved)
> > [    0.000000]  BIOS-e820: 00000000000e8000 - 0000000000100000 (reserved)
> > [    0.000000]  BIOS-e820: 0000000000100000 - 000000000fff0000 (usable)
> > [    0.000000]  BIOS-e820: 000000000fff0000 - 0000000010000000 (ACPI data)
> > [    0.000000]  BIOS-e820: 00000000fffbc000 - 0000000100000000 (reserved)
> >    
> 
> That's only 256MB (0xfff0000).

Oh, yes, sorry, I booted with 256 now since that worked too. 512 yields
[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  BIOS-e820: 0000000000000000 - 000000000009f000 (usable)
[    0.000000]  BIOS-e820: 000000000009f000 - 00000000000a0000 (reserved)
[    0.000000]  BIOS-e820: 00000000000e8000 - 0000000000100000 (reserved)
[    0.000000]  BIOS-e820: 0000000000100000 - 000000001fff0000 (usable)
[    0.000000]  BIOS-e820: 000000001fff0000 - 0000000020000000 (ACPI data)
[    0.000000]  BIOS-e820: 00000000fffbc000 - 0000000100000000 (reserved)


johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: BUG: using smp_processor_id() in preemptible

[Avi Kivity]

On 06/29/2009 01:32 PM, Johannes Berg wrote:
>
>> Do you really have a 100MB kernel?
>>      
>
> Hmm. Maybe?
>
>     text	   data	    bss		    dec		    hex	filename
> 5635486	5482828	84287488	95405802	5afc6ea	vmlinux
> -rwxr-xr-x 1 johannes johannes 86242879 2009-06-29 11:31 vmlinux
> -rwxr-xr-x 1 johannes johannes 83M 2009-06-29 11:31 vmlinux
>
> But that's with complete debug info.
>    

debug info is not loaded IIUC.  But you have 84MB of BSS, that looks 
very broken.  Maybe you have a zillion cpus configured, though that 
should be dynamic now.


-- 
error compiling committee.c: too many arguments to function

Re: BUG: using smp_processor_id() in preemptible

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 886 bytes --]

On Mon, 2009-06-29 at 13:39 +0300, Avi Kivity wrote:
> On 06/29/2009 01:32 PM, Johannes Berg wrote:
> >
> >> Do you really have a 100MB kernel?
> >>      
> >
> > Hmm. Maybe?
> >
> >     text	   data	    bss		    dec		    hex	filename
> > 5635486	5482828	84287488	95405802	5afc6ea	vmlinux
> > -rwxr-xr-x 1 johannes johannes 86242879 2009-06-29 11:31 vmlinux
> > -rwxr-xr-x 1 johannes johannes 83M 2009-06-29 11:31 vmlinux
> >
> > But that's with complete debug info.
> >    
> 
> debug info is not loaded IIUC.  But you have 84MB of BSS, that looks 
> very broken.  Maybe you have a zillion cpus configured, though that 
> should be dynamic now.

I had MAXSMP configured, which makes it 4096, but if I turn off MAXSMP
and make NR_CPUS 8, then I get
   text	   data	    bss	   	 dec		    hex	filename
5594521	2562924	12726272	20883717	13ea905	vmlinux

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: BUG: using smp_processor_id() in preemptible

[Avi Kivity]

On 06/29/2009 01:55 PM, Johannes Berg wrote:
> I had MAXSMP configured, which makes it 4096, but if I turn off MAXSMP
> and make NR_CPUS 8, then I get
>     text	   data	    bss	   	 dec		    hex	filename
> 5594521	2562924	12726272	20883717	13ea905	vmlinux
>    

12MB bss is still pretty bloated, but I guess that's beyond the scope of 
this thread.

-- 
error compiling committee.c: too many arguments to function

Re: BUG: using smp_processor_id() in preemptible

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 553 bytes --]

On Mon, 2009-06-29 at 14:38 +0300, Avi Kivity wrote:
> On 06/29/2009 01:55 PM, Johannes Berg wrote:
> > I had MAXSMP configured, which makes it 4096, but if I turn off MAXSMP
> > and make NR_CPUS 8, then I get
> >     text	   data	    bss	   	 dec		    hex	filename
> > 5594521	2562924	12726272	20883717	13ea905	vmlinux
> >    
> 
> 12MB bss is still pretty bloated, 

Yeah, should probably check where that's coming from.

> but I guess that's beyond the scope of 
> this thread.

Indeed. In any case, thanks for your help.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: BUG: using smp_processor_id() in preemptible

[Avi Kivity]

On 06/29/2009 12:54 PM, Johannes Berg wrote:
> On Mon, 2009-06-29 at 12:08 +0300, Avi Kivity wrote:
>
>    
>>>> You can see where it hangs using the monitor 'info registers' and 'x/30i
>>>> $eip' commands.
>>>>
>>>>          
>>> not much luck since it doesn't hang at a specific instruction:
>>>
>>>        
>> You can try mapping these with gdb (in fact, you can have gdb connect to
>> qemu and do source level debugging).
>>      
>
> It's actually panic'ed because it ran out of memory, but didn't print
> that to my vga/console/curses/...
>    

Maybe we give it crappy numa table, try booting with numa disabled.  
Also make sure you boot with the bios provided by qemu-kvm, not some 
random qemu.

-- 
error compiling committee.c: too many arguments to function

Re: BUG: using smp_processor_id() in preemptible

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 477 bytes --]

On Mon, 2009-06-29 at 12:59 +0300, Avi Kivity wrote:

> > It's actually panic'ed because it ran out of memory, but didn't print
> > that to my vga/console/curses/...
> >    
> 
> Maybe we give it crappy numa table, try booting with numa disabled.  

# CONFIG_NUMA is not set   

> Also make sure you boot with the bios provided by qemu-kvm, not some 
> random qemu.

I, uh, have no idea how to ensure that -- I'm simply using the debian
kvm package.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: BUG: using smp_processor_id() in preemptible

[Avi Kivity]

On 06/29/2009 01:00 PM, Johannes Berg wrote:

>> Also make sure you boot with the bios provided by qemu-kvm, not some
>> random qemu.
>>      
>
> I, uh, have no idea how to ensure that -- I'm simply using the debian
> kvm package.
>    

That should be a good way to ensure it.  What version is it?

-- 
error compiling committee.c: too many arguments to function

Re: BUG: using smp_processor_id() in preemptible

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 421 bytes --]

On Mon, 2009-06-29 at 13:04 +0300, Avi Kivity wrote:
> On 06/29/2009 01:00 PM, Johannes Berg wrote:
> 
> >> Also make sure you boot with the bios provided by qemu-kvm, not some
> >> random qemu.
> >>      
> >
> > I, uh, have no idea how to ensure that -- I'm simply using the debian
> > kvm package.
> >    
> 
> That should be a good way to ensure it.  What version is it?

Version: 85+dfsg-4

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 801 bytes --]