[Qemu-devel] kqemu freebsd host smp problems?

[Qemu-devel] kqemu freebsd host smp problems?

[Juergen Lock]

Hi!

 Is kqemu and the freebsd wrapper smp aware?  I just saw this panic
report again,
	http://lists.freebsd.org/pipermail/freebsd-current/2005-May/050161.html
and noticed it apparently happened with an smp kernel.

 Curious...
	Juergen

Re: [Qemu-devel] kqemu freebsd host smp problems?

[Bakul Shah]

Lock writes:
>  Is kqemu and the freebsd wrapper smp aware?  I just saw this panic
> report again,
> 	http://lists.freebsd.org/pipermail/freebsd-current/2005-May/050161.html
> and noticed it apparently happened with an smp kernel.

My guess is

	.d_flags = D_NEEDGIANT,

needs to be added to the initializer of kqemu_cdevsw for the
freebsd-current case.  AFAIK this flag ensures only one
thread can be in this driver at a time (but caveat emptor: I
don't play in the kernel these days).

Re: [Qemu-devel] kqemu freebsd host smp problems?

[Norikatsu Shigemura]

On Sun, 03 Jul 2005 17:37:42 -0700
Bakul Shah <bakul@BitBlocks.com> wrote:
> Lock writes:
> >  Is kqemu and the freebsd wrapper smp aware?  I just saw this panic
> > report again,
> > 	http://lists.freebsd.org/pipermail/freebsd-current/2005-May/050161.html
> > and noticed it apparently happened with an smp kernel.
> My guess is
> 	.d_flags = D_NEEDGIANT,
> needs to be added to the initializer of kqemu_cdevsw for the
> freebsd-current case.  AFAIK this flag ensures only one
> thread can be in this driver at a time (but caveat emptor: I
> don't play in the kernel these days).

	I confirmed that qemu on latest FreeBSD 6-current got more
	stability!!, but more little slowly:-( and a panic:-( too.


	Now I'm testing improved qemu port:
	http://tmp.ninth-nine.com/qemu/qemu.20050708-2.port.tar.bz2

	1. Merge /dev/kqemu cloning support to kmod_bsd.c.
	   Obtained from:	http://lists.gnu.org/archive/html/qemu-devel/2005-06/msg00135.html
	   Submitted by:	Craig Boston <craig@xfoil.gank.org>

	> $ fstat /dev/kqemu*
	> USER     CMD          PID   FD MOUNT      INUM MODE         SZ|DV R/W NAME
	> nork     qemu       33805    5 /dev        168 crw-rw----  #C:0:0x0 rw  /dev/kqemu1
	> root     qemu       20779    6 /dev        152 crw-rw----  #C:0:0x0 rw  /dev/kqemu0
	   In this time, I'm installing Windows XP SP2 and FreeBSD 5.4-R.

	2. Giant-lock kqemu.ko.
	   Obtained from:	http://lists.gnu.org/archive/html/qemu-devel/2005-07/msg00070.html
	   Suggested by:	Bakul Shah <bakul@BitBlocks.com>

	3. Add experimental IDE WDMA support.
	   Obtained from:	I forgot:-(
	   Submitted by(AFAIK):	Juergen Lock <qemu-l@jelal.kn-bremen.de>

	4. Utilize BSDMakefile to compile kqemu.ko, and cosmetic change.


	I contacted a panic.  Please check following message.

PANIC MESSAGE ON CONSOLE:
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
FreeBSD/i386 (nadesico.ninth-nine.com) (dcons)

login: info: [drm] Loading R200 Microcode
info: [drm] Loading R200 Microcode
panic: vm_page_insert: page already inserted
cpuid = 0
KDB: enter: panic
[thread pid 49875 tid 100270 ]
Stopped at      kdb_enter+0x30: leave
db> tr
Tracing pid 49875 tid 100270 td 0xc3374480
kdb_enter(c06672e3,0,c067582a,efc55870,c50b1b60) at kdb_enter+0x30
panic(c067582a,c3374480,2,c1a99178,2) at panic+0x14e
vm_page_insert(c1a99178,c60e19cc,5d2c,0,c11d3030) at vm_page_insert+0x2a
vm_page_alloc(c60e19cc,5d2c,0,222,d65549d8) at vm_page_alloc+0x348
allocbuf(d65549d8,4000,0,4000,3863b048) at allocbuf+0x721
getblk(c50b1aa0,174b,0,4000,0) at getblk+0x5ad
ffs_balloc_ufs2(c50b1aa0,5d2c000,0,1000,c345d300) at ffs_balloc_ufs2+0x1b67
ffs_write(efc55be8,efc55b94,4,0,0) at ffs_write+0x389
VOP_WRITE_APV(c069ade0,efc55be8,c3374480,c2325c00,0) at VOP_WRITE_APV+0xec
vn_write(c3259a20,efc55cb0,c345d300,0,c3374480) at vn_write+0x240
dofilewrite(c3374480,4,c3259a20,efc55cb0,ffffffff) at dofilewrite+0x85
kern_writev(c3374480,4,efc55cb0,804e000,1000) at kern_writev+0x65
write(c3374480,efc55d04,c,c3374480,efc55d2c) at write+0x4f
syscall(3b,3b,bfbf003b,1000,682b2258) at syscall+0x370
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (4, FreeBSD ELF32, write), eip = 0x682148bf, esp = 0xbfbfd8fc, ebp =
 0xbfbfd918 ---
db> show pcpu 0
cpuid        = 0
curthread    = 0xc3374480: pid 49875 "fetch"
curpcb       = 0xefc55d90
fpcurthread  = none
idlethread   = 0xc22d7900: pid 12 "idle: cpu0"
APIC ID      = 0
currentldt   = 0x50
db> show pcpu 1
cpuid        = 1
curthread    = 0xc328e900: pid 20779 "qemu"
curpcb       = 0xefbbfd90
fpcurthread  = none
idlethread   = 0xc22d7780: pid 11 "idle: cpu1"
APIC ID      = 1
currentldt   = 0x50
db> call doadump()
Dumping 1023 MB (2 chunks)
  chunk 0: 1MB (159 pages) ... ok
  chunk 1: 1023MB (261872 pages) 1007 991 975 959 943 927 911 895 879 863 847 831 815 799 783 767 751 735 719 703 687 671 655 639 623 607 591 575 559 543 527 511 495 479 463 447 431 415 399 383 367 351 335 319 303 287 271 255 239 223 207 191 175 159 143 127 111 95 79 63 47 31 15 ... ok

Dump complete
= 0xf
db> reset
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -


# kgdb /boot/kernel/kernel.debug /var/crash/vmcore.0
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(snip)
(kgdb) where
#0  doadump () at pcpu.h:165
#1  0xc0431b66 in db_fncall (dummy1=-272279900, dummy2=0, dummy3=9600, 
    dummy4=0xefc55688 "`\004p潜\003") at /usr/src/sys/ddb/db_command.c:489
#2  0xc04318e2 in db_command (last_cmdp=0xc06a85e4, cmd_table=0x0, 
    aux_cmd_tablep=0xc067cb40, aux_cmd_tablep_end=0xc067cb44)
    at /usr/src/sys/ddb/db_command.c:349
#3  0xc04319f5 in db_command_loop () at /usr/src/sys/ddb/db_command.c:455
#4  0xc0433ba5 in db_trap (type=3, code=0) at /usr/src/sys/ddb/db_main.c:221
#5  0xc04d5ece in kdb_trap (type=0, code=0, tf=0xefc557e8)
    at /usr/src/sys/kern/subr_kdb.c:473
#6  0xc06434c8 in trap (frame=
      {tf_fs = 8, tf_es = 40, tf_ds = -272302040, tf_edi = 256, tf_esi = 1, tf_ebp = -272279504, tf_isp = -272279532, tf_ebx = -272279440, tf_edx = 0, tf_ecx = -1056755712, tf_eax = 18, tf_trapno = 3, tf_err = 0, tf_eip = -1068672096, tf_cs = 32, tf_eflags = 642, tf_esp = -1067021478, tf_ss = -1067027741})
    at /usr/src/sys/i386/i386/trap.c:600
#7  0xc062ec5a in calltrap () at /usr/src/sys/i386/i386/exception.s:139
#8  0x00000008 in ?? ()
#9  0x00000028 in ?? ()
#10 0xefc50028 in ?? ()
#11 0x00000100 in ?? ()
#12 0x00000001 in ?? ()
#13 0xefc55830 in ?? ()
#14 0xefc55814 in ?? ()
#15 0xefc55870 in ?? ()
#16 0x00000000 in ?? ()
#17 0xc1033000 in ?? ()
#18 0x00000012 in ?? ()
#19 0x00000003 in ?? ()
#20 0x00000000 in ?? ()
#21 0xc04d5ba0 in kdb_enter (msg=0x0) at cpufunc.h:60
#22 0xc04b553e in panic (
    fmt=0xc067582a "vm_page_insert: page already inserted")
    at /usr/src/sys/kern/kern_shutdown.c:537
#23 0xc05fb8fa in vm_page_insert (m=0xc1a99178, object=0xc60e19cc, pindex=Unhandled dwarf expression opcode 0x93
)
    at /usr/src/sys/vm/vm_page.c:539
#24 0xc05fc018 in vm_page_alloc (object=0xc60e19cc, pindex=23852, req=546)
    at /usr/src/sys/vm/vm_page.c:867
#25 0xc0510241 in allocbuf (bp=0xd65549d8, size=16384)
    at /usr/src/sys/kern/vfs_bio.c:2770
#26 0xc050fa9d in getblk (vp=0xc50b1aa0, blkno=5963, size=16384, slpflag=0, 
    slptimeo=0, flags=0) at /usr/src/sys/kern/vfs_bio.c:2541
#27 0xc05b73e7 in ffs_balloc_ufs2 (vp=0xc50b1aa0, startoffset=Unhandled dwarf expression opcode 0x93
)
    at /usr/src/sys/ufs/ffs/ffs_balloc.c:817
---Type <return> to continue, or q <return> to quit---
#28 0xc05d1b49 in ffs_write (ap=0xefc55be8)
    at /usr/src/sys/ufs/ffs/ffs_vnops.c:662
#29 0xc064f5bc in VOP_WRITE_APV (vop=0xc069ade0, a=0xefc55be8)
    at vnode_if.c:698
#30 0xc0530660 in vn_write (fp=0xc3259a20, uio=0xefc55cb0, 
    active_cred=0xc345d300, flags=0, td=0xc3374480) at vnode_if.h:372
#31 0xc04e29e5 in dofilewrite (td=0xc3374480, fd=0, fp=0xc3259a20, 
    auio=0xefc55cb0, offset=Unhandled dwarf expression opcode 0x93
) at file.h:246
#32 0xc04e2805 in kern_writev (td=0xc3374480, fd=4, auio=0x0)
    at /usr/src/sys/kern/sys_generic.c:402
#33 0xc04e26bf in write (td=0x0, uap=0x0)
    at /usr/src/sys/kern/sys_generic.c:326
#34 0xc0643f80 in syscall (frame=
      {tf_fs = 59, tf_es = 59, tf_ds = -1078001605, tf_edi = 4096, tf_esi = 1747657304, tf_ebp = -1077946088, tf_isp = -272278172, tf_ebx = 1747579940, tf_edx = 134537216, tf_ecx = 134574080, tf_eax = 4, tf_trapno = 0, tf_err = 2, tf_eip = 1747011775, tf_cs = 51, tf_eflags = 534, tf_esp = -1077946116, tf_ss = 59})
    at /usr/src/sys/i386/i386/trap.c:985
#35 0xc062ecaf in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:200
#36 0x0000003b in ?? ()
#37 0x0000003b in ?? ()
#38 0xbfbf003b in ?? ()
#39 0x00001000 in ?? ()
#40 0x682b2258 in ?? ()
#41 0xbfbfd918 in ?? ()
#42 0xefc55d64 in ?? ()
#43 0x6829f424 in ?? ()
#44 0x0804e000 in ?? ()
#45 0x08057000 in ?? ()
#46 0x00000004 in ?? ()
#47 0x00000000 in ?? ()
#48 0x00000002 in ?? ()
#49 0x682148bf in ?? ()
#50 0x00000033 in ?? ()
#51 0x00000216 in ?? ()
#52 0xbfbfd8fc in ?? ()
#53 0x0000003b in ?? ()
#54 0x49806749 in ?? ()
#55 0x67498067 in ?? ()
#56 0x80674980 in ?? ()
#57 0x49806749 in ?? ()
#58 0x19b2c000 in ?? ()
#59 0xc255ea3c in ?? ()
#60 0xc3374480 in ?? ()
---Type <return> to continue, or q <return> to quit---
#61 0xefc55c94 in ?? ()
#62 0xefc55c78 in ?? ()
#63 0xc22d8300 in ?? ()
#64 0xc04cc0c0 in sched_switch (td=0x682b2258, newtd=0x6829f424, flags=Cannot access memory at address 0xbfbfd928
)
    at /usr/src/sys/kern/sched_4bsd.c:973
Previous frame inner to this frame (corrupt stack?)
(kgdb) x/72b 0xc1a99178
0xc1a99178:     0x40    0xab    0x4c    0xc1    0x60    0xf5    0x6f    0xc0
0xc1a99180:     0xc0    0xa1    0xff    0xc1    0x38    0xfd    0xe8    0xc1
0xc1a99188:     0x30    0xfd    0xe8    0xc1    0xc0    0xa1    0xff    0xc1
0xc1a99190:     0x38    0x97    0x70    0xc3    0x1a    0xd4    0x00    0x00
0xc1a99198:     0x00    0x00    0x00    0x00    0x00    0x80    0x27    0x24
0xc1a991a0:     0x01    0x00    0x00    0x00    0x88    0x52    0x22    0xe1
0xc1a991a8:     0x90    0x52    0x22    0xe1    0x00    0x00    0x00    0x00
0xc1a991b0:     0x78    0x00    0x01    0x00    0x00    0x00    0x00    0x00
0xc1a991b8:     0x00    0x00    0x00    0x00    0x00    0xff    0x00    0x00
(kgdb)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Re: [Qemu-devel] kqemu freebsd host smp problems?

[Antony T Curtis]

On Sun, 2005-07-10 at 13:39 +0900, Norikatsu Shigemura wrote:
> On Sun, 03 Jul 2005 17:37:42 -0700
> Bakul Shah <bakul@BitBlocks.com> wrote:
> > Lock writes:
> > >  Is kqemu and the freebsd wrapper smp aware?  I just saw this panic
> > > report again,
> > > 	http://lists.freebsd.org/pipermail/freebsd-current/2005-May/050161.html
> > > and noticed it apparently happened with an smp kernel.
> > My guess is
> > 	.d_flags = D_NEEDGIANT,
> > needs to be added to the initializer of kqemu_cdevsw for the
> > freebsd-current case.  AFAIK this flag ensures only one
> > thread can be in this driver at a time (but caveat emptor: I
> > don't play in the kernel these days).
> 
> 	I confirmed that qemu on latest FreeBSD 6-current got more
> 	stability!!, but more little slowly:-( and a panic:-( too.

IMO, That flag is not the cause of the panics and that it should(tm)
work without requiring GIANT...

I think it is possible that the kqemu code is freeing a page without
unlocking it so that when another process does file IO which requires
pages to be allocated, attempts to wire those pages results in failure
and so a panic occurrs.

Perhaps if a different method for allocating memory rather than
contigmalloc/contigfree should be used by the kernel module.

<snip>

Offtopic - but am I the only person who has modified the if_tap driver
to permit opening by non-superuser?

-- 
Antony T Curtis, BSc.                   UNIX, Linux, *BSD, Networking
antony.t.curtis@ntlworld.com            C++, J2EE, Perl, MySQL, Apache
                                        IT Consultancy.