Re: FreeBSD-amd64 fails to start with SMP on quemu-kvm

["Michael S. Tsirkin" <mst@redhat.com>]

On Wed, Jan 16, 2013 at 03:05:50PM +0200, Gleb Natapov wrote:
> 
> Copying Michael.
> 
> On Tue, Jan 15, 2013 at 12:18:59PM +0100, Artur Samborski wrote:
> > W dniu 13.01.2013 10:23, Gleb Natapov pisze:
> > >On Fri, Jan 11, 2013 at 03:57:05PM +0100, Artur Samborski wrote:
> > >>W dniu 09.01.2013 11:15, Gleb Natapov pisze:
> > >>>On Tue, Jan 08, 2013 at 09:45:35PM +0100, Artur Samborski wrote:
> > >>>>W dniu 08.01.2013 00:00, Marcelo Tosatti pisze:
> > >>>>>On Mon, Jan 07, 2013 at 06:13:22PM +0100, Artur Samborski wrote:
> > >>>>>>Hello,
> > >>>>>>
> > >>>>>>When i try to run FreeBSD-amd64 on more than 1 vcpu in quemu-kvm
> > >>>>>>(Fedora Core 17) eg. to run FreeBSD-9.0-RELEASE-amd64 with:
> > >>>>>>
> > >>>>>>qemu-kvm -m 1024m -cpu host -smp 2 -cdrom
> > >>>>>>/storage/iso/FreeBSD-9.0-RELEASE-amd64-dvd1.iso
> > >>>>>>
> > >>>>>>it freezes KVM with:
> > >>>>>>
> > >>>>>>KVM internal error. Suberror: 1
> > >>>>>>emulation failure
> > >>>>>>RAX=ffffffff80b0d4c0 RBX=000000000009f000 RCX=00000000c0000080
> > >>>>>>RDX=0000000000000000
> > >>>>>>RSI=000000000000d238 RDI=0000000000000000 RBP=0000000000000000
> > >>>>>>RSP=0000000000000000
> > >>>>>>R8 =0000000000000000 R9 =0000000000000000 R10=0000000000000000
> > >>>>>>R11=0000000000000000
> > >>>>>>R12=0000000000000000 R13=0000000000000000 R14=0000000000000000
> > >>>>>>R15=0000000000000000
> > >>>>>>RIP=000000000009f076 RFL=00010086 [--S--P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
> > >>>>>>ES =0000 0000000000000000 0000ffff 0000f300 DPL=3 DS16 [-WA]
> > >>>>>>CS =0008 0000000000000000 00000000 00209900 DPL=0 CS64 [--A]
> > >>>>>>SS =9f00 000000000009f000 0000ffff 0000f300 DPL=3 DS16 [-WA]
> > >>>>>>DS =0018 0000000000000000 ffffffff 00c09300 DPL=0 DS   [-WA]
> > >>>>>>FS =0000 0000000000000000 0000ffff 0000f300 DPL=3 DS16 [-WA]
> > >>>>>>GS =0000 0000000000000000 0000ffff 0000f300 DPL=3 DS16 [-WA]
> > >>>>>>LDT=0000 0000000000000000 0000ffff 00008200 DPL=0 LDT
> > >>>>>>TR =0000 0000000000000000 0000ffff 00008b00 DPL=0 TSS64-busy
> > >>>>>>GDT=     000000000009f080 00000020
> > >>>>>>IDT=     0000000000000000 0000ffff
> > >>>>>>CR0=80000011 CR2=0000000000000000 CR3=000000000009c000 CR4=00000030
> > >>>>>>DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000
> > >>>>>>DR3=0000000000000000
> > >>>>>>DR6=00000000ffff0ff0 DR7=0000000000000400
> > >>>>>>EFER=0000000000000501
> > >>>>>>Code=00 00 00 80 0f 22 c0 ea 70 f0 09 00 08 00 48 b8 c0 d4 b0 80
> > >>>>>><ff> ff ff ff ff e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > >>>>>>00 99 20 00 ff ff 00 00
> > >>>>>
> > >>>>>Artur,
> > >>>>>
> > >>>>>Can you check whether
> > >>>>>
> > >>>>>https://patchwork-mail.kernel.org/patch/1942681/
> > >>>>>
> > >>>>>fixes your problem
> > >>>>>
> > >>>>
> > >>>>Hi, thanks for the reply.
> > >>>>
> > >>>>Unfortunately, the patch does not help. Attempt to start FreeBSD
> > >>>>amd64 via quemu-kvm with -smp parameter greater than 1 fails in
> > >>>>exactly the same way as before.
> > >>>>
> > >>>>The patch was applied to the kernel from the 3.6.11-1.fc17.src.rpm package.
> > >>>>
> > >>>>Do I need some additional patches?
> > >>>>
> > >>>>
> > >>>Can you try queue branch from kvm.git?
> > >>>
> > >>
> > >>Thanks for the advice - I used kernel from kvm.git(queue) and was
> > >>able to run FreeBSD-amd guest with SMP on my kvm-host server without
> > >>previous problems. Unfortunately, after few hours server was hung
> > >>up. Case requires further investigations, but generally speaking we
> > >>went forward. Unfortunately, experiments on the server are
> > >>difficult, because it is used for everyday work.
> > >>
> > >Thanks for testing. This hang indeed looks like completely different
> > >problem. What workload the VM was running?
> > >
> > 
> > First test:
> > 
> >  - kvm.git kernel
> >  - 2 kvm guest running:
> >    - Linux (in idle)
> >    - Freebsd-amd64 (high load, about 7 -- continuous FreeBSD world
> > and kernel build)
> >  - KVM host hangs after about 5 hours
> >  - nothing special in system logs
> >  - message caught on one of the active SSH session:
> > 
> > kernel:[24742.127690] BUG: soft lockup - CPU#2 stuck for 22s!
> > [vhost-3686:3700]
> > 
> > Second test:
> > 
> >  - kvm.git kernel
> >  - 1 kvm guest running:
> >    - Linux (at the time of hang -- in idle)
> >    - about 10 minutes before KVM host hangs -- load about 6 (kernel build)
> >  - in system logs:
> > 
> > BUG: soft lockup - CPU#0 stuck for 22s! [vhost-1771:1800]
> > Modules linked in: binfmt_misc ip6table_filter ip6_tables
> > ebtable_nat ebtables lockd sunrpc nf_conntrack_ipv4 nf_defrag_ipv4
> > xt_state nf_conntrack xt_CHECKSUM iptable_mangle bridge stp llc
> > be2iscsi iscsi_boot_sysfs bnx2i cnic uio cxgb4i cxgb4 cxgb3i cxgb3
> > mdio libcxgbi ib_iser bnep bluetooth rfkill rdma_cm ib_addr iw_cm
> > ib_cm ib_sa ib_mad ib_core iscsi_tcp libiscsi_tcp libiscsi
> > scsi_transport_iscsi ioatdma vhost_net iTCO_wdt iTCO_vendor_support
> > ses lpc_ich tun macvtap macvlan mfd_core enclosure bnx2 joydev
> > i7core_edac dca edac_core wmi coretemp dcdbas kvm_intel pcspkr
> > crc32c_intel kvm serio_raw acpi_power_meter microcode uinput
> > ipmi_devintf ipmi_si ipmi_msghandler megaraid_sas
> > CPU 0
> > Pid: 1800, comm: vhost-1771 Not tainted 3.7.0+ #2 Dell Inc.
> > PowerEdge R610/086HF8
> > RIP: 0010:[<ffffffff8152876f>]  [<ffffffff8152876f>]
> > skb_flow_dissect+0xbf/0x3e0
> > RSP: 0018:ffff88042145dbd8  EFLAGS: 00000246
> > RAX: 0000000000000000 RBX: ffff8807fa489c00 RCX: f7ab0c277df5b6fd
> > RDX: ffff880820c59800 RSI: ffff88042145dc58 RDI: ffff8807fa489c00
> > RBP: ffff88042145dc48 R08: 0000000000000404 R09: 0000000000000412
> > R10: ffff8807fa489c00 R11: 0000000000000412 R12: ffffffff81522a57
> > R13: 0000000000000000 R14: ffffffff81521fdc R15: ffff88042145db78
> > FS:  0000000000000000(0000) GS:ffff88083fc00000(0000) knlGS:0000000000000000
> > CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> > CR2: 0000000001c0e9f4 CR3: 0000000827973000 CR4: 00000000000027e0
> > DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> > DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> > Process vhost-1771 (pid: 1800, threadinfo ffff88042145c000, task
> > ffff880421151740)
> > Stack:
> >  ffff88042145dc78 0000000000000412 0000000000000000 0000000000000412
> >  000000000000000c ffff880422570800 ffff88042145dc78 ffffffff81525881
> >  0000000000000000 00000000de057f32 ffff8807fa489c00 0000000000000412
> > Call Trace:
> >  [<ffffffff81525881>] ? skb_copy_datagram_from_iovec+0x61/0x280
> >  [<ffffffff8152a06a>] __skb_get_rxhash+0x1a/0xd0
> >  [<ffffffffa01109e0>] tun_get_user+0x3e0/0x760 [tun]
> >  [<ffffffffa0110dba>] tun_sendmsg+0x5a/0x80 [tun]
> >  [<ffffffffa013cd2a>] handle_tx+0x28a/0x680 [vhost_net]
> >  [<ffffffffa013d155>] handle_tx_kick+0x15/0x20 [vhost_net]
> >  [<ffffffffa013995d>] vhost_worker+0xed/0x190 [vhost_net]
> >  [<ffffffffa0139870>] ? vhost_work_flush+0x110/0x110 [vhost_net]
> >  [<ffffffff81081750>] kthread+0xc0/0xd0
> >  [<ffffffff81010000>] ? ftrace_define_fields_xen_mc_entry+0x50/0xf0
> >  [<ffffffff81081690>] ? kthread_create_on_node+0x120/0x120
> >  [<ffffffff8163feac>] ret_from_fork+0x7c/0xb0
> >  [<ffffffff81081690>] ? kthread_create_on_node+0x120/0x120
> > Code: 68 41 2b 44 24 6c 29 d8 83 f8 13 0f 8e eb 00 00 00 48 63 d3 49
> > 03 94 24 e8 00 00 00 48 85 d2 74 b0 31 c0 66 f7 42 06 3f ff 75 04
> > <0f> b6 42 09 48 8b 4a 0c 49 89 0e 0f b6 12 83 e2 0f 8d 1c 93 eb
> > 
> > repeated several times with similar content
> > 
> > -- 
> > Artur


Looks like this?
https://git.kernel.org/?p=linux/kernel/git/davem/net.git;a=commit;h=76fe45812a3b134c39170ca32dfd4b7217d33145

> --
> 			Gleb.