From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36419) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WiVNR-0007of-HH for qemu-devel@nongnu.org; Thu, 08 May 2014 16:57:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1WiVNJ-0004X0-8B for qemu-devel@nongnu.org; Thu, 08 May 2014 16:57:13 -0400 Received: from mail.windriver.com ([147.11.1.11]:50017) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1WiVNI-0004Wt-Ug for qemu-devel@nongnu.org; Thu, 08 May 2014 16:57:05 -0400 Message-ID: <536BEF9D.1020806@windriver.com> Date: Thu, 8 May 2014 14:57:01 -0600 From: Chris Friesen MIME-Version: 1.0 References: <53693FA4.3000306@windriver.com> <5369D504.2070101@redhat.com> <536AB2EE.6010504@windriver.com> <20140508130231.GA24978@grmbl.mre> <536B8332.9070705@redhat.com> <20140508133052.GA25602@grmbl.mre> <536B9524.8070307@windriver.com> <536B95E3.3010808@redhat.com> <536BA95C.8070306@windriver.com> <536BAA86.2090109@redhat.com> In-Reply-To: <536BAA86.2090109@redhat.com> Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] virtio-serial-pci very expensive during live migration List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Paolo Bonzini , Amit Shah Cc: qemu-devel@nongnu.org, armbru@redhat.com On 05/08/2014 10:02 AM, Paolo Bonzini wrote: > Il 08/05/2014 17:57, Chris Friesen ha scritto: >>>> >>>> The fact remains that qemu crashes when I apply the patch. I also >>>> tried >>>> patching it as below in virtio_pci_vmstate_change(). That would allow >>>> the VM to boot, but it would crash when I tried to do a live migration. >>> >>> Can you give us your command line and a backtrace? >> >> For the backtrace, you mean with a core file? Or is there a better way? > > You can attach gdb to the qemu process, and run "thread apply all bt" > when QEMU crashes. For the commandline, it's coming from OpenStack via libvirt, so it's fairly complicated: /usr/bin/kvm -c 0x00000000000000000000000000000001 -n 4 --proc-type=secondary --file-prefix=vs -- -enable-dpdk -name instance-00000007 -S -machine pc-i440fx-1.4,accel=kvm,usb=off -m 512 -mem-prealloc -mem-path /mnt/huge-2048kB/libvirt/qemu -smp 2,sockets=2,cores=1,threads=1 -uuid b277d785-9c31-45be-91d3-15b7b1aca974 -smbios type=1,manufacturer=OpenStack Foundation,product=OpenStack Nova,version=2013.2.2,serial=00000000-0000-3848-3342-4e5037313536,uuid=b277d785-9c31-45be-91d3-15b7b1aca974 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/instance-00000007.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -no-kvm-pit-reinjection -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x3 -drive file=/etc/nova/instances/b277d785-9c31-45be-91d3-15b7b1aca974/disk,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -chardev file,id=charserial0,path=/etc/nova/instances/b277d785-9c31-45be-91d3-15b7b1aca974/console.log -device isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/cgcs.heartbeat.instance-00000007.sock,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=cgcs.heartbeat -device usb-tablet,id=input0 -vnc 0.0.0.0:0 -k en-us -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -device avp,port=4a05cedd-3868-449a-9728-899bbea10863,id=avp4a05cedd-38 As for the backtrace, I can attach gdb to the qemu process, but when I hit "c" to continue it gives an error: Reading symbols from /lib64/libnss_files.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libnss_files.so.2 warning: File "/lib64/libthread_db-1.0.so" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load". warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available. 0x00007f56c6366d23 in select () from /lib64/libc.so.6 (gdb) c Continuing. Warning: Cannot insert breakpoint -1. Error accessing memory address 0x2f7c50: Input/output error. Chris