From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:36192) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1S7Sk0-0006Wk-Jt for qemu-devel@nongnu.org; Tue, 13 Mar 2012 10:30:30 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1S7Sjl-0007lx-CD for qemu-devel@nongnu.org; Tue, 13 Mar 2012 10:30:20 -0400 Received: from mail-lb0-f173.google.com ([209.85.217.173]:48488) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1S7Sjk-0007hh-Tg for qemu-devel@nongnu.org; Tue, 13 Mar 2012 10:30:05 -0400 Received: by lbon3 with SMTP id n3so277129lbo.4 for ; Tue, 13 Mar 2012 07:30:02 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <4F5F34D1.40403@redhat.com> References: <20120313103602.8741.71939.stgit@dhcp-8-167.nay.redhat.com> <4F5F34D1.40403@redhat.com> Date: Tue, 13 Mar 2012 14:30:02 +0000 Message-ID: From: Stefan Hajnoczi Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] [PATCH 0/2] virtio-pci: fix abort when fail to allocate ioeventfd List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Amos Kong Cc: aliguori@us.ibm.com, stefanha@linux.vnet.ibm.com, kvm@vger.kernel.org, "Michael S. Tsirkin" , mtosatti@redhat.com, qemu-devel@nongnu.org, avi@redhat.com On Tue, Mar 13, 2012 at 11:51 AM, Amos Kong wrote: > On 13/03/12 19:23, Stefan Hajnoczi wrote: >> >> On Tue, Mar 13, 2012 at 10:42 AM, Amos Kong =A0wrote: >>> >>> Boot up guest with 232 virtio-blk disk, qemu will abort for fail to >>> allocate ioeventfd. This patchset changes kvm_has_many_ioeventfds(), >>> and check if available ioeventfd exists. If not, virtio-pci will >>> fallback to userspace, and don't use ioeventfd for io notification. > > > Hi Stefan, > > >> Please explain how it fails with 232 devices. =A0Where does it abort and >> why? > > > (gdb) bt > #0 =A00x00007ffff48c8885 in raise () from /lib64/libc.so.6 > #1 =A00x00007ffff48ca065 in abort () from /lib64/libc.so.6 > #2 =A00x00007ffff7e89a3d in kvm_io_ioeventfd_add (section=3D0x7fffbfbf561= 0, > match_data=3Dtrue, data=3D0, fd=3D461) at /home/devel/qemu/kvm-all.c:778 > #3 =A00x00007ffff7e89b3f in kvm_eventfd_add (listener=3D0x7ffff82ebe80, > section=3D0x7fffbfbf5610, match_data=3Dtrue, data=3D0, fd=3D461) at > /home/devel/qemu/kvm-all.c:802 > #4 =A00x00007ffff7e9bcf7 in address_space_add_del_ioeventfds > (as=3D0x7ffff8b278a0, fds_new=3D0x7fffb80106f0, fds_new_nb=3D201, > fds_old=3D0x7fffb800db20, fds_old_nb=3D200) > =A0 =A0at /home/devel/qemu/memory.c:612 > #5 =A00x00007ffff7e9c04f in address_space_update_ioeventfds > (as=3D0x7ffff8b278a0) at /home/devel/qemu/memory.c:645 > #6 =A00x00007ffff7e9caa0 in address_space_update_topology (as=3D0x7ffff8b= 278a0) > at /home/devel/qemu/memory.c:726 > #7 =A00x00007ffff7e9cb95 in memory_region_update_topology (mr=3D0x7fffdeb= 179b0) > at /home/devel/qemu/memory.c:746 > #8 =A00x00007ffff7e9e802 in memory_region_add_eventfd (mr=3D0x7fffdeb179b= 0, > addr=3D16, size=3D2, match_data=3Dtrue, data=3D0, fd=3D461) at > /home/devel/qemu/memory.c:1220 > #9 =A00x00007ffff7d9e832 in virtio_pci_set_host_notifier_internal > (proxy=3D0x7fffdeb175a0, n=3D0, assign=3Dtrue) at > /home/devel/qemu/hw/virtio-pci.c:175 > #10 0x00007ffff7d9ea5f in virtio_pci_start_ioeventfd (proxy=3D0x7fffdeb17= 5a0) > at /home/devel/qemu/hw/virtio-pci.c:230 > #11 0x00007ffff7d9ee51 in virtio_ioport_write (opaque=3D0x7fffdeb175a0, > addr=3D18, val=3D7) at /home/devel/qemu/hw/virtio-pci.c:325 > #12 0x00007ffff7d9f37b in virtio_pci_config_writeb (opaque=3D0x7fffdeb175= a0, > addr=3D18, val=3D7) at /home/devel/qemu/hw/virtio-pci.c:457 > #13 0x00007ffff7e9ac23 in memory_region_iorange_write > (iorange=3D0x7fffb8005cc0, offset=3D18, width=3D1, data=3D7) at > /home/devel/qemu/memory.c:427 > #14 0x00007ffff7e857e2 in ioport_writeb_thunk (opaque=3D0x7fffb8005cc0, > addr=3D61970, data=3D7) at /home/devel/qemu/ioport.c:212 > #15 0x00007ffff7e85197 in ioport_write (index=3D0, address=3D61970, data= =3D7) at > /home/devel/qemu/ioport.c:83 > #16 0x00007ffff7e85d9a in cpu_outb (addr=3D61970, val=3D7 '\a') at > /home/devel/qemu/ioport.c:289 > #17 0x00007ffff7e8a70a in kvm_handle_io (port=3D61970, data=3D0x7ffff7c11= 000, > direction=3D1, size=3D1, count=3D1) at /home/devel/qemu/kvm-all.c:1123 > #18 0x00007ffff7e8ad0a in kvm_cpu_exec (env=3D0x7fffc1688010) at > /home/devel/qemu/kvm-all.c:1271 > #19 0x00007ffff7e595fc in qemu_kvm_cpu_thread_fn (arg=3D0x7fffc1688010) a= t > /home/devel/qemu/cpus.c:733 > #20 0x00007ffff63687f1 in start_thread () from /lib64/libpthread.so.0 > #21 0x00007ffff497b92d in clone () from /lib64/libc.so.6 > (gdb) frame 2 > #2 =A00x00007ffff7e89a3d in kvm_io_ioeventfd_add (section=3D0x7fffbfbf561= 0, > match_data=3Dtrue, data=3D0, fd=3D461) at /home/devel/qemu/kvm-all.c:778 > 778 =A0 =A0 =A0 =A0 =A0 =A0 abort(); > (gdb) l > 773 =A0 =A0 =A0 =A0 assert(match_data && section->size =3D=3D 2); > 774 > 775 =A0 =A0 =A0 =A0 r =3D kvm_set_ioeventfd_pio_word(fd, > section->offset_within_address_space, > 776 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0data, true); > 777 =A0 =A0 =A0 =A0 if (r < 0) { > 778 =A0 =A0 =A0 =A0 =A0 =A0 abort(); > 779 =A0 =A0 =A0 =A0 } This is where graceful fallback code needs to be added. I have added Avi because I'm not very familiar with the new memory API and how it does ioeventfd. Basically we need a way for ioeventfd to fail if we hit rlimit or the in-kernel io bus device limit. Suggestions? (The reason I don't like using has_many_ioeventfds() is because it's ugly and inefficient to open and then close file descriptors - especially in a multithreaded program where file descriptor availability can change while we're executing.) Stefan