From mboxrd@z Thu Jan 1 00:00:00 1970 From: Tom Lyon Subject: Re: BUG: virtio_mmio multi-queue competely broken -- virtio *registers* considered harmful Date: Thu, 02 May 2013 08:17:41 -0700 Message-ID: <51828395.8040701@lyon-about.com> References: <5181E046.7000309@lyon-about.com> <1367494640.25984.54.camel@hornet> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: "rusty@rustcorp.com.au" , "mst@redhat.com" , "virtualization@lists.linux-foundation.org" , "kvm@vger.kernel.org" To: Pawel Moll Return-path: Received: from qmta15.emeryville.ca.mail.comcast.net ([76.96.27.228]:49286 "EHLO qmta15.emeryville.ca.mail.comcast.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1757211Ab3EBPYv (ORCPT ); Thu, 2 May 2013 11:24:51 -0400 In-Reply-To: <1367494640.25984.54.camel@hornet> Sender: kvm-owner@vger.kernel.org List-ID: On 05/02/2013 04:37 AM, Pawel Moll wrote: > Hi Tom, > > On Thu, 2013-05-02 at 04:40 +0100, Tom Lyon wrote: >> Virtiio_mmio attempts to mimic the layout of some control registers = from >> virtio_pci. These registers, in particular VIRTIO_MMIO_QUEUE_SEL an= d >> VIRTIO_PCI_QUEUE_SEL, >> are active in nature, and not just passive like a normal memory >> location. Thus, the host side must react immediately upon write of >> these registers to map some other registers (queue address, size, et= c) >> to queue-specific locations. This is just not possible for mmio, an= d, I >> would argue, not desirable for PCI either. > Could you, please, elaborate more about the environment you are talki= ng > about here? > > The intention of the MMIO device is to behave like a normal memory > mapped peripheral, say serial port. In the world of architecture with= out > separate I/O address space (like ARM, MIPS, SH-4 to name only those I > know anything about), such peripherals are usually mapped into the > virtual address space with special attributes, eg. guaranteeing > transactions order. That's why the host can "react immediately" and t= o > my knowledge multi-queue virtio devices work just fine. > > I'd love see comments from someone with x86 expertise in such areas. > Maybe we are missing some memory barriers here? So the host > implementation would have a chance to react to the QUEUE_SEL access > before servicing other transactions? > > Regards > > Pawe=C5=82 > > Ah, my mistake. I was assuming that mmio just used shared memory=20 regions, not that there was emulated IO registers. I was driven to tha= t assumption by looking at the rpmsg use case in which a main processor=20 talks to satellite processors - there seems to be no hypervisor and therefore no emulated IO in that case. However, looking= =20 deeper, it seems that rpmsg has its own notify mechanism anyways, so it is not so cleanly layered on virtio. Also, for my needs, I want rpmsg=20 and virtio_net. In my desire to use a PCIe card to emulate virtio I am also talking=20 about a non-hypervisor use case - just leveraging the virtio framework= =20 for inter-processor communication. Virtio is so close to being the answer it woud be a=20 shame if it wasn't. So I would propose that virtio be restructured such= =20 that the configuration layout is identical among virtio transports and acts like memory, the=20 configuration location is transport-specific, and the notification=20 mechanism is transport specific - but I could live writing to some queue specific memory/io location (for both=20 pci and mmio).