Re: BUG: virtio_mmio multi-queue competely broken -- virtio *registers* considered harmful

[Tom Lyon <pugs@lyon-about.com>]

On 05/02/2013 04:37 AM, Pawel Moll wrote:
> Hi Tom,
>
> On Thu, 2013-05-02 at 04:40 +0100, Tom Lyon wrote:
>> Virtiio_mmio attempts to mimic the layout of some control registers from
>> virtio_pci.  These registers, in particular VIRTIO_MMIO_QUEUE_SEL and
>> VIRTIO_PCI_QUEUE_SEL,
>> are active in nature, and not just passive like a normal memory
>> location.  Thus, the host side must react immediately upon write of
>> these registers to map some other registers (queue address, size, etc)
>> to queue-specific locations.  This is just not possible for mmio, and, I
>> would argue, not desirable for PCI either.
> Could you, please, elaborate more about the environment you are talking
> about here?
>
> The intention of the MMIO device is to behave like a normal memory
> mapped peripheral, say serial port. In the world of architecture without
> separate I/O address space (like ARM, MIPS, SH-4 to name only those I
> know anything about), such peripherals are usually mapped into the
> virtual address space with special attributes, eg. guaranteeing
> transactions order. That's why the host can "react immediately" and to
> my knowledge multi-queue virtio devices work just fine.
>
> I'd love see comments from someone with x86 expertise in such areas.
> Maybe we are missing some memory barriers here? So the host
> implementation would have a chance to react to the QUEUE_SEL access
> before servicing other transactions?
>
> Regards
>
> Paweł
>
>
Ah, my mistake. I was assuming that mmio just used shared memory 
regions, not that there was emulated IO registers.  I was driven to that
assumption by looking at the rpmsg use case in which a main processor 
talks to satellite processors - there seems to be no
hypervisor and therefore no emulated IO in that case.  However, looking 
deeper, it seems that rpmsg has its own notify mechanism anyways, so it
is not so cleanly layered on virtio.  Also, for my needs, I want rpmsg 
and virtio_net.

In my desire to use a PCIe card to emulate virtio I am also talking 
about a non-hypervisor use case - just  leveraging the virtio framework 
for inter-processor
communication.  Virtio is so close to being the answer it woud be a 
shame if it wasn't. So I would propose that virtio be restructured such 
that the configuration
layout is identical among virtio transports and acts like memory, the 
configuration location is transport-specific, and the notification 
mechanism is transport specific - but I
could live writing to some queue specific memory/io location (for both 
pci and mmio).