Re: [PATCH v1 0/4] virtio-msg transport layer

["Michael S. Tsirkin" <mst@redhat.com>]

On Wed, Feb 25, 2026 at 08:03:48AM +0000, Bertrand Marquis wrote:
> Hi Manivannan,
> 
> > On 25 Feb 2026, at 08:45, Manivannan Sadhasivam <mani@kernel.org> wrote:
> >
> > Hi Bertrand,
> >
> > On Fri, Feb 20, 2026 at 09:02:12AM +0000, Bertrand Marquis wrote:
> >> Hi Parav,
> >>
> >>> On 20 Feb 2026, at 07:13, Parav Pandit <parav@nvidia.com> wrote:
> >>>
> >>>
> >>>
> >>>> From: Michael S. Tsirkin <mst@redhat.com>
> >>>> Sent: 20 February 2026 05:25 AM
> >>>>
> >>>> On Fri, Feb 13, 2026 at 01:52:06PM +0000, Parav Pandit wrote:
> >>>>> Hi Bill,
> >>>>>
> >>>>>> From: Bill Mills <bill.mills@linaro.org>
> >>>>>> Sent: 26 January 2026 10:02 PM
> >>>>>>
> >>>>>> This series adds the virtio-msg transport layer.
> >>>>>>
> >>>>>> The individuals and organizations involved in this effort have had difficulty in
> >>>>>> using the existing virtio-transports in various situations and desire to add one
> >>>>>> more transport that performs its transport layer operations by sending and
> >>>>>> receiving messages.
> >>>>>>
> >>>>>> Implementations of virtio-msg will normally be done in multiple layers:
> >>>>>> * common / device level
> >>>>>> * bus level
> >>>>>>
> >>>>>> The common / device level defines the messages exchanged between the driver
> >>>>>> and a device. This common part should lead to a common driver holding most
> >>>>>> of the virtio specifics and can be shared by all virtio-msg bus implementations.
> >>>>>> The kernel implementation in [3] shows this separation. As with other transport
> >>>>>> layers, virtio-msg should not require modifications to existing virtio device
> >>>>>> implementations (virtio-net, virtio-blk etc). The common / device level is the
> >>>>>> main focus of this version of the patch series.
> >>>>>>
> >>>>>> The virtio-msg bus level implements the normal things a bus defines
> >>>>>> (enumeration, dma operations, etc) but also implements the message send and
> >>>>>> receive operations. A number of bus implementations are envisioned,
> >>>>>> some of which will be reusable and general purpose. Other bus implementations
> >>>>>> might be unique to a given situation, for example only used by a PCIe card
> >>>>>> and its driver.
> >>>>>>
> >>>>>> The standard bus messages are an effort to avoid different bus implementations
> >>>>>> doing the same thing in different ways for no good reason. However the
> >>>>>> different environments will require different things. Instead of trying to
> >>>>>> anticipate all needs and provide something very abstract, we think
> >>>>>> implementation specific messages will be needed at the bus level. Over time,
> >>>>>> if we see similar messages across multiple bus implementations, we will move to
> >>>>>> standardize a bus level message for that.
> >>>>>>
> >>>>>
> >>>>> I would review more, had first round of sparse review.
> >>>>> Please find few comments/questions below.
> >>>>
> >>>> I'd like to comment that I think it makes sense to have a basic simple transport and
> >>>> then add performance features on top as appropriate.
> >>> Sounds good. Simple but complete is needed.
> >>
> >> Agree.
> >>
> >>>
> >>>> So one way to address some of these comments is to show how
> >>>> they can be addressed with a feature bit down the road.
> >>>>
> >>>>
> >>>>> 1. device number should be 32-bit in struct virtio_msg_header.
> >>>>>> From SIOV_R2 experiences, we learnt that some uses have use case for more than 64k devices.
> >>>>> Also mapping PCI BDF wont be enough in 16-bits considering domain field.
> >>>>>
> >>>>> 2. msg_size of 16-bits for 64KB-8 bytes is too less for data transfer.
> >>>>> For example, a TCP stream wants to send 64KB of data + payload, needs more than 64KB data.
> >>>>> Needs 32-bits.
> >>>>>
> >>>>> 3. BUS_MSG_EVENT_DEVICE to have symmetric name as ADDED and REMOVED (instead of READY)
> >>>>> But more below.
> >>>>>
> >>>>> 4. I dont find the transport messages to read and write to the driver memory supplied in VIRTIO_MSG_SET_VQUEUE addresses to operate
> >>>> the virtqueues.
> >>>>> Dont we need VIRTIO_MEM_READ, VIRTIO_MEM_WRITE request and response?
> >>>>
> >>>> surely this can be an optional transport feature bit.
> >>>>
> >>> How is this optional?
> >>
> >> As said in a previous mail, we have messages already for that.
> >> Please confirm if that answer your question.
> >>
> >>> How can one implement a transport without defining the basic data transfer semantics?
> >>
> >> We did a lot of experiments and we are feature equivalent to PCI, MMIO or Channel I/O.
> >> If anything is missing, we are more than happy to discuss it and solve the issue.
> >>
> >
> > I'd love to have this transport over PCI because it addresses the shortcomings
> > of the existing PCI transport which just assumes that every config space access\
> > is trap and emulate.
> 
> Agree and AMD did exactly that in their demonstrator.
> I will give you answers here as i know them but Edgar will probably give you more
> details (and probably fix my mistakes).
> 
> >
> > But that being said, I somewhat agree with Parav that we should define the bus
> > implementations in the spec to avoid fixing the ABI in the implementations. For
> > instance, if we try to use this transport over PCI, we've got questions like:
> >
> > 1. How the device should be bind to the virtio-msg-pci bus driver and not with
> > the existing virtio-pci driver? Should it use a new Vendor ID or Sub-IDs?
> 
> One bus is appearing as one pci device with its own Vendor ID,


It might be pretty handy to include, as a separate RFC, a quick
description of that binding.


> >
> > 2. How the Virtio messages should be transferred? Is it through endpoint config
> > space or through some other means?
> 
> The virtio messages are transfered using FIFOs stored in the BAR of the PCI
> device (ending up being memory shared between both sides)
> 
> >
> > 3. How the notification be delivered from the device to the host? Through
> > INT-X/MSI/MSI-X or even polling?
> 
> Notifications are delivered through MSI.
> 
> >
> > And these are just a few questions that comes to the top of my head. There could
> > be plenty more.
> >
> > How can we expect all the virtio-msg bus implementations to adhere to the same
> > format so that the interoperability offered by the Virtio spec is guaranteed?
> 
> We spent a lot of time thinking on that (this started around 2 years ago) and we
> discussed several use cases and did some PoC to try to have everything covered
> (secure to non secure and vm to vm using ffa, system to system over PCI or hardware
> messaging system, PCI, Xen specific implementation) to check the needs and try to
> cover as much as we can.
> 
> Now there might be cases we missed but we think that having a purely message based
> interface between the bus and the transport and split responsibilities the way we did
> is allowing lots of different bus implementations without affecting the transport and
> driver/device implementations on top.
> 
> We identified that a common use case will be for the bus to transfer messages using
> FIFOs to optimize speed (at the end you need to have a way to share memory between
> both sides so why not using a part of it to transfer the messages to and reduce the number
> of data exchanges and copies) and this will be used by PCI, Xen, FF-A and others in
> practice (so we might standardize the FIFO format in the future to allow even more code
> reuse between busses).
> 
> If you have any questions or doubts, or if you have a use case that should be investigated
> please tell us.
> 
> Cheers
> Bertrand
> 
> 
> >
> > - Mani
> >
> > --
> > மணிவண்ணன் சதாசிவம்
> 
> IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.