Re: what should a virtio-mmio transport without a backend look like?

Re: what should a virtio-mmio transport without a backend look like?

[Pawel Moll]

On Thu, 2013-06-20 at 11:29 +0100, Peter Maydell wrote:
> I'm (finally) trying to add virtio-mmio support properly to
> QEMU now Fred has put all the refactoring foundations in place.
> 
> 1. One question I've run into is: what should a virtio-mmio transport
> with no backend look like to the guest OS? The spec as written
> seems to assume that there's always some backend present.
> (The idea is that QEMU might just always instantiate say 8
> mmio transports, and then whether they actually have a
> blk/net/whatever backend depends on user options).
> 
> It looks as if the current linux driver insists (if it sees a
> device tree node) that the MagicValue register at least is
> correct (otherwise it complains). So one possibility would
> be "MagicValue/Version/VendorID must read as usual, DeviceID
> should read as some special "nothing here" value (0?), everything
> else can RAZ/WI".
> 
> We could just say "all RAZ/WI", since this merely causes Linux
> currently to print a warning about the bad magic number, and
> then subsequently make Linux less alarmist about the zero.
> 
> We could also define that the transport should look as if
> there's some sort of 'null' backend plugged in. This would
> be more effort on the qemu side though, I think.

There are two aspects of the problem:

1. Current implementation of the virtio core won't do anything to the
device if the device/vendor IDs don't match with any of the drivers.

2. All that current virtio-mmio implementation will do is:
* read magic
* read device & vendor id
* write page size

So, a device behaving as you mentioned - magic ok, all register read as
zero, all writes ignored, will do exactly what you want.

Now, as to mandating this:

1. We could mandate device ID 0 (zero) as "NOOP". This is in line with
current ID numbers allocation, just needs formalizing at the top level
of the spec.

2. I have no problem with adding the magic/RAZ/WI behaviour to the
current appendix X, however I must say that the "write page size" will
disappear in the next version of the spec (no more page numbers, just
normal 64-bit addresses), so defining device ID as above will cover your
use case, I believe (as in: correct magic == correct device, NOOP device
== no further actions).

> 2. What was the rationale behind prohibiting interrupt
> line sharing between two virtio-mmio transports? ("device
> operation" section of appendix X). Lack of spare IRQs
> turns out to be the major limit on how many transports you
> can have.

Hm. Simplicity was the intention, really, no hidden agenda. I've never
actually tried to have two platform devices sharing an interrupt, so I'm
not sure how will the generic infrastructure behave in such situation.

> (Is there a mailing list I should be asking this question on?)

virtualization@lists.linux-foundation.org (now on copy)

Paweł


_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: what should a virtio-mmio transport without a backend look like?

[Christopher Covington]

Hi Peter,

On 06/20/2013 07:08 AM, Pawel Moll wrote:
> On Thu, 2013-06-20 at 11:29 +0100, Peter Maydell wrote:
>> I'm (finally) trying to add virtio-mmio support properly to
>> QEMU now Fred has put all the refactoring foundations in place.
>>
>> 1. One question I've run into is: what should a virtio-mmio transport
>> with no backend look like to the guest OS? The spec as written
>> seems to assume that there's always some backend present.
>> (The idea is that QEMU might just always instantiate say 8
>> mmio transports, and then whether they actually have a
>> blk/net/whatever backend depends on user options).
>>
>> It looks as if the current linux driver insists (if it sees a
>> device tree node) that the MagicValue register at least is
>> correct (otherwise it complains). So one possibility would
>> be "MagicValue/Version/VendorID must read as usual, DeviceID
>> should read as some special "nothing here" value (0?), everything
>> else can RAZ/WI".

Might it be reasonably easy to just not enumerate unused transports in the
device tree or kernel parameters?

Regards,
Christopher

-- 
Employee of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by the Linux Foundation.

Re: what should a virtio-mmio transport without a backend look like?

[Peter Maydell]

On 20 June 2013 13:58, Christopher Covington <cov@codeaurora.org> wrote:
>> On Thu, 2013-06-20 at 11:29 +0100, Peter Maydell wrote:
>>> 1. One question I've run into is: what should a virtio-mmio transport
>>> with no backend look like to the guest OS? The spec as written
>>> seems to assume that there's always some backend present.
>>> (The idea is that QEMU might just always instantiate say 8
>>> mmio transports, and then whether they actually have a
>>> blk/net/whatever backend depends on user options).

> Might it be reasonably easy to just not enumerate unused transports
> in the device tree or kernel parameters?

At least for QEMU, the backend that plugs into the transport
isn't created until after the machine model has created
transports and put together the device tree blob. So we don't
really have the information about what devices are going to
appear at the point we're doing this.

thanks
-- PMM

Re: what should a virtio-mmio transport without a backend look like?

[Christopher Covington]

Hi Peter,

On 06/21/2013 11:23 AM, Peter Maydell wrote:
> On 20 June 2013 13:58, Christopher Covington <cov@codeaurora.org> wrote:
>>> On Thu, 2013-06-20 at 11:29 +0100, Peter Maydell wrote:
>>>> 1. One question I've run into is: what should a virtio-mmio transport
>>>> with no backend look like to the guest OS? The spec as written
>>>> seems to assume that there's always some backend present.
>>>> (The idea is that QEMU might just always instantiate say 8
>>>> mmio transports, and then whether they actually have a
>>>> blk/net/whatever backend depends on user options).
> 
>> Might it be reasonably easy to just not enumerate unused transports
>> in the device tree or kernel parameters?
> 
> At least for QEMU, the backend that plugs into the transport
> isn't created until after the machine model has created
> transports and put together the device tree blob. So we don't
> really have the information about what devices are going to
> appear at the point we're doing this.

Would using CONFIG_VIRTIO_MMIO_CMDLINE_DEVICES enumeration instead of device
tree be any easier?

How does the back end know which devices to create?

Thanks,
Christopher

-- 
Employee of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by the Linux Foundation.

Re: what should a virtio-mmio transport without a backend look like?

[Peter Maydell]

On 21 June 2013 17:02, Christopher Covington <cov@codeaurora.org> wrote:
> Would using CONFIG_VIRTIO_MMIO_CMDLINE_DEVICES enumeration
> instead of device tree be any easier?

My general view is that the kernel command line is
the user's to manipulate, and that QEMU shouldn't
touch it at all (just pass it through). (Conversely,
QEMU shouldn't require the user to specify odd
kernel command line arguments in order to make things
work.)

(Having talked with Grant Likely I now have a reasonable
plan for manipulating the dtb to add virtio nodes, so
I'm happy that that's the right way to go.)

As it happens, if you use the command line to specify
a virtio device it doesn't make the same complaint about
bad magic number as if you specify it via dtb, but that
should probably be fixed in the kernel :-)

> How does the back end know which devices to create?

Anything specified via -device on the QEMU command line
is created after the board model has initialized. When a
virtio-blk-backend (or whatever) is created it's automatically
plugged into the first available matching bus (in the same
way that if you create a PCI card via -device it plugs
into the first available PCI slot).

-- PMM

Re: what should a virtio-mmio transport without a backend look like?

[Pawel Moll]

On Fri, 2013-06-21 at 17:41 +0100, Peter Maydell wrote:
> On 21 June 2013 17:02, Christopher Covington <cov@codeaurora.org> wrote:
> > Would using CONFIG_VIRTIO_MMIO_CMDLINE_DEVICES enumeration
> > instead of device tree be any easier?
> 
> My general view is that the kernel command line is
> the user's to manipulate, and that QEMU shouldn't
> touch it at all (just pass it through). (Conversely,
> QEMU shouldn't require the user to specify odd
> kernel command line arguments in order to make things
> work.)

I couldn't agree more. Command line is for the user and user only. And
the mention config option is optional ;-) (and, interestingly enough it
wasn't there in the original version of the driver).

> As it happens, if you use the command line to specify
> a virtio device it doesn't make the same complaint about
> bad magic number as if you specify it via dtb, but that
> should probably be fixed in the kernel :-)

I don't really see how this would be possible - the "complaining code"
is just a normal platform device probe function. And whether specified
in the command line or in the tree, it's the same - platform - device.

Are you sure you haven't misspelled the parameter name or something in
the definition syntax? ;-)

Paweł


_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: what should a virtio-mmio transport without a backend look like?

[Peter Maydell]

On 21 June 2013 17:47, Pawel Moll <pawel.moll@arm.com> wrote:
> On Fri, 2013-06-21 at 17:41 +0100, Peter Maydell wrote:
>> As it happens, if you use the command line to specify
>> a virtio device it doesn't make the same complaint about
>> bad magic number as if you specify it via dtb, but that
>> should probably be fixed in the kernel :-)
>
> I don't really see how this would be possible - the "complaining code"
> is just a normal platform device probe function.

Sorry, you're correct and I misremembered. I just retested
with specifying via command line and it behaves the same
way as via dtb.

-- PMM

Re: what should a virtio-mmio transport without a backend look like?

[Christopher Covington]

Hi Peter,

On 06/21/2013 01:01 PM, Peter Maydell wrote:
> On 21 June 2013 17:47, Pawel Moll <pawel.moll@arm.com> wrote:
>> On Fri, 2013-06-21 at 17:41 +0100, Peter Maydell wrote:
>>> As it happens, if you use the command line to specify
>>> a virtio device it doesn't make the same complaint about
>>> bad magic number as if you specify it via dtb, but that
>>> should probably be fixed in the kernel :-)
>>
>> I don't really see how this would be possible - the "complaining code"
>> is just a normal platform device probe function.
> 
> Sorry, you're correct and I misremembered. I just retested
> with specifying via command line and it behaves the same
> way as via dtb.

I was going to say something. I see that complaint all the time.

Anyhow, I just did a quick experiment with 0-size block devices, and they seem
to work for me, although trying to mount the device yields the confusing
message, "mount: mounting /dev/vda on mount failed: Invalid argument".

Regards,
Christopher

-- 
Employee of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by the Linux Foundation.

Re: what should a virtio-mmio transport without a backend look like?

[Peter Maydell]

On 21 June 2013 19:01, Christopher Covington <cov@codeaurora.org> wrote:
> Anyhow, I just did a quick experiment with 0-size block devices, and they seem
> to work for me, although trying to mount the device yields the confusing
> message, "mount: mounting /dev/vda on mount failed: Invalid argument".

I'm confused -- what's the significance of zero size
block devices?

thanks
-- PMM

Re: what should a virtio-mmio transport without a backend look like?

[Christopher Covington]

On 06/21/2013 02:28 PM, Peter Maydell wrote:
> On 21 June 2013 19:01, Christopher Covington <cov@codeaurora.org> wrote:
>> Anyhow, I just did a quick experiment with 0-size block devices, and they seem
>> to work for me, although trying to mount the device yields the confusing
>> message, "mount: mounting /dev/vda on mount failed: Invalid argument".
> 
> I'm confused -- what's the significance of zero size
> block devices?

You were proposing to use a valid/existing MagicValue/Version/VendorID with a
special DeviceID that does nothing. I'm saying why not use a valid/existing
MagicValue/Version/VendorID/DeviceID with a special parameter setting, size=0,
that does nothing?

Christopher

-- 
Employee of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by the Linux Foundation.

Re: what should a virtio-mmio transport without a backend look like?

[Peter Maydell]

On 21 June 2013 19:45, Christopher Covington <cov@codeaurora.org> wrote:
> You were proposing to use a valid/existing MagicValue/Version/VendorID with a
> special DeviceID that does nothing. I'm saying why not use a valid/existing
> MagicValue/Version/VendorID/DeviceID with a special parameter setting, size=0,
> that does nothing?

Ah, I see. Well, it sounds from your quoted mount message as if they
do actually do something (ie the kernel finds a block device and
creates a /dev/vda for it), rather than nothing, which seems like
a bad thing. Also, it's mixing a detail of the backend layer (what
a zero-sized disk happens to look like) with the transport layer,
which seems a bit ugly spec-wise.

(Implementation wise I'm not crazy about it either since it would
be way more complicated than saying "no backend? OK, RAZ/WI".)

-- PMM

Re: what should a virtio-mmio transport without a backend look like?

[Christopher Covington]

On 06/22/2013 06:51 AM, Peter Maydell wrote:
> On 21 June 2013 19:45, Christopher Covington <cov@codeaurora.org> wrote:
>> You were proposing to use a valid/existing MagicValue/Version/VendorID with a
>> special DeviceID that does nothing. I'm saying why not use a valid/existing
>> MagicValue/Version/VendorID/DeviceID with a special parameter setting, size=0,
>> that does nothing?

[...]

> Also, it's mixing a detail of the backend layer (what
> a zero-sized disk happens to look like) with the transport layer,
> which seems a bit ugly spec-wise.

I don't think that has to be the case. From what I understand of your
architecture, the device layer is completely opaque to the transport layer,
and the transport layer is immutable, but surely the device layer will at
least know how many transports are available? As long as that's true, can't
the device layer just create real devices and hook them up to transports, and
then create no-op devices and hook them up to any remaining transports?

> (Implementation wise I'm not crazy about it either since it would
> be way more complicated than saying "no backend? OK, RAZ/WI".)

(I thought virtio block devices were already implemented.)

Christopher

-- 
Employee of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by the Linux Foundation.

Re: what should a virtio-mmio transport without a backend look like?

[Peter Maydell]

On 24 June 2013 13:26, Christopher Covington <cov@codeaurora.org> wrote:
> On 06/22/2013 06:51 AM, Peter Maydell wrote:
>> On 21 June 2013 19:45, Christopher Covington <cov@codeaurora.org> wrote:
>>> You were proposing to use a valid/existing MagicValue/Version/VendorID with a
>>> special DeviceID that does nothing. I'm saying why not use a valid/existing
>>> MagicValue/Version/VendorID/DeviceID with a special parameter setting, size=0,
>>> that does nothing?
>
> [...]
>
>> Also, it's mixing a detail of the backend layer (what
>> a zero-sized disk happens to look like) with the transport layer,
>> which seems a bit ugly spec-wise.
>
> I don't think that has to be the case. From what I understand of your
> architecture, the device layer is completely opaque to the transport layer,
> and the transport layer is immutable, but surely the device layer will at
> least know how many transports are available? As long as that's true, can't
> the device layer just create real devices and hook them up to transports, and
> then create no-op devices and hook them up to any remaining transports?

No, the device creation code just creates all the devices the user
asks for. At the point where it can't find a transport to
plug one into, the device-creation fails with an error message.
It doesn't attempt to find buses with nothing plugged in (since
for instance it's entirely reasonable to have a PCI slot with
no card in it).

>> (Implementation wise I'm not crazy about it either since it would
>> be way more complicated than saying "no backend? OK, RAZ/WI".)
>
> (I thought virtio block devices were already implemented.)

Yes, but "what does a transport do if there's no backend" is
transport code, whereas the block device is backend code.
I actually think the creation of a spurious /dev/vda makes
'zero-sized block backend' a non-starter anyway, though.
It's also neither conceptually pure nor pragmatically easy,
so it falls between two stools.

I could see the rationale for a 'cleanliness of spec' position
that we should handle this with a 'null' backend with a new
DeviceID which is completely functional (including all the host
feature bitmaps, setting up queues, interrupts, etc) but just
doesn't do anything. That would be more complicated to implement
than RAZ/WI, but not impossible.

thanks
-- PMM

Re: what should a virtio-mmio transport without a backend look like?

[Paolo Bonzini]

Il 21/06/2013 20:28, Peter Maydell ha scritto:
> On 21 June 2013 19:01, Christopher Covington <cov@codeaurora.org> wrote:
>> Anyhow, I just did a quick experiment with 0-size block devices, and they seem
>> to work for me, although trying to mount the device yields the confusing
>> message, "mount: mounting /dev/vda on mount failed: Invalid argument".
> 
> I'm confused -- what's the significance of zero size
> block devices?

I wouldn't be sad if QEMU forbade creation of zero-sized block devices. :)

Paolo

Re: what should a virtio-mmio transport without a backend look like?

[Rusty Russell]

Pawel Moll <pawel.moll@arm.com> writes:
> On Thu, 2013-06-20 at 11:29 +0100, Peter Maydell wrote:
>> I'm (finally) trying to add virtio-mmio support properly to
>> QEMU now Fred has put all the refactoring foundations in place.
>> 
>> 1. One question I've run into is: what should a virtio-mmio transport
>> with no backend look like to the guest OS? The spec as written
>> seems to assume that there's always some backend present.
>> (The idea is that QEMU might just always instantiate say 8
>> mmio transports, and then whether they actually have a
>> blk/net/whatever backend depends on user options).
>> 
>> It looks as if the current linux driver insists (if it sees a
>> device tree node) that the MagicValue register at least is
>> correct (otherwise it complains). So one possibility would
>> be "MagicValue/Version/VendorID must read as usual, DeviceID
>> should read as some special "nothing here" value (0?), everything
>> else can RAZ/WI".
>> 
>> We could just say "all RAZ/WI", since this merely causes Linux
>> currently to print a warning about the bad magic number, and
>> then subsequently make Linux less alarmist about the zero.
>> 
>> We could also define that the transport should look as if
>> there's some sort of 'null' backend plugged in. This would
>> be more effort on the qemu side though, I think.
>
> There are two aspects of the problem:
>
> 1. Current implementation of the virtio core won't do anything to the
> device if the device/vendor IDs don't match with any of the drivers.
>
> 2. All that current virtio-mmio implementation will do is:
> * read magic
> * read device & vendor id
> * write page size
>
> So, a device behaving as you mentioned - magic ok, all register read as
> zero, all writes ignored, will do exactly what you want.
>
> Now, as to mandating this:
>
> 1. We could mandate device ID 0 (zero) as "NOOP". This is in line with
> current ID numbers allocation, just needs formalizing at the top level
> of the spec.

FWIW, I'm happy to bless 0 as "no device present".

Cheers,
Rusty.