[VirtIO] Support for various devices in Xen

[VirtIO] Support for various devices in Xen

[Andrei Cherechesu (OSS)]

[-- Attachment #1: Type: text/plain, Size: 2312 bytes --]

Hello,

As I've mentioned in previous discussion threads in the xen-devel
community, we are running Xen 4.17 (uprev to 4.18 in progress) on NXP
S32G automotive processors (Cortex-A53 cores) and we wanted to know more
about the support for various VirtIO device types in Xen.

In the Xen 4.17 release notes, the VirtIO standalone backends mentioned
as supported and tested are: virtio-disk, virtio-net, virtio-i2c and
virtio-gpio.

However, we've only managed to successfully set up and try some
use-cases with the virtio-disk standalone backend [0] (which Olexandr
provided) based on the virtio-mmio transport.

As such, we have a few questions, which we haven't been able to figure
out from the mailing list discussions and/or code:
    1. Are there any plans for the virtio-disk repo to have a stable
    version? Is it going to be long-term hosted and maintained in the
    xen-troops github repo? Or was it just an one-time PoC implementation
    and the strategy for future VirtIO devices will be based on a more generic
    approach (i.e., without need for a specific standalone app)?

    2. With regards to the other backends, we want to try out and provide PV
    networking to a DomU based on virtio-net, but we haven't found any available
    resources for it (e.g., the standalone backend implementation if needed for
    control plane, configuration examples, presentations, demos, docs). Does it
    rely on the QEMU virtio-net or vhost implementation? Are there any examples
    on how to set it up? Any required Xen/Linux Kernel/QEMU versions?

    3. What other VirtIO device types are there planned to be supported in Xen?
    I'm supposing libxl will also need changes to accomodate new configuration
    parameters for each of them. Or is there something I'm missing?

    4. Also, while we're at it, are there any plans regarding SCMI
    awareness for Xen (e.g., SCMI Mediator - where the RFC thread from 2022
    seems discontinued)? Or is the preferred approach for sharing SCMI access
    to guests through virtio-scmi?

Thank you very much for the support, once again, and we're also looking
forward to the progress on the rust-vmm initiative.

Regards,
Andrei Cherechesu,
NXP Semiconductors

[0] https://github.com/xen-troops/virtio-disk

[-- Attachment #2: Type: text/html, Size: 4964 bytes --]

Re: [VirtIO] Support for various devices in Xen

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 3199 bytes --]

Hi Andrei,

Great that you are working with Xen and virtio. We at AMD are very
interested in it as well.

I'll let Artem (CCed) and his team comment about the xen-troops
repository.

I can share that from our side we are working toward adding complete
support for using QEMU as virtio-backends provider. Vikram (CCed)
managed to get virtio-net to work with the backend in QEMU. He should be
able to provide instructions. We want to get those patches upstream in
Xen and QEMU as soon as possible.

The libxl changes are lagging behind a bit and you might have to use
device_model_args to enable virtio backends in QEMU.

For SCMI, I'll let Bertrand (CCed) comment.

Cheers,

Stefano


On Thu, 11 Jan 2024, Andrei Cherechesu (OSS) wrote:
> Hello,
> 
> As I've mentioned in previous discussion threads in the xen-devel
> community, we are running Xen 4.17 (uprev to 4.18 in progress) on NXP
> S32G automotive processors (Cortex-A53 cores) and we wanted to know more
> about the support for various VirtIO device types in Xen.
> 
> In the Xen 4.17 release notes, the VirtIO standalone backends mentioned
> as supported and tested are: virtio-disk, virtio-net, virtio-i2c and
> virtio-gpio.
> 
> However, we've only managed to successfully set up and try some
> use-cases with the virtio-disk standalone backend [0] (which Olexandr
> provided) based on the virtio-mmio transport.
> 
> As such, we have a few questions, which we haven't been able to figure
> out from the mailing list discussions and/or code:
>     1. Are there any plans for the virtio-disk repo to have a stable
>     version? Is it going to be long-term hosted and maintained in the
>     xen-troops github repo? Or was it just an one-time PoC implementation
> 
>     and the strategy for future VirtIO devices will be based on a more generic
> 
>     approach (i.e., without need for a specific standalone app)?
> 
> 
>     2. With regards to the other backends, we want to try out and provide PV
> 
>     networking to a DomU based on virtio-net, but we haven't found any available
> 
>     resources for it (e.g., the standalone backend implementation if needed for
> 
>     control plane, configuration examples, presentations, demos, docs). Does it
> 
>     rely on the QEMU virtio-net or vhost implementation? Are there any examples
> 
>     on how to set it up? Any required Xen/Linux Kernel/QEMU versions?
> 
> 
>     3. What other VirtIO device types are there planned to be supported in Xen?
> 
>     I'm supposing libxl will also need changes to accomodate new configuration
> 
>     parameters for each of them. Or is there something I'm missing?
> 
> 
>     4. Also, while we're at it, are there any plans regarding SCMI
>     awareness for Xen (e.g., SCMI Mediator - where the RFC thread from 2022
> 
>     seems discontinued)? Or is the preferred approach for sharing SCMI access
> 
>     to guests through virtio-scmi?
> 
> Thank you very much for the support, once again, and we're also looking
> forward to the progress on the rust-vmm initiative.
> 
> Regards,
> Andrei Cherechesu,
> NXP Semiconductors
> 
> [0] https://github.com/xen-troops/virtio-disk
> 
> 
> 

Re: [VirtIO] Support for various devices in Xen

[Vikram Garhwal]

Hi Andrei & Stefano,
On Thu, Jan 11, 2024 at 04:35:22PM -0800, Stefano Stabellini wrote:
> Hi Andrei,
> 
> Great that you are working with Xen and virtio. We at AMD are very
> interested in it as well.
> 
> I'll let Artem (CCed) and his team comment about the xen-troops
> repository.
> 
> I can share that from our side we are working toward adding complete
> support for using QEMU as virtio-backends provider. Vikram (CCed)
> managed to get virtio-net to work with the backend in QEMU. He should be
> able to provide instructions. We want to get those patches upstream in
> Xen and QEMU as soon as possible.
Actually, QEMU patches are already upstreamed for virtio-blk and virtio-net
devices available in v8.2.0.
For virtio with grants, the patches are WiP.

On Xen side, we are yet to upstream xen-tools patches which basically generate
the right arguments when invoking QEMU.
Here are down stream patches if you want:
1. https://github.com/Xilinx/xen/commit/be35b46e907c7c78fd23888d837475eb28334638
2. For Virtio disk backend:
    https://github.com/Xilinx/xen/commit/947280803294bbb963f428423f679d074c60d632
3. For Virtio-net:
    https://github.com/Xilinx/xen/commit/32fcc702718591270e5c8928b7687d853249c882
4. For changing the machine name to Xenpvh(to align with QEMU changes):
    https://github.com/Xilinx/xen/commit/5f669949c9ffdb1947cb47038956b5fb8eeb072a
> 
> The libxl changes are lagging behind a bit and you might have to use
> device_model_args to enable virtio backends in QEMU.
But QEMU 8.2.0 can still be used for virtio-net on ARM.

@Andrei here is an example on how to use virtio-net with QEMU:
    -device virtio-net-device,id=nic0,netdev=net0,mac=00:16:3e:4f:43:05 \
    -netdev type=tap,id=net0,ifname=vif1.0-emu,br=xenbr0,script=no,downscript=no\
    -machine xenpvh

Please make sure to use xenpvh as QEMU machine.

Regards,
Vikram
> 
> For SCMI, I'll let Bertrand (CCed) comment.
> 
> Cheers,
> 
> Stefano
> 
> 
> On Thu, 11 Jan 2024, Andrei Cherechesu (OSS) wrote:
> > Hello,
> > 
> > As I've mentioned in previous discussion threads in the xen-devel
> > community, we are running Xen 4.17 (uprev to 4.18 in progress) on NXP
> > S32G automotive processors (Cortex-A53 cores) and we wanted to know more
> > about the support for various VirtIO device types in Xen.
> > 
> > In the Xen 4.17 release notes, the VirtIO standalone backends mentioned
> > as supported and tested are: virtio-disk, virtio-net, virtio-i2c and
> > virtio-gpio.
> > 
> > However, we've only managed to successfully set up and try some
> > use-cases with the virtio-disk standalone backend [0] (which Olexandr
> > provided) based on the virtio-mmio transport.
> > 
> > As such, we have a few questions, which we haven't been able to figure
> > out from the mailing list discussions and/or code:
> >     1. Are there any plans for the virtio-disk repo to have a stable
> >     version? Is it going to be long-term hosted and maintained in the
> >     xen-troops github repo? Or was it just an one-time PoC implementation
> > 
> >     and the strategy for future VirtIO devices will be based on a more generic
> > 
> >     approach (i.e., without need for a specific standalone app)?
> > 
> > 
> >     2. With regards to the other backends, we want to try out and provide PV
> > 
> >     networking to a DomU based on virtio-net, but we haven't found any available
> > 
> >     resources for it (e.g., the standalone backend implementation if needed for
> > 
> >     control plane, configuration examples, presentations, demos, docs). Does it
> > 
> >     rely on the QEMU virtio-net or vhost implementation? Are there any examples
> > 
> >     on how to set it up? Any required Xen/Linux Kernel/QEMU versions?
> > 
> > 
> >     3. What other VirtIO device types are there planned to be supported in Xen?
> > 
> >     I'm supposing libxl will also need changes to accomodate new configuration
> > 
> >     parameters for each of them. Or is there something I'm missing?
> > 
> > 
> >     4. Also, while we're at it, are there any plans regarding SCMI
> >     awareness for Xen (e.g., SCMI Mediator - where the RFC thread from 2022
> > 
> >     seems discontinued)? Or is the preferred approach for sharing SCMI access
> > 
> >     to guests through virtio-scmi?
> > 
> > Thank you very much for the support, once again, and we're also looking
> > forward to the progress on the rust-vmm initiative.
> > 
> > Regards,
> > Andrei Cherechesu,
> > NXP Semiconductors
> > 
> > [0] https://github.com/xen-troops/virtio-disk
> > 
> > 
> > 

Re: [VirtIO] Support for various devices in Xen

[Andrei Cherechesu]

Hi Stefano, Vikram, Viresh,

Thank you for your answers and support, and sorry for my late reply.


On 12/01/2024 02:56, Vikram Garhwal wrote:
> Hi Andrei & Stefano,
>
> Actually, QEMU patches are already upstreamed for virtio-blk and virtio-net
> devices available in v8.2.0.
> For virtio with grants, the patches are WiP.
>
> On Xen side, we are yet to upstream xen-tools patches which basically generate
> the right arguments when invoking QEMU.
> Here are down stream patches if you want:
> 1. https://github.com/Xilinx/xen/commit/be35b46e907c7c78fd23888d837475eb28334638
> 2. For Virtio disk backend:
>     https://github.com/Xilinx/xen/commit/947280803294bbb963f428423f679d074c60d632
> 3. For Virtio-net:
>     https://github.com/Xilinx/xen/commit/32fcc702718591270e5c8928b7687d853249c882
> 4. For changing the machine name to Xenpvh(to align with QEMU changes):
>     https://github.com/Xilinx/xen/commit/5f669949c9ffdb1947cb47038956b5fb8eeb072a
>> The libxl changes are lagging behind a bit and you might have to use
>> device_model_args to enable virtio backends in QEMU.
> But QEMU 8.2.0 can still be used for virtio-net on ARM.
>
> @Andrei here is an example on how to use virtio-net with QEMU:
>     -device virtio-net-device,id=nic0,netdev=net0,mac=00:16:3e:4f:43:05 \
>     -netdev type=tap,id=net0,ifname=vif1.0-emu,br=xenbr0,script=no,downscript=no\
>     -machine xenpvh
>
> Please make sure to use xenpvh as QEMU machine.
>
> Regards,
> Vikram
I've managed to successfully get a DomU up and running with the rootfs based on virtio-blk. I'm running QEMU 8.2.1, Xen 4.18 + Vikram's downstream patches, Linux 6.6.12-rt, built through yocto with some changes to xen-tools and QEMU recipes.

However, when also enabling PV networking through virtio-net, it seems that DomU cannot successfully boot. The device model args passed by xen-tools when invoking QEMU look exactly like what Vikram said they should.

While executing `xl -v create ..` I can see some error regarding the device model crashing:

        libxl: debug: libxl_exec.c:127:libxl_report_child_exitstatus: domain 1 device model (dying as expected) [300] died due to fatal signal Killed

But the error is not fatal and the DomU spawn goes on, it boots but never reaches prompt. It seems that kernel crashes silently at some point. Though, the networking interface is present since udev tries to rename it right before boot hangs:

        [    4.376715] vif vif-0 enX0: renamed from eth1

Why would the QEMU DM process be killed, though? Invalid memory access?

Here are the full logs for the "xl create" command [0] and for DomU's dmesg [1].
Any ideas as to why that might happen, some debugging insights, or maybe some configuration details I could have overlooked?

Thank you very much for your help once again.

Regards,
Andrei Cherechesu

[0] https://privatebin.net/?0fc1db27433dbcb5#4twCBMayizr7x89pxPzNqQ198z92q8YxVheHvNDsVAtd
[1] https://privatebin.net/?ec3cb13fe2a086a1#F1zynLYQJCUDfZiwikZtRBEPJTACR2GZX6jn2ShXxmae
>> For SCMI, I'll let Bertrand (CCed) comment.
>>
>> Cheers,
>>
>> Stefano
>>
>>
>> On Thu, 11 Jan 2024, Andrei Cherechesu (OSS) wrote:
>>> Hello,
>>>
>>> As I've mentioned in previous discussion threads in the xen-devel
>>> community, we are running Xen 4.17 (uprev to 4.18 in progress) on NXP
>>> S32G automotive processors (Cortex-A53 cores) and we wanted to know more
>>> about the support for various VirtIO device types in Xen.
>>>
>>> In the Xen 4.17 release notes, the VirtIO standalone backends mentioned
>>> as supported and tested are: virtio-disk, virtio-net, virtio-i2c and
>>> virtio-gpio.
>>>
>>> However, we've only managed to successfully set up and try some
>>> use-cases with the virtio-disk standalone backend [0] (which Olexandr
>>> provided) based on the virtio-mmio transport.
>>>
>>> As such, we have a few questions, which we haven't been able to figure
>>> out from the mailing list discussions and/or code:
>>>     1. Are there any plans for the virtio-disk repo to have a stable
>>>     version? Is it going to be long-term hosted and maintained in the
>>>     xen-troops github repo? Or was it just an one-time PoC implementation
>>>
>>>     and the strategy for future VirtIO devices will be based on a more generic
>>>
>>>     approach (i.e., without need for a specific standalone app)?
>>>
>>>
>>>     2. With regards to the other backends, we want to try out and provide PV
>>>
>>>     networking to a DomU based on virtio-net, but we haven't found any available
>>>
>>>     resources for it (e.g., the standalone backend implementation if needed for
>>>
>>>     control plane, configuration examples, presentations, demos, docs). Does it
>>>
>>>     rely on the QEMU virtio-net or vhost implementation? Are there any examples
>>>
>>>     on how to set it up? Any required Xen/Linux Kernel/QEMU versions?
>>>
>>>
>>>     3. What other VirtIO device types are there planned to be supported in Xen?
>>>
>>>     I'm supposing libxl will also need changes to accomodate new configuration
>>>
>>>     parameters for each of them. Or is there something I'm missing?
>>>
>>>
>>>     4. Also, while we're at it, are there any plans regarding SCMI
>>>     awareness for Xen (e.g., SCMI Mediator - where the RFC thread from 2022
>>>
>>>     seems discontinued)? Or is the preferred approach for sharing SCMI access
>>>
>>>     to guests through virtio-scmi?
>>>
>>> Thank you very much for the support, once again, and we're also looking
>>> forward to the progress on the rust-vmm initiative.
>>>
>>> Regards,
>>> Andrei Cherechesu,
>>> NXP Semiconductors
>>>
>>> [0] https://github.com/xen-troops/virtio-disk
>>>
>>>
>>>

Re: [VirtIO] Support for various devices in Xen

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 6127 bytes --]

-Vikram +Edgar

On Thu, 11 Apr 2024, Andrei Cherechesu wrote:
> Hi Stefano, Vikram, Viresh,
> 
> Thank you for your answers and support, and sorry for my late reply.
> 
> 
> On 12/01/2024 02:56, Vikram Garhwal wrote:
> > Hi Andrei & Stefano,
> >
> > Actually, QEMU patches are already upstreamed for virtio-blk and virtio-net
> > devices available in v8.2.0.
> > For virtio with grants, the patches are WiP.
> >
> > On Xen side, we are yet to upstream xen-tools patches which basically generate
> > the right arguments when invoking QEMU.
> > Here are down stream patches if you want:
> > 1. https://github.com/Xilinx/xen/commit/be35b46e907c7c78fd23888d837475eb28334638
> > 2. For Virtio disk backend:
> >     https://github.com/Xilinx/xen/commit/947280803294bbb963f428423f679d074c60d632
> > 3. For Virtio-net:
> >     https://github.com/Xilinx/xen/commit/32fcc702718591270e5c8928b7687d853249c882
> > 4. For changing the machine name to Xenpvh(to align with QEMU changes):
> >     https://github.com/Xilinx/xen/commit/5f669949c9ffdb1947cb47038956b5fb8eeb072a
> >> The libxl changes are lagging behind a bit and you might have to use
> >> device_model_args to enable virtio backends in QEMU.
> > But QEMU 8.2.0 can still be used for virtio-net on ARM.
> >
> > @Andrei here is an example on how to use virtio-net with QEMU:
> >     -device virtio-net-device,id=nic0,netdev=net0,mac=00:16:3e:4f:43:05 \
> >     -netdev type=tap,id=net0,ifname=vif1.0-emu,br=xenbr0,script=no,downscript=no\
> >     -machine xenpvh
> >
> > Please make sure to use xenpvh as QEMU machine.
>
> I've managed to successfully get a DomU up and running with the rootfs based on virtio-blk. I'm running QEMU 8.2.1, Xen 4.18 + Vikram's downstream patches, Linux 6.6.12-rt, built through yocto with some changes to xen-tools and QEMU recipes.
> 
> However, when also enabling PV networking through virtio-net, it seems that DomU cannot successfully boot. The device model args passed by xen-tools when invoking QEMU look exactly like what Vikram said they should.
> 
> While executing `xl -v create ..` I can see some error regarding the device model crashing:
> 
>         libxl: debug: libxl_exec.c:127:libxl_report_child_exitstatus: domain 1 device model (dying as expected) [300] died due to fatal signal Killed
> 
> But the error is not fatal and the DomU spawn goes on, it boots but never reaches prompt. It seems that kernel crashes silently at some point. Though, the networking interface is present since udev tries to rename it right before boot hangs:
> 
>         [    4.376715] vif vif-0 enX0: renamed from eth1
> 
> Why would the QEMU DM process be killed, though? Invalid memory access?
> 
> Here are the full logs for the "xl create" command [0] and for DomU's dmesg [1].
> Any ideas as to why that might happen, some debugging insights, or maybe some configuration details I could have overlooked?
> 
> Thank you very much for your help once again.

Edgar (CCed) has recently setup a working system with QEMU and the
xenpvh machine for ARM. He should be able to help you.

Cheers,

Stefano


> [0] https://privatebin.net/?0fc1db27433dbcb5#4twCBMayizr7x89pxPzNqQ198z92q8YxVheHvNDsVAtd
> [1] https://privatebin.net/?ec3cb13fe2a086a1#F1zynLYQJCUDfZiwikZtRBEPJTACR2GZX6jn2ShXxmae
> >> For SCMI, I'll let Bertrand (CCed) comment.
> >>
> >> Cheers,
> >>
> >> Stefano
> >>
> >>
> >> On Thu, 11 Jan 2024, Andrei Cherechesu (OSS) wrote:
> >>> Hello,
> >>>
> >>> As I've mentioned in previous discussion threads in the xen-devel
> >>> community, we are running Xen 4.17 (uprev to 4.18 in progress) on NXP
> >>> S32G automotive processors (Cortex-A53 cores) and we wanted to know more
> >>> about the support for various VirtIO device types in Xen.
> >>>
> >>> In the Xen 4.17 release notes, the VirtIO standalone backends mentioned
> >>> as supported and tested are: virtio-disk, virtio-net, virtio-i2c and
> >>> virtio-gpio.
> >>>
> >>> However, we've only managed to successfully set up and try some
> >>> use-cases with the virtio-disk standalone backend [0] (which Olexandr
> >>> provided) based on the virtio-mmio transport.
> >>>
> >>> As such, we have a few questions, which we haven't been able to figure
> >>> out from the mailing list discussions and/or code:
> >>>     1. Are there any plans for the virtio-disk repo to have a stable
> >>>     version? Is it going to be long-term hosted and maintained in the
> >>>     xen-troops github repo? Or was it just an one-time PoC implementation
> >>>
> >>>     and the strategy for future VirtIO devices will be based on a more generic
> >>>
> >>>     approach (i.e., without need for a specific standalone app)?
> >>>
> >>>
> >>>     2. With regards to the other backends, we want to try out and provide PV
> >>>
> >>>     networking to a DomU based on virtio-net, but we haven't found any available
> >>>
> >>>     resources for it (e.g., the standalone backend implementation if needed for
> >>>
> >>>     control plane, configuration examples, presentations, demos, docs). Does it
> >>>
> >>>     rely on the QEMU virtio-net or vhost implementation? Are there any examples
> >>>
> >>>     on how to set it up? Any required Xen/Linux Kernel/QEMU versions?
> >>>
> >>>
> >>>     3. What other VirtIO device types are there planned to be supported in Xen?
> >>>
> >>>     I'm supposing libxl will also need changes to accomodate new configuration
> >>>
> >>>     parameters for each of them. Or is there something I'm missing?
> >>>
> >>>
> >>>     4. Also, while we're at it, are there any plans regarding SCMI
> >>>     awareness for Xen (e.g., SCMI Mediator - where the RFC thread from 2022
> >>>
> >>>     seems discontinued)? Or is the preferred approach for sharing SCMI access
> >>>
> >>>     to guests through virtio-scmi?
> >>>
> >>> Thank you very much for the support, once again, and we're also looking
> >>> forward to the progress on the rust-vmm initiative.
> >>>
> >>> Regards,
> >>> Andrei Cherechesu,
> >>> NXP Semiconductors
> >>>
> >>> [0] https://github.com/xen-troops/virtio-disk
> >>>
> >>>
> >>>
> 

Re: [VirtIO] Support for various devices in Xen

[Edgar E. Iglesias]

On Fri, Apr 12, 2024 at 1:23 AM Stefano Stabellini
<sstabellini@kernel.org> wrote:
>
> -Vikram +Edgar
>
> On Thu, 11 Apr 2024, Andrei Cherechesu wrote:
> > Hi Stefano, Vikram, Viresh,
> >
> > Thank you for your answers and support, and sorry for my late reply.
> >
> >
> > On 12/01/2024 02:56, Vikram Garhwal wrote:
> > > Hi Andrei & Stefano,
> > >
> > > Actually, QEMU patches are already upstreamed for virtio-blk and virtio-net
> > > devices available in v8.2.0.
> > > For virtio with grants, the patches are WiP.
> > >
> > > On Xen side, we are yet to upstream xen-tools patches which basically generate
> > > the right arguments when invoking QEMU.
> > > Here are down stream patches if you want:
> > > 1. https://github.com/Xilinx/xen/commit/be35b46e907c7c78fd23888d837475eb28334638
> > > 2. For Virtio disk backend:
> > >     https://github.com/Xilinx/xen/commit/947280803294bbb963f428423f679d074c60d632
> > > 3. For Virtio-net:
> > >     https://github.com/Xilinx/xen/commit/32fcc702718591270e5c8928b7687d853249c882
> > > 4. For changing the machine name to Xenpvh(to align with QEMU changes):
> > >     https://github.com/Xilinx/xen/commit/5f669949c9ffdb1947cb47038956b5fb8eeb072a
> > >> The libxl changes are lagging behind a bit and you might have to use
> > >> device_model_args to enable virtio backends in QEMU.
> > > But QEMU 8.2.0 can still be used for virtio-net on ARM.
> > >
> > > @Andrei here is an example on how to use virtio-net with QEMU:
> > >     -device virtio-net-device,id=nic0,netdev=net0,mac=00:16:3e:4f:43:05 \
> > >     -netdev type=tap,id=net0,ifname=vif1.0-emu,br=xenbr0,script=no,downscript=no\
> > >     -machine xenpvh
> > >
> > > Please make sure to use xenpvh as QEMU machine.
> >
> > I've managed to successfully get a DomU up and running with the rootfs based on virtio-blk. I'm running QEMU 8.2.1, Xen 4.18 + Vikram's downstream patches, Linux 6.6.12-rt, built through yocto with some changes to xen-tools and QEMU recipes.
> >
> > However, when also enabling PV networking through virtio-net, it seems that DomU cannot successfully boot. The device model args passed by xen-tools when invoking QEMU look exactly like what Vikram said they should.
> >
> > While executing `xl -v create ..` I can see some error regarding the device model crashing:
> >
> >         libxl: debug: libxl_exec.c:127:libxl_report_child_exitstatus: domain 1 device model (dying as expected) [300] died due to fatal signal Killed
> >
> > But the error is not fatal and the DomU spawn goes on, it boots but never reaches prompt. It seems that kernel crashes silently at some point. Though, the networking interface is present since udev tries to rename it right before boot hangs:
> >
> >         [    4.376715] vif vif-0 enX0: renamed from eth1
> >
> > Why would the QEMU DM process be killed, though? Invalid memory access?
> >
> > Here are the full logs for the "xl create" command [0] and for DomU's dmesg [1].
> > Any ideas as to why that might happen, some debugging insights, or maybe some configuration details I could have overlooked?
> >
> > Thank you very much for your help once again.

Hi Andrei,

I'll share some info about my setup:
I'm using:

Xen upstream/master + virtio patches that Vikram shared
Commit 63f66058b5 on this repo/branch:
https://github.com/edgarigl/xen/tree/edgar/virtio-base

QEMU 02e16ab9f4 upstream/master
Linux 09e5c48fea17 upstream/master (from March)
Yocto rootfs.

I had a look at your logs but I can't tell why it's failing on your side.
I've not tried using a virtio-blk as a rootfs on my side, perhaps related.
It would be useful to see a diff of your Xen tree compared to plain
4.18 or whatever base you've got.
You probably don't have
https://github.com/edgarigl/xen/commit/63f66058b508180107963ea37217bc88d813df8f
but if that was the problem, I'd thought virtio wouldn't work at all
with your kernel it could also be related.

My guest config looks like this:
name = "g0"
memory = 1024
vcpus = 1
kernel = "Image"
ramdisk = "core-image-minimal-qemuarm64.rootfs.cpio.gz"
extra = "root=/dev/ram0 console=ttyAMA0"
vif = [ 'model=virtio-net,type=ioemu,bridge=xenbr0' ]
disk = [ '/etc/xen/file.img,,xvda,backendtype=qdisk,specification=virtio' ]

xl launches QEMU with the following args:
/usr/bin/qemu-system-aarch64 -xen-domid 1 -no-shutdown -chardev
socket,id=libxl-cmd,path=/var/run/xen/qmp-libxl-1,server=on,wait=off
-mon chardev=libxl-cmd,mode=control -chardev
socket,id=libxenstat-cmd,path=/var/run/xen/qmp-libxenstat-1,server=on,wait=off
-mon chardev=libxenstat-cmd,mode=control -nodefaults -no-user-config
-xen-attach -name g0 -vnc none -display none -nographic -global
virtio-mmio.force-legacy=false -device
virtio-net-device,id=nic0,netdev=net0,mac=00:16:3e:13:86:9c,iommu_platform=on
-netdev type=tap,id=net0,ifname=vif1.0-emu,br=xenbr0,script=no,downscript=no
-machine xenpvh -m 1024 -device
virtio-blk-device,drive=image,iommu_platform=on -drive
if=none,id=image,format=raw,file=/etc/xen/file.img -global
virtio-mmio.force-legacy=false

Cheers,
Edgar


>
> Edgar (CCed) has recently setup a working system with QEMU and the
> xenpvh machine for ARM. He should be able to help you.
>
> Cheers,
>
> Stefano
>
>
> > [0] https://privatebin.net/?0fc1db27433dbcb5#4twCBMayizr7x89pxPzNqQ198z92q8YxVheHvNDsVAtd
> > [1] https://privatebin.net/?ec3cb13fe2a086a1#F1zynLYQJCUDfZiwikZtRBEPJTACR2GZX6jn2ShXxmae
> > >> For SCMI, I'll let Bertrand (CCed) comment.
> > >>
> > >> Cheers,
> > >>
> > >> Stefano
> > >>
> > >>
> > >> On Thu, 11 Jan 2024, Andrei Cherechesu (OSS) wrote:
> > >>> Hello,
> > >>>
> > >>> As I've mentioned in previous discussion threads in the xen-devel
> > >>> community, we are running Xen 4.17 (uprev to 4.18 in progress) on NXP
> > >>> S32G automotive processors (Cortex-A53 cores) and we wanted to know more
> > >>> about the support for various VirtIO device types in Xen.
> > >>>
> > >>> In the Xen 4.17 release notes, the VirtIO standalone backends mentioned
> > >>> as supported and tested are: virtio-disk, virtio-net, virtio-i2c and
> > >>> virtio-gpio.
> > >>>
> > >>> However, we've only managed to successfully set up and try some
> > >>> use-cases with the virtio-disk standalone backend [0] (which Olexandr
> > >>> provided) based on the virtio-mmio transport.
> > >>>
> > >>> As such, we have a few questions, which we haven't been able to figure
> > >>> out from the mailing list discussions and/or code:
> > >>>     1. Are there any plans for the virtio-disk repo to have a stable
> > >>>     version? Is it going to be long-term hosted and maintained in the
> > >>>     xen-troops github repo? Or was it just an one-time PoC implementation
> > >>>
> > >>>     and the strategy for future VirtIO devices will be based on a more generic
> > >>>
> > >>>     approach (i.e., without need for a specific standalone app)?
> > >>>
> > >>>
> > >>>     2. With regards to the other backends, we want to try out and provide PV
> > >>>
> > >>>     networking to a DomU based on virtio-net, but we haven't found any available
> > >>>
> > >>>     resources for it (e.g., the standalone backend implementation if needed for
> > >>>
> > >>>     control plane, configuration examples, presentations, demos, docs). Does it
> > >>>
> > >>>     rely on the QEMU virtio-net or vhost implementation? Are there any examples
> > >>>
> > >>>     on how to set it up? Any required Xen/Linux Kernel/QEMU versions?
> > >>>
> > >>>
> > >>>     3. What other VirtIO device types are there planned to be supported in Xen?
> > >>>
> > >>>     I'm supposing libxl will also need changes to accomodate new configuration
> > >>>
> > >>>     parameters for each of them. Or is there something I'm missing?
> > >>>
> > >>>
> > >>>     4. Also, while we're at it, are there any plans regarding SCMI
> > >>>     awareness for Xen (e.g., SCMI Mediator - where the RFC thread from 2022
> > >>>
> > >>>     seems discontinued)? Or is the preferred approach for sharing SCMI access
> > >>>
> > >>>     to guests through virtio-scmi?
> > >>>
> > >>> Thank you very much for the support, once again, and we're also looking
> > >>> forward to the progress on the rust-vmm initiative.
> > >>>
> > >>> Regards,
> > >>> Andrei Cherechesu,
> > >>> NXP Semiconductors
> > >>>
> > >>> [0] https://github.com/xen-troops/virtio-disk
> > >>>
> > >>>
> > >>>
> >

Re: [VirtIO] Support for various devices in Xen

[Andrei Cherechesu]

[-- Attachment #1: Type: text/plain, Size: 5028 bytes --]

On 12/04/2024 11:35, Edgar E. Iglesias wrote:
> On Fri, Apr 12, 2024 at 1:23 AM Stefano Stabellini
> <sstabellini@kernel.org> wrote:
>
> -Vikram +Edgar
>
> On Thu, 11 Apr 2024, Andrei Cherechesu wrote:
>>> I've managed to successfully get a DomU up and running with the rootfs based on virtio-blk. I'm running QEMU 8.2.1, Xen 4.18 + Vikram's downstream patches, Linux 6.6.12-rt, built through yocto with some changes to xen-tools and QEMU recipes.
>>>
>>> However, when also enabling PV networking through virtio-net, it seems that DomU cannot successfully boot. The device model args passed by xen-tools when invoking QEMU look exactly like what Vikram said they should.
>>>
>>> While executing `xl -v create ..` I can see some error regarding the device model crashing:
>>>
>>>         libxl: debug: libxl_exec.c:127:libxl_report_child_exitstatus: domain 1 device model (dying as expected) [300] died due to fatal signal Killed
>>>
>>> But the error is not fatal and the DomU spawn goes on, it boots but never reaches prompt. It seems that kernel crashes silently at some point. Though, the networking interface is present since udev tries to rename it right before boot hangs:
>>>
>>>         [    4.376715] vif vif-0 enX0: renamed from eth1
>>>
>>> Why would the QEMU DM process be killed, though? Invalid memory access?
>>>
>>> Here are the full logs for the "xl create" command [0] and for DomU's dmesg [1].
>>> Any ideas as to why that might happen, some debugging insights, or maybe some configuration details I could have overlooked?
>>>
>>> Thank you very much for your help once again.
> Hi Andrei,
>
> I'll share some info about my setup:
> I'm using:
>
> Xen upstream/master + virtio patches that Vikram shared
> Commit 63f66058b5 on this repo/branch:
> https://github.com/edgarigl/xen/tree/edgar/virtio-base
>
> QEMU 02e16ab9f4 upstream/master
> Linux 09e5c48fea17 upstream/master (from March)
> Yocto rootfs.
> I had a look at your logs but I can't tell why it's failing on your side.
> I've not tried using a virtio-blk as a rootfs on my side, perhaps related.
> It would be useful to see a diff of your Xen tree compared to plain
> 4.18 or whatever base you've got.
> You probably don't have
> https://github.com/edgarigl/xen/commit/63f66058b508180107963ea37217bc88d813df8f
> but if that was the problem, I'd thought virtio wouldn't work at all
> with your kernel it could also be related.
>
> My guest config looks like this:
> name = "g0"
> memory = 1024
> vcpus = 1
> kernel = "Image"
> ramdisk = "core-image-minimal-qemuarm64.rootfs.cpio.gz"
> extra = "root=/dev/ram0 console=ttyAMA0"
> vif = [ 'model=virtio-net,type=ioemu,bridge=xenbr0' ]
> disk = [ '/etc/xen/file.img,,xvda,backendtype=qdisk,specification=virtio' ]
Hi Edgar, Stefano,


Thank you for your support. Just wanted to let you know that I've managed to run everything just fine. After some more debugging, I figured the fix I needed was actually in QEMU, one which had been already merged upstream (available in v9.0.0) by Peng Fan [0], following a discussion a few months ago on this list. And since I'm running v8.2.1, my QEMU did not have it.

So I can confirm granting DomU access to rootfs from an SDCard partition over virtio-blk also works. However, if I use the usual Xen PV drivers for block + virtio-net, it does not work (device model fails to spawn). But if I'm using virtio-blk + Xen PV Net, it works :). Also both VirtIOs at the same time work.

This is the configuration:
    vif = [ 'model=virtio-net,type=ioemu' ]
    disk = [ 'phy:/dev/mmcblk0p3,xvda,backendtype=qdisk,specification=virtio' ]
    extra = "root=/dev/vda ..."

Also tested them with foreign mappings and with grant based transports.

Are there any other virtio device types you managed to test so far besides these ones (over virtio-mmio/virtio-grant)? Has anyone tested the rust-vmm vhost-device backends from Viresh with Xen?


Regards,
Andrei C

[0] https://github.com/qemu/qemu/commit/9253d83062268209533df4b29859e5b51a2dc324

>
> xl launches QEMU with the following args:
> /usr/bin/qemu-system-aarch64 -xen-domid 1 -no-shutdown -chardev
> socket,id=libxl-cmd,path=/var/run/xen/qmp-libxl-1,server=on,wait=off
> -mon chardev=libxl-cmd,mode=control -chardev
> socket,id=libxenstat-cmd,path=/var/run/xen/qmp-libxenstat-1,server=on,wait=off
> -mon chardev=libxenstat-cmd,mode=control -nodefaults -no-user-config
> -xen-attach -name g0 -vnc none -display none -nographic -global
> virtio-mmio.force-legacy=false -device
> virtio-net-device,id=nic0,netdev=net0,mac=00:16:3e:13:86:9c,iommu_platform=on
> -netdev type=tap,id=net0,ifname=vif1.0-emu,br=xenbr0,script=no,downscript=no
> -machine xenpvh -m 1024 -device
> virtio-blk-device,drive=image,iommu_platform=on -drive
> if=none,id=image,format=raw,file=/etc/xen/file.img -global
> virtio-mmio.force-legacy=false
>
> Cheers,
> Edgar
>
>
>> Edgar (CCed) has recently setup a working system with QEMU and the
>> xenpvh machine for ARM. He should be able to help you.
>>
>> Cheers,
>>
>> Stefano
>>
>>

[-- Attachment #2: Type: text/html, Size: 7591 bytes --]

Re: [VirtIO] Support for various devices in Xen

[Viresh Kumar]

On 30-04-24, 03:11, Andrei Cherechesu wrote:
> Are there any other virtio device types you managed to test so far
> besides these ones (over virtio-mmio/virtio-grant)? Has anyone
> tested the rust-vmm vhost-device backends from Viresh with Xen?

I have tested them earlier with Xen emulated with the help of Qemu.
Steps are mentioned here:

https://github.com/vireshk/xen-vhost-frontend

-- 
viresh

Re: [VirtIO] Support for various devices in Xen

[Viresh Kumar]

+ Few Linaro folks.

Hi Andrei,

On 11-01-24, 15:32, Andrei Cherechesu (OSS) wrote:
> Hello,
> 
> As I've mentioned in previous discussion threads in the xen-devel
> community, we are running Xen 4.17 (uprev to 4.18 in progress) on NXP
> S32G automotive processors (Cortex-A53 cores) and we wanted to know more
> about the support for various VirtIO device types in Xen.
> 
> In the Xen 4.17 release notes, the VirtIO standalone backends mentioned
> as supported and tested are: virtio-disk, virtio-net, virtio-i2c and
> virtio-gpio.

We worked on I2C and GPIO for Linaro's Project Orko [1].

> However, we've only managed to successfully set up and try some
> use-cases with the virtio-disk standalone backend [0] (which Olexandr
> provided) based on the virtio-mmio transport.
> 
> As such, we have a few questions, which we haven't been able to figure
> out from the mailing list discussions and/or code:
>     1. Are there any plans for the virtio-disk repo to have a stable
>     version? Is it going to be long-term hosted and maintained in the
>     xen-troops github repo? Or was it just an one-time PoC implementation
>     and the strategy for future VirtIO devices will be based on a more generic
>     approach (i.e., without need for a specific standalone app)?
> 
>     2. With regards to the other backends, we want to try out and provide PV
>     networking to a DomU based on virtio-net, but we haven't found any available
>     resources for it (e.g., the standalone backend implementation if needed for
>     control plane, configuration examples, presentations, demos, docs). Does it
>     rely on the QEMU virtio-net or vhost implementation? Are there any examples
>     on how to set it up? Any required Xen/Linux Kernel/QEMU versions?
> 
>     3. What other VirtIO device types are there planned to be supported in Xen?
>     I'm supposing libxl will also need changes to accomodate new configuration
>     parameters for each of them. Or is there something I'm missing?
> 
>     4. Also, while we're at it, are there any plans regarding SCMI
>     awareness for Xen (e.g., SCMI Mediator - where the RFC thread from 2022
>     seems discontinued)? Or is the preferred approach for sharing SCMI access
>     to guests through virtio-scmi?
> 
> Thank you very much for the support, once again, and we're also looking
> forward to the progress on the rust-vmm initiative.

We have tested our work mostly with the rust-vmm community. We have
already upstreamed support for Xen's Foreign and Grant mapping in
there. We are maintaining a separate crate [2] inside of rust-vmm, for
all the backends, which can also be tested easily with Xen I suppose
(now that we have support for a generic Virtio-device in there).

> Regards,
> Andrei Cherechesu,
> NXP Semiconductors
> 
> [0] https://github.com/xen-troops/virtio-disk

-- 
viresh

[1] https://linaro.atlassian.net/wiki/spaces/ORKO/overview
[2] https://github.com/rust-vmm/vhost-device