Re: [RFC 0/3] Introduce a new Qemu machine for RISC-V

[Atish Patra <atishp@atishpatra.org>]

On Tue, Apr 19, 2022 at 5:26 PM Atish Patra <atishp@atishpatra.org> wrote:
>
> On Tue, Apr 19, 2022 at 9:51 AM Daniel P. Berrangé <berrange@redhat.com> wrote:
> >
> > On Mon, Apr 11, 2022 at 07:10:06PM -0700, Atish Patra wrote:
> > >
> > > The RISC-V virt machine has helped RISC-V software eco system to evolve at a
> > > rapid pace even in absense of the real hardware. It is definitely commendable.
> > > However, the number of devices & commandline options keeps growing as a result
> > > of that as well. That adds flexibility but will also become bit difficult
> > > to manage in the future as more extension support will be added. As it is the
> > > most commonly used qemu machine, it needs to support all kinds of device and
> > > interrupts as well. Moreover, virt machine has limitations on the maximum
> > > number of harts it can support because of all the MMIO devices it has to support.
> > >
> > > The RISC-V IMSIC specification allows to develop machines completely relying
> > > on MSI and don't care about the wired interrupts at all. It just requires
> > > all the devices to be present behind a PCI bus or present themselves as platform
> > > MSI device. The former is a more common scenario in x86 world where most
> > > of the devices are behind PCI bus. As there is very limited MMIO device
> > > support, it can also scale to very large number of harts.
> > >
> > > That's why, this patch series introduces a minimalistic yet very extensible
> > > forward looking machine called as "RISC-V Mini Computer" or "minic". The
> > > idea is to build PC or server like systems with this machine. The machine can
> > > work with or without virtio framework. The current implementation only
> > > supports RV64. I am not sure if building a RV32 machine would be of interest
> > > for such machines. The only mmio device it requires is clint to emulate
> > > the mtimecmp.
> >

Any other thoughts ?

> > I would ask what you see as the long term future usage for 'virt' vs
> > 'minic' machine types ? Would you expect all existing users of 'virt'
> > to ultimately switch to 'minic', or are there distinct non-overlapping
> > use cases for 'virt' vs 'minic' such that both end up widely used ?
> >
>
> Nope. I don't expect existing 'virt' users to switch to 'minic' as
> they aim to cater to different users.
>
> Here are the major differences
> 1. virt machine supports MMIO devices & wired interrupts. Minic doesn't
> 2. virt machine doesn't support the MSI only option yet (can be added
> though[1]). Minic does.
> 3. Number of cpu supported by virt machine are limited because of the
> MMIO devices. Minic can scale to very
> large numbers of cpu.
> 4. 'Minic' only supports PCI based MSI capable devices. Thus, MSI is a
> mandatory requirement for 'minic' while
> it is optional for 'virt'.
>
> 'Minic' aims towards the users who want to create virtual machines
> that are MSI based and don't care about
> a million options that virt machines provide.  Virt machine is more
> complex so that it can be flexible in terms of
> what it supports. Minic is a minimalistic machine which doesn't need
> to be expanded a lot in the future given that
> most of the devices can be behind PCI.
>
> [1] https://github.com/atishp04/qemu/tree/virt_imsic_only
>
> > Is 'minic' intended to be able to mimic real physical hardware at all,
> > or is it still intended as a purely virtual machine, like a 'virt-ng' ?
> >
>
> Any future hardware that relies only on PCI-MSI based devices, they
> can be created on top of minic.
> At that point, minic will provide a useful abstract for all those
> machines as well. minic doesn't need a virtio framework.
> Thus, it can closely emulate such hardware as well.
>
> > Essentially 'virt' was positioned as the standard machine to use if
> > you want to run a virtual machine, without any particular match to
> > physical hardware. It feels like 'minic' is creating a second machine
> > type to fill the same purpose, so how do users decide which to use ?
> >
>
> I envision 'minic' to be a standard machine for a specific set of user
> requirements (x86 style PCI based
> machines). Virt machine will continue to be a standard machine for
> more generic use cases with MMIO devices.
>
> > > "Naming is hard". I am not too attached with the name "minic".
> > > I just chose least bad one out of the few on my mind :). I am definitely
> > > open to any other name as well.
> > >
> > > The other alternative to provide MSI only option to aia in the
> > > existing virt machine to build MSI only machines. This is certainly doable
> > > and here is the patch that supports that kind of setup.
> > >
> > > https://github.com/atishp04/qemu/tree/virt_imsic_only
> > >
> > > However, it even complicates the virt machine even further with additional
> > > command line option, branches in the code. I believe virt machine will become
> > > very complex if we continue this path. I am interested to learn what everyone
> > > else think.
> > >
> > > It is needless to say that the current version of minic machine
> > > is inspired from virt machine and tries to reuse as much as code possible.
> > > The first patch in this series adds MSI support for serial-pci device so
> > > console can work on such a machine. The 2nd patch moves some common functions
> > > between minic and the virt machine to a helper file. The PATCH3 actually
> > > implements the new minic machine.
> > >
> > > I have not added the fw-cfg/flash support. We probably should add those
> > > but I just wanted to start small and get the feedback first.
> > > This is a work in progress and have few more TODO items before becoming the
> > > new world order :)
> > >
> > > 1. OpenSBI doesn't have PCI support. Thus, no console support for OpenSBI
> > > for now.
> > > 2. The ns16550 driver in OpenSBI also need to support MSI/MSI-X.
> > > 3. Add MSI-X support for serial-pci device.
> > >
> > > This series can boot Linux distros with the minic machine with or without virtio
> > > devices with out-of-tree Linux kernel patches[1]. Here is an example commandline
> > >
> > > Without virtio devices (nvme, serial-pci & e1000e):
> > > =====================================================
> > > /scratch/workspace/qemu/build/qemu-system-riscv64 -cpu rv64 -M minic -m 1G -smp 4 -nographic -nodefaults \
> > > -display none -bios /scratch/workspace/opensbi/build/platform/generic/firmware/fw_dynamic.elf \
> > > -kernel /scratch/workspace/linux/arch/riscv/boot/Image \
> > > -chardev stdio,mux=on,signal=off,id=charconsole0 \
> > > -mon chardev=charconsole0,mode=readline \
> > > -device pci-serial,msi=true,chardev=charconsole0 \
> > > -drive id=disk3,file=/scratch/workspace/rootfs_images//fedora/Fedora-Developer-Rawhide-20211110.n.0-sda.raw,format=raw,if=none,id=drive-system-disk,cache=none,format=raw \
> > > -device nvme,serial=deadbeef,drive=disk3 \
> > > -netdev user,id=usernet,hostfwd=tcp::10000-:22 -device e1000e,netdev=usernet,bus=pcie.0 \
> > > -append 'root=/dev/nvme0n1p2 rw loglevel=8 memblock=debug console=ttyS0 earlycon' -d in_asm -D log.txt -s
> > >
> > > With virtio devices (virtio-scsi-pci, serial-pci & virtio-net-pci)
> > > ==================================================================
> > > /scratch/workspace/qemu/build/qemu-system-riscv64 -cpu rv64 -M minic -m 1G -smp 4 -nographic -nodefaults \
> > > -display none -bios /scratch/workspace/opensbi/build/platform/generic/firmware/fw_dynamic.elf \
> > > -kernel /scratch/workspace/linux/arch/riscv/boot/Image \
> > > -chardev stdio,mux=on,signal=off,id=charconsole0 \
> > > -mon chardev=charconsole0,mode=readline \
> > > -device pci-serial,msi=true,chardev=charconsole0 \
> > > -drive file=/scratch/workspace/rootfs_images//fedora/Fedora-Developer-Rawhide-20211110.n.0-sda.raw,format=raw,if=none,id=drive-system-disk,cache=none \
> > > -device virtio-scsi-pci,id=scsi0 -device scsi-hd,bus=scsi0.0,drive=drive-system-disk,id=system-disk,bootindex=1 \
> > > -netdev user,id=n1,hostfwd=tcp::10000-:22 -device virtio-net-pci,netdev=n1 \
> > > -append 'root=/dev/sda2 rw loglevel=8 memblock=debug console=ttyS0 earlycon'
> > >
> > > The objective of this series is to engage the community to solve this problem.
> > > Please suggest if you have another alternatve solution.
> > >
> > > [1] https://github.com/atishp04/linux/tree/msi_only_console
> > >
> > > Atish Patra (3):
> > > serial: Enable MSI capablity and option
> > > hw/riscv: virt: Move common functions to a separate helper file
> > > hw/riscv: Create a new qemu machine for RISC-V
> > >
> > > configs/devices/riscv64-softmmu/default.mak |   1 +
> > > hw/char/serial-pci.c                        |  36 +-
> > > hw/riscv/Kconfig                            |  11 +
> > > hw/riscv/machine_helper.c                   | 417 +++++++++++++++++++
> > > hw/riscv/meson.build                        |   2 +
> > > hw/riscv/minic.c                            | 438 ++++++++++++++++++++
> > > hw/riscv/virt.c                             | 403 ++----------------
> > > include/hw/riscv/machine_helper.h           |  87 ++++
> > > include/hw/riscv/minic.h                    |  65 +++
> > > include/hw/riscv/virt.h                     |  13 -
> > > 10 files changed, 1090 insertions(+), 383 deletions(-)
> > > create mode 100644 hw/riscv/machine_helper.c
> > > create mode 100644 hw/riscv/minic.c
> > > create mode 100644 include/hw/riscv/machine_helper.h
> > > create mode 100644 include/hw/riscv/minic.h
> > >
> > > --
> > > 2.25.1
> > >
> > >
> >
> > With regards,
> > Daniel
> > --
> > |: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
> > |: https://libvirt.org         -o-            https://fstop138.berrange.com :|
> > |: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
> >
> >
>
>
> --
> Regards,
> Atish



-- 
Regards,
Atish