Re: [PATCH RFCv3 9/9] s390x: initial support for virtio-mem

[Cornelia Huck <cohuck@redhat.com>]

On Fri, 24 Jul 2020 16:37:50 +0200
David Hildenbrand <david@redhat.com> wrote:

> Let's wire up the initial, basic virtio-mem implementation in QEMU. It will
> have to see some important extensions (esp., resizeable allocations)
> before it can be considered production ready. Also, the focus on the Linux
> driver side is on memory hotplug, there are a lot of things optimize in
> the future to improve memory unplug capabilities. However, the basics
> are in place.
> 
> Block migration for now, as we'll have to take proper care of storage
> keys and storage attributes. Also, make sure to not hotplug huge pages
> to a setup without huge pages.
> 
> With a Linux guest that supports virtio-mem (and has
> CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE set for now), a basic example.
> 
> 1. Start a VM with 2G initial memory and a virtio-mem device with a maximum
>    capacity of 18GB (and an initial size of 300M):
>     sudo qemu-system-s390x \
>         --enable-kvm \
>         -m 2G,maxmem=20G \
>         -smp 4 \
>         -nographic \
>         -chardev socket,id=monitor,path=/var/tmp/monitor,server,nowait \
>         -mon chardev=monitor,mode=readline \
>         -net nic -net user \
>         -hda s390x.cow2 \
>         -object memory-backend-ram,id=mem0,size=18G \
>         -device virtio-mem-ccw,id=vm0,memdev=mem0,requested-size=300M
> 
> 2. Query the current size of virtio-mem device:
>     (qemu) info memory-devices
>     Memory device [virtio-mem]: "vm0"
>       memaddr: 0x80000000
>       node: 0
>       requested-size: 314572800
>       size: 314572800
>       max-size: 19327352832
>       block-size: 1048576
>       memdev: /objects/mem0
> 
> 3. Request to grow it to 8GB:
>     (qemu) qom-set vm0 requested-size 8G
>     (qemu) info memory-devices
>     Memory device [virtio-mem]: "vm0"
>       memaddr: 0x80000000
>       node: 0
>       requested-size: 8589934592
>       size: 8589934592
>       max-size: 19327352832
>       block-size: 1048576
>       memdev: /objects/mem0
> 
> 4. Request to shrink it to 800M (might take a while, might not fully
>    succeed, and might not be able to remove memory blocks in Linux):
>   (qemu) qom-set vm0 requested-size 800M
>   (qemu) info memory-devices
>   Memory device [virtio-mem]: "vm0"
>     memaddr: 0x80000000
>     node: 0
>     requested-size: 838860800
>     size: 838860800
>     max-size: 19327352832
>     block-size: 1048576
>     memdev: /objects/mem0
> 
> Note 1: Due to lack of resizeable allocations, we will go ahead and
> reserve a 18GB vmalloc area + size the QEMU RAM slot + KVM mamory slot
> 18GB. echo 1 > /proc/sys/vm/overcommit_memory might be required for
> now. In the future, this area will instead grow on actual demand and shrink
> when possible.
> 
> Note 2: Although virtio-mem-pci is wired up as well, it does not seem to
> work currently on s390x due to lack of MSI-X.

IIRC, you can trick virtio-pci into using msi-x via nvectors. Might be
interesting to try.

> 
> Signed-off-by: David Hildenbrand <david@redhat.com>
> ---
>  hw/s390x/Kconfig           |   1 +
>  hw/s390x/Makefile.objs     |   1 +
>  hw/s390x/s390-virtio-ccw.c | 121 ++++++++++++++++++++++++++++++++++++-
>  hw/virtio/virtio-mem.c     |   2 +
>  4 files changed, 123 insertions(+), 2 deletions(-)