Re: [Qemu-devel] [PULL 00/48] ivshmem series

["Marc-André Lureau" <mlureau@redhat.com>]

Hi

----- Original Message -----
> Hello!
> 
>  I have merged this PULL into our local repository and my colleague gave it a
>  try.
>  It seems to work, but looks like ivshmem + memdev + hugetlbfs doesn't do the
>  right thing. Memdev does attach itself to a hugetlb, but we cannot actually
>  share data with it.
>  Our command line was:
> --- cut ---
> /usr/local/bin/qemu-system-aarch64 \
>         -name ivshmem1_huge \
>         -machine virt,accel=kvm,usb=off,gic-version=3 \
>         -cpu host -m 2048 \
>         -realtime mlock=off \
>         -smp 8,sockets=8,cores=1,threads=1 \
>         -nographic \
>         -kernel /var/lib/libvirt/images/Image_uio \
>         -append console='ttyAMA0,115200n8 root=/dev/sda4 rootwait
>         hugepages=128 earlycon=pl011,0x9000000' \
>         -device i82801b11-bridge,id=pci.1,bus=pcie.0,addr=0x1 \
>         -device pci-bridge,chassis_nr=2,id=pci.2,bus=pci.1,addr=0x1 \
>         -device virtio-scsi-pci,id=scsi0,bus=pci.2,addr=0x1 \
>         -drive
>         file=/var/lib/libvirt/images/sda3_pool/ivshmem1_huge.qcow2,if=none,id=drive-scsi0-0-0-0,format=qcow2
>         \
>         -device
>         scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1
>         \
>         -object
>         memory-backend-file,size=4G,mem-path=/dev/huge,id=mb1,share=on \
>         -device ivshmem,memdev=mb1 \
>         -msg timestamp=on
> --- cut ---
> 
>  The problem is that memdev creates a file as temporary, and then immediately
>  unlinks it. Therefore, we cannot feed the same file to another VM. These
>  are open files, used by two running VMs, which are supposed to exchange
>  data via ivshmem.
> --- cut ---
> [root@thunderx-2 igor]# ls -l /proc/34845/fd
> total 0
> lrwx------ 1 root root 64 Oct  9 06:59 0 -> /dev/pts/2
> lrwx------ 1 root root 64 Oct  9 06:59 1 -> /dev/pts/2
> ...
> lrwx------ 1 root root 64 Oct  9 06:59 7 -> anon_inode:[eventfd]
> lrwx------ 1 root root 64 Oct  9 06:59 8 ->
> /dev/huge/qemu_back_mem._objects_mb1.lwEeDc (deleted)
> lrwx------ 1 root root 64 Oct  9 06:59 9 -> /dev/kvm
> [root@thunderx-2 igor]# ls -l /proc/34866/fd
> total 0
> lrwx------ 1 root root 64 Oct  9 06:59 0 -> /dev/pts/0
> lrwx------ 1 root root 64 Oct  9 06:59 1 -> /dev/pts/0
> ...
> lrwx------ 1 root root 64 Oct  9 06:59 7 -> anon_inode:[eventfd]
> lrwx------ 1 root root 64 Oct  9 06:59 8 ->
> /dev/huge/qemu_back_mem._objects_mb1.0Myp8M (deleted)
> lrwx------ 1 root root 64 Oct  9 06:59 9 -> /dev/kvm
> --- cut ---
>  Due to the same reason, we cannot also exchange data between host and VM.
>  Is it a flaw or do we just do something wrong?

file_ram_alloc() only allocates with temporary files, even when using share=true.

We could teach it to use an existing file instead. This is an additional feature that could be considered after this series, since it's not a regression. Also, the prefered way to share memory (including hugetlb) is to use the ivshmem-server (that way you have a clear owner, and you can signal vm2vm).

> Tested-by: Igor Skalkin <i.skalkin@samsung.com>

Thanks a lot for testing.

> 
> Kind regards,
> Pavel Fedin
> Expert Engineer
> Samsung Electronics Research center Russia
> 
> 
>