Re: QEMU unexpectedly closed the monitor

["Alex Bennée" <alex.bennee@linaro.org>]

Itaru Kitayama <itaru.kitayama@linux.dev> writes:

> Hi Alex,
>
>> On Jul 30, 2024, at 22:25, Alex Bennée <alex.bennee@linaro.org> wrote:
>> 
>> Itaru Kitayama <itaru.kitayama@linux.dev> writes:
>> 
>>> Hi,
>>> 
>>> Executing virt-install with the following options:
>>> 
>>> sudo virt-install --machine=virt --arch=aarch64 --name=test8 --disk
>>> path=/var/lib/libvirt/images/jammy.qcow2,format=qcow2,device=disk,bus=virtio,cache=none
>>> --memory=2048 --vcpu=1 --nographic --check all=off --features acpi=off
>>> --import --os-variant=ubuntu22.04 --virt-type kvm --boot
>>> kernel=Image-guest-cca-v4,initrd=rootfs.cpio,kernel_args='earlycon
>>> console=ttyAMA0 rdinit=/sbin/init rw root=/dev/vda acpi=off' --network
>>> none
>>> 
>>> receives an error:
>>> 
>>> Starting install...
>>> ERROR    internal error: QEMU unexpectedly closed the monitor (vm='test8'): 2024-07-29T06:23:04.717118Z qemu-system-aarch64: could not load kernel '/home/realm/Image-guest-cca-v4’
>>> 
>>> I am not sure how to investigate as the kernel Image (Image-guest-cca-v4) can work if it is used directly with the qemu-system-aarch64, like:
>>> 
>>> sudo qemu-system-aarch64 -kernel Image-guest-cca-v4 -initrd
>>> rootfs.cpio -cpu host -M virt,gic-version=3,acpi=off -enable-kvm -smp
>>> 2 -m 2048M -nographic -append 'earlycon console=ttyAMA0
>>> rdinit=/sbin/init rw root=/dev/vda acpi=off' -net none
>> 
>> Is there any way to get virt-install to dump its QEMU command line?
>> There obviously seems to be something off if it works when you build the
>> command line by hand.
>
> Not in a form we wish, but structured way, see the attached log with
> the —debug option.

You can extract the XML into a file and then do:

  virsh domxml-to-native qemu-argv test.xml 

which will give you an excessive command line like this:

  /usr/bin/qemu-system-aarch64 -name guest=test8,debug-threads=on -object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/home/alex/.config/libvirt/qemu/lib/domain--1-test8/master-key.aes"}' -machine virt-6.2,usb=off,dump-guest-core=off,gic-version=3,memory-backend=mach-virt.ram -accel kvm -cpu host -m 2048 -object '{"qom-type":"memory-backend-ram","id":"mach-virt.ram","size":2147483648}' -overcommit mem-lock=off -smp 1,sockets=1,cores=1,threads=1 -uuid b32a6305-44e2-470e-b514-2c0aff76549b -display none -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/home/alex/.config/libvirt/qemu/lib/domain--1-test8/monitor.sock,server=on,wait=off -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -no-acpi -boot strict=on -kernel /home/realm/Image-v6.10 -initrd /home/realm/rootfs.cpio -append 'earlycon console=ttyAMA0 rdinit=/sbin/init rw root=/dev/vda acpi=on' -device pcie-root-port,port=8,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x1 -device pcie-root-port,port=9,chassis=2,id=pci.2,bus=pcie.0,addr=0x1.0x1 -device pcie-root-port,port=10,chassis=3,id=pci.3,bus=pcie.0,addr=0x1.0x2 -device pcie-root-port,port=11,chassis=4,id=pci.4,bus=pcie.0,addr=0x1.0x3 -device pcie-root-port,port=12,chassis=5,id=pci.5,bus=pcie.0,addr=0x1.0x4 -device pcie-root-port,port=13,chassis=6,id=pci.6,bus=pcie.0,addr=0x1.0x5 -device pcie-root-port,port=14,chassis=7,id=pci.7,bus=pcie.0,addr=0x1.0x6 -device pcie-root-port,port=15,chassis=8,id=pci.8,bus=pcie.0,addr=0x1.0x7 -device pcie-root-port,port=16,chassis=9,id=pci.9,bus=pcie.0,multifunction=on,addr=0x2 -device pcie-root-port,port=17,chassis=10,id=pci.10,bus=pcie.0,addr=0x2.0x1 -device pcie-root-port,port=18,chassis=11,id=pci.11,bus=pcie.0,addr=0x2.0x2 -device pcie-root-port,port=19,chassis=12,id=pci.12,bus=pcie.0,addr=0x2.0x3 -device pcie-root-port,port=20,chassis=13,id=pci.13,bus=pcie.0,addr=0x2.0x4 -device pcie-root-port,port=21,chassis=14,id=pci.14,bus=pcie.0,addr=0x2.0x5 -device qemu-xhci,p2=15,p3=15,id=usb,bus=pci.1,addr=0x0 -device virtio-serial-pci,id=virtio-serial0,bus=pci.2,addr=0x0 -blockdev '{"driver":"file","filename":"/var/lib/libvirt/images/jammy.qcow2","node-name":"libvirt-1-storage","cache":{"direct":true,"no-flush":false},"auto-read-only":true,"discard":"unmap"}' -blockdev '{"node-name":"libvirt-1-format","read-only":false,"cache":{"direct":true,"no-flush":false},"driver":"qcow2","file":"libvirt-1-storage"}' -device virtio-blk-pci,bus=pci.3,addr=0x0,drive=libvirt-1-format,id=virtio-disk0,bootindex=1,write-cache=on -chardev pty,id=charserial0 -serial chardev:charserial0 -chardev socket,id=charchannel0,path=/home/alex/.config/libvirt/qemu/channel/target/domain--1-test8/org.qemu.guest_agent.0,server=on,wait=off -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -audiodev '{"id":"audio1","driver":"none"}' -device virtio-balloon-pci,id=balloon0,bus=pci.4,addr=0x0 -object '{"qom-type":"rng-random","id":"objrng0","filename":"/dev/urandom"}' -device virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.5,addr=0x0 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on

You then need to manually strip out all the various chardevs for libvirt
control sockets and you can an equivalent command line you can run from
the console. One thing that did jump out as a bit weird to me was:

 -rtc base=utc -no-shutdown -no-acpi -boot strict=on \
 -kernel /home/realm/Image-v6.10 \
 -initrd /home/realm/rootfs.cpio \
 -append 'earlycon console=ttyAMA0 rdinit=/sbin/init rw root=/dev/vda acpi=on'

which seems to be both disabling ACPI and then enabling it in the kernel
options. Maybe that is a firmware thing?

>
> Itaru.
>
> [2. application/octet-stream; log]...
>
>
>> 
>>> 
>>> Userland is Ubuntu 24.04, and the host and guest kernels are for the protected VM (Realm) execution.
>>> Any help would be appreciated.
>>> 
>>> Thanks,
>>> Itaru.
>> 
>> -- 
>> Alex Bennée
>> Virtualisation Tech Lead @ Linaro

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro