Running x86 hypervisors inside QEMU x86 TCG outer guest?

Running x86 hypervisors inside QEMU x86 TCG outer guest?

[Edgar E. Iglesias]

Hi!

I'm trying to figure out if QEMU supports running x86 hypervisors (KVM and Xen) inside emulated (TCG) x86_64 guests.
I.e, using TCG emulated x86 hardware virtualization (Intel VMX/EPT or AMD SVM/NPT).

So far, I've been able to run nested x86 KVM when the host is on KVM but no luck with TCG.
With intel TCG cores kvm-ok in the guest says there's no KVM support.
With AMD TCG cores, kvm-ok looks fine but as soon as I try to run a guest, it doesn't run anything.

Am I doing something wrong or is this expected?

Some of the command-lines I've tried:
This is the command-line I use for AMD like outer TCG guest:
${QEMU} -M q35,accel=tcg -m 6G \
        -cpu "EPYC-Rome-v2" \
        -smp 1 \
        -display none \
        -serial mon:stdio \
        -kernel ubuntu-23.10-server-cloudimg-amd64-vmlinuz-generic \
        -append "root=/dev/sda1 console=ttyS0 intel_iommu=on" \
        -initrd ubuntu-23.10-server-cloudimg-amd64-initrd-generic \
        -device intel-iommu,intremap=on,device-iotlb=on \
        -device virtio-net-pci,netdev=net0,romfile="" \
        -netdev type=user,id=net0,hostfwd=tcp::2228-:22 \
        -drive file=ubuntu-23.10-server-cloudimg-amd64.img \
        -drive file=cloudimg-user-data.img,format=raw,snapshot=yes \
        -drive file=hd.qcow2 \
        $*

This is the one for Intel TCG guests:
${QEMU} -M q35,accel=tcg -m 6G \
        -cpu "Haswell,vmx=on" \
        -smp 1 \
        -display none \
        -serial mon:stdio \
        -kernel ubuntu-23.10-server-cloudimg-amd64-vmlinuz-generic \
        -append "root=/dev/sda1 console=ttyS0 intel_iommu=on" \
        -initrd ubuntu-23.10-server-cloudimg-amd64-initrd-generic \
        -device intel-iommu,intremap=on,device-iotlb=on \
        -device virtio-net-pci,netdev=net0,romfile="" \
        -netdev type=user,id=net0,hostfwd=tcp::2228-:22 \
        -drive file=ubuntu-23.10-server-cloudimg-amd64.img \
        -drive file=cloudimg-user-data.img,format=raw,snapshot=yes \
        -drive file=hd.qcow2 \
        $*

This is the one that works (nested KVM):
${QEMU} -M q35,accel=kvm,kernel-irqchip=split -m 6G \
        -cpu host \
        -smp 1 \
        -display none \
        -serial mon:stdio \
        -kernel ubuntu-23.10-server-cloudimg-amd64-vmlinuz-generic \
        -append "root=/dev/sda1 console=ttyS0 intel_iommu=on" \
        -initrd ubuntu-23.10-server-cloudimg-amd64-initrd-generic \
        -device intel-iommu,intremap=on,device-iotlb=on \
        -device virtio-net-pci,netdev=net0,romfile="" \
        -netdev type=user,id=net0,hostfwd=tcp::2228-:22 \
        -drive file=ubuntu-23.10-server-cloudimg-amd64.img \
        -drive file=cloudimg-user-data.img,format=raw,snapshot=yes \
        -drive file=hd.qcow2 \
        $*

Best regards,
Edgar