No virtio devices in SeaBIOS VMs

No virtio devices in SeaBIOS VMs

[Dario Faggioli]

[-- Attachment #1: Type: text/plain, Size: 2169 bytes --]

Hello everyone,

With QEMU 8.2, guests that:
 - use SeaBIOS
 - use something different than "-cpu host" OR don't use "host-phys-
bits=on"
 - have more than 2815 MB of RAM

have problems with their virtio devices and, hence, malfunction in
various ways (e.g., if they're using a virtio disk, they don't find it,
and the VM does not boot).

This broke all of a sudden, as soon as we updated QEMU to 8.2.0 in
openSUSE, and we got some bugreports about it. E.g., this one [1]
includes some logs and info, but I can provide more, if helpful.

I did try master (instead of 8.2, although it was master as of last
week), but the problem was still there.

I've then bisected it to SeaBIOS commit:
96a8d130a8c2e908e357ce62cd713f2cc0b0a2eb ("be less conservative with
the 64bit pci io window"). In fact, falling back to an earlier SeaBIOS
version, before that commit, or even just reverting it [1], solves the
issue.

UEFI guests seem not to be affected in any way, no matter amount of RAM
or CPU model (well, of course, since it's a SeaBIOS commit! :-D What I
mean is that there seems to be nothing in edk2 that induces the same
behavior).

A way of working this around (beside switching to UEFI or to cpu=host)
is to turn on host-phys-bits, e.g., with '<maxphysaddr
mode="passthrough"/>' in the XML.

It is, however, a bit impractical to have to do this for all the VMs
that one may have... Especially if they're a lot! :-)

I know that there have been issues and discussions (and they were also
related to virtio, I think) about these changes already. I don't know
if it's the same or a related problem but is there a way to avoid
having to ask people to change all their VMs' config?

Thanks and Regards,
Dario

[1] https://bugzilla.suse.com/show_bug.cgi?id=1219977
[2] well, I actually reverted a6ed6b701f0a57db0569ab98b0661c12a6ec3ff8
too, for convenience
-- 
Dario Faggioli, Ph.D
http://about.me/dario.faggioli
Virtualization Software Engineer
SUSE Labs, SUSE https://www.suse.com/
-------------------------------------------------------------------
<<This happens because _I_ choose it to happen!>> (Raistlin Majere)

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: No virtio devices in SeaBIOS VMs

[Gerd Hoffmann]

  Hi,

> UEFI guests seem not to be affected in any way, no matter amount of RAM
> or CPU model (well, of course, since it's a SeaBIOS commit! :-D What I
> mean is that there seems to be nothing in edk2 that induces the same
> behavior).

That used to be a problem with UEFI too.

> A way of working this around (beside switching to UEFI or to cpu=host)
> is to turn on host-phys-bits, e.g., with '<maxphysaddr
> mode="passthrough"/>' in the XML.

Sounds like the phys-bits of your vcpus is larger than the value the
host actually supports.  So if the firmware tries to use the whole
address space available things break.

Both UEFI and SeaBIOS have a similar heuristic to figure whenever they
can trust phys-bits or not, and those checks consider upstream qemu
behavior (use phys-bits=40 for all cpu types except 'host').

When this came up with UEFI the root cause turned out to be that suse
qemu derived from upstream qemu.  There have been phys-bits values other
than 40 which where not valid (i.e. larger than supported by the host).

I don't know how that was solved in the end.  But given that we see
similar problems again with SeaBIOS I suspect it was patched in suse
OVMF not suse qemu.

> It is, however, a bit impractical to have to do this for all the VMs
> that one may have... Especially if they're a lot! :-)

I'd actually recommend to run all VMs with host-phys-bits=on (and use
host-phys-bits-limit=value if you need phys-bits being equal on all
machines of a heterogeneous cluster for live migration compatibility).

phys-bits being too big never was a valid configuration.  It only
happened to work because the firmware was very conservative with address
space usage.  That strategy became increasingly problematic though.
These days GPUs and NPUs can have gigabytes of device memory and equally
large pci memory bars ...

take care,
  Gerd